Event camera based time domain mapping encryption imaging method and system

Abstract

The application provides an event camera-based time domain mapping encryption imaging method and system, and belongs to the field of computer vision hardware encryption imaging. The method comprises the following steps: a receiver builds an asynchronous time domain mapping imaging system and constructs a spatial coding key sequence, then obtains an SLM modulation pattern sequence, and obtains a noise SLM modulation pattern sequence and sends it to a sender; the receiver further obtains a spatial decoding key sequence based on the SLM modulation pattern sequence. The sender obtains imaging event information and an encrypted time decoding key sequence based on the noise SLM modulation pattern sequence and sends them to the receiver. The receiver obtains a fragmented grayscale image based on the spatial decoding key sequence and the received information, and then obtains a dense and complete grayscale image through a video completion network based on the fragmented grayscale image. The application can realize real-time encryption and reduce the risk of leakage before the transmission of encrypted information; moreover, the application is not sensitive to the collection environment, has high decoding efficiency and low implementation difficulty.

Description

Technical Field

[0001] This invention belongs to the field of computer vision hardware encrypted imaging, specifically relating to a temporal mapping encrypted imaging method and system based on an event camera. Background Technology

[0002] An event camera is an image sensor that asynchronously acquires brightness change information with extremely high temporal resolution. Compared to traditional image sensors, its advantages lie in its extremely high visual information acquisition frame rate and high dynamic range, while its disadvantage is that it cannot directly acquire grayscale information. However, event camera temporal mapping imaging technology establishes a mapping relationship between light intensity (i.e., grayscale) information and the timestamp of the event by modulating the transmittance of the light incident on the event camera, thereby achieving high dynamic range imaging using the event camera. In addition to using synchronous transmittance modulation devices such as aperture shutters and liquid crystal light valves to synchronously acquire full-frame grayscale images of all pixels, spatial light modulators can also be used to asynchronously modulate the transmittance of different pixels, thereby asynchronously acquiring the grayscale information of each pixel. The acquired asynchronous grayscale information is essentially constructed from the time difference between "the time when each pixel is gated" and "the timestamp of the event generated by that pixel after gated". The information directly acquired by the event camera is only the latter, that is, a series of chaotic event information. Only when the "gating moment" corresponding to each event is clearly defined can grayscale information be correctly decoded. The "gating moment of each pixel" essentially constitutes a spatiotemporal decoding key. This unique temporal mapping encrypted imaging process occurs before the grayscale information is formed; that is, the information collector does not acquire a "grayscale image" in the general sense, but rather a series of discrete "temporal mapping events." The spatiotemporal decoding key is encapsulated in the encrypted imaging acquisition system, and the information collector can be completely unaware of it. Only when the information acquirer uses the correct key to decode can the grayscale information be correctly presented, which greatly reduces the risk of information leakage at the acquisition end.

[0003] Traditional visual information encryption methods can be divided into two categories: software algorithm encryption and hardware imaging encryption. This invention falls under the latter category. Software algorithm encryption methods share the common feature of first acquiring complete grayscale information (images or videos), then using various algorithms and keys for encryption before data transmission. This encryption step aims to prevent leakage during data transmission, but cannot guarantee that information will not be leaked before transmission. Furthermore, this method of acquiring information first and then encrypting it cannot achieve real-time encryption. Hardware imaging encryption, on the other hand, occurs during the grayscale information formation process, with a typical method being the "double random phase coding" optical encryption method. While this method solves the two major problems of leakage risk before transmission and the inability to encrypt in real time, it has high requirements for system stability and poor fault tolerance. Summary of the Invention

[0004] To address the problems in the prior art, this invention proposes a temporal mapping encrypted imaging method and system based on an event camera.

[0005] The technical solution adopted in this invention is as follows:

[0006] In a first aspect, the present invention discloses a temporal mapping encrypted imaging method based on an event camera, comprising:

[0007] 1) The receiver builds an asynchronous time-domain mapping imaging system for acquiring imaging event information. The system includes an imaging objective, a front relay mirror, a spatial light modulator, a rear relay mirror, and an event camera.

[0008] 2) The receiver constructs a spatial coding key sequence, obtains an SLM modulation pattern sequence based on the spatial coding key sequence, and then adds time and space noise to the SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence, which is then sent to the sender.

[0009] 3) The receiver's spatial light modulator plays an SLM modulation pattern sequence, and at the same time, the system is used to photograph a uniformly illuminated flat panel to obtain the spatial decoding key sequence.

[0010] 4) The sender builds the same asynchronous time-domain mapping imaging system as the receiver. The sender's spatial light modulator plays the noise SLM modulation pattern sequence and simultaneously uses its own system to capture the target scene. The event camera records the imaging event information and the time when the noise SLM modulation pattern in the noise SLM modulation pattern sequence is switched. The time when the noise SLM modulation pattern is switched is the time decoding key sequence required for encrypted imaging decoding. The sender then encrypts the time decoding key sequence and finally sends the imaging event information and the encrypted time decoding key sequence to the receiver.

[0011] 5) The receiver decodes the encrypted time decoding key sequence, and decrypts the imaging event information based on the decoding result and the spatial decoding key sequence to obtain a fragmented grayscale image.

[0012] 6) The receiver obtains a dense and complete grayscale image based on the fragmented grayscale image through a video completion network, thus obtaining the image corresponding to the target scene.

[0013] In a second aspect, the present invention discloses a temporal mapping imaging system for implementing the method, comprising a modulation pattern acquisition module, a spatial decoding key sequence acquisition module, an acquisition module, a fragmented grayscale image acquisition module, and a dense grayscale image acquisition module;

[0014] The modulation pattern acquisition module is used to build an asynchronous time-domain mapping imaging system, which includes an imaging objective, a front relay mirror, a spatial light modulator, a rear relay mirror, and an event camera.

[0015] And construct a spatial coding key sequence, obtain an SLM modulation pattern sequence based on the spatial coding key sequence, and then add time and space noise to the SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence and send it to the acquisition module;

[0016] The spatial decoding key sequence acquisition module is used to enable the spatial light modulator to play the SLM modulation pattern sequence, and at the same time use the asynchronous time domain mapping imaging system to photograph a uniformly illuminated flat plate to calibrate and obtain the spatial decoding key sequence.

[0017] The acquisition module is used to build an asynchronous time-domain mapping imaging system and enable its spatial light modulator to play the noise SLM modulation pattern sequence. Simultaneously, it uses its own asynchronous time-domain mapping imaging system to capture the target scene. The event camera records imaging event information and the time when the noise SLM modulation pattern in the noise SLM modulation pattern sequence is switched. The time when the noise SLM modulation pattern is switched is the time decoding key sequence required for encrypted imaging decoding. This time decoding key sequence is then encrypted. Finally, the imaging event information and the encrypted time decoding key sequence are sent to the fragmented grayscale image acquisition module.

[0018] The fragmented grayscale image acquisition module is used to decode the encrypted temporal decoding key sequence, and decrypt the imaging event information by combining the decoding result and the spatial decoding key sequence to obtain the fragmented grayscale image.

[0019] The dense grayscale image acquisition module is used to obtain a dense and complete grayscale image based on the fragmented grayscale image through a video completion network, that is, to obtain the image corresponding to the target scene.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] This invention first constructs an asynchronous temporal mapping imaging system at the receiver, then constructs a spatial coding key sequence and obtains an SLM modulation pattern sequence based on the spatial coding key sequence. The SLM modulation pattern sequence is then used to control the conduction position of the spatial light modulator, enabling asynchronous transmittance modulation of the incident light by the spatial light modulator, and calibrating to obtain a spatial decoding key sequence. Simultaneously, temporal and spatial noise are added to the obtained SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence, which is then sent to the sender. The sender constructs the same asynchronous temporal mapping imaging system as the receiver, with its spatial light modulator playing the noisy SLM modulation pattern sequence. Simultaneously, the sender uses its own system to capture images of the target scene. An event camera records imaging event information and the time of switching the noisy SLM modulation pattern in the noisy SLM modulation pattern sequence. The time of switching the noisy SLM modulation pattern is the temporal decoding key sequence required for encrypted imaging decoding, achieving real-time spatiotemporal encryption of visual information. Since there is a certain risk of leakage when the temporal decoding key sequence is transmitted to the receiver along with the imaging event information, it is necessary to encrypt the temporal decoding key sequence before transmission. The receiver, by combining the temporal and spatial decoding key sequences, can correctly decode the imaging event information to obtain sparse, fragmented grayscale information. This fragmented grayscale information is then fed into a video completion network to obtain a dense, complete grayscale image, thus obtaining the image corresponding to the target scene and completing the visual information decryption. Compared to current hardware-encrypted imaging methods, this invention has lower system stability requirements, higher fault tolerance, and extends the encryption dimension to the temporal dimension, further improving its anti-decryption capability. Currently used optical hardware-encrypted imaging methods, such as "dual random phase coding," introduce random masks in the spatial and spectral domains to modulate the input data. Decryption requires knowing the same random mask. This hardware-encrypted imaging method has high requirements for the stability of the optical system, mainly because the encryption and decryption process relies on the precise performance of the optical components and strict phase matching: 1. The core of dual random phase coding is the random phase mask; any phase error (such as thermal expansion of optical components or small changes caused by dust in the optical path) will affect the accuracy of encryption and decryption. 2. During decryption, it is crucial to ensure that the mask used during encryption is completely identical; otherwise, the image will be unrecoverable or severely distorted. 3. Dual-random phase encoding relies on Fourier transform, and optical Fourier transform systems require strict alignment of the optical path. Even slight misalignments of optical components (such as lenses and mirrors) can distort the Fourier transform, affecting the quality of both the encrypted and decrypted images. Mechanical vibrations or external interference can also cause optical component misalignment. 4. Distortion, defocus, or aberrations in the Fourier transform lens can distort information in the encrypted image. Furthermore, this hardware encryption method essentially processes visual information in the spatial domain and lacks temporal encryption capabilities.This invention essentially involves temporal control of light rays within the field of view, making it insensitive to assembly errors in the optical system. Therefore, it has lower stability requirements and higher fault tolerance. Furthermore, the encryption method of this invention essentially encrypts visual information in both time and space dimensions; the missing decoding key in either dimension will prevent the correct decryption information from being obtained. In conclusion, this invention plays a crucial role in protecting the security of visual information such as images and videos. Attached Figure Description

[0022] Figure 1 This is a system diagram of the DMD modulation asynchronous time-domain mapping imaging system of the present invention;

[0023] Figure 2 This is a system diagram of the LCoS modulated asynchronous time-domain mapping imaging system of the present invention;

[0024] Figure 3 This is a diagram corresponding to the checkerboard-style spatial encoding key in a specific embodiment of the present invention;

[0025] Figure 4 This is a diagram corresponding to the Vironri-form spatial coding key in another specific embodiment of the present invention;

[0026] Figure 5 This is a process diagram of obtaining the SLM modulation pattern sequence according to the present invention;

[0027] Figure 6 This is a diagram of the spatial decoding key obtained by the present invention and the result diagram after morphological processing;

[0028] Figure 7 This is a process diagram of the present invention for capturing encrypted imaging event information and recording the timing of switching noise SLM modulation patterns;

[0029] Figure 8 This is a process diagram of the present invention to obtain a fragmented grayscale image using a time-decoding key sequence and a spatial-decoding key sequence;

[0030] Figure 9 This is a process diagram of the present invention to complete the fragmented grayscale image;

[0031] Figure 10 This is a diagram illustrating the effect of encrypted imaging in this invention. Detailed Implementation

[0032] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0033] To overcome the risk of data leakage before transmission and the inability to encrypt in real time in visual software algorithms, and to overcome the problems of high system stability requirements and poor fault tolerance of optical encryption methods, this invention proposes a temporal mapping encryption imaging method and system based on an event camera.

[0034] The method of this invention is implemented based on an asynchronous temporal mapping imaging system, which includes an imaging objective, a relay mirror, a spatial light modulator (SLM), an event camera, and optional polarizers or polarization beam splitters (PBS). The imaging objective images the external scene onto a virtual image plane; the front relay mirror images the image on this virtual image plane onto the surface of the spatial light modulator; the spatial light modulator can be high-speed modulated, selecting some of its pixels to conduct, thereby continuing to send part of the field of view light to the rear relay mirror; the rear relay mirror receives the field of view light modulated by the spatial light modulator and images it onto the event camera. The spatial light modulator can be implemented as a digital micromirror device (DMD) or a liquid crystal on silicon (LCoS). Due to their different characteristics, different spatial light modulators require slightly different optical system arrangements. Figure 1 and Figure 2 As shown, these are schematic diagrams of two representative asynchronous time-domain mapping imaging systems provided by this invention.

[0035] Figure 1 This invention utilizes a DMD-modulated asynchronous temporal mapping imaging system, with a digital micromirror device (DMM) 3 as the spatial light modulator. The imaging objective 1 images the natural scene onto a virtual image plane; the front relay mirror 2 images the image from the virtual image plane onto the DMM 3; the DMM 3 modulates the field of view of the pixels by changing the deflection angle of the micromirrors on different pixels, allowing the field of view rays from the white pixels to be reflected into the subsequent optical path, while the field of view rays from the black pixels are excluded from the optical system, thus achieving spatial modulation; the rear relay mirror 4 images the modulated field of view rays onto the image plane of the event camera; and the event camera 5 acquires brightness change information on the image plane of the event camera to obtain imaging event information.

[0036] Figure 2This invention utilizes an LCoS-modulated asynchronous temporal mapping imaging system, with a spatial light modulator being a silicon-based liquid crystal 8 (SLC). The imaging objective 1 images the natural scene onto a virtual image plane. A front relay lens 2 sequentially passes the image from the virtual image plane through a first polarizer 6 and a polarizing prism 7 to further image it onto the surface of the SLC. It is necessary to control the polarization direction of the first polarizer 6 to be consistent with the polarization direction of the light transmitted through the polarizing prism 7. The SLC changes the polarization direction of the incident light from each pixel, causing the polarization direction of the white pixels to change by 90 degrees, while the polarization direction of the black pixels remains unchanged. The polarization direction of the second polarizer 9 is perpendicular to that of the first polarizer 6, allowing field rays whose polarization direction has changed by 90 degrees after modulation by the SLC to pass through the second polarizer 9, while field rays whose polarization direction has not changed cannot pass through the second polarizer 9. A rear relay lens 4 images the field rays modulated by the SLC onto the image plane of an event camera. The event camera 5 collects brightness change information on the image plane of the event camera to obtain imaging event information.

[0037] Figure 1 and Figure 2 The spatial light modulator is conjugate to both the virtual image plane and the image plane of the event camera. Its role in the optical path is to regulate external light, ensuring it illuminates only some pixels while blocking the rest. By using the modulation pattern of the spatial light modulator to select different pixels for imaging at different times, the spatiotemporal hardware encryption of this invention is achieved. The newly added conductive area at each moment is the spatial key required for decoding, and the switching moment of each mask is the temporal key required for decoding. The spatial key is set before data acquisition, is independent of the acquisition process, is a static key, is kept by the information receiver, is invisible to the information collector, and is not transmitted with the data. The temporal key is related to the acquisition process, is a dynamic key, is visible to both the information collector and the receiver, and is transmitted with the data.

[0038] The selection of spatial light modulators must meet the following requirements:

[0039] 1. The switching frequency shall not be lower than 1000Hz.

[0040] 2. The contrast ratio should not be lower than 100:1. The switching frequency directly affects the information acquisition frequency and encryption frequency of the asynchronous time-domain mapping imaging system; the contrast ratio refers to the ratio of the transmittance of the light-transmitting part to the transmittance of the light-blocking part in the modulation mask, that is, the contrast ratio of the spatial light modulator refers to the ratio of the maximum light intensity to the minimum light intensity that the spatial light modulator can achieve when modulating the light signal; too low a contrast ratio will affect the signal-to-noise ratio of the acquired grayscale information.

[0041] The spatial light modulator and the event camera are hard synchronized, meaning the event camera can record the moment when the spatial light modulator switches the modulation mask each time, thereby acquiring an accurate time key.

[0042] The temporal mapping encrypted imaging method based on event cameras provided by this invention can be implemented according to the following steps:

[0043] Step 1: The recipient, according to the present invention example Figure 1 Or example Figure 2 Build an asynchronous time-domain mapping imaging system.

[0044] Step 2: The receiver designs a spatial coding key sequence, consisting of p×p spatial coding keys arranged in a preset order. These keys control the position of newly activated pixels in the spatial light modulator. Each spatial coding key has a unique activation position, and all activation positions collectively cover all pixels, ensuring complete activation of the spatial light modulator and guaranteeing no overlap or omission of activation positions. These spatial coding keys cycle sequentially, and their activation areas represent newly selected pixels when the spatial light modulator switches modulation patterns.

[0045] Step 3: The receiver uses the spatial coding key sequence obtained in Step 2 to generate an SLM modulation pattern sequence; then, time and spatial noise are added to the SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence, which is then sent to the sender.

[0046] Step four: The receiver's spatial light modulator plays the SLM modulation pattern sequence obtained in step three and uses its asynchronous time-domain mapping imaging system to photograph a uniformly illuminated flat panel, calibrating the spatial decoding key sequence. This calibrated spatial decoding key sequence is used to decode the imaging event information recorded by the event camera.

[0047] Step 5: The sender sets up an asynchronous time-domain mapping imaging system identical to the receiver's. Its spatial light modulator plays the noisy SLM modulation pattern sequence obtained in Step 3, and it uses its own asynchronous time-domain mapping imaging system to capture the target scene. The event camera records the imaging event information and the time when the noisy SLM modulation pattern in the noisy SLM modulation pattern sequence switches. The time when the modulation pattern switches is the time decoding key sequence required for encrypted imaging decoding. Since the time decoding key sequence and the imaging event information need to be transmitted to the receiver together, and there is a certain risk of leakage when the time decoding key sequence is transmitted together with the encrypted imaging event information, the time decoding key sequence needs to be encrypted before transmission. Finally, the imaging event information and the encrypted time decoding key sequence are sent to the receiver.

[0048] Step Six: After the receiver obtains the imaging event information captured in Step Five and the encrypted time decoding key sequence, it first restores the encrypted time decoding key sequence, and then combines the restored time decoding key sequence with the spatial decoding key sequence obtained in Step Four. Using the principle of time-domain mapping imaging, the receiver decrypts the imaging event information to obtain a fragmented grayscale image.

[0049] Step 7: The receiver obtains a dense and complete grayscale image based on the fragmented grayscale image through a video completion network, thus obtaining the image corresponding to the target scene.

[0050] Specifically: First, fragmented grayscale images are stacked and synthesized into asynchronous grayscale frames. Then, a biased inter-frame coarse optical flow is estimated for the asynchronous grayscale frames. Next, the biased inter-frame coarse optical flow is corrected and refined by combining the intra-frame temporal deviation indicated by the spatial decoding key, resulting in a bias-free intra-frame optical flow. Finally, the fragmented grayscale images and the bias-free intra-frame optical flow are used to reconstruct the missing parts of the fragmented grayscale images using a video completion network, resulting in a dense and complete grayscale image.

[0051] In one specific embodiment of the present invention, step two specifically includes:

[0052] 2.1) Suppose a spatial coding key is an image with width W and height H. Divide the entire image into p×p seed regions. Each spatial coding key selects one type of seed region for conduction, and de-conducts in other seed regions. One form of spatial coding key is a checkerboard pattern. Design a total of p×p spatial coding keys according to the following steps:

[0053] 2.2) Divide the spatially encoded key into grid blocks with a width of W / (s×p) and a height of H / (s×p). Each grid block is (s×p)×(s×p) in size, and further subdivide each grid block into p×p conductive blocks, each of size s×s. In this embodiment, W is 1920, H is 1080, s is 6, and p is 5. For each spatially encoded key, randomly select one conductive block from the p×p conductive blocks of each grid block and set it to 1 (this ensures the block is conductive), while the remaining positions are 0 (this ensures the remaining blocks are not conductive). Different spatially encoded keys cannot select the same conductive block within the same grid block; that is, different spatially encoded keys have different conductive blocks in the same grid block, thus ensuring that the conductive positions of each spatially encoded key are unique. After the above steps, p×p spatially encoded keys with neither repeated nor omitted conductive positions can be obtained. Figure 3 The effect of assigning different colors to the open areas of this p×p spatial encoding key and overlaying them on a single image is shown. It can be seen that the open areas are neither repeated nor omitted.

[0054] In another specific embodiment of the present invention, the specific content of step two is as follows:

[0055] 2.1) Assume that the width of the spatial encoding key is W and the height is H for an image. Divide the entire image into p×p seed regions. For each spatial encoding key, one of the seed regions is selected to be conductive, and the other seed regions are non-conductive. The second form of the spatial encoding key is the Voronoi form. Design a total of p×p spatial encoding keys according to the following steps:

[0056] 2.2) Randomly generate d points within the W×H rectangular region, covering the entire rectangular region, where the d points include the 4 endpoints of the rectangular region. And 0.001×W×H < d < 0.01×W×H. In this embodiment, W is 1920, H is 1080, d is 10000, and p is 5. Use the scipy.spatial.Voronoi method in the python scipy library to generate a Voronoi diagram for these points. The Voronoi diagram divides the plane into several regions, and each region corresponds to a point, representing the region closest to that point.

[0057] 2.3) Group and evenly distribute the points. Divide all the points into p×p groups on average. The regions corresponding to the points with the same serial number are set to 1, representing the conductive regions of the spatial encoding key with that serial number, and the rest are non-conductive. Figure 4 shows the effect of assigning different colors to the conductive regions of these p×p spatial encoding keys and superimposing them on one picture. It can be seen that the conductive regions neither overlap nor遗漏.

[0058] Combined with Figure 5 In a specific embodiment of the present invention, the specific content of step three is as follows:

[0059] 3.1) Assume that the switching time interval of the SLM modulation pattern in the SLM modulation pattern sequence by the spatial light modulator (SLM) is τ. In this embodiment, τ is 2 ms. In the asynchronous time-domain mapping imaging system, the conduction duration of each pixel each time is M×τ, and the reset duration is N×τ, where 0 < M, N < p×p, and M + N = p×p. To achieve the above functions, design the SLM modulation pattern sequence according to the following steps:

[0060] 3.2) The initial image of the SLM modulation pattern sequence is an all-0 image, representing that all pixels are in a non-conductive state.

[0061] 3.3) The k-th image (0 < k ≤ M) in the SLM modulation pattern sequence is the superposition result of the first k spatial encoding keys in the spatial encoding key sequence.

[0062] 3.4) The k-th image (k > M) in the modulation pattern sequence is the superposition result of the (k - M) % (p × p)-th to the k % (p × p)-th spatial encoding keys in the spatial encoding key sequence.

[0063] 3.5) Through steps 3.2) to 3.4), the SLM modulation pattern sequence is obtained.

[0064] 3.6) Add temporal and spatial noise to the SLM modulation pattern sequence to enhance the security of the encryption imaging method. Specifically: Add random spatial noise to each SLM modulation pattern in the SLM modulation pattern sequence. The form of the spatial noise is r discrete pixels with a value of 1, where 0 < r < (W × H) × N / (10 × (M + N)). In this embodiment, r is taken as 5000. These random spatial noise positions will cause the pixels in the reset state to conduct randomly for a period of time, thus generating some additional noise events, thereby increasing the chaos of the encrypted information. In order to ensure that these random spatial noises do not affect the normal temporal mapping gray-scale acquisition, it is necessary to ensure that no noise appears in each pixel for at least (N / 2) × τ time before effective conduction to ensure sufficient reset time. Therefore, the appearance positions of these random spatial noises are limited to the regions outside the lower N / 2 spatial key conduction regions; that is, the random spatial noises are added to the pixels in the reset state and within the first (N / 2) × τ time of the reset duration.

[0065] Add temporal noise to the switching moments of the SLM modulation pattern sequence that has already been added with spatial noise. The specific method is to change the switching time interval of the SLM modulation pattern from τ to where the meaning is a Gaussian random number with a mean of 0 and a range of ±σ. σ should be less than τ / 2 and is taken as 200 μs in this embodiment. Since the switching moment of each mask is the time decoding key sequence required for decoding, the purpose of adding temporal noise to the switching moment is to increase the chaos of the time decoding key and reduce the risk of brute-force deciphering the time decoding key sequence. Finally, the noisy SLM modulation pattern sequence is obtained and sent to the sender.

[0066] In a specific implementation of the present invention, the specific content of step four is:

[0067] 4.1) Place a uniformly illuminated flat plate in front of the imaging objective lens 1 of the asynchronous temporal mapping imaging system, and adjust the brightness of the flat plate so that each pixel that enters the conduction state on the image plane of the event camera in the asynchronous temporal mapping imaging system can trigger a positive event within one τ time. Among them, when the brightness change on the event camera pixel reaches the trigger threshold of the event camera, an event will be triggered. A brighter change represents a positive event, and a darker change represents a negative event.

[0068] 4.2) The spatial light modulator plays the SLM modulation pattern sequence obtained in step 3.5) and captures the moment of each modulation pattern switching by an event camera that is hard synchronized with the spatial light modulator, as well as the positive events triggered by each pixel on the image plane of the event camera during the switching of the SLM modulation pattern.

[0069] 4.3) Extract the switching time t of the i-th SLM modulation pattern. i Until the next switching time t i The positive events generated between +τ, the spatial location of these positive events is time t. i The newly added conduction positions in the SLM modulation pattern are used to construct a binarized image, where the spatial positions of these positive events are marked as 1, and the remaining positions are marked as 0, thus identifying the spatial decoding key D of the i-th image. i It is worth noting that the spatial encoding key designed in step two and the spatial decoding key obtained in step four are similar, but not entirely the same. The spatial encoding key designed in step two is the new conductive area added each time the spatial light modulator switches modulation patterns, while the spatial decoding key obtained in step four is the new conductive area added to the image plane of the event camera at each modulation pattern switch. The former is imaged to the latter through the rear repeater. Since the magnification of the rear repeater is not necessarily 1, and the pixel size of the spatial light modulator is not exactly the same as the pixel size of the event camera, the spatial encoding key designed in step two and the spatial decoding key obtained in step four are similar, but not entirely the same.

[0070] 4.4) Sequentially label the spatial decoding keys for p×p switching times.

[0071] 4.5) Some spatial light modulators may have spatial defects during modulation, such as gaps between the edges of micromirrors in a digital micromirror device (DMD) or some micromirrors not responding. These spatial defects can cause holes or bright spot noise in the spatial decoding key obtained in step 4.4). Therefore, this invention uses morphological processing operations to filter out bright spot noise and fill holes. Specifically, a morphological opening operation is first performed on the spatial decoding key obtained in step 4.4) using a 5×5 rectangular kernel to filter out bright spot noise; then a morphological closing operation is performed using a 5×5 rectangular kernel to fill holes. Figure 6 The effects of morphological operations are demonstrated. The spatial decoding keys, after undergoing morphological operations, are arranged in order to form a spatial decoding key sequence.

[0072] Simultaneously, starting from the first corrected spatial decoding key in the spatial decoding key sequence, an intra-frame time deviation indicator is generated for every p×p corrected spatial decoding keys; the intra-frame time deviation indicator is a matrix, and for the on-state of the i-th corrected spatial decoding key in every p×p corrected spatial decoding keys, the corresponding position of its intra-frame time deviation indicator is assigned the value i×τ.

[0073] Combination Figure 7 In one specific embodiment of the present invention, step five specifically includes:

[0074] 5.1) The sender builds the same asynchronous time-domain mapping imaging system as the receiver. Then, the sender's spatial light modulator loads the noise SLM modulation pattern sequence obtained in step 3.6) and simultaneously starts its own asynchronous time-domain mapping imaging system to capture the target scene. During the shooting process, the event camera, which is hard-synchronized with the spatial light modulator, can record encrypted imaging event information and the time when the noise SLM modulation pattern in the noise SLM modulation pattern sequence is switched.

[0075] 5.2) Each time the noise SLM modulation pattern is switched, pixels of the event camera in a ratio of 1 / (p×p) enter the effective conduction state, while a small number of pixels enter the ineffective conduction state due to the noise added in step 3.6). That is, pixels that should not have been conducting become conducting due to the noise. These pixel positions will experience a process of light intensity increasing from 0, thus triggering a positive event. Figure 7 The events in the imaging event information are marked in red. Additionally, approximately 1 / (p×p) of the event camera pixels transition from the on state to the reset state; these trigger negative events upon entering the reset state and are marked in blue. In a specific embodiment of the invention, only positive events generated after entering the on state are considered, while negative events generated after entering the reset state are filtered out to reduce the amount of data during data transmission.

[0076] 5.3) In a specific embodiment of the present invention, the time decoding key sequence is encrypted using the AES symmetric encryption method before being transmitted to the receiver along with the imaging event information. The encryption process is as follows:

[0077] 5.3.1) Generate a random 256-bit key for AES-256 encryption.

[0078] 5.3.2) Convert the time decoding key sequence into a byte stream data_bytes and pad it with data to ensure that its length is a multiple of the block size AES.block_size (16 bytes).

[0079] 5.3.3) Create an AES cipher AES.new(key,AES.MODE_CBC) and call cipher.encrypt(pad(data_bytes,AES.block_size)) to encrypt the data and obtain the ciphertext. Here, AES.MODE_CBC indicates that AES-CBC mode is used.

[0080] 5.3.4) After encryption is complete, the generated initialization vector (IV) and ciphertext are written into a binary file and transmitted to the receiver along with the key. The IV is random data generated during each encryption operation to improve encryption security.

[0081] Combination Figure 8 In one specific embodiment of the present invention, step six specifically includes:

[0082] 7.1) The receiver decrypts the ciphertext in step 5.3.4) based on the key in step 5.3.1) and the initialization vector IV in step 5.3.4), removes padding, and restores the time decoding key sequence obtained in step 5.2).

[0083] 7.2) Let c be the sequence number of the switching time of a certain noise SLM modulation pattern in the entire time decoding key sequence, then the switching time is t. c In step 4.5), select the spatial decoding key with sequence number i = c%(p×p) after morphological operations from the spatial decoding key sequence. In the encrypted imaging event, the acquisition time range is [t]. c ,t c +Mτ] is a set of positive events whose spatial positions are set to 1 in the spatial decoding key after morphological operations. In this set, for a given spatial position (x, y), there exist one or more positive events with different timestamps, where the event whose timestamp is closest to t is the positive event. c Apart from the initial positive event (IPE), the remaining events are called trailing events. Trailing events can be filtered out during correct decoding, but will become interference information during incorrect decoding.

[0084] 7.3) For any spatial location (x, y) where the spatial decoding key is 1 in step 7.2), filter out the trailing events at that location from the positive event set, extract the IPE, and calculate the IPE timestamp and the switching modulation pattern time t. c Time difference t *(x, y). Based on the temporal mapping imaging principle (Bao Y, Sun L, Ma Y, et al. Temporal-mapping photography for event cameras [C] / / European Conference on Computer Vision. Springer, Cham, 2025: 55-72.), the grayscale value I(x, y) at the location where the positive event occurred is calculated using the following formula:

[0085]

[0086] Where C PD V is the capacitance of the photodiode in the event camera. ref V is the reference voltage for the event camera, which is 0 because all pixels are blocked in the reset state; I(x, y) is the grayscale value (light intensity) of pixel (x, y) when it is turned on; TR(t) is the light intensity modulation function, which can be referenced to the response function of the spatial light modulator used. In the specific embodiment of this invention, since the response speed of the spatial light modulator is extremely fast, this response function can be considered as a unit step function ε(t) that steps to 1 at t=0; thd The preset threshold voltage for the event camera. (Numerator) It is a constant for all pixels, and therefore can be normalized to 1. β is a constant that adjusts the position of the inverse proportional mapping interval, and in the specific embodiment of this invention, it is selected as 5τ.

[0087] 7.4) For each position (x, y) where the spatial decoding key is 1 after morphological operations in step 7.2), a grayscale value I(x, y) can be obtained. This grayscale value is normalized according to the range [1 / (M×τ+β), 1 / β] to obtain the normalized grayscale value.

[0088]

[0089] All normalized grayscale values ​​in a spatial decoding key after morphological operations This constructs a fragmented grayscale image, which represents the fragmented grayscale image of the switching moments of the noise SLM modulation pattern corresponding to the spatial decoding key after morphological operations. The fragmented grayscale image means that in the entire grayscale image, only the conduction positions of the morphologically operated spatial decoding key have grayscale values, while the remaining positions are empty. For each switching moment t of the noise SLM modulation pattern... c Each can produce a fragmented grayscale image I c . Figure 8The process of obtaining a fragmented grayscale image using the temporal decoding key sequence and the spatial decoding key sequence in step seven of the hardware encryption imaging event is demonstrated.

[0090] Combination Figure 9 In one specific embodiment of the present invention, step eight specifically includes:

[0091] 8.1) Divide the fragmented grayscale images obtained in step 7.4) into groups of p×p, and stack the p×p fragmented grayscale images in each group to synthesize a complete asynchronous grayscale frame. The asynchronous grayscale frame is denoted as sum. q Its timestamp is the timestamp t of the first fragmented grayscale image in the group. q The frame rate of an asynchronous grayscale frame is 1 / (p×p×τ).

[0092] 8.2) The asynchronous grayscale frames obtained in step 8.1) are used to estimate optical flow using the RAFT model (Reference: Zachary Teed and Jia Deng. Raft: Recurrent all-pairs optical transforms for optical flow. In ECCV, 2020.) to obtain the biased inter-frame coarse optical flow coarse_flow. q The frame rate of the inter-frame coarse optical flow is 1 / (p×p×ττ), the width is W, the height is H, and the number of channels is 2. (Inter-frame coarse optical flow coarse_flow) q The meaning is asynchronous grayscale frame sum q to the next asynchronous grayscale frame sum q+1 The motion vector of each pixel.

[0093] 8.3) The inter-frame coarse optical flow and intra-frame temporal offset indication obtained in step 8.2) are fed into the optical flow correction and refinement module to obtain the offset-free intra-frame optical flow. The frame rate of the intra-frame optical flow is 1 / τ, the width is W, the height is H, and the number of channels is 2. An inter-frame coarse optical flow (coarse_flow) is generated. q Corresponding to p×p intra-frame optical flow floW q flow q+1 ...,flow q+p×p→1 Intra-frame optical flow refers to the motion vector of each pixel in the time dimension, subdivided into p×p, from the next asynchronous grayscale frame to the next. The intra-frame temporal deviation indicator is generated from the spatial decoding key after morphological operations in step 4.5). The intra-frame temporal deviation indicator is a matrix of width W and height H, where the deviation is calculated for the i-th spatial decoding key D. i The conduction position is assigned a value of i×τ in the intra-frame time offset indication matrix. Figure 9Different values ​​are identified by different colors. The specific implementation method of the optical flow correction and thinning module is as follows: Each asynchronous grayscale frame and its corresponding intra-frame temporal deviation are first normalized, and then concatenated into an image stacking tensor with width W, height H, and number of channels 2; the image stacking tensor is processed by one input convolutional layer input_conv1 and three residual convolutional blocks ResidualBlock to extract features, resulting in a feature tensor with width W, height H, and number of middle channels; the feature tensor is then compared with the inter-frame coarse optical flow coarse_flow obtained in step 8.2). q The input tensor is stacked into a single optical flow correction tensor with width W, height H, and middle channels + 2. This input tensor is then passed through one input convolutional layer (input_conv2), one residual convolutional block (ResidualBlock), and one output convolutional layer (output_conv), ultimately yielding an optical flow correction output tensor with width W, height H, and 2×p×p channels. This output tensor is then reshaped into p×p debiased intra-frame optical flow vectors with width W, height H, and 2 channels each. q flow q+1 ...,flow q+p×p-1 The module uses an input convolutional layer (input_conv1) with 2 input channels and 3 middle channels for output; a residual convolutional block (ResidualBlock) with 3 middle channels for both input and output; an input convolutional layer (input_conv2) with 3 middle channels + 2 and 3 middle channels for output; and an output convolutional layer (output_conv) with 3 middle channels and 2×p×p output. The kernel size (kernel_size) of input_conv1 and input_conv2, and the kernel size (stride) of the output convolutional layers are 3, with a padding of 1, and activation using the LeakyReLU function. The kernel size (ResidualBlock) of the residual convolutional block is 3, with a stride of 1, padding of 1, and activation using the ReLU function. The optical flow correction and thinning module uses intra-frame optical flow pseudo-labels for supervision. Intra-frame optical flow pseudo-labels are obtained by estimating the optical flow from the ground truth values ​​of adjacent intra-frame images using the RAFT model.

[0094] 8.4) Pair the fragmented grayscale image obtained in step 7.4) with the intra-frame optical flow obtained in step 8.3) and feed them into the video completion network (Reference: Zhou S, Li C, Chan KCK, et al. Propainter: Improving propagation and transformer for video inpainting[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision.2023:10477-10486.) to generate a complete dense grayscale image with a frame rate of 1 / τ.

[0095] Figure 10 This demonstrates the effectiveness of the encrypted imaging technology of this invention. The first column shows two visualization forms of the collected real event data: event point clouds and event frames. For ease of display, the event point clouds contain events within the time period [0, 25τ], while the event frames display events within the time period [0, 5τ]. The texture of the grayscale image is difficult to discern from the raw data, unlike the image contour texture typically provided by event cameras, thus highlighting the privacy protection function of encrypted imaging events.

[0096] The true grayscale image is a full-frame grayscale image obtained using synchronous transmittance modulation and the principle of temporal mapping imaging. When the method described in this invention is used, and the correct temporal decoding key sequence and spatial decoding key sequence are used for decoding, the correctly decoded grayscale image is very close to the true grayscale image. In practical applications, if the hard synchronization connection between the event camera and spatial light modulation is not set correctly, causing a fixed deviation between the switching time of the temporal decoding key sequence and the actual modulation pattern, the correct decoding process will be interfered with to some extent. In this case, the decoded image will have a certain brightness deviation from the true grayscale image, but the texture information will not be interfered with. In addition, when the optical path of the asynchronous temporal mapping imaging system is interfered with, causing a slight misalignment between the area of ​​the event camera image plane that is turned on at each moment and the spatial decoding key, only the misaligned area will be unable to decode the correct grayscale value, while the other areas will not be affected, unlike optical hardware encryption methods such as "double random phase coding" which will cause all grayscale values ​​to be unable to be decoded correctly. This demonstrates the robustness of this invention when the encryption process is interfered with.

[0097] Without either the temporal decoding key or the spatial decoding key, grayscale images cannot be decoded correctly. Without the temporal decoding key, it's impossible to know when each spatial location is activated, making it impossible to obtain the correct grayscale value through temporal mapping imaging principles. Without the spatial decoding key, it's impossible to know which spatial locations are activated at a given modulation pattern switching moment. If it's assumed that all locations are activated at that moment, positive events generated after that moment will be used to calculate the grayscale information at that moment. This will cause some areas activated earlier to exhibit brighter grayscale values, and areas activated later to exhibit darker grayscale values. Similarly, using an incorrect random spatial decoding key will also lead to most decoded grayscale values ​​being incorrect. The dual protection of the temporal and spatial decoding keys, along with the unique asynchronous grayscale information acquisition method of this invention, ensures the security of visual information during transmission and acquisition. This dual protection feature of the temporal and spatial decoding keys significantly distinguishes it from current optical hardware encryption methods.

[0098] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A temporal mapping-based encrypted imaging method based on an event camera, characterized in that, include: 1) The receiver builds an asynchronous time-domain mapping imaging system for acquiring imaging event information. The system includes an imaging objective, a front relay mirror, a spatial light modulator, a rear relay mirror, and an event camera. 2) The receiver constructs a spatial coding key sequence, obtains an SLM modulation pattern sequence based on the spatial coding key sequence, and then adds time and space noise to the SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence, which is then sent to the sender. 3) The receiver's spatial light modulator plays an SLM modulation pattern sequence, and at the same time, the system is used to photograph a uniformly illuminated flat panel to obtain the spatial decoding key sequence. 4) The sender builds the same asynchronous time-domain mapping imaging system as the receiver. The sender's spatial light modulator plays the noise SLM modulation pattern sequence and simultaneously uses its own system to capture the target scene. The event camera records the imaging event information and the time when the noise SLM modulation pattern in the noise SLM modulation pattern sequence is switched. The time when the noise SLM modulation pattern is switched is the time decoding key sequence required for encrypted imaging decoding. The sender then encrypts the time decoding key sequence and finally sends the imaging event information and the encrypted time decoding key sequence to the receiver. 5) The receiver decodes the encrypted time decoding key sequence, and decrypts the imaging event information based on the decoding result and the spatial decoding key sequence to obtain a fragmented grayscale image. 6) The receiver obtains a dense and complete grayscale image based on the fragmented grayscale image through a video completion network, thus obtaining the image corresponding to the target scene.

2. The temporal mapping encrypted imaging method based on an event camera according to claim 1, characterized in that, In the asynchronous time-domain mapping imaging system, the imaging objective image the external scene onto a virtual image plane. The front relay mirror images the image on the virtual image plane onto the surface of the spatial light modulator. The spatial light modulator can select some pixels on its surface to conduct, thereby sending part of the field of view light to the rear relay mirror. The rear relay mirror receives the field of view light modulated by the spatial light modulator and images it onto the image plane of the event camera. The event camera collects the brightness change information on the image plane of the event camera to obtain imaging event information. The switching frequency of the spatial light modulator is greater than or equal to 1000Hz, and the contrast ratio of the spatial light modulator is greater than or equal to 100:

1. The spatial light modulator is a digital micromirror device.

3. The method according to claim 1, characterized in that, In step 2), a spatially encoded key sequence is constructed, including: The spatial coding key sequence consists of p×p spatial coding keys arranged in a preset order. The spatial coding keys are used to control the position of newly activated pixels in the spatial light modulator. Let the spatial encoding key be an image with width \(W\) and height \(H\). Divide the spatial encoding key into grid blocks with a total number of \(W\times H / (s\times p\times s\times p)\). Each grid block has a size of \((s\times p)\times(s\times p)\). Then split each grid block into \(p\times p\) conductive blocks, each conductive block having a size of \(s\times s\). For each spatial encoding key, among the \(p\times p\) conductive blocks in each grid block, randomly select one conductive block to be conductive, that is, set this conductive block to 1, and the remaining conductive blocks are non - conductive, that is, set the remaining conductive blocks to 0; where different spatial encoding keys have different conductive blocks in the same grid block; finally, obtain \(p\times p\) spatial encoding keys with non - repeating and non - missing conductive positions.

4. The method according to claim 1, characterized in that, In step 2), construct a spatial encoding key sequence, including: The spatial encoding key sequence is composed of \(p\times p\) spatial encoding keys arranged in a preset order. The spatial encoding key is used to control the positions of the newly conductive pixels of the spatial light modulator. Let the spatial encoding key be an image with width \(W\) and height \(H\). Randomly generate \(d\) points in the image with width \(W\) and height \(H\), where the \(d\) points include the 4 endpoints of the image, and \(0.001\times W\times H < d < 0.01\times W\times H\); then generate a Voronoi diagram for the \(d\) points. The Voronoi diagram divides the image into \(d\) regions, each region corresponding to a point; divide all points into \(p\times p\) groups on average, and set the regions corresponding to the points with the same serial number to 1, representing the conductive regions of the spatial encoding key of that serial number, and the remaining regions to 0, representing non - conductive; finally, obtain \(p\times p\) spatial encoding keys with non - repeating and non - missing conductive positions.

5. The method according to claim 3, characterized in that, In step 2), let the switching time interval of the SLM modulation pattern in the SLM modulation pattern sequence of the spatial light modulator be \(\tau\). In the system, the conduction duration of each pixel of the spatial light modulator each time is \(M\times\tau\), and the reset duration is \(N\times\tau\), where \(0 < M,N < p\times p\), and \(M + N=p\times p\); Therefore, the initial SLM modulation pattern in the SLM modulation pattern sequence is an image where all pixels are non - conductive, that is, the SLM modulation pattern at this time is an all - 0 image; The \(k\) - th image in the SLM modulation pattern sequence is the superposition result of the first \(k\) spatial encoding keys, where \(0 < k\leq M\); When \(k > M\), the \(k\) - th SLM modulation pattern in the SLM modulation pattern sequence is the superposition result of the \((k - M)\%(p\times p)\) - th spatial encoding key to the \(k\%(p\times p)\) - th spatial encoding key; Add time and spatial noise to the SLM modulation pattern sequence, including: Add random spatial noise to each SLM modulation pattern in the SLM modulation pattern sequence. The form of the random spatial noise is \(r\) discrete pixel points with a value of 1, where \(0 < r <(W\times H)\times N / (10\times(M + N))\); the random spatial noise is added to the pixels in the reset state and within the first \((N / 2)\times\tau\) time of the reset duration; Change the switching time interval of the SLM modulation pattern from τ to in, It is a Gaussian random number with a mean of 0 and a range of ±σ, where σ is less than τ / 2.

6. The method according to claim 5, characterized in that, The said step 3) includes: 3.1) Place a uniformly illuminated flat plate in front of the imaging objective of the asynchronous time-domain mapping imaging system at the receiver, and adjust the brightness of the flat plate so that each pixel in the event camera image plane of the asynchronous time-domain mapping imaging system that enters the conduction state can trigger a positive event within a time τ. The positive event refers to the event generated when the pixel becomes brighter and reaches the event camera trigger threshold. 3.2) The spatial light modulator plays the SLM modulation pattern sequence and captures the moment of each SLM modulation pattern switching and the positive events triggered by each pixel on the image plane of the event camera during the SLM modulation pattern switching process through an event camera that is hard synchronized with the spatial light modulator. 3.3) Extract the switching time t of the i-th SLM modulation pattern. i Until the next switching time t i For positive events generated between +τ, construct a binary image, set the spatial position of the positive event to 1, and set the other positions to 0, to obtain the spatial decoding key D for the i-th image. i ; 3.4) Obtain the spatial decoding key at p×p switching times in sequence, use morphological opening operation to filter out bright noise in the spatial decoding key, and then use morphological closing operation to fill the holes in the spatial decoding key to obtain the corrected spatial decoding key. All the corrected spatial decoding keys are arranged in sequence to form a spatial decoding key sequence. Simultaneously, starting from the first corrected spatial decoding key in the spatial decoding key sequence, an intra-frame time deviation indicator is generated for every p×p corrected spatial decoding keys; the intra-frame time deviation indicator is a matrix, and for the on-state of the i-th corrected spatial decoding key in every p×p corrected spatial decoding keys, the corresponding position of its intra-frame time deviation indicator is assigned the value i×τ.

7. The method according to claim 1, characterized in that, In step 4), the time decoding key sequence is encrypted using the AES symmetric encryption method.

8. The method according to claim 6, characterized in that, Step 5) includes: 5.1) The receiver decodes the encrypted time decoding key sequence to obtain the time decoding key sequence; 5.2) Let c be the sequence number of the switching time of a certain noise SLM modulation pattern in the entire time decoding key sequence, then the switching time is t. c Then, in the spatial decoding key sequence obtained in step 3), select the corrected spatial decoding key with sequence number i = c%(p×p); 5.3) Obtain the time range [t] from the imaging event information. c ,t c +Mτ] and the set of positive events whose spatial position is set to 1 in the corrected spatial decoding key; obtain the set of events located at spatial position (x,y) with timestamps closest to t. c For a positive event, calculate the timestamp of the positive event and the switching time t. c Time difference t * (x,y), and then calculate the gray value I(x,y) at the location where the positive event occurred according to the principle of temporal mapping imaging. The calculation formula is: Among them, C PD The capacitance of the photodiode in the event camera; V ref V is the reference voltage of the event camera, which is 0; TR(t) is the intensity modulation function, and ε(t) is the unit step function that jumps to 1 at t=0; thd The preset threshold voltage for the event camera; β is a constant for adjusting the position of the inverse mapping interval; 5.4) In a corrected spatial decoding key, each pixel with a spatial position set to 1 has a corresponding grayscale value I(x,y). These grayscale values ​​are normalized according to the range [1 / (M×τ+β), 1 / β] to obtain normalized grayscale values. All normalized grayscale values ​​in the corrected spatial decoding key A fragmented grayscale image is constructed, which is the fragmented grayscale image of the switching moment of the noise SLM modulation pattern corresponding to the corrected spatial decoding key; wherein, a fragmented grayscale image can be obtained for each switching moment of the noise SLM modulation pattern.

9. The method according to claim 8, characterized in that, Step 6) includes: 6.1) Starting from the first fragmented grayscale image, each p×p fragmented grayscale image is divided into a group, and then the p×p fragmented grayscale images in each group are stacked and synthesized into a complete asynchronous grayscale frame, and finally multiple asynchronous grayscale frames are obtained; among them, each asynchronous grayscale frame corresponds to p×p corrected spatial decoding keys, that is, it corresponds to an intra-frame time deviation indicator. 6.2) Use the RAFT model to estimate the optical flow of asynchronous grayscale frames to obtain the inter-frame coarse optical flow with bias; 6.3) Normalize each asynchronous grayscale frame and its corresponding intra-frame temporal deviation, then stack them into an image stacking tensor. Input the image stacking tensor into one input convolutional layer and three residual convolutional layers to extract features and obtain a feature tensor. Stack the feature tensor with the corresponding biased inter-frame coarse optical flow to form an optical flow correction input tensor. Then, input the optical flow correction input tensor into one input convolutional layer, one residual convolutional layer, and one output convolutional layer to obtain an optical flow correction output tensor with width W, height H, and 2×p×p channels. Reorganize the optical flow correction output tensor into p×p bias-free intra-frame optical flows with width W, height H, and 2 channels. 6.4) Pair the fragmented grayscale image obtained in step 5.4) with the debiased intra-frame optical flow obtained in step 6.3) one by one, and input them together into the video completion network to obtain a dense and complete grayscale image, that is, to obtain the image corresponding to the target scene.

10. A time-domain mapping imaging system implementing the method of claim 1, characterized in that, It includes a modulation pattern acquisition module, a spatial decoding key sequence acquisition module, an acquisition module, a fragmented grayscale image acquisition module, and a dense grayscale image acquisition module; The modulation pattern acquisition module is used to build an asynchronous time-domain mapping imaging system, which includes an imaging objective, a front relay mirror, a spatial light modulator, a rear relay mirror, and an event camera. And construct a spatial coding key sequence, obtain an SLM modulation pattern sequence based on the spatial coding key sequence, and then add time and space noise to the SLM modulation pattern sequence to obtain a noisy SLM modulation pattern sequence and send it to the acquisition module; The spatial decoding key sequence acquisition module is used to enable the spatial light modulator to play the SLM modulation pattern sequence, and at the same time use the asynchronous time domain mapping imaging system to photograph a uniformly illuminated flat plate to calibrate and obtain the spatial decoding key sequence. The acquisition module is used to build an asynchronous time-domain mapping imaging system and enable its spatial light modulator to play the noise SLM modulation pattern sequence. Simultaneously, it uses its own asynchronous time-domain mapping imaging system to capture the target scene. The event camera records imaging event information and the time when the noise SLM modulation pattern in the noise SLM modulation pattern sequence is switched. The time when the noise SLM modulation pattern is switched is the time decoding key sequence required for encrypted imaging decoding. This time decoding key sequence is then encrypted. Finally, the imaging event information and the encrypted time decoding key sequence are sent to the fragmented grayscale image acquisition module. The fragmented grayscale image acquisition module is used to decode the encrypted temporal decoding key sequence, and decrypt the imaging event information by combining the decoding result and the spatial decoding key sequence to obtain the fragmented grayscale image. The dense grayscale image acquisition module is used to obtain a dense and complete grayscale image based on the fragmented grayscale image through a video completion network, that is, to obtain the image corresponding to the target scene.

Images