Falling object imaging recovery method fusing image frame and event

Abstract

The invention relates to a falling object imaging recovery method fusing an image frame and an event. The method comprises the following steps: converting a sparse event stream of a falling object collected by an event camera into event features with time sequence and spatial features; extracting RGB features on the RGB image of the falling object collected by the RGB camera; the event features and the RGB features are mapped to a unified feature space, and deep fusion is carried out by adopting an attention mechanism or a guide filtering mode; the fusion features are decoded through a multi-scale decoding network, motion constraint loss and texture enhancement loss are introduced in the decoding stage, and a clear image of the falling object is obtained through gradual recovery. According to the method, high time resolution and dynamic response characteristics of the event camera and texture and color information of the RGB camera are fully utilized, and cross-modal alignment and fusion are combined, so that the problem of how to realize clear, complete and detail-rich image restoration of a small target in a high-speed falling scene is solved; the problem of limitation of a single sensor and an existing fusion method is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and pattern recognition, and particularly relates to a falling object imaging recovery method fusing image frames and events. BACKGROUND

[0002] With the development of artificial intelligence and computer vision, visual perception technology is widely used in industrial safety monitoring, urban public safety, and nuclear power equipment operation guarantee. In actual application scenarios, the detection and identification of falling small targets (such as screws, tools, and debris) is an extremely important task. Such targets have the characteristics of small size, high speed, and irregular motion trajectory. Traditional frame-based image acquisition methods are prone to severe motion blur or insufficient exposure, which leads to a decrease in imaging quality and further affects subsequent detection and analysis.

[0003] In recent years, as a new type of brain-like visual sensor, event cameras have attracted attention from the academic and industrial communities due to their high temporal resolution (microsecond level), low latency, and high sensitivity to dynamic scenes. Event cameras trigger event outputs by capturing changes in pixel intensity, avoiding the collection of redundant static information, and are particularly suitable for the perception of high-speed moving objects.

[0004] However, event cameras also have inherent limitations: their output is a sparse event stream that lacks texture and color information, and direct use for image restoration often only includes edges and motion contours, making it difficult to meet high-fidelity imaging requirements. On the other hand, traditional RGB cameras can provide rich texture and color information, but images often appear blurred and trailing in high-speed motion scenarios.

[0005] Therefore, how to fuse the advantages of event cameras and RGB cameras, taking into account dynamic clarity and texture fidelity, has become an important research direction in the field of visual perception. In particular, in the task of falling small target detection and image reconstruction, fusing event and RGB information is expected to overcome the limitations of a single sensor and achieve high-definition restoration of high-speed moving small targets.

[0006] Currently, there are mainly the following technical solutions for imaging and restoration of high-speed moving small targets: 1. High-speed imaging and deblurring method based on a single RGB camera Traditional industrial cameras or high-speed cameras reduce motion blur by increasing frame rate and shortening exposure time, but this method requires high light conditions and is still prone to low signal-to-noise ratio and image blur in low light or extremely small target conditions.

[0007] Some image processing-based deblurring algorithms (such as blind deblurring methods based on convolutional neural networks) can restore clear images to some extent, but often rely on accurate estimation of the motion blur kernel and perform poorly in the presence of irregular trajectories and sudden speed changes of falling small targets.

[0008] 2. Image reconstruction method based on event camera In recent years, a series of image reconstruction methods based on event data have been proposed in the academic community. An event camera is a new type of bionic vision sensor that asynchronously outputs pixel coordinates, timestamps, and event polarity when the brightness of a pixel changes beyond a threshold. Event cameras have the characteristics of low latency, low power consumption, high temporal resolution, and high dynamic range. Common event data representation methods include event frames and event time surfaces. In addition, there are studies that use deep neural networks to directly reconstruct grayscale images from event streams.

[0009] Event cameras have high sensitivity in dynamic scenes and can avoid motion blur. However, these reconstruction methods also have some problems: the reconstruction results usually only contain the edges and motion trajectories of the target, lacking texture details; they cannot provide color information, and the reconstruction results are relatively monotonous; in complex background scenes, event noise can easily interfere with the true outline of small targets.

[0010] 3. Fusion method based on events and RGB In order to make full use of the advantages of traditional RGB cameras and event cameras, some research works attempt to fuse event information with RGB frames to improve the dynamic performance of images. For example, using edge information from event cameras to assist in deblurring RGB images (also known as visible light images); in low-light scenes, using event data to improve target detection accuracy; in target recognition tasks, using a multi-modal feature fusion network of events and RGB to improve classification accuracy.

[0011] In summary, the existing technology has the following shortcomings: The existing technology still has obvious shortcomings in the imaging and restoration of falling small targets.

[0012] Firstly, methods based on a single RGB camera cannot fundamentally avoid motion blur. For high-speed falling small targets, due to the small size and fast motion of the target, conventional imaging often results in severely blurred images or even images that are almost unusable. Although there are some deblurring algorithms that can restore clarity to some extent, these methods generally rely on accurate estimation of the motion trajectory, and in scenes where small targets have random trajectories and sudden acceleration, traditional deblurring techniques are difficult to work effectively. In short, in dynamic scenes where small targets fall at high speed, images captured by a single RGB camera are severely blurred, and deblurring algorithms have limited effectiveness.

[0013] Secondly, the image reconstruction method of single event camera also has limitations. Although the sparse event stream output by the event camera can reflect the dynamic edge and motion trajectory, the reconstruction result lacks the necessary texture and color information, and the overall visual quality is insufficient. At the same time, in the case of complex background and noise interference, the event data is easily affected by false triggering, causing the distortion of the true form of the target and reducing the usability of the restored image. In short, in the dynamic scene of small target falling at high speed, the single event camera can only provide the edge information of the target motion, lacks texture and color, and the reconstructed image quality is poor.

[0014] Thirdly, the existing event and RGB fusion method also fails to solve the above problems well. Most research works focus on detection and classification tasks rather than clear image restoration at the image level. Even if image fusion is involved, the method is usually simple and lacks deep cross-modal feature alignment and fusion mechanism. This makes the fusion result limited in image clarity and detail recovery. Especially in the scene of small target falling at high speed, the existing method lacks targeted optimization, and it is often difficult to achieve ideal results in the restoration of small target, blur suppression and detail fidelity. In short, in the dynamic scene of small target falling at high speed, the existing event and RGB fusion method focuses on detection and recognition tasks and fails to effectively realize high-fidelity image restoration, and lacks special optimization for small target falling at high speed.

[0015] In summary, the existing technology cannot effectively solve the clear image restoration problem in the scene of falling small target. Therefore, it is urgent to propose a new event and RGB fusion algorithm to fully utilize the advantages of event camera in dynamic clarity and the advantages of RGB camera in texture fidelity, so as to improve the overall imaging quality of high-speed moving target. SUMMARY

[0016] Therefore, the present application provides a falling object imaging recovery method for fusing image frames and events, which fully utilizes the high temporal resolution and dynamic response characteristics of event cameras and the texture and color information of RGB cameras, realizes high-fidelity image restoration of small target falling at high speed through cross-modal alignment and fusion, and solves the problem of how to realize clear, complete and detailed image restoration of small target in the scene of falling at high speed, thereby breaking through the limitations of single sensor and existing fusion method.

[0017] The first aspect of the present application provides a falling object imaging recovery method for fusing image frames and events, which comprises: Step S10: Convert the sparse event stream of falling objects captured by the event camera into event features with temporal and spatial characteristics, and combine the event features to capture the motion trajectory and edge information of the falling objects. The event features include event time surface or event volume. Step S20: Use a deep convolutional network to extract RGB features from the RGB image of the falling object captured by the RGB camera, and explicitly model the blur characteristics. The RGB features include low-level texture features and high-level semantic features. Step S30: Map the event features and RGB features to a unified feature space, and perform deep fusion using an attention mechanism or guided filtering to obtain fused features; Step S40: Decode the fused features through a multi-scale decoding network, and introduce motion constraint loss and texture enhancement loss in the decoding stage to gradually recover a clear image of the falling object, so as to output a clear image restoration result of the falling object.

[0018] In one specific embodiment of this application, step S10 includes: Step S11: Collect sparse event streams of falling objects from the event camera. According to timestamp Mapped to voxel representation, forming an event volume V, which satisfies: in, It is the number of voxel blocks that are divided into. Indicates that the current event is assigned to two adjacent blocks. ; Step S12: Use a lightweight convolutional network to extract the motion edge features of the event volume to obtain the motion trajectory and edge information of the falling object.

[0019] In one specific embodiment of this application, step S20 includes: Step S21: Based on the free fall characteristics of the falling object, perform motion modeling to obtain the fuzzy kernel prior and the world coordinates of the falling object in three-dimensional space. With time t, it approximately satisfies free fall motion, that is in, The initial world coordinates of the falling object. It is the initial velocity vector. It is the gravitational acceleration vector. After mapping to pixel coordinates satisfy: in, It is the camera intrinsic parameter matrix. It is the camera extrinsic parameter matrix. This represents the homogeneous projection and normalization operation. Step S22: Assuming the pixel coordinates of the falling object change only in the y-direction, then the projected displacement vector... Finally, the fuzzy kernel of the prior estimate is obtained. : ; Step S23: Extract RGB features from the RGB image using a convolutional neural network (CNN) or a visual transformer (ViT); Step S24: Add a pooling fuzzy perception module during feature extraction. Used to estimate image blur kernel. in, RGB features are input at a certain scale. It is the fuzzy kernel of network estimation; Step S25: Combine the fuzzy kernel estimated prior to the given information. The final fuzzy kernel is obtained by linear weighting. , .

[0020] In one specific embodiment of this application, step S30 includes: Step S31: Consistently match event features and RGB features across the spatiotemporal dimensions using a feature alignment network. Let the event features and RGB features at a certain scale be... and After feature alignment network The result of back alignment is: ; Step S32: Use event features to guide the deblurring and detail restoration of RGB features, and simultaneously use RGB features to supplement the event features with texture and color information to obtain fused features. in, Based on event characteristics Guided gating unit, The adjustment weights of the gating unit, To enhance RGB features, To enhance event characteristics.

[0021] A second aspect of this application provides a computer apparatus including a processor and a memory. The processor is used to execute a falling object imaging restoration method that fuses image frames and events according to the first aspect of this application. The memory is used to store executable instructions of the processor.

[0022] A third aspect of this application provides a computer-readable storage medium having executable instructions stored thereon. When executed by a processor, the executable instructions implement a falling object imaging reconstruction method that fuses image frames and events, as described in the first aspect of this application.

[0023] The fourth aspect of this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements a falling object imaging restoration method that fuses image frames and events according to the first aspect of this application.

[0024] The beneficial effects of the technical solution of this application are as follows: by making full use of the high temporal resolution and dynamic capture advantages of the event camera, clear target motion structure information is provided; by combining the rich texture and color information provided by the RGB camera, the detail fidelity of the image is guaranteed; through cross-modal alignment and deep fusion methods, effective complementarity between event data and RGB images is achieved; and for the high-speed motion characteristics of falling objects, motion constraints and detail enhancement mechanisms are introduced to improve the performance of falling objects in terms of sharpness, edge sharpness and texture restoration. Attached Figure Description

[0025] Figure 1 The diagram shown is a flowchart illustrating a method for imaging and restoring a falling object by fusing image frames and events, according to an embodiment of this application.

[0026] Figure 2 The diagram shown is a simplified flowchart of a falling object imaging restoration method that fuses image frames and events according to an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] At least one embodiment of this application provides a method for imaging and restoring a falling object by fusing image frames and events, referencing... Figure 1 and Figure 2 The method for imaging and restoring a falling object by fusing image frames and events includes the following steps S10 to S40.

[0029] Step S10: Convert the sparse event stream of falling objects captured by the event camera into event features with temporal and spatial characteristics, and combine the event features to capture the motion trajectory and edge information of the falling objects. Event features include event time surface or event volume grid.

[0030] It should be noted that the falling object is a small target, and can be referred to as a small target or small object.

[0031] Step S20: Use a deep convolutional network to extract RGB features from the RGB image of the falling object captured by the RGB camera, and explicitly model the blurring characteristics. RGB features include low-level texture features and high-level semantic features.

[0032] It should be noted that sparse event streams are a type of event data, while RGB images are a type of RGB data.

[0033] Step S30: Map the event features and RGB features to a unified feature space, and use an attention mechanism or guided filtering to perform deep fusion to obtain fused features.

[0034] Step S40: Decode the fused features through a multi-scale decoding network, and introduce motion constraint loss and texture enhancement loss in the decoding stage to gradually recover a clear image of the falling object, so as to output a clear image restoration result of the falling object.

[0035] It should be noted that step S10 is the event feature representation step, which captures the motion trajectory and edge information of the falling object. Step S20 is the RGB blurred image feature extraction step. Step S30 is the cross-modal feature alignment and fusion step, which can take into account both the dynamic structural information of the event and the texture information of the RGN. Step S40 is the image reconstruction and detail enhancement step, which inputs the fused features into the decoding network to recover the clear image of the target layer by layer, and uses a dual loss function (i.e., motion constraint loss and texture enhancement loss) in the decoding stage. This ensures that the recovered target edge matches the event trajectory, reducing texture enhancement loss. This ensures that the texture of the restored result is consistent with the details of the RGB image, thereby enhancing the restoration effect of the fallen object in a clear image of the fallen object.

[0036] The embodiments of this application can significantly alleviate the image blurring problem caused by high-speed motion. By introducing the high temporal resolution characteristics of an event camera, the dynamic edges and motion trajectories of small targets can be accurately captured during image restoration, avoiding the strong dependence of traditional deblurring methods on motion estimation, thus maintaining clear imaging even in scenes with random trajectories and abrupt acceleration changes.

[0037] Meanwhile, the embodiments of this application can restore rich texture and color details while ensuring dynamic clarity. Through cross-modal feature alignment and deep fusion mechanism, event features and RGB features complement each other in a unified space: event features provide structural constraints, and RGB features provide texture information, thereby avoiding the problems of texture loss and target shape distortion in the reconstruction results of a single event camera.

[0038] Furthermore, the embodiments of this application are specifically optimized for the characteristics of falling small targets. Motion constraint loss and texture enhancement loss are introduced during the image decoding stage to ensure the clarity, edge sharpness, and detail fidelity of small targets in the reconstructed image, thereby improving the ability to recover small-sized targets. Compared to existing simple superposition fusion methods, the multi-scale cross-modal fusion of this invention can more effectively handle complex situations involving small target sizes and high movement speeds.

[0039] Furthermore, the embodiments of this application have good versatility and scalability. Their framework can be adapted to different event representation methods and different deep feature extraction networks, which facilitates their application in other dynamic vision tasks, such as safety monitoring in industrial production, debris detection of spacecraft modules, and dynamic perception of unmanned systems.

[0040] In summary, the embodiments of this application fully utilize the complementary advantages of event cameras and RGB cameras to achieve high-definition image restoration of small targets in high-speed falling scenarios. This solves the problem that existing technologies cannot effectively balance dynamic clarity and texture fidelity, and significantly improves the reliability and practicality of visual perception of small targets in dynamic scenes.

[0041] In at least one embodiment of this application, steps S11 and S12 are specific manifestations of step S10.

[0042] Step S11: Collect sparse event streams of falling objects from the event camera. According to timestamp Mapped to voxel representation, forming an event volume V, which satisfies: in, It is the number of voxel blocks that are divided into. Indicates that the current event is assigned to two adjacent blocks. ; Step S12: Use a lightweight convolutional network to extract the motion edge features of the event volume to obtain the motion trajectory and edge information of the falling object.

[0043] It should be noted that the trajectory and edge information of the falling object are also known as the dynamic outline of the falling object.

[0044] In at least one embodiment of this application, steps S21 to S25 are a specific manifestation of step S20.

[0045] Step S21: Based on the free-fall characteristics of the falling object, perform motion modeling to obtain the fuzzy kernel prior. World coordinates of the falling object in 3D space. With time t, it approximately satisfies free fall motion, that is in, The initial world coordinates of the falling object. It is the initial velocity vector. It is the gravitational acceleration vector.

[0046] After mapping to pixel coordinates satisfy: in, It is the camera intrinsic parameter matrix. It is the camera extrinsic parameter matrix. This represents the homogeneous projection and normalization operation.

[0047] Step S22: Assuming the pixel coordinates of the falling object change only in the y-direction, then the projected displacement vector... Finally, the fuzzy kernel of the prior estimate is obtained. : .

[0048] Step S23: Extract RGB features from the RGB image using a convolutional neural network (CNN) or a visual transformer (ViT); Step S24: Add a pooling fuzzy perception module during feature extraction. Used to estimate image blur kernel. in, RGB features are input at a certain scale. It is the fuzzy kernel of network estimation.

[0049] Step S25: Combine the fuzzy kernel estimated prior to the given information. The final fuzzy kernel is obtained by linear weighting. , .

[0050] It should be noted that the main function of step S30 is to determine the final fuzzy kernel. Used to guide deep fusion of event features and RGB features, ensuring that the fusion process takes into account fuzziness degradation.

[0051] The function of step S40: final fuzzy kernel It directly participates in the image restoration process in multi-scale decoding networks, especially in combination with motion constraint loss and texture enhancement loss, to ensure the output of clear images.

[0052] In at least one embodiment of this application, steps S31 and S32 are a specific implementation of step S30.

[0053] Step S31: Use a feature alignment network to ensure consistency between event features and RGB features in the spatiotemporal dimension. Let the event features and RGB features at a certain scale be respectively... and After feature alignment network The result of back alignment is: .

[0054] It should be noted that the feature alignment network can be based on alignment modules that combine channel attention and spatial attention.

[0055] Step S32: Use event features to guide the deblurring and detail restoration of RGB features, and simultaneously use RGB features to supplement the event features with texture and color information to obtain fused features. in, Based on event characteristics Guided gating unit, The adjustment weights of the gating unit, To enhance RGB features, To enhance event characteristics.

[0056] In the above embodiments, a multi-scale fusion strategy is used to achieve layer-by-layer interaction between event features and RGB features at different resolutions.

[0057] At least one embodiment of this application also provides a computer device including a processor and a memory. The processor is used to execute a method for imaging and restoring a fallen object by fusing image frames and events, provided in any of the above embodiments of this application. The memory is used to store executable instructions for the processor, such as application programs. There can be one or more processors. The application programs stored in the memory can include one or more modules, each corresponding to a set of instructions. Furthermore, the processor is configured to execute instructions to perform the above-described method for imaging and restoring a fallen object by fusing image frames and events.

[0058] The computer device may also include a power supply component configured for power management, a wired or wireless network interface configured to connect the computer device to a network, and an input / output (I / O) interface. The computer device can operate on an operating system stored in memory, such as Windows Server. TM Mac OSX TM Unix TM Linux TM FreeBSD TM Or similar.

[0059] At least one embodiment of this application also provides a computer-readable storage medium storing executable instructions for a computer thereon. When executed by a processor, the executable instructions implement a falling object imaging restoration method that fuses image frames and events, as provided in any of the above embodiments of this application.

[0060] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the computer device, enables the computer device to perform the fallen object imaging restoration method that fuses image frames and events. The fallen object imaging restoration method that fuses image frames and events is executed by an agent program.

[0061] Those skilled in the art will recognize that the algorithmic steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0062] At least one embodiment of this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements a falling object imaging restoration method that fuses image frames and events provided in any of the above embodiments of this application.

[0063] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a computer program product. This computer program product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the falling object imaging restoration method that fuses image frames and events according to various embodiments of this application. The aforementioned storage medium includes various media capable of storing program verification codes, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] It should be noted that the combination of the technical features in the embodiments of this application is not limited to the combination methods described in the embodiments of this application or the combination methods described in specific embodiments. All technical features described in this application can be freely combined or combined in any way, unless they contradict each other.

[0065] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the term "comprising" only indicates that it includes the explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0066] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0067] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A falling object imaging recovery method fusing image frames and events, characterized in that, The method comprises the following steps: Step S10, converting the sparse event stream of the falling object collected by the event camera into event features with timing and spatial characteristics, and capturing the motion trajectory and edge information of the falling object in combination with the event features, the event features including event time surfaces or event volumes; Step S20, extracting RGB features on the RGB image of the falling object collected by the RGB camera by using a deep convolutional network, and explicitly modeling the blur characteristics, the RGB features including low-level texture features and high-level semantic features; Step S30, mapping the event features and the RGB features to a unified feature space, and performing deep-level fusion by using an attention mechanism or a guided filtering method to obtain fused features; Step S40, decoding the fused features by using a multi-scale decoding network, and introducing motion constraint loss and texture enhancement loss in the decoding stage to gradually recover a clear image of the falling object, and outputting a clear image restoration result of the falling object.

2. The method of claim 1, wherein, Step S10 comprises the following steps: Step S11, mapping the sparse event stream of the falling object captured by the event camera According to the timestamps to a voxel representation, forming an event volume V, the event volume V satisfying: wherein, is the number of voxel blocks into which the current event is divided, indicates that the current event is divided into two adjacent blocks ; Step S12, extracting motion edge features of the event volume by using a lightweight convolutional network to obtain the motion trajectory and edge information of the falling object.

3. The method of claim 1, wherein, Step S20 comprises the following steps: Step S21, motion modeling is performed according to the free fall characteristics of the falling object to obtain a blur kernel prior, world coordinates of the falling object in a three-dimensional space approximately satisfies the free fall motion over time t, that is, wherein, is the initial world coordinate of the falling object, is the initial velocity vector, is the gravity acceleration vector, After mapping to pixel coordinates satisfies: wherein, is a camera intrinsic matrix, is a camera extrinsic matrix, represents a homographic projection and normalization operation; Step S22, assuming that the pixel coordinates of the falling object only change in the y direction, the projection displacement vector , the final blur kernel is obtained : ； Step S23, extracting RGB features on the RGB image by using a convolutional neural network (CNN) or a visual Transformer (ViT); Step S24, adding a pooling blur perception module in the feature extraction process for estimating an image blur kernel, wherein, RGB features for a certain scale input, is the blur kernel estimated by the network; Step S25, combining the blur kernel with the a priori estimate , linearly weighted to obtain the final blur kernel , 。 4. The method of claim 1 to 3, wherein, Step S30 comprises the following steps: Step S31, the consistency of the event feature and the RGB feature in the space-time dimension is performed through the feature alignment network, and the event feature and the RGB feature in a certain scale are respectively denoted as and The alignment result after the feature alignment network is: ； Step S32, using the event features to guide the deblurring and detail recovery of the RGB features, and using the RGB features to supplement the texture and color information of the event features to obtain fused features, wherein, is a gating unit guided by event features is a gating unit guided by event features is a gating unit guided by event features is a gating unit guided by event features is a gating unit guided by event features 5. A computer apparatus, characterized by The method comprises the following steps: A processor is configured to execute the method for restoring an image of a falling object by fusing image frames and events according to any one of claims 1 to 4. A memory is configured to store executable instructions of the processor. The executable instructions are executed by the processor to implement the method for restoring an image of a falling object by fusing image frames and events according to any one of claims 1 to 4.

6. A computer readable storage medium having stored thereon executable instructions of a computer, characterized in that, The computer program / instructions are executed by the processor to implement the method for restoring an image of a falling object by fusing image frames and events according to any one of claims 1 to 4.

7. A computer program product comprising computer programs / instructions, characterized in that, ​

Images