An event camera-based image processing method, device, equipment and medium

Abstract

The application belongs to the technical field of image processing, and discloses an image processing method and device based on an event camera, equipment and a medium. The method comprises the following steps: acquiring event data of a blurred image collected by an event camera; constructing an initial PINN model, embedding an event generation equation in the initial PINN model; inputting the event data into the PINN model to obtain a predicted luminance value, performing self-supervision of a time derivative based on the predicted luminance value and an actual luminance value, introducing a Tikhonov regularization constraint condition to optimize the PINN model, and determining an optimized PINN model; inputting event data of an image to be processed into the optimized PINN model to obtain log luminance values of three color channels of the image to be processed; and converting the log luminance values into a reconstructed image frame by using tone mapping. The method can reduce noise and artifacts of the reconstructed image and improve the quality of the model reconstructed image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and medium based on an event camera. Background Technology

[0002] With the continuous advancement of computer vision technology, traditional frame cameras have encountered challenges in handling dynamic scenes and high dynamic range (HDR) environments. To overcome these challenges, event cameras have emerged as a new type of sensor, gaining increasing attention and becoming a new hot topic in computer vision research due to their low power consumption, high dynamic range, and high temporal resolution. Unlike traditional cameras, event cameras can capture instantaneous changes in a scene with microsecond-level temporal resolution, generating sparse and asynchronous event stream data. This data format differs significantly from traditional frame images, bringing new challenges to information processing and algorithm design.

[0003] Existing event stream processing methods primarily rely on supervised learning, which requires a large amount of labeled data to train the model. However, synthetic training data often differs from real-world scenes, leading to unsatisfactory performance in practical applications. Furthermore, existing methods often lack robustness to noise when handling complex lighting conditions and high dynamic range scenes, affecting the quality of the reconstruction results. Summary of the Invention

[0004] Therefore, embodiments of this application provide an image processing method, apparatus, device, and medium based on an event camera, which reduces the dependence on labeled data and enables high-quality reconstruction of blurred images.

[0005] Firstly, this application provides an image processing method based on an event camera.

[0006] This application is achieved through the following technical solution:

[0007] An image processing method based on an event camera, the image processing method comprising:

[0008] Acquire event data from blurred images captured by the event camera;

[0009] An initial PINN model is constructed, and an event generation equation is embedded in the initial PINN model. The input of the initial PINN model is event data, and the output is the predicted brightness value that satisfies the event generation equation.

[0010] The event data of the blurred image is input into the initial PINN model to obtain the predicted brightness value of the event data. Self-supervision of the time derivative is performed based on the predicted brightness value and the actual brightness value. Tikhonov regularization constraint is introduced to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, the optimized PINN model is determined.

[0011] The event data of the image to be processed is input into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed.

[0012] Tone mapping is used to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames.

[0013] In a preferred example of this application, the method may be further configured to use tone mapping to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames, including:

[0014] The logarithmic brightness values ​​of the three color channels are converted into HDR range brightness values ​​using an exponential function;

[0015] The brightness and contrast of the image are adjusted using the Reinhard function, converting HDR range brightness values ​​to LDR range brightness values.

[0016] Reconstructed image frames are generated based on the LDR range brightness values.

[0017] In a preferred example of this application, the event data of the image to be processed can be further configured to be input into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed, including:

[0018] The time coordinates of the event data are input into the optimized PINN model to obtain the logarithmic brightness values ​​of the red channel, the green channel, and the blue channel.

[0019] In a preferred example of this application, the hidden layers of the PINN model can be further configured as multi-layer fully connected neural networks, and the initial PINN model uses randomized parameter values.

[0020] In a preferred example of this application, it can be further configured to perform self-supervised temporal derivative calculations based on predicted and actual brightness values, including:

[0021] Calculate the difference in brightness change between the predicted brightness value output by the PINN model and the actual brightness value calculated by the event generation equation:

[0022] The mean squared error loss function is determined based on the difference in brightness variation, and the time derivative of the PINN model is self-supervised based on the mean squared error loss function.

[0023] In a preferred example of this application, the initial PINN model can be further optimized by introducing Tikhonov regularization constraints, including:

[0024] The spatial gradient of logarithmic brightness is constrained by Tikhonov regularization, expressed as follows:

[0025]

[0026] in, This represents a regularization constraint. This represents the derivative in the x-axis. This represents the derivative in the y-axis.

[0027] In a preferred example of this application, it can be further set as follows:

[0028] The formula for the event generation equation is expressed as:

[0029]

[0030] Where L represents the brightness change, P i The direction of the brightness change is indicated by θ, which represents the brightness change threshold that triggers the event, and δ(tt) i ) represents the Dirac function, and t1 and t2 represent time coordinates.

[0031] Secondly, this application provides an image processing apparatus based on an event camera.

[0032] This application is achieved through the following technical solution:

[0033] An image processing apparatus based on an event camera is provided for executing the image processing method based on an event camera as described in the first aspect above, the image processing apparatus comprising:

[0034] The data acquisition module is used to acquire event data from blurry images captured by the event camera;

[0035] The model building module is used to build an initial PINN model, in which an event generation equation is embedded. The input of the initial PINN model is event data, and the output is the predicted brightness value that satisfies the event generation equation.

[0036] The model optimization module is used to input the event data of the blurred image into the initial PINN model to obtain the predicted brightness value of the event data. Based on the predicted brightness value and the actual brightness value, the time derivative is self-supervised, and the Tikhonov regularization constraint is introduced to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, the optimized PINN model is determined.

[0037] The brightness value prediction module is used to input the event data of the image to be processed into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed.

[0038] The tone mapping module is used to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames using tone mapping.

[0039] Thirdly, this application achieves its goal through the following technical solutions:

[0040] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described event camera-based image processing methods.

[0041] Fourthly, this application provides a computer-readable storage medium.

[0042] This application is achieved through the following technical solution:

[0043] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described image processing methods based on an event camera.

[0044] In summary, compared with the prior art, the beneficial effects of the technical solution provided by the embodiments of this application include at least the following: This application acquires event data of blurred images captured by an event camera; constructs an initial PINN model and embeds an event generation equation into the initial PINN model; inputs the event data of the blurred image into the initial PINN model to obtain the predicted brightness value of the event data, performs self-supervision of the time derivative based on the predicted brightness value and the actual brightness value, and introduces Tikhonov regularization constraints to optimize the initial PINN model. Once the prediction performance of the PINN model reaches a preset standard, the optimized PINN model is determined; the event data of the image to be processed is input into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed; and uses tone mapping to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames. This application utilizes the PINN model to directly solve the physical equations in the event generation model through self-supervised learning, reducing the dependence on a large amount of labeled data, thereby significantly reducing the cost and time investment in data labeling and improving the applicability of the model in real-world scenarios; by utilizing the dual constraints of time derivative self-supervision and spatial regularization, the prediction error of the model is improved, noise and artifacts in the reconstructed image are reduced, and the quality of the reconstructed image is improved. Attached Figure Description

[0045] Figure 1 This is a schematic flowchart of an image processing method based on an event camera provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of an image processing device based on an event camera provided in an embodiment of this application;

[0047] Explanation of reference numerals in the attached figures:

[0048] Data acquisition module - 101, model building module - 201, model optimization module - 301, brightness value prediction module - 401, tone mapping module - 501. Detailed Implementation

[0049] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this application, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0052] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0053] The embodiments of this application will be described in further detail below with reference to the accompanying drawings. The image processing method based on an event camera of this application can be applied to multiple application scenarios such as low-light enhancement for autonomous driving, image stabilization, and efficient compression of video data.

[0054] like Figure 1 As shown, the image processing method based on an event camera proposed in this application includes:

[0055] S10: Acquire event data from the blurred image captured by the event camera.

[0056] An event camera is a new type of visual sensor. Unlike traditional cameras that continuously capture images, each pixel in an event camera is an independent sensor that responds to changes in brightness signals. When the brightness change of a pixel exceeds a preset threshold, an event is triggered, generating a series of sparse and asynchronous event sequences (also known as an event stream). Event cameras output data asynchronously in units of events, without waiting for the acquisition and transmission of the entire frame, thus exhibiting extremely low latency. In practical implementations, any of the following event cameras can be used: DAVIS346, DVXplorer, PROPHESEE Metavision EVK4, or PROPHESEE Metavision EVK5. Each event in the event sequence is represented as: (x i ,y i ,t i ,P i ), where x i The x-coordinate of the pixel point i represents the current event i, y i t represents the y-coordinate of the pixel point of the current event i. i P represents the time coordinate corresponding to the current event i. i P represents the direction (also called polarity) of the brightness change of the pixel in the current event i. i This indicates an increase or decrease in brightness. P i The value of P can be negative one or positive one. i When the value of P is negative one, it indicates that the brightness of the pixel has decreased; i When the value is positive one, it indicates that the brightness of the pixel increases.

[0057] S20: Construct an initial PINN model and embed an event generation equation into it. The input of the initial PINN model is event data, and the output is the predicted brightness value that satisfies the event generation equation.

[0058] First, this application constructs an initial PINN model (Physics-Informed Neural Networks). The PINN model takes event data as input, specifically the timestamp (time coordinate) of the current event; the output is the predicted brightness values ​​for the three color channels corresponding to the pixels of the current event: the predicted brightness values ​​for the red, green, and blue channels. The PINN model structure includes an input layer, hidden layers, and an output layer. The hidden layers are multi-layer fully connected neural networks, consisting of multiple layers, each containing multiple neurons. Randomized parameter values ​​are used when constructing the initial PINN model. Subsequently, during training, the parameter values ​​of the PINN model are optimized and adjusted.

[0059] In some embodiments, to simplify analysis, this application assumes that the brightness change of a single pixel is within a fixed spatial coordinate system; that is, it ignores changes in pixel position within the image and focuses only on the brightness value of each pixel. The brightness of each pixel can vary, and different pixel brightness values ​​represent different colors. The Dirac function δ can be used to describe the event e on a pixel in the event data. i :

[0060] e i =P i θδ(tt i ), where θ represents the brightness change threshold that triggers the event;

[0061] The brightness increment can be represented by the accumulation of events on the pixel within a preset time interval [t1, t2].

[0062] ΔL=L(t2)-L(t1),

[0063]

[0064] Assuming the time interval [t1, t2] is short and noise is negligible, the nonlinear brightness change can be transformed into a logarithmic form using Taylor expansion, namely the first time derivative of the brightness increment, representing the rate of logarithmic brightness change over time:

[0065]

[0066] Therefore, the event generation equation can be expressed as:

[0067]

[0068] Where L represents the brightness change, P i The direction of the brightness change is indicated by θ, which represents the brightness change threshold that triggers the event, and δ(tt) i) represents the Dirac function, and t1 and t2 represent time coordinates.

[0069] The event generation equation is embedded into the PINN model to serve as a physical constraint when the PINN model processes event data, ensuring that the model's output satisfies the constraint conditions of the physical equation.

[0070] When constructing the initial PINN model, randomized parameter values ​​are used. The PINN model represents complex information through a parameterized fully connected multilayer perceptron (MLP). By incorporating the aforementioned event generation equation into the training process of the neural network, it guides the network's learning, enabling effective training and prediction even under conditions of scarce data or high noise.

[0071] S30: Input the event data of the blurred image into the initial PINN model to obtain the predicted brightness value of the event data. Perform self-supervision of the time derivative based on the predicted brightness value and the actual brightness value, and introduce Tikhonov regularization constraints to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, determine the optimized PINN model.

[0072] In some embodiments, the predicted brightness value output by the PINN model is directly minimized. The difference in brightness change between the brightness value ΔL calculated by the event generation equation and the value of brightness change:

[0073]

[0074] The mean squared error loss function is determined based on the difference in brightness variation, and the time derivative of the PINN model is self-supervised based on the mean squared error loss function:

[0075]

[0076] Although time derivative self-supervised estimation can estimate the brightness function, considering that the PINN model lacks prior knowledge of the initial brightness value of each pixel, this application further introduces a Tikhonov regularization constraint into the loss function to eliminate unnatural artifacts that may be included in the estimation results, ensuring that the model output remains within the natural image space. The specific steps are as follows:

[0077] The spatial gradient of logarithmic brightness is constrained by Tikhonov regularization, expressed as follows:

[0078]

[0079] in, This represents a regularization constraint. This represents the derivative in the x-axis. This represents the derivative in the y-axis.

[0080] Tikhonov regularization limits model complexity by penalizing large parameter variations, which helps the model perform better on unknown data and reduces the risk of overfitting. Compared to some complex regularization methods, such as CNN denoisers, Tikhonov regularization is computationally simpler and more efficient, does not require a large-scale network model, and thus significantly reduces training time.

[0081] The overall objective function of the PINN model during training is: The PINN model is trained using gradient descent based on this objective function. When the value of the objective function decreases to a preset threshold, or when the value of the objective function no longer decreases significantly after a certain number of epochs, the model can be considered to have converged and training can be terminated.

[0082] S40: Input the event data of the image to be processed into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed.

[0083] The event data of the image to be processed is input into the optimized PINN model. The PINN model predicts the brightness values ​​of three color channels (red, green, and blue channels) based on the time coordinate t of the event data. The PINN model constructed in this application is not limited to single brightness prediction, but can be extended to brightness prediction of three color channels (RGB).

[0084] S50: Use tone mapping to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames.

[0085] Specifically, the logarithmic brightness values ​​of the three color channels are converted into HDR (High Dynamic Range) brightness values ​​using an exponential function:

[0086] I(x,y,t)=exp(F Θ (x,y,t));

[0087] The Reinhard function adjusts the image's brightness and contrast, converting logarithmic brightness values ​​in the HDR range to logarithmic brightness values ​​in the LDR range. The specific expression is as follows:

[0088]

[0089] I represents the logarithmic luminance value for the HDR range, Γ(I) represents the logarithmic luminance value for the LDR range, and γ represents the hyperparameter used to control contrast.

[0090] Reconstructed image frames are generated based on the LDR range brightness values.

[0091] In some feasible embodiments, an event camera is mounted on an autonomous vehicle and positioned at the front of the vehicle to more realistically capture images obtained during the vehicle's operation. During nighttime driving, the event camera captures event data in low-light conditions, including information on brightness changes of roads, pedestrians, and vehicles. This event data is first input into a Physical Neural Network (PINN) model. The PINN model extracts features from the event data and optimizes it using time derivative self-supervision and spatial regularization constraints to minimize the mean square error between the predicted brightness changes and the actual event data, while also reducing noise and artifacts in the reconstructed images. The optimized PINN model is then used for low-light enhancement processing to generate high-quality image frames. Tone mapping is then used to convert high dynamic range (HDR) brightness values ​​to low dynamic range image frames. The resulting image clarity and brightness are significantly enhanced during nighttime driving, enabling accurate identification of roads, pedestrians, and other vehicles, thus improving the perception capabilities and safety of the autonomous driving system in low-light environments.

[0092] In some embodiments, the processing method of this application is applicable to image processing when shooting dynamic scenes, and can remove image blur caused by camera shake to generate clear image frames. Event data recording shake and blur is acquired through the data acquisition software or interface of the event camera. This event data contains temporal and spatial information of pixel brightness changes. The information from the event data is used as input to the PINN model. The PINN model is optimized through time derivative self-supervision and spatial regularization. Time derivative self-supervision minimizes the mean square error between the predicted brightness change and the actual event data, and spatial regularization reduces noise and artifacts in the reconstructed image. The optimized PINN model is then used for shake removal processing to generate clear image frames. High dynamic range (HDR) brightness values ​​are converted to low dynamic range image frames using tone mapping. By removing shake and blur from the image, the generated image frames are clearer, suitable for high-quality image processing in dynamic scenes, and improve image detail preservation and visual effects.

[0093] In some embodiments, the processing method of this application is applicable to scenarios requiring efficient compression of video data, reducing the bandwidth required for storage and transmission while maintaining high-quality video reconstruction. Video event data captured by an event camera is acquired through data acquisition software or an interface of the event camera. This video event data records brightness change information within video frames. The information from the event data is used as input to the PINN model, which is then optimized using time derivative self-supervision and spatial regularization. Time derivative self-supervision minimizes the mean square error between the predicted brightness change and the actual event data, while Tikhonov regularization reduces noise and artifacts in the reconstructed image. The optimized PINN model is used to compress the video data, converting the brightness information in the video frames into parameters of the PINN model for efficient compression. Decompression restores the compressed representation to high-quality video frames. Tone mapping is used to convert high dynamic range (HDR) brightness values ​​into low dynamic range image frames. This achieves high-compression video compression while maintaining high-quality image reconstruction, significantly reducing the bandwidth and storage space required for video storage and transmission, and improving the economy and practicality of video applications.

[0094] Another embodiment of this application also provides an image processing apparatus for an event camera, which is used to perform the above-described processing method, such as... Figure 2 As shown, the processing apparatus includes:

[0095] The data acquisition module 101 is used to acquire event data from the blurred image captured by the event camera;

[0096] The model building module 201 is used to build an initial PINN model. An event generation equation is embedded in the initial PINN model. The input of the initial PINN model is event data, and the output is the predicted brightness value that satisfies the event generation equation.

[0097] The model optimization module 301 is used to input the event data of the blurred image into the initial PINN model, obtain the predicted brightness value of the event data, perform self-supervision of the time derivative based on the predicted brightness value and the actual brightness value, and introduce Tikhonov regularization constraints to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, the optimized PINN model is determined.

[0098] The brightness value prediction module 401 is used to input the event data of the image to be processed into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed.

[0099] The tone mapping module 501 is used to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames using tone mapping.

[0100] In some embodiments, the tone mapping module 501 is used to convert the logarithmic brightness values ​​of the three color channels into HDR range brightness values ​​using an exponential function: I(x,y,t)=exp(F Θ (x,y,t));

[0101] The Reinhard function adjusts the image's brightness and contrast, converting logarithmic brightness values ​​in the HDR range to logarithmic brightness values ​​in the LDR range. The specific expression is as follows:

[0102]

[0103] I represents the logarithmic luminance value for the HDR range, Γ(I) represents the logarithmic luminance value for the LDR range, and γ represents the hyperparameter used to control contrast.

[0104] The specific limitations of the image processing device for the event camera provided in this embodiment can be found in the embodiments of the image processing method based on the event camera described above, and will not be repeated here. Each module in the above-described image processing device for the event camera can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0105] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform the steps of the event-camera-based image processing method as described in any of the above embodiments.

[0106] The working process, working details, and technical effects of the computer device provided in this embodiment can be found in the embodiment of the image processing method based on the event camera described above, and will not be repeated here.

[0107] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the event-camera-based image processing method as described in any of the above embodiments. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0108] The working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiment of the image processing method based on the event camera described above, and will not be repeated here.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system described in this application can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. An image processing method based on an event camera, characterized in that, include: Acquire event data from blurred images captured by the event camera; An initial PINN model is constructed, and an event generation equation is embedded in the initial PINN model. The input of the initial PINN model is event data, and the output is the predicted brightness values ​​of the three color channels that satisfy the event generation equation. The event data of the blurred image is input into the initial PINN model to obtain the predicted brightness value of the event data. Self-supervision of the time derivative is performed based on the predicted brightness value and the actual brightness value. Tikhonov regularization constraint is introduced to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, the optimized PINN model is determined. The event data of the image to be processed is input into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed. Tone mapping is used to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames, including: The logarithmic brightness values ​​of the three color channels are converted into HDR range brightness values ​​using an exponential function; The brightness and contrast of the image are adjusted using the Reinhard function, converting HDR range brightness values ​​to LDR range brightness values. Reconstructed image frames are generated based on the LDR range brightness values.

2. The image processing method based on an event camera according to claim 1, characterized in that, The event data of the image to be processed is input into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed, including: The time coordinates of the event data are input into the optimized PINN model to obtain the logarithmic brightness values ​​of the red channel, the green channel, and the blue channel.

3. The image processing method based on an event camera according to claim 1, characterized in that, The hidden layers of the PINN model are multi-layer fully connected neural networks, and the initial PINN model uses randomized parameter values.

4. The image processing method based on an event camera according to claim 1, characterized in that, Self-supervised time derivative calculation based on predicted and actual brightness values ​​includes: Calculate the difference in brightness variation between the predicted brightness value output by the PINN model and the actual brightness value calculated by the event generation equation: The mean squared error loss function is determined based on the difference in brightness variation, and the time derivative of the PINN model is self-supervised based on the mean squared error loss function.

5. The image processing method based on an event camera according to claim 3, characterized in that, The initial PINN model is optimized by introducing Tikhonov regularization constraints, including: The spatial gradient of logarithmic brightness is constrained by Tikhonov regularization, expressed as follows: ， in, This represents a regularization constraint. This represents the derivative in the x-axis. This represents the derivative in the y-axis.

6. The image processing method based on an event camera according to any one of claims 1 to 5, characterized in that, The formula for the event generation equation is expressed as: ， in Indicates changes in brightness. Indicates the direction of brightness change. This indicates the brightness change threshold that triggers the event. Represents the Dirac function, and Represents the time coordinate.

7. An image processing device based on an event camera, characterized in that, For performing the image processing method as described in any one of claims 1 to 5, comprising: The data acquisition module is used to acquire event data from blurry images captured by the event camera; The model building module is used to build an initial PINN model, in which an event generation equation is embedded. The input of the initial PINN model is event data, and the output is the predicted brightness value that satisfies the event generation equation. The model optimization module is used to input the event data of the blurred image into the initial PINN model to obtain the predicted brightness value of the event data. Based on the predicted brightness value and the actual brightness value, the time derivative is self-supervised, and the Tikhonov regularization constraint is introduced to optimize the initial PINN model. Once the prediction performance of the PINN model reaches the preset standard, the optimized PINN model is determined. The brightness value prediction module is used to input the event data of the image to be processed into the optimized PINN model to obtain the logarithmic brightness values ​​of the three color channels of the image to be processed. The tone mapping module is used to convert the logarithmic brightness values ​​of the three color channels into reconstructed image frames using tone mapping. Specifically, it is used to convert the logarithmic brightness values ​​of the three color channels into HDR range brightness values ​​using an exponential function. The brightness and contrast of the image are adjusted using the Reinhard function, converting HDR range brightness values ​​to LDR range brightness values. Reconstructed image frames are generated based on the LDR range brightness values.

8. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

Images