Image deblurring method, apparatus, electronic device, and storage medium

Abstract

The application relates to an image deblurring method and device, an electronic device and a storage medium. The image deblurring method comprises the following steps: acquiring a to-be-processed blurred image and an exposure time of the to-be-processed blurred image; slicing the exposure time into a preset number of time points, and determining image frames corresponding to the time points and DVS event information corresponding to the image frames; the image frame corresponding to the middle time point is corresponding to first DVS event information, and the image frame corresponding to the time point other than the middle time point is corresponding to second DVS event information; determining a first relationship coefficient of the first DVS event information and the second DVS event information; and performing deblurring processing on the to-be-processed blurred image according to the first relationship coefficient. Through the application, the problem of poor image deblurring effect in the related art is solved, and the image deblurring effect is improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to image deblurring methods, apparatus, electronic devices, and storage media. Background Technology

[0002] In recent years, the increasing popularity of portable and lightweight digital imaging devices has made it easier for people to capture moments in life. However, during the image or video shooting process, the movement of the object being photographed or camera shake can cause relative motion between the camera and the target, resulting in blurry images. How to recover a clear image from a blurry one, providing high-quality images for further image processing and subsequent intelligent computer analysis, has become an urgent problem to be solved.

[0003] Currently, image deblurring in related technologies mainly utilizes deconvolution techniques. However, the principle of deconvolution deblurring requires assumptions based on statistical experiments because the features of the clear image and the causes of degradation in the blurred image are unknown beforehand. When the actual image does not match the assumptions, the deblurring will not achieve the desired effect.

[0004] There is currently no effective solution to the problem of poor image deblurring in related technologies. Summary of the Invention

[0005] This embodiment provides an image deblurring method, apparatus, electronic device, and storage medium to solve the problem of poor image deblurring effect in related technologies.

[0006] Firstly, this embodiment provides an image deblurring method, including:

[0007] Obtain the blurred image to be processed and its exposure time;

[0008] The exposure time is sliced ​​into a preset number of time points, and the image frame corresponding to each time point and the DVS event information corresponding to the image frame are determined; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information.

[0009] Determine the first relationship coefficient between the first DVS event information and the second DVS event information;

[0010] Based on the first relationship coefficient, the blurred image to be processed is deblurred.

[0011] In some embodiments, obtaining the blurred image to be processed and the exposure time of the blurred image to be processed includes:

[0012] The event camera acquires the blurred image to be processed and its exposure time, as well as the DVS event information within the exposure time.

[0013] In some embodiments, determining the first relationship coefficient between the first DVS event information and the second DVS event information includes:

[0014] Based on the first DVS event information and the second DVS event information corresponding to the image frame at each time point other than the intermediate time point, determine multiple sub-first relationship coefficients between the first DVS event information and the second DVS event information corresponding to the image frame at each time point other than the intermediate time point;

[0015] The first relation coefficient is obtained by summing up the multiple sub-first relation coefficients.

[0016] In some embodiments, blurring the image to be processed according to the first relationship coefficient includes:

[0017] The first relationship coefficient is subjected to guided filtering to obtain the second relationship coefficient;

[0018] Subtracting the second relationship coefficient from the first relationship coefficient yields the third relationship coefficient;

[0019] Gaussian filtering is applied to the third relation coefficient to obtain the fourth relation coefficient;

[0020] The fourth relation coefficient is added to the second relation coefficient to obtain the target relation coefficient;

[0021] The blurred image to be processed is blurred according to the target relation coefficient.

[0022] In some embodiments, after determining a first relationship coefficient between the first DVS event information and the second DVS event information, the method further includes:

[0023] Based on the first relationship coefficient between the first DVS event information and the second DVS event information, a fifth relationship coefficient is determined between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points other than its own time point.

[0024] Based on the fifth relationship coefficient between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points in each time point (excluding its own time point), the blurred image to be processed is deblurred sequentially to obtain the latent image corresponding to each time point.

[0025] The latent image corresponding to each of the time points is input into a preset image processing model to obtain a clear image of the blurred image to be processed, wherein the preset image processing model is trained to predict the clear image of the latent image within the exposure time.

[0026] In some embodiments, the latent image corresponding to each of the stated time points is input into a preset image processing model to obtain a clear image of the blurred image to be processed, including:

[0027] The latent images corresponding to each of the aforementioned time points are arranged sequentially according to the order of the exposure times;

[0028] The latent images corresponding to each of the arranged time points are input into a preset image processing model to obtain a clear image of the blurred image to be processed.

[0029] In some embodiments, the preset image processing model includes an RNN deep learning model.

[0030] Secondly, this embodiment provides an image deblurring device, comprising:

[0031] The first acquisition module is used to acquire the blurred image to be processed and the exposure time of the blurred image to be processed;

[0032] The slicing module is used to slice the exposure time into a preset number of time points and determine the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information.

[0033] The first determining module is used to determine a first relationship coefficient between the first DVS event information and the second DVS event information;

[0034] The first processing module is used to perform deblurring processing on the blurred image to be processed according to the first relationship coefficient.

[0035] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image deblurring method described in the first aspect above.

[0036] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the image deblurring method described in the first aspect above.

[0037] Compared with related technologies, the image deblurring method, apparatus, electronic device, and storage medium provided in this embodiment solve the problem of poor image deblurring effect in related technologies and improve the image deblurring effect by acquiring the blurred image to be processed and its exposure time; slicing the exposure time into a preset number of time points and determining the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information; determining a first relationship coefficient between the first DVS event information and the second DVS event information; and performing deblurring processing on the blurred image to be processed based on the first relationship coefficient.

[0038] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a hardware structure block diagram of the terminal for the image deblurring method in this embodiment;

[0041] Figure 2 This is a flowchart of the image deblurring method in this embodiment;

[0042] Figure 3 This is a schematic diagram of the exposure time slice in this embodiment;

[0043] Figure 4 This is a structural block diagram of the image deblurring device in this embodiment. Detailed Implementation

[0044] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0045] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0046] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the image deblurring method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0047] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image deblurring method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0048] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0049] Among related technologies, some employ EDI (dual integral) models and event cameras to establish blurred images, events, and latent images. Figure 3 The EDI model assumes that the blurred image is an integral of a series of latent images, and that events reflect the changes between these latent images. Therefore, when the blurred image and its corresponding event information within the exposure time are known, the latent image at the intermediate moment can be obtained, i.e., the sharp image at the intermediate moment, thus achieving deblurring. The EDI model is purely theoretically derived and easily portable. However, its operating speed of 1.5 m / s makes it unsuitable for systems with high real-time requirements because the EDI model needs to integrate the latent image corresponding to each event within the exposure time to obtain the sharp image, further reducing deblurring efficiency. Additionally, its sensitivity to noise leads to artifacts in the processing results, resulting in poor deblurring performance.

[0050] The Dynamic Vision Sensor (DVS) is a novel biomimetic vision sensor, also known as an event camera. Unlike traditional frame-based RGB cameras, DVS captures dynamic information about the scene. Its output is a DVS event, which represents the changes that occur within the camera lens over microseconds. The high temporal resolution of DVS better characterizes the image blurring process, providing a new approach to deblurring, especially for blurring caused by rapid motion.

[0051] To address the aforementioned problems, this embodiment provides an image deblurring method. Figure 2 This is a flowchart of the image deblurring method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0052] Step S201: Obtain the blurred image to be processed and its exposure time.

[0053] In this step, the blurred image to be processed can be acquired in real time or obtained from a database storing blurred images to be processed. The exposure time is associated with the blurred image to be processed, and each blurred image to be processed has a corresponding exposure time.

[0054] Step S202: Slice the exposure time into a preset number of time points, and determine the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information.

[0055] In this step, preset time points can be automatically and evenly divided based on the exposure time length. By voxelizing the exposure time into time slices, the subsequent deblurring process can be accelerated.

[0056] Step S203: Determine the first relationship coefficient between the first DVS event information and the second DVS event information.

[0057] In this step, the first relationship coefficient is determined by the events between the first DVS event information and the second DVS event information. The first relationship coefficient is used to represent the correlation of the number of events between the first DVS event information and the second DVS event information.

[0058] It should be noted that the first DVS event information has a corresponding first correlation coefficient with other time points.

[0059] Step S204: Deblur the blurred image to be processed according to the first relation coefficient.

[0060] In this step, the blurred image to be processed is deblurred by using the relationship model between the latent image and the blurred image at the midpoint of the exposure (i.e., the first relationship coefficient).

[0061] Based on steps S201 to S204, the method of deblurring the blurred image to be processed uses the first relationship coefficient between the first DVS event information corresponding to the image frame at the intermediate time point and the second DVS event information corresponding to the image frames at other time points. This method eliminates the need for integration processing of the latent image corresponding to each event. Instead, it obtains the clear image at the intermediate time point by slicing the exposure time, thus achieving deblurring. It also eliminates the need for assumptions based on statistical experiments, solving the problem of poor image deblurring effect in related technologies and improving the image deblurring effect. Furthermore, it solves the problem of low system operating speed caused by the dual integral model processing in related technologies, accelerating the efficiency of image deblurring.

[0062] In some embodiments, the device may acquire information including: acquiring a blurred image to be processed and the exposure time of the blurred image to be processed via an event camera, as well as DVS event information within the exposure time.

[0063] In this embodiment, the event camera can acquire the exposure time of the blurred image to be processed and the DVS event information within the exposure time. The camera has the characteristics of high dynamic range, high temporal resolution and low latency.

[0064] In some embodiments, determining the first relationship coefficient between the first DVS event information and the second DVS event information includes: determining multiple sub-first relationship coefficients for the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point, based on the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point; and summing the multiple sub-first relationship coefficients to obtain the first relationship coefficient.

[0065] In this embodiment, by using the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point, multiple sub-first relation coefficients of the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point are realized, so as to determine the first relation coefficient of the first DVS event information at the intermediate time point, thereby realizing the conversion between the image frames at the intermediate time point and the image frames at other times according to the first relation coefficient.

[0066] It should be noted that the event polarity accumulation map before the intermediate time point needs to be inverted, because it describes the change in pixel brightness from the intermediate time point to the previous time point, while the actual event camera describes the change in brightness from the next time point to the previous time point.

[0067] The following example describes and explains the determination of the first relationship coefficient between the first DVS event information and the second DVS event information in this embodiment.

[0068] In one embodiment, determining the first relationship coefficient between the first DVS event information and the second DVS event information includes the following steps:

[0069] Step 1: Divide the exposure time into N equal parts (i.e., the preset number of time points in the above embodiment), where N is an odd number, and the specific value depends on the trade-off between computational complexity and the desired result. For example... Figure 3 As shown, N is set to 9, so there will be 9 latent images.

[0070] Step 2: Using the second integral in the EDI model, the relationship between the latent images at intermediate times and other times is obtained through the event information between time points, denoted as Latent_ti=Latent_tmid*gain_i, where gain_i is the coefficient between the two latent images.

[0071] Step 3: The blurred image Blur is a superposition of latent images. Using the relationship between latent images, we obtain the calculation formulas for the blurred image Blur and the latent image Latend_tmid at the intermediate time point:

[0072]

[0073] Step 4: Using the above formula, the latent image Latend_tmid at the intermediate time point can be calculated, where gain is the cumulative sum of gain_i:

[0074] Latent_tmid = Blur / gain;

[0075] In some embodiments, blurring the image to be processed according to the first relation coefficient includes: performing guided filtering on the first relation coefficient to obtain a second relation coefficient; subtracting the second relation coefficient from the first relation coefficient to obtain a third relation coefficient; performing Gaussian filtering on the third relation coefficient to obtain a fourth relation coefficient; adding the fourth relation coefficient to the second relation coefficient to obtain a target relation coefficient; and blurring the image to be processed according to the target relation coefficient.

[0076] In this embodiment, the method of obtaining the target relation coefficient by combining guided filtering and Gaussian filtering can further reduce the impact of noise and help to recover a high-quality, clear image.

[0077] In some embodiments, after determining the first relationship coefficient between the first DVS event information and the second DVS event information, a fifth relationship coefficient can be determined between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points within each time point, excluding the time point itself, based on the first relationship coefficient between the first DVS event information and the second DVS event information. Based on the fifth relationship coefficient between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points within each time point, the blurred image to be processed is sequentially deblurred to obtain the latent image corresponding to each time point. The latent image corresponding to each time point is input into a preset image processing model to obtain a clear image of the blurred image to be processed, wherein the preset image processing model is trained to predict the clear image of the latent image within the exposure time.

[0078] In this embodiment, by obtaining the corresponding latent image at each time point and then inputting the latent image at each time point into a preset image processing model, a clear image of the blurred image to be processed is obtained, thereby further improving the clarity of the latent image and compensating for the blurring problem caused by rapid motion that AI algorithms cannot solve.

[0079] In some embodiments, inputting the latent images corresponding to each time point into a preset image processing model to obtain a clear image of the blurred image to be processed includes: arranging the latent images corresponding to each time point in chronological order of exposure time; and inputting the arranged latent images corresponding to each time point into a preset image processing model to obtain a clear image of the blurred image to be processed.

[0080] In this embodiment, by inputting the latent images corresponding to each time point after arrangement into a preset image processing model, a clear image of the blurred image to be processed is obtained, thus avoiding the problem of poor image deblurring effect caused by image sequence errors.

[0081] In some of these embodiments, the preset image processing model includes an RNN deep learning model.

[0082] In this embodiment, the RNN deep learning method cannot restore a blurred image to a clear image. However, this embodiment uses the EDI method to process rapidly blurred images to obtain intermediate results, restore the main outline of the blurred image at each time point, reduce the degree of blur, and then input it into the RNN deep learning model for further deblurring to obtain the final clear image, thereby making up for the inability of AI algorithms to solve the blur problem caused by rapid motion.

[0083] It should be noted that the preset image processing model is not limited to RNN deep learning models; other AI algorithms used for deblurring are also acceptable.

[0084] This embodiment also provides an image deblurring device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0085] Figure 4 This is a structural block diagram of the image deblurring device in this embodiment, as shown below. Figure 4 As shown, the device includes:

[0086] The first acquisition module 41 is used to acquire the blurred image to be processed and the exposure time of the blurred image to be processed;

[0087] The slicing module 42, coupled to the first acquisition module 41, is used to slice the exposure time into a preset number of time points and determine the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information.

[0088] The first determining module 43, coupled to the slicing module 42, is used to determine the first relationship coefficient between the first DVS event information and the second DVS event information;

[0089] The first processing module 44 is coupled to the first determining module 43 and is used to perform deblurring processing on the blurred image to be processed according to the first relation coefficient.

[0090] In some embodiments, the first acquisition module 41 includes an acquisition unit, configured to acquire the blurred image to be processed and the exposure time of the blurred image to be processed, as well as DVS event information within the exposure time, through the DVS camera.

[0091] In some embodiments, the first determining module 43 includes: a determining unit, configured to determine multiple sub-first relationship coefficients of the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point, based on the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point; and an accumulating unit, configured to accumulate the multiple sub-first relationship coefficients to obtain a first relationship coefficient.

[0092] In some embodiments, the first processing module 44 includes: a first filtering unit for performing guided filtering on the first relation coefficient to obtain a second relation coefficient; a first calculation unit for subtracting the second relation coefficient from the first relation coefficient to obtain a third relation coefficient; a second filtering unit for performing Gaussian filtering on the third relation coefficient to obtain a fourth relation coefficient; a second calculation unit for adding the fourth relation coefficient to the second relation coefficient to obtain a target relation coefficient; and a processing unit for performing blurring processing on the blurred image to be processed according to the target relation coefficient.

[0093] In some embodiments, the device further includes: a second determining module, configured to determine, based on a first relationship coefficient between the DVS event information of the image frame corresponding to each time point and the DVS event information of image frames at other time points (excluding the time point itself) within each time point; a second processing module, configured to sequentially perform deblurring processing on the blurred image to be processed based on the DVS event information of the image frame corresponding to each time point and the fifth relationship coefficient between the DVS event information of the image frame corresponding to each time point and the DVS event information of image frames at other time points (excluding the time point itself) within each time point, to obtain the latent image corresponding to each time point; and an input module, configured to input the latent image corresponding to each time point into a preset image processing model to obtain a clear image of the blurred image to be processed, wherein the preset image processing model is trained to predict the clear image of the latent image within the exposure time.

[0094] In some embodiments, the input module includes: a sorting unit for arranging the latent images corresponding to each time point in chronological order of exposure time; and an input unit for inputting the sorted latent images corresponding to each time point into a preset image processing model to obtain a clear image of the blurred image to be processed.

[0095] In some of these embodiments, the preset image processing model includes an RNN deep learning model.

[0096] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0097] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0098] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0099] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0100] Step S21: Obtain the blurred image to be processed and its exposure time.

[0101] Step S22: Slice the exposure time into a preset number of time points, and determine the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information.

[0102] Step S23: Determine the first relationship coefficient between the first DVS event information and the second DVS event information.

[0103] Step S24: Deblur the blurred image to be processed according to the first relation coefficient.

[0104] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0105] Furthermore, in conjunction with the image deblurring methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the image deblurring methods described in the above embodiments.

[0106] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0107] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0108] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0109] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An image deblurring method, characterized in that, include: Obtain the blurred image to be processed and its exposure time; The exposure time is sliced ​​into a preset number of time points, and the image frame corresponding to each time point and the DVS event information corresponding to the image frame are determined; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information. Determine a first relationship coefficient between the first DVS event information and the second DVS event information; determining the first relationship coefficient between the first DVS event information and the second DVS event information includes: determining multiple sub-first relationship coefficients between the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point, based on the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point; summing the multiple sub-first relationship coefficients to obtain the first relationship coefficient; the first relationship coefficient represents the association between the first DVS event information and the second DVS event information; Based on the first relationship coefficient, the blurred image to be processed is deblurred.

2. The image deblurring method according to claim 1, characterized in that, The acquisition of the blurred image to be processed and the exposure time of the blurred image to be processed includes: The event camera acquires the blurred image to be processed and its exposure time, as well as the DVS event information within the exposure time.

3. The image deblurring method according to claim 1, characterized in that, The blurring process for the image to be processed, based on the first relationship coefficient, includes: The first relationship coefficient is subjected to guided filtering to obtain the second relationship coefficient; Subtracting the second relationship coefficient from the first relationship coefficient yields the third relationship coefficient; Gaussian filtering is applied to the third relation coefficient to obtain the fourth relation coefficient; The fourth relation coefficient is added to the second relation coefficient to obtain the target relation coefficient; The blurred image to be processed is blurred according to the target relation coefficient.

4. The image deblurring method according to claim 1, characterized in that, After determining the first relationship coefficient between the first DVS event information and the second DVS event information, the method further includes: Based on the first relationship coefficient between the first DVS event information and the second DVS event information, a fifth relationship coefficient is determined between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points other than its own time point. Based on the fifth relationship coefficient between the DVS event information of the image frame corresponding to each time point and the DVS event information of the image frame corresponding to other time points in each time point (excluding its own time point), the blurred image to be processed is deblurred sequentially to obtain the latent image corresponding to each time point. The latent image corresponding to each of the time points is input into a preset image processing model to obtain a clear image of the blurred image to be processed, wherein the preset image processing model is trained to predict the clear image of the latent image within the exposure time.

5. The image deblurring method according to claim 4, characterized in that, The latent image corresponding to each of the aforementioned time points is input into a preset image processing model to obtain a clear image of the blurred image to be processed, including: The latent images corresponding to each of the aforementioned time points are arranged sequentially according to the order of the exposure times; The latent images corresponding to each of the arranged time points are input into a preset image processing model to obtain a clear image of the blurred image to be processed.

6. The image deblurring method according to claim 4, characterized in that, The preset image processing model includes: RNN deep learning model.

7. An image deblurring device, characterized in that, include: The first acquisition module is used to acquire the blurred image to be processed and the exposure time of the blurred image to be processed; The slicing module is used to slice the exposure time into a preset number of time points and determine the image frame corresponding to each time point and the DVS event information corresponding to the image frame; wherein, the image frame at the middle time point corresponds to the first DVS event information, and the image frames at other time points other than the middle time point correspond to the second DVS event information. A first determining module is configured to determine a first relationship coefficient between the first DVS event information and the second DVS event information; the determination of the first relationship coefficient between the first DVS event information and the second DVS event information includes: determining multiple sub-first relationship coefficients between the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point, based on the first DVS event information and the second DVS event information corresponding to the image frames at each time point other than the intermediate time point; summing the multiple sub-first relationship coefficients to obtain the first relationship coefficient; the first relationship coefficient represents the association between the first DVS event information and the second DVS event information; The first processing module is used to perform deblurring processing on the blurred image to be processed according to the first relationship coefficient.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the image deblurring method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image deblurring method according to any one of claims 1 to 6.

Images