Motion blur detection method and device, electronic equipment and storage medium

Abstract

The invention relates to a motion blur detection method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a frame image of a target object; obtaining target event information of the target object collected by an event camera in a time period from the ith frame to the (i + 1) th frame, i being a positive integer; and based on the target event information, determining whether motion blur occurs in the (i + 1) th frame of image. Therefore, the event information collected in the time period between the two adjacent frames can be considered, motion blur detection is carried out on the next frame of image in the two adjacent frames, the information amount of the needed event information is small, the calculation amount is small, the detection efficiency of motion blur can be improved, and then the image processing efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a motion blur detection method, device, electronic device and storage medium. Background Art

[0002] Motion blur, also known as dynamic blur, refers to the obvious blur and dragging traces caused by fast-moving objects in a static scene or a series of images (such as movies, animations, etc.), which has a great impact on image quality. In the image preprocessing stage, most images are subjected to motion blur detection. However, the motion blur detection method in the related art has the problem of low detection efficiency. Summary of the invention

[0003] The present disclosure provides a motion blur detection method, device, electronic device, computer-readable storage medium, and computer program product to at least solve the problem of low detection efficiency in the motion blur detection method in the related art. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a method for detecting motion blur is provided, comprising: acquiring a frame image of a target object; acquiring target event information of the target object collected by an event camera in a time period from an i-th frame to an i+1-th frame, wherein i is a positive integer; and determining whether motion blur occurs in the i+1-th frame image based on the target event information.

[0005] In one embodiment of the present disclosure, determining whether motion blur occurs in the i+1th frame image based on the target event information includes: obtaining motion information of the target object based on the target event information; and determining whether motion blur occurs in the i+1th frame image based on the motion information.

[0006] In one embodiment of the present disclosure, the motion information includes motion speed, and determining whether the i+1th frame image has motion blur based on the motion information includes: if the motion speed is greater than a set threshold, determining that the i+1th frame image has motion blur; if the motion speed is less than or equal to the set threshold, determining that the i+1th frame image has no motion blur.

[0007] In one embodiment of the present disclosure, determining whether motion blur occurs in the i+1th frame image based on the target event information includes: inputting the target event information into a target model, and having the target model output a target recognition result of the i+1th frame image, wherein the target recognition result includes whether motion blur occurs.

[0008] In one embodiment of the present disclosure, the method further includes: acquiring frame sample images of the sample object; acquiring sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and the label of the j+1th frame sample image, wherein j is a positive integer and the label includes whether motion blur occurs; inputting the sample event information into an initial model, and the initial model outputting a predicted recognition result of the j+1th frame sample image, wherein the predicted recognition result includes whether motion blur occurs; training the initial model based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image to obtain the target model.

[0009] In one embodiment of the present disclosure, determining whether motion blur occurs in the i+1th frame image based on the target event information includes: obtaining a functional relationship between candidate event information and a candidate recognition result of the i+1th frame image, wherein the candidate event information is event information of the target object acquired by an event camera during a time period from the i-th frame to the i+1-th frame, and the candidate recognition result includes whether motion blur occurs; determining whether motion blur occurs in the i+1th frame image based on the target event information and the functional relationship.

[0010] In one embodiment of the present disclosure, obtaining the functional relationship includes: obtaining frame sample images of the sample object; obtaining sample event information of the sample object collected by an event camera in a time period from the jth frame to the j+1th frame, and a label of the j+1th frame sample image, wherein j is a positive integer and the label includes whether motion blur occurs; obtaining the functional relationship based on the sample event information and the label of the j+1th frame sample image.

[0011] In one embodiment of the present disclosure, the method further includes: if motion blur occurs in the (i+1)th frame image, deblurring the (i+1)th frame image.

[0012] According to a second aspect of an embodiment of the present disclosure, a motion blur detection device is provided, comprising: a first acquisition module, configured to execute acquisition of frame images of a target object; a second acquisition module, configured to execute acquisition of target event information of the target object collected by an event camera in a time period from an i-th frame to an i+1-th frame, wherein i is a positive integer; and a detection module, configured to determine whether motion blur occurs in the i+1-th frame image based on the target event information.

[0013] In one embodiment of the present disclosure, the detection module is further configured to execute: obtaining motion information of the target object based on the target event information; and determining whether motion blur occurs in the (i+1)th frame image based on the motion information.

[0014] In one embodiment of the present disclosure, the motion information includes motion speed, and the detection module is further configured to execute: if the motion speed is greater than a set threshold, determining that motion blur occurs in the i+1th frame image; if the motion speed is less than or equal to the set threshold, determining that no motion blur occurs in the i+1th frame image.

[0015] In one embodiment of the present disclosure, the detection module is further configured to execute: inputting the target event information into a target model, and having the target model output a target recognition result of the i+1th frame image, wherein the target recognition result includes whether motion blur occurs.

[0016] In one embodiment of the present disclosure, the device also includes: a training module, which is configured to execute: obtaining frame sample images of a sample object; obtaining sample event information of the sample object collected by an event camera in a time period from the jth frame to the j+1th frame, and a label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; inputting the sample event information into an initial model, and the initial model outputs a predicted recognition result of the j+1th frame sample image, wherein the predicted recognition result includes whether motion blur occurs; training the initial model based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image to obtain the target model.

[0017] In one embodiment of the present disclosure, the detection module is further configured to perform: obtaining a functional relationship between candidate event information and a candidate recognition result of the i+1th frame image, wherein the candidate event information is event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, and the candidate recognition result includes whether motion blur occurs; based on the target event information and the functional relationship, determining whether motion blur occurs in the i+1th frame image.

[0018] In one embodiment of the present disclosure, the detection module is further configured to perform: acquiring frame sample images of the sample object; acquiring sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and the label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; and obtaining the functional relationship based on the sample event information and the label of the j+1th frame sample image.

[0019] In one embodiment of the present disclosure, the detection module is further configured to execute: if motion blur occurs in the (i+1)th frame image, deblurring the (i+1)th frame image.

[0020] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method described in the first aspect of the embodiment of the present disclosure are implemented.

[0022] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor of an electronic device, the computer program implements the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0023] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: obtaining a frame image of a target object, obtaining target event information of the target object collected by an event camera in a time period from the i-th frame to the i+1-th frame, wherein i is a positive integer, and determining whether motion blur occurs in the i+1-th frame image based on the target event information. Thus, considering the event information collected in the time period between two adjacent frames, motion blur detection can be performed on the latter frame image of the two adjacent frames, and the amount of information and calculation required for the event information is small, which helps to improve the detection efficiency of motion blur, and thus improve the efficiency of image processing.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0026] Figure 1 The figure is a flow chart of a method for detecting motion blur according to an exemplary embodiment.

[0027] Figure 2 The figure is a flow chart of a method for detecting motion blur according to another exemplary embodiment.

[0028] Figure 3 The figure is a flow chart of a method for detecting motion blur according to another exemplary embodiment.

[0029] Figure 4 The figure is a flow chart of a method for detecting motion blur according to another exemplary embodiment.

[0030] Figure 5 The invention is a block diagram of a motion blur detection device according to an exemplary embodiment.

[0031] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] The acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant laws and regulations.

[0035] Figure 1 is a flow chart of a motion blur detection method according to an exemplary embodiment. Figure 1 As shown, the motion blur detection method of the embodiment of the present disclosure includes the following steps.

[0036] S101, obtaining a frame image of a target object.

[0037] It should be noted that the subject of the motion blur detection method of the embodiment of the present disclosure is an electronic device, such as a mobile phone, a notebook, a desktop computer, a vehicle-mounted terminal, a smart home appliance, a wearable device, etc. Among them, the wearable device may include a wrist-worn device (such as a smart watch, a smart bracelet), a head-worn device, a foot-worn device, etc. The motion blur detection method of the embodiment of the present disclosure can be executed by the motion blur detection device of the embodiment of the present disclosure, and the motion blur detection device of the embodiment of the present disclosure can be configured in any electronic device to execute the motion blur detection method of the embodiment of the present disclosure.

[0038] It should be noted that the target object refers to the object photographed by the image acquisition device, and there is no excessive limitation on the image acquisition device. For example, it may include a monochrome camera, a color camera, etc.

[0039] In one implementation, a frame image of a target object captured by an RGB camera may be obtained. In this case, the frame image is an RGB image. It should be noted that the RGB camera is a color camera, and the color of each pixel in the RGB image is composed of three channels: R (Red), G (Green), and B (Blue).

[0040] In one implementation, the frame image of the target object may be acquired according to a set frame rate. It should be noted that the set frame rate may be a fixed value or may be dynamically updated, and no further limitation is made here.

[0041] S102, obtaining target event information of the target object collected by the event camera in a time period from the i-th frame to the (i+1)-th frame, where i is a positive integer.

[0042] It should be noted that event cameras are also called DVS (Dynamic Vision Sensor), and any event camera in the relevant technology can be used, and there is no excessive limitation here. If the brightness change value of a certain pixel is greater than the set threshold, the event camera will generate event information for the pixel. If the brightness change value of a certain pixel is less than or equal to the set threshold, the event camera will not generate event information for the pixel, that is, the event camera only outputs event information of local pixels.

[0043] It can be understood that if the brightness change values ​​of all pixels at a certain acquisition moment are less than or equal to the set threshold, no event information is generated for all pixels at that acquisition moment, and the event camera does not output the event information at that acquisition moment, that is, the output frequency of the event camera is not fixed, and it has the advantages of high temporal resolution, low latency, and low power consumption.

[0044] It should be noted that there are no excessive restrictions on the event information of the pixel point, for example, it may include the identification, position, brightness increase or decrease, brightness change value, acquisition time, etc. of the pixel point. Among them, brightness increase or brightness decrease can be represented by a numerical value, for example, 1 is used to represent brightness increase, and 0 is used to represent brightness decrease.

[0045] It should be noted that the target event information refers to the event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame. It is the event information actually collected and may include the event information of the target object collected at multiple collection moments in the above time period. The event information of the target object collected at any collection moment may include the event information of at least one pixel point.

[0046] S103, based on the target event information, determining whether motion blur occurs in the (i+1)th frame image.

[0047] For example, taking i=1 as an example, the target event information A of the target object collected by the event camera in the time period from the first frame to the second frame can be obtained, and based on the target event information A, it is determined whether motion blur occurs in the second frame image.

[0048] Taking i=2 as an example, the target event information B of the target object collected by the event camera in the time period from the second frame to the third frame can be obtained, and based on the target event information B, it is determined whether motion blur occurs in the third frame image.

[0049] Taking i=3 as an example, the target event information C of the target object collected by the event camera in the time period from the 3rd frame to the 4th frame can be obtained, and based on the target event information C, it is determined whether motion blur occurs in the 4th frame image.

[0050] In one embodiment, based on the target event information, determining whether motion blur occurs in the i+1th frame image includes identifying whether the target object is in motion based on the target event information; if the target object is in motion, determining that motion blur occurs in the i+1th frame image; if the target object is not in motion, determining that motion blur does not occur in the i+1th frame image.

[0051] In some examples, based on the target event information, whether the target object is in motion is identified, including if the number of pixels in the target event information is greater than a set threshold, indicating that the number of pixels with larger brightness changes is larger, then the target object is more likely to be in motion, and the target object can be identified as being in motion; conversely, if the number of pixels in the target event information is less than or equal to the set threshold, indicating that the number of pixels with larger brightness changes is smaller, then the target object is less likely to be in motion, and the target object can be identified as not being in motion.

[0052] In some examples, based on the target event information, whether the target object is in motion is identified, including if the average value of the brightness change values ​​of multiple pixels in the target event information is greater than a set threshold, indicating that the brightness change of the pixel points is large, then the target object is more likely to be in motion, and the target object can be identified as being in motion; conversely, if the average value of the brightness change values ​​of multiple pixels in the target event information is less than or equal to the set threshold, indicating that the brightness change of the pixel points is small, then the target object is less likely to be in motion, and it can be identified that the target object is not in motion.

[0053] In one embodiment, the method further includes deblurring the i+1th frame image if motion blur occurs in the i+1th frame image. Thus, the frame image is deblurred only when motion blur occurs in the frame image, thereby avoiding deblurring the frame image that does not have motion blur, that is, the frame image that does not have motion blur can skip the deblurring step, thereby improving image processing efficiency and reducing the amount of calculation.

[0054] The motion blur detection method provided by the embodiment of the present disclosure obtains a frame image of a target object, obtains target event information of the target object collected by an event camera in a time period from the i-th frame to the i+1-th frame, wherein i is a positive integer, and determines whether the i+1-th frame image has motion blur based on the target event information. Thus, the event information collected in the time period between two adjacent frames can be taken into account to perform motion blur detection on the latter frame image of the two adjacent frames. The amount of event information required is small, and the amount of calculation is small, which helps to improve the detection efficiency of motion blur, and thus improve the efficiency of image processing.

[0055] Figure 2 is a flow chart of a motion blur detection method according to another exemplary embodiment. Figure 2 As shown, the motion blur detection method of the embodiment of the present disclosure includes the following steps.

[0056] S201, obtaining a frame image of a target object.

[0057] S202 , obtaining target event information of the target object collected by the event camera in a time period from the i-th frame to the (i+1)-th frame, where i is a positive integer.

[0058] For the relevant contents of steps S201 - S202 , please refer to the above embodiment and will not be described again here.

[0059] S203: Obtain motion information of the target object based on the target event information.

[0060] It should be noted that there are no excessive restrictions on the motion information, for example, it may include motion speed, motion acceleration, motion direction, motion trajectory, etc. The motion information may include the motion information of the target object in the time period from the i-th frame to the i+1-th frame, and may also include the motion information of the target object in the i+1-th frame.

[0061] In one embodiment, based on the target event information, the motion information of the target object is obtained, including inputting the target event information into an optical flow algorithm, and the optical flow algorithm outputs the motion information. It should be noted that the optical flow algorithm can adopt any optical flow algorithm in the relevant technology, such as Lucas-Kanade Method, Horn Schunck Method, etc., which are not limited here.

[0062] In one embodiment, obtaining the motion information of the target object based on the target event information includes obtaining the number of pixels in the target event information and obtaining the motion information based on the number of pixels. For example, the motion speed and motion acceleration are positively correlated with the number of pixels.

[0063] In one embodiment, the motion information of the target object is obtained based on the target event information, including obtaining the average value of the brightness change values ​​of multiple pixels in the target event information, and obtaining the motion information based on the average value. For example, the motion speed and motion acceleration are positively correlated with the average value.

[0064] In one embodiment, the target event information includes event information of the target object collected at N collection moments, and the motion information of the target object is obtained based on the target event information, including obtaining the position of the target object at the pth collection moment based on the event information of the target object collected at the pth collection moment, and fitting the position of the target object at the N collection moments to obtain the motion trajectory of the target object. Wherein, p is a positive integer not greater than N, and N is a positive integer.

[0065] In one embodiment, based on the target event information, the motion information of the target object is obtained, including inputting the target event information into a motion model, and the motion model outputs the motion information. It should be noted that the motion model is not too limited, for example, it may include a deep learning model.

[0066] S204: Determine whether motion blur occurs in the (i+1)th frame image based on the motion information.

[0067] In one embodiment, the motion information includes motion speed, and based on the motion information, it is determined whether motion blur occurs in the i+1th frame image, including determining that motion blur occurs in the i+1th frame image if the motion speed is greater than a set threshold, and determining that motion blur does not occur in the i+1th frame image if the motion speed is less than or equal to the set threshold.

[0068] In one embodiment, the motion information includes motion acceleration, and based on the motion information, it is determined whether motion blur occurs in the i+1th frame image, including if the motion acceleration is greater than a set threshold, determining that motion blur occurs in the i+1th frame image, and if the motion acceleration is less than or equal to the set threshold, determining that motion blur does not occur in the i+1th frame image.

[0069] In one embodiment, the motion information includes a motion trajectory. Based on the motion information, determining whether motion blur occurs in the i+1th frame image includes obtaining the distance between any two adjacent trajectory points in the motion trajectory. If the distance between any two adjacent trajectory points is greater than a set threshold, determining that motion blur occurs in the i+1th frame image; if the distance between any two adjacent trajectory points is less than or equal to the set threshold, determining that motion blur does not occur in the i+1th frame image.

[0070] In one embodiment, the motion information includes a motion direction, and based on the motion information, it is determined whether motion blur occurs in the i+1th frame image, including determining that motion blur occurs in the i+1th frame image if the motion direction of the target object is different from the motion direction of the image acquisition device, and determining that motion blur does not occur in the i+1th frame image if the motion direction of the target object is the same as the motion direction of the image acquisition device.

[0071] The motion blur detection method provided by the embodiment of the present disclosure obtains the motion information of the target object based on the target event information, and determines whether the i+1th frame image has motion blur based on the motion information. Thus, the target event information can be taken into account to obtain the motion information of the target object, and motion blur detection can be performed on the latter frame image of two adjacent frames.

[0072] Figure 3 is a flow chart of a motion blur detection method according to another exemplary embodiment. Figure 3 As shown, the motion blur detection method of the embodiment of the present disclosure includes the following steps.

[0073] S301, obtaining a frame image of a target object.

[0074] S302, obtaining target event information of the target object collected by the event camera in a time period from the i-th frame to the i+1-th frame, where i is a positive integer.

[0075] For the relevant contents of steps S301 - S302 , please refer to the above embodiments and will not be described again here.

[0076] S303, inputting the target event information into the target model, and the target model outputs the target recognition result of the (i+1)th frame image, wherein the target recognition result includes whether motion blur occurs.

[0077] It should be noted that there are not too many restrictions on the target model, for example, it can include deep learning models.

[0078] In one embodiment, the method further includes acquiring a frame sample image of the sample object, acquiring sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and a label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs, inputting the sample event information into an initial model, and the initial model outputting a predicted recognition result of the j+1th frame sample image, wherein the predicted recognition result includes whether motion blur occurs, and training the initial model based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image to obtain a target model. Thus, the training of the target model can be achieved based on the sample event information and the label of the frame sample image.

[0079] It should be noted that the label can be represented by a numerical value, for example, 1 is used to represent the presence of motion blur, and 0 is used to represent the absence of motion blur.

[0080] It should be noted that the initial model is trained based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image to obtain the target model. Any model training method in the relevant technology can be used to achieve this, and no excessive restrictions are made here.

[0081] In some examples, based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image, the initial model is trained to obtain the target model, which may include obtaining a loss function based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image, and training the initial model based on the loss function to obtain the target model. It should be noted that there are no excessive restrictions on the loss function, for example, it may include CE (Cross Entropy), MSE (Mean-Square Error), KL (Kullback-Leibler) divergence, contrast loss function, etc.

[0082] In one embodiment, the target recognition result, the label, also includes the image area where motion blur occurs.

[0083] The motion blur detection method provided by the embodiment of the present disclosure inputs the target event information into the target model, and the target model outputs the target recognition result of the i+1th frame image, wherein the target recognition result includes whether motion blur occurs. Thus, the target event information can be processed by the target model, and the motion blur detection can be performed on the latter frame image of two adjacent frames.

[0084] Figure 4is a flow chart of a motion blur detection method according to another exemplary embodiment. Figure 4 As shown, the motion blur detection method of the embodiment of the present disclosure includes the following steps.

[0085] S401, acquiring a frame image of a target object.

[0086] S402, obtaining target event information of the target object collected by the event camera in a time period from the i-th frame to the (i+1)-th frame, where i is a positive integer.

[0087] For the relevant contents of steps S401 - S402 , please refer to the above embodiment and will not be described again here.

[0088] S403, obtaining a functional relationship between candidate event information and candidate recognition results of the i+1th frame image, wherein the candidate event information is event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, and the candidate recognition results include whether motion blur occurs.

[0089] It should be noted that the candidate event information refers to the event information of the target object that may be collected by the event camera in the time period from the i-th frame to the i+1-th frame, not the actual collected event information, and the candidate recognition result of the i+1-th frame image refers to the possible recognition result of the i+1-th frame image, not the actual recognition result.

[0090] It should be noted that the functional relationship refers to the functional relationship between the candidate event information as the independent variable and the candidate recognition result of the i+1th frame image as the dependent variable. There are no excessive restrictions on the functional relationship, for example, it can include a linear functional relationship, an inverse proportional functional relationship, a quadratic functional relationship, an exponential functional relationship, etc.

[0091] In one embodiment, the functional relationship is preset, for example, the functional relationship can be calibrated during the design, manufacture, and use of the event camera or image acquisition device. The number of the functional relationship is at least one.

[0092] In one implementation, acquiring the functional relationship includes obtaining the functional relationship based on a mapping relationship between an identifier of the event camera and / or an identifier of the image acquisition device and the functional relationship.

[0093] In one embodiment, obtaining the functional relationship includes obtaining a frame sample image of the sample object, obtaining sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and a label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs, and obtaining the functional relationship based on the sample event information and the label of the j+1th frame sample image. Thus, the functional relationship can be obtained based on the sample event information and the label of the frame sample image.

[0094] In some examples, a functional relationship is obtained based on sample event information and a label of the j+1th frame sample image, including determining the jth sample point in a set coordinate system based on the sample event information and the label of the j+1th frame sample image, fitting multiple sample points to obtain a fitting curve, and using the functional relationship corresponding to the fitting curve as the final functional relationship.

[0095] For example, taking the coordinate system as a two-dimensional coordinate system, the sample event information can be converted into the X coordinate in the two-dimensional coordinate system, and the label of the j+1th frame sample image can be converted into the Y coordinate in the two-dimensional coordinate system, and the coordinate point corresponding to the X coordinate and Y coordinate in the two-dimensional coordinate system is used as the jth sample point.

[0096] S404: Determine whether motion blur occurs in the (i+1)th frame image based on the target event information and the functional relationship.

[0097] In one embodiment, determining whether motion blur occurs in the i+1th frame image based on the target event information and the functional relationship includes obtaining a target recognition result of the i+1th frame image based on the target event information and the functional relationship to determine whether motion blur occurs in the i+1th frame image. The target recognition result includes whether motion blur occurs.

[0098] In some examples, based on the target event information and the functional relationship, the target recognition result of the i+1th frame image is obtained, including substituting the target event information as an independent variable into the functional relationship to obtain the dependent variable of the functional relationship as the target recognition result of the i+1th frame image.

[0099] In some examples, obtaining the target recognition result of the i+1th frame image based on the target event information and the functional relationship includes selecting a target functional relationship from a plurality of functional relationships, and obtaining the target recognition result of the i+1th frame image based on the target event information and the target functional relationship. Thus, if there are multiple functional relationships, the target functional relationship can be selected to obtain the target recognition result of the i+1th frame image.

[0100] For example, selecting a target function relationship from multiple function relationships includes identifying whether the setting conditions of the function relationship are currently met, and if the setting conditions of the function relationship are currently met, using the function relationship as the target function relationship. It should be noted that there are no excessive restrictions on the setting conditions of the function relationship, and different function relationships can correspond to different setting conditions.

[0101] The motion blur detection method provided by the embodiment of the present disclosure obtains the functional relationship between the candidate event information and the candidate recognition result of the i+1th frame image, wherein the candidate event information is the event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, and the candidate recognition result includes whether motion blur occurs. Based on the target event information and the functional relationship, it is determined whether motion blur occurs in the i+1th frame image. Therefore, the target event information and the functional relationship can be comprehensively considered to perform motion blur detection on the latter frame image of two adjacent frames.

[0102] Figure 5 is a block diagram of a motion blur detection device according to an exemplary embodiment. Figure 5 The motion blur detection device 100 of the embodiment of the present disclosure includes: a first acquisition module 110, a second acquisition module 120 and a detection module 130.

[0103] A first acquisition module 110 is configured to acquire a frame image of a target object;

[0104] The second acquisition module 120 is configured to acquire target event information of the target object collected by the event camera in a time period from the i-th frame to the i+1-th frame, where i is a positive integer;

[0105] The detection module 130 is configured to determine whether motion blur occurs in the (i+1)th frame image based on the target event information.

[0106] In one embodiment of the present disclosure, the detection module 130 is further configured to execute: obtaining motion information of the target object based on the target event information; and determining whether motion blur occurs in the (i+1)th frame image based on the motion information.

[0107] In one embodiment of the present disclosure, the motion information includes motion speed, and the detection module 130 is further configured to execute: if the motion speed is greater than a set threshold, determining that motion blur occurs in the i+1th frame image; if the motion speed is less than or equal to the set threshold, determining that no motion blur occurs in the i+1th frame image.

[0108] In one embodiment of the present disclosure, the detection module 130 is further configured to execute: inputting the target event information into a target model, and having the target model output a target recognition result of the i+1th frame image, wherein the target recognition result includes whether motion blur occurs.

[0109] In one embodiment of the present disclosure, the motion blur detection device 100 also includes: a training module, which is configured to execute: obtaining frame sample images of a sample object; obtaining sample event information of the sample object collected by an event camera in a time period from the jth frame to the j+1th frame, and a label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; inputting the sample event information into an initial model, and the initial model outputs a predicted recognition result of the j+1th frame sample image, wherein the predicted recognition result includes whether motion blur occurs; training the initial model based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image to obtain the target model.

[0110] In one embodiment of the present disclosure, the detection module 130 is further configured to perform: obtaining a functional relationship between candidate event information and a candidate recognition result of the i+1th frame image, wherein the candidate event information is event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, and the candidate recognition result includes whether motion blur occurs; based on the target event information and the functional relationship, determining whether motion blur occurs in the i+1th frame image.

[0111] In one embodiment of the present disclosure, the detection module 130 is further configured to perform: acquiring frame sample images of the sample object; acquiring sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and the label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; and obtaining the functional relationship based on the sample event information and the label of the j+1th frame sample image.

[0112] In one embodiment of the present disclosure, the detection module 130 is further configured to execute: if motion blur occurs in the (i+1)th frame image, deblurring the (i+1)th frame image.

[0113] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0114] The motion blur detection device provided by the embodiment of the present disclosure obtains the frame image of the target object, obtains the target event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, wherein i is a positive integer, and determines whether the i+1-th frame image has motion blur based on the target event information. Thus, the event information collected in the time period between two adjacent frames can be taken into account to perform motion blur detection on the latter frame image of the two adjacent frames. The amount of event information required is small, and the amount of calculation is small, which helps to improve the detection efficiency of motion blur, and thus improve the efficiency of image processing.

[0115] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment.

[0116] like Figure 6 As shown, the electronic device 200 includes:

[0117] The memory 210 and the processor 220, a bus 230 connecting different components (including the memory 210 and the processor 220), the memory 210 stores a computer program, and when the processor 220 executes the program, the motion blur detection method described in the embodiment of the present disclosure is implemented.

[0118] Bus 230 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0119] The electronic device 200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 200, including volatile and non-volatile media, removable and non-removable media.

[0120] The memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. The electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 260 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 230 via one or more data medium interfaces. The memory 210 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0121] A program / utility 280 having a set (at least one) of program modules 270 may be stored, for example, in the memory 210, such program modules 270 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 270 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0122] The electronic device 200 may also communicate with one or more external devices 290 (e.g., keyboards, pointing devices, displays 291, etc.), one or more devices that enable a user to interact with the electronic device 200, and / or any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 292. Furthermore, the electronic device 200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 293. Figure 6 As shown, the network adapter 293 communicates with other modules of the electronic device 200 via the bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] The processor 220 executes various functional applications and data processing by running the programs stored in the memory 210 .

[0124] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the motion blur detection method of the embodiment of the present disclosure, and will not be repeated here.

[0125] The electronic device provided by the embodiment of the present disclosure can execute the motion blur detection method as described above, obtain the frame image of the target object, obtain the target event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, where i is a positive integer, and determine whether the i+1-th frame image has motion blur based on the target event information. Therefore, considering the event information collected in the time period between two adjacent frames, the latter frame image in the two adjacent frames can be subjected to motion blur detection, and the amount of information and calculation required for the event information is small, which helps to improve the detection efficiency of motion blur, and thus improve the efficiency of image processing.

[0126] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the motion blur detection method provided by the present disclosure when the program instructions are executed by a processor.

[0127] Alternatively, the computer readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0128] In order to implement the above embodiments, the present disclosure further provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the motion blur detection method as described above is implemented.

[0129] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0130] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting motion blur, It is characterized in that include: Acquire a frame image of a target object; Obtain target event information of the target object collected by the event camera in the time period from the i-th frame to the i+1-th frame, where i is a positive integer; Based on the target event information, it is determined whether motion blur occurs in the (i+1)th frame image.

2. The method according to claim 1, It is characterized in that The determining, based on the target event information, whether motion blur occurs in the (i+1)th frame image includes: Based on the target event information, obtaining motion information of the target object; Based on the motion information, it is determined whether motion blur occurs in the (i+1)th frame image.

3. The method according to claim 2, It is characterized in that The motion information includes a motion speed, and determining whether the (i+1)th frame image has motion blur based on the motion information includes: If the motion speed is greater than a set threshold, it is determined that motion blur occurs in the i+1th frame image; If the motion speed is less than or equal to the set threshold, it is determined that the (i+1)th frame image does not have motion blur.

4. The method according to claim 1, It is characterized in that The determining, based on the target event information, whether motion blur occurs in the (i+1)th frame image includes: The target event information is input into a target model, and the target model outputs a target recognition result of the (i+1)th frame image, wherein the target recognition result includes whether motion blur occurs.

5. The method according to claim 4, It is characterized in that The method further comprises: Get the frame sample image of the sample object; Obtaining sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and the label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; Inputting the sample event information into an initial model, and having the initial model output a predicted recognition result of the j+1th frame sample image, wherein the predicted recognition result includes whether motion blur occurs; Based on the label of the j+1th frame sample image and the predicted recognition result of the j+1th frame sample image, the initial model is trained to obtain the target model.

6. The method according to claim 1, It is characterized in that The determining, based on the target event information, whether motion blur occurs in the (i+1)th frame image includes: Acquire a functional relationship between candidate event information and a candidate recognition result of the i+1th frame image, wherein the candidate event information is event information of the target object collected by the event camera in a time period from the i-th frame to the i+1-th frame, and the candidate recognition result includes whether motion blur occurs; Based on the target event information and the functional relationship, it is determined whether motion blur occurs in the (i+1)th frame image.

7. The method according to claim 6, It is characterized in that Obtaining the functional relationship includes: Get the frame sample image of the sample object; Obtaining sample event information of the sample object collected by the event camera in the time period from the jth frame to the j+1th frame, and the label of the j+1th frame sample image, wherein j is a positive integer, and the label includes whether motion blur occurs; The functional relationship is obtained based on the sample event information and the label of the j+1th frame sample image.

8. The method according to any one of claims 1 to 7, It is characterized in that The method further comprises: If the i+1th frame image is motion blurred, a deblurring process is performed on the i+1th frame image.

9. A motion blur detection device, It is characterized in that include: A first acquisition module is configured to acquire a frame image of a target object; A second acquisition module is configured to acquire target event information of the target object collected by the event camera in a time period from the i-th frame to the i+1-th frame, wherein i is a positive integer; The detection module is configured to determine whether motion blur occurs in the (i+1)th frame image based on the target event information.

10. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1 to 8.

11. A computer readable storage medium having computer program instructions stored thereon, It is characterized in that When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images