A method and device for removing ghosting and an electronic device

Abstract

The application discloses a method and device for removing ghosting, an electronic device and a computer readable storage medium. The method comprises: determining a ghosting area in a to-be-processed image through a ghosting detection model; determining target ghosting pixels in each ghosting sub-area according to a reference image, wherein the reference image and the to-be-processed image come from the same video stream, the ghosting sub-area is obtained by dividing the ghosting area, and the number of the ghosting sub-areas is more than two; screening a target ghosting sub-area according to the number of the target ghosting pixels in each ghosting sub-area; and replacing each target ghosting sub-area in the to-be-processed image with a corresponding area of the reference image to obtain a first target image. Through the application, the damage to the original image background during ghosting removal can be reduced, and the difference between the image after ghosting removal and the original image can be avoided.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method for removing ghosting, a device for removing ghosting, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Ghosting, also known as motion blur, refers to the phenomenon where, in low-light environments, due to the camera's poor light sensitivity, the trajectory of a moving object cannot converge in time during imaging, resulting in a ghost image (or ghosting) appearing behind the moving object in the image. In the field of security monitoring, ghosting increases the difficulty of target recognition in the monitored image, affecting the effectiveness and real-time performance of security measures.

[0003] Currently, several motion blur removal schemes have been proposed to ensure image quality, such as motion tracking-based motion blur removal schemes. However, these schemes do not fully consider the real-world scene and are prone to damaging the background of the original image during motion blur removal, resulting in significant differences between the removed image and the original image. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and computer-readable storage medium for removing ghosting, which can reduce the damage to the original image background during ghosting removal and avoid excessive differences between the ghosting-removed image and the original image.

[0005] Firstly, this application provides a method for removing ghosting, including:

[0006] The motion blur region in the image to be processed is determined by the motion blur detection model;

[0007] Based on the reference image, the target trailing pixels within each trailing region are determined. The reference image and the image to be processed come from the same video stream. The trailing regions are obtained by dividing the trailing regions, and the number of trailing regions is more than two.

[0008] The target trailing regions are obtained by filtering based on the number of target trailing pixels in each trailing region.

[0009] The shadow regions of each target in the image to be processed are replaced with the corresponding regions in the reference image to obtain the first target image.

[0010] Secondly, this application provides a motion blur removal device, comprising:

[0011] The first determining module is used to determine the motion blur region in the image to be processed by the motion blur detection model;

[0012] The second determining module is used to determine the target trailing pixels in each trailing region based on the reference image. The reference image and the image to be processed come from the same video stream. The trailing regions are obtained by dividing the trailing regions, and the number of trailing regions is two or more.

[0013] The filtering module is used to filter the target shadow regions based on the number of target shadow pixels in each shadow region.

[0014] The replacement module is used to replace the shadow regions of each target in the image to be processed with the corresponding regions of the reference image to obtain the first target image.

[0015] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in the first aspect.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0017] Fifthly, this application provides a computer program product comprising a computer program that, when executed by one or more processors, implements the steps of the method described in the first aspect.

[0018] The beneficial effects of this application compared to existing technologies are as follows: In this application, for an image to be processed that exhibits motion blur, the electronic device can first determine the motion blur region in the image to be processed using a motion blur detection model. It can be understood that this motion blur region provides an initial limitation on the scope of motion blur removal. Subsequently, the electronic device determines the target motion blur pixels within each motion blur region based on a reference image. The reference image and the image to be processed originate from the same video stream, and the motion blur regions are obtained by dividing the motion blur regions, with at least two motion blur regions. Next, the electronic device can filter and obtain the target motion blur regions based on the number of target motion blur pixels within each motion blur region. It can be understood that this target motion blur region provides a final limitation on the scope of motion blur removal. Finally, the electronic device can replace each target motion blur region in the image to be processed with the corresponding region of the reference image. After the replacement is completed, the first target image is obtained. In the above process, on the one hand, this application does not directly perform ghosting removal on the entire ghosting region. Instead, it further filters out the target ghosting regions that truly require ghosting removal and only performs ghosting removal on these target ghosting regions, ensuring that the first target image obtained after ghosting removal does not exhibit blocky phenomena. On the other hand, considering the adaptability and consistency of human visual perception, the real scene is fully considered during ghosting removal processing. Specifically, the target ghosting regions are replaced with corresponding regions of the reference image. In summary, the solution of this application can reduce the damage to the original image background during ghosting removal while ensuring the ghosting removal effect, avoiding excessive differences between the ghosting-removed image and the original image.

[0019] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating the implementation process of the ghosting removal method provided in the embodiments of this application;

[0022] Figure 2 This is an example image of the first target image obtained after removing ghosting from the image to be processed based on a reference image, as provided in the embodiments of this application.

[0023] Figure 3This is an example diagram illustrating the data flow when an image to be processed is processed in the order of de-ghosting, de-blurring, and de-noising, as provided in the embodiments of this application.

[0024] Figure 4 This is a quantitative example diagram of the motion blur level provided in the embodiments of this application;

[0025] Figure 5 This is a structural block diagram of the ghosting removal device provided in the embodiments of this application;

[0026] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0028] Currently, there are existing ghosting removal schemes based on motion tracking or motion segmentation. However, due to the complexity of backgrounds in real-world scenes, these schemes introduce numerous interfering factors, such as swaying leaves, flowing rivers, and / or flickering lights, reducing their robustness and making it difficult to accurately identify ghosting regions. Consequently, existing ghosting removal schemes typically remove detected ghosting regions directly, which damages the original image's background, resulting in significant differences between the ghosted and original images. Therefore, this application provides a ghosting removal method, device, electronic device, and computer-readable storage medium that reduces damage to the original image's background during ghosting removal and avoids excessive differences between the ghosted and original images. To illustrate the technical solution proposed in this application, specific embodiments are described below.

[0029] The ghosting removal method provided in this application is applied to electronic devices with image processing capabilities. This is merely an example; the electronic device can be a personal computer (PC), smartphone, tablet, surveillance camera, or server, etc., and is not limited thereto. Please refer to [link to relevant documentation]. Figure 1 The implementation process of this ghosting removal method is detailed below:

[0030] Step 101: Determine the motion blur region in the image to be processed using the motion blur detection model.

[0031] Electronic devices can identify non-first frame images in a received video stream as images to be processed. It can be understood that when an electronic device receives a frame from a video stream, if that frame is not the first frame, it can be considered that the image may have motion blur, and therefore is considered an image to be processed.

[0032] Considering that motion blur often occurs in dimly lit environments, to conserve processing resources and reduce unnecessary motion blur detection operations, in some examples, the electronic device can identify non-first frame images from the received video stream that were captured within a preset nighttime period as images to be processed. It can be understood that when the electronic device receives a non-first frame image from the video stream, it can also obtain the image's capture time and detect whether the capture time falls within the preset nighttime period; if the capture time is within this nighttime period, the image is considered to have a high probability of motion blur and can be identified as an image to be processed. Alternatively, in other examples, the electronic device can also identify non-first frame images from the received video stream whose brightness is below a preset brightness threshold as images to be processed. It can be understood that when the electronic device receives a non-first frame image from the video stream, it can detect the image brightness of that image; if the detection result indicates that the image brightness is below the preset brightness threshold, the light at the time the image was captured was dim, the image is considered to have a high probability of motion blur, and the image can be identified as an image to be processed.

[0033] After obtaining the image to be processed, the electronic device can input the image to a motion blur detection model. The output of the motion blur detection model is a motion blur region detection box of the input image to be processed. It can be understood that the area enclosed by this motion blur region detection box in the image to be processed is the motion blur region in the image. For ease of distinction, the pixels within the motion blur region can be called motion blur pixels; these motion blur pixels are pixels that are highly likely to be affected by the motion blur phenomenon.

[0034] For illustrative purposes only, the detection bounding box for the ghosting region is rectangular. Of course, the ghosting detection region can also be represented by other shapes, which are not limited here.

[0035] As an example, this ghosting detection model can be constructed using a one-stage Convolutional Neural Network (CNN) containing convolutional layers, pooling layers, and fully connected layers. In practical applications in security monitoring, the number of layers in this CNN does not exceed 24. This is because 24 layers is sufficient to achieve good network performance; further increasing the network depth would not significantly improve performance and would also severely increase training and detection time. Of course, other neural networks can also be used to construct this ghosting detection model; this is not a limitation here.

[0036] It is understood that before putting the motion blur detection model into application, the motion blur detection model should be trained in advance using the corresponding training set; that is, in this embodiment, the motion blur detection model used is a pre-trained model.

[0037] Of course, smart devices can also determine the motion blur region in other ways, including but not limited to frame difference method and motion segmentation method, which are not limited here.

[0038] Step 102: Based on the reference image, determine the target trailing pixel points within each trailing shadow area.

[0039] The electronic device can pre-determine a reference image. This reference image and the image to be processed originate from the same video stream, and the reference image contains no motion blur (i.e., objects in the reference image are kept as still as possible). By comparing the reference image and the image to be processed, the electronic device can identify the target motion blur pixel from the motion blur pixels in the motion blur region. This target motion blur pixel is the pixel ultimately determined to be affected by the motion blur phenomenon.

[0040] The electronic device can divide the ghosting region of the image to be processed into two or more non-overlapping ghosting regions. The shapes of the different ghosting regions can be the same or different, without limitation; and the areas of the different ghosting regions can be the same or different, without limitation. To improve the subsequent ghosting removal effect, the area of ​​the ghosting region can be set to a relatively small value, controlled within a certain range. As an example only, the area of ​​the ghosting region can be set to not exceed a preset area threshold. Combining the divided ghosting regions in the image to be processed, and the target ghosting pixels in the image to be processed, the electronic device can determine the target ghosting pixels within each ghosting region.

[0041] Step 103: Filter out the target shadow regions based on the number of target shadow pixels in each shadow region.

[0042] For any given ghosting region, the more target ghosting pixels within that region, the more severe the ghosting; conversely, the fewer target ghosting pixels, the less severe the ghosting. In other words, the number of target ghosting pixels within a ghosting region reflects the degree of ghosting. Based on this, electronic devices can filter out ghosting regions with more severe ghosting based on the number of target ghosting pixels in each ghosting region. For ease of distinction, such ghosting regions can be denoted as target ghosting regions.

[0043] Step 104: Replace each target shadow region in the image to be processed with the corresponding region in the reference image to obtain the first target image.

[0044] Regarding the ghosting areas in the image to be processed, the electronic device can retain the non-target ghosting areas and only perform ghosting removal processing on the target ghosting areas. It's understandable that because the ghosting in the target ghosting areas is more severe, it's necessary for the electronic device to perform ghosting removal processing on them; while the ghosting in the non-target ghosting areas is less severe. If the electronic device also performs ghosting removal processing on the non-target ghosting areas, it would result in the ghosting removal processing covering too large an area, causing the parts corresponding to the ghosting areas in the processed image to appear as obvious blocks, resulting in an unrealistic and unnatural image effect.

[0045] Specifically, since the image to be processed is a frame in a video stream, to ensure the adaptability and consistency of the image after ghosting removal in the human eye when the user watches the video stream, the ghosting removal process performed by the electronic device on the target ghosting region can be specifically as follows: each target ghosting region in the image to be processed is replaced with a corresponding region in the reference image. It can be understood that for any target ghosting region, the boundary coordinates of the target ghosting region are known to the electronic device. Therefore, the electronic device can find the corresponding region in the reference image based on the boundary coordinates of the target ghosting region and directly replace the target ghosting region in the image to be processed with the corresponding region. It can be understood that the coordinate information (position information) of the target ghosting region and its corresponding region are completely consistent; the only difference is that the target ghosting region originates from the image to be processed, and the corresponding region originates from the reference image.

[0046] After removing the ghosting from all target shadow areas in the image to be processed, the ghosted image, also known as the first target image, is obtained. This first target image can be output to the display screen of an electronic device for user viewing.

[0047] For example only, please refer to Figure 2 , Figure 2 An example of the first target image obtained after removing ghosting from the image to be processed based on a reference image is given. The ghosting region of the image to be processed is divided into four ghosting regions: A1, A2, A3, and A4. Among them, A3 and A4 are identified as the target ghosting regions. In the first target image obtained after ghosting removal, A3 is replaced with the corresponding region a3 in the reference image, and A4 is replaced with the corresponding region a4 in the reference image. It should be noted that... Figure 2 In the image, the reference image is represented in white, the image to be processed is represented in gray, and the gray part in the first target image represents the image to be processed, while the white part represents the reference image.

[0048] In some embodiments, to avoid misidentification of the reference image and ensure that the obtained reference image can represent the actual static scene, the electronic device may determine the reference image through the following steps:

[0049] A1. In the video stream, acquire multiple consecutive frames of images.

[0050] In situations where real-time performance is critical (e.g., when real-time video streaming is required), the consecutive frames obtained in this step should precede the image to be processed in the video stream. This ensures that the electronic device can immediately begin motion blur detection and removal of the image to be processed upon receiving it. Conversely, in situations where real-time performance is less critical (e.g., when playing video streams offline), there is no need to restrict the timing of the consecutive frames and the image to be processed within the video stream.

[0051] To further ensure consistency in human visual perception, and to help accurately identify target trailing pixels within the trailing shadow area, the electronic device can specifically acquire multiple consecutive frames of images preceding the image to be processed in the video stream, and the interval between the first frame of these multiple consecutive frames and the image to be processed is within a preset interval threshold.

[0052] A2. In a series of consecutive images, detect the inter-frame differences between adjacent images.

[0053] It can be understood that in a series of consecutive images, the first and second frames are adjacent, the second and third frames are adjacent, and so on. Let N be the number of frames in a series of consecutive images, then there are N-1 pairs of adjacent images. For each pair of adjacent images, the electronic device can detect their inter-frame differences; that is, it can detect the inter-frame differences between the first and second frames, the second and third frames, ..., and the (N-1)th and Nth frames. Specifically, the inter-frame difference in this step refers to the difference in pixel values, that is, the absolute value of the grayscale difference.

[0054] A3. If the inter-frame differences of all adjacent images meet the preset difference conditions, then any frame in a series of consecutive frames will be determined as the reference image.

[0055] The electronic device can set a first pixel value difference threshold of t1 and a pixel count threshold of q. For any pair of adjacent images, based on their inter-frame difference and the first pixel value difference threshold t1, pixels with significant differences can be identified, i.e., pixels whose inter-frame difference exceeds the first pixel value difference threshold t1. After dilation and erosion of these pixels, the final connected component can be obtained. If the number of pixels x in the connected component exceeds the pixel count threshold q, then the connected component represents a moving target. Based on this, it can be determined that the inter-frame difference of the pair of adjacent images does not meet the difference condition, i.e., the pair of adjacent images shows a significant image change. As an example only, the first pixel value difference threshold t1 can be 0.

[0056] If the inter-frame differences of all adjacent images satisfy the difference condition, then the consecutive frames can be considered to be essentially the same, meaning the corresponding scene is approximately a static scene. In this case, any frame in the consecutive frames can be determined as the reference image.

[0057] In some embodiments, to quickly determine the target trailing pixel within the trailing shadow area, step 102 may specifically include:

[0058] B1. Generate a frame difference image based on the inter-frame differences between the reference image and the image to be processed.

[0059] The electronic device can calculate the inter-frame difference between a reference image and the image to be processed, which includes the difference in pixel value for each pixel; based on the inter-frame difference, the electronic device can generate a corresponding frame difference image.

[0060] As an example only, the pixel value of the (x,y) pixel in the reference image is pixel1, and the pixel value of the (x,y) pixel in the image to be processed is pixel2. Then the pixel value of the (x,y) pixel in the frame difference image is |pixel1-pixel2|.

[0061] B2. Binarize the frame difference image to obtain a binarized image.

[0062] Based on a preset second pixel value difference threshold t2, the electronic device can binarize the frame difference image. It can be understood that for pixels with pixel values ​​greater than or equal to the second pixel value difference threshold t2, they can be binarized into a first pixel value (255); for pixels with pixel values ​​less than the second pixel value difference threshold t2, they can be binarized into a second pixel value (0).

[0063] B3. The pixels corresponding to the target binarized pixels in the binarized image within each shadow region are determined as target shadow pixels. The target binarized pixels are: pixels with a preset pixel value.

[0064] Here, the preset pixel value refers to the first pixel value (255). The electronic device can identify the pixels in the binarized image whose pixel value is the preset pixel value as the target binarized pixels. It can be understood that the pixels in the image to be processed that correspond to these target binarized pixels are the pixels that differ significantly from the reference image. However, not all pixels that differ significantly from the reference image are caused by ghosting. In fact, the area outside the ghosting shadow area can be considered as the background area, and the electronic device is not interested in these areas during the ghosting detection and removal process. Based on this, for any ghosting shadow area, the electronic device can identify the pixels within the ghosting shadow area that correspond to the target binarized pixels as the target ghosting pixels.

[0065] In some embodiments, to achieve accurate filtering of the target shadow region, step 103 may include:

[0066] C1. Determine the trailing density of each trailing region based on the number of target trailing pixels within each trailing region.

[0067] In one application scenario, the areas of all resulting trailing shadow regions are equal. In this scenario, since the areas of all trailing shadow regions are equal, the trailing shadow density is only related to the number of target trailing shadow pixels within each region; specifically, the number of target trailing shadow pixels within a region is directly proportional to the trailing shadow density. To simplify the operation, the number of target trailing shadow pixels can be directly considered equivalent to the trailing shadow density; that is, the trailing shadow density of a region can be directly represented by the number of target trailing shadow pixels within that region.

[0068] In another application scenario, there are two or more trailing regions with different areas among all the resulting trailing regions. In this scenario, the trailing density is related not only to the number of target trailing pixels within the trailing region but also to the area of ​​the trailing region. The area of ​​the trailing region can be represented by the total number of pixels within the trailing region. Therefore, the formula for calculating the trailing density is as follows:

[0069]

[0070] Where p represents the ghost density of the ghost region; q represents the number of target ghost pixels within the ghost region; and area represents the area of ​​the ghost region, which is also the total number of pixels in the ghost region.

[0071] C2. Define the trailing shadow region with a trailing shadow density greater than the preset density threshold as the target trailing shadow region.

[0072] The electronic device can compare the ghost density of each ghost region with a preset density threshold. For any ghost region, if the ghost density of the ghost region is greater than the preset density threshold, it is considered that the proportion of target ghost pixels in the ghost region is large, and the ghost region can be identified as the target ghost region.

[0073] It should be noted that in the first application scenario proposed in step C1, the density threshold should be a positive integer greater than 1; in the second application scenario proposed in step C1, the density threshold should be a decimal greater than 0 and less than 1.

[0074] In some embodiments, analysis of video streams in the field of security monitoring reveals that, for moving objects in images, in addition to motion blur, there is often also blurring. That is, in low-light scenes, images captured by a camera of moving objects may simultaneously exhibit both motion blur and blurring. The blurring problem typically manifests as the shape of some areas of the moving object being distorted or even disappearing. Therefore, to further improve the quality of the video stream, in addition to motion blur removal processing, the electronic device can also perform blur removal processing. The motion blur removal method proposed in this application embodiment may further include:

[0075] D1. Determine the blurred regions in the image to be processed using a blur detection model.

[0076] After obtaining the image to be processed, the electronic device can input the image to a blur detection model. The output of the blur detection model is a bounding box for the blurred region in the input image to be processed. It can be understood that the area enclosed by this bounding box is the blurred region in the image to be processed.

[0077] For illustrative purposes only, the detection bounding box for the blurred region is rectangular. Of course, the blurred detection region can also be represented in other shapes; this is not a limitation here.

[0078] As an example, this blurring detection model can be constructed using a one-stage Convolutional Neural Network (CNN) containing convolutional layers, pooling layers, and fully connected layers. In practical applications in security monitoring, the number of layers in this CNN does not exceed 24. This is because 24 layers are sufficient to achieve good network performance; further increasing the network depth makes it difficult to achieve even better performance and significantly increases training and detection time. Of course, other neural networks can also be used to construct this blurring detection model; this is not a limitation here.

[0079] It is understood that step D1 can be executed in parallel with step 101, or it can be executed before or after step 101. This application embodiment does not limit the execution order of step D1.

[0080] D2. Based on the position of the blurred region in the image to be processed, the first target image is processed by a blur removal model to obtain the second target image.

[0081] Step D1 has identified the region in the image to be processed that has a blurring problem, i.e., the blurred region. The electronic device can use the position parameters of this blurred region and the first target image as input to the de-blurring model. The de-blurring model performs targeted de-blurring processing on the first target image, thereby obtaining the de-blurred image output by the de-blurring model, i.e., the second target image. This second target image can be output to the display screen of the electronic device for user viewing.

[0082] In some examples, the network structure used in the de-virtualization model can be a generative adversarial network (GAN) or other networks, which are not limited here.

[0083] It is understood that before putting the blur detection model and the blur removal model into application, they should be trained in advance using their respective training sets; that is, in this embodiment, the blur detection model and the blur removal model used are specifically pre-trained models.

[0084] In some embodiments, analysis of video streams in the field of security monitoring reveals that, for moving objects in the image, in addition to the ghosting problem, there is also a noise problem. That is, in low-light scenes, images captured by a camera of moving objects may simultaneously contain ghosting and noise. The noise problem typically occurs around and on the moving object in the image. Therefore, to further improve the quality of the video stream, in addition to ghosting removal processing, the electronic device can also perform noise reduction processing. The ghosting removal method proposed in this application embodiment may further include:

[0085] E1. Determine the noise region in the image to be processed using a noise detection model;

[0086] After obtaining the image to be processed, the electronic device can input the image to a noise detection model. The output of the noise detection model is a noise region detection box in the input image to be processed. It can be understood that the area enclosed by this noise region detection box is the noise region in the image to be processed.

[0087] For illustrative purposes only, the noise detection area is shown as a rectangle. Of course, the noise detection area can also be represented by other shapes, which are not limited here.

[0088] As an example, this noise detection model can be constructed using a one-stage Convolutional Neural Network (CNN) containing convolutional layers, pooling layers, and fully connected layers. In practical applications in security monitoring, the number of layers in this CNN does not exceed 24. This is because 24 layers is sufficient to achieve good network performance; further increasing the network depth makes it difficult to achieve even better performance and significantly increases training and detection time. Of course, other neural networks can also be used to construct this noise detection model; this is not a limitation here.

[0089] It is understood that step E1 can be executed in parallel with step 101, or it can be executed before or after step 101. This application embodiment does not limit the execution order of step E1.

[0090] E2. Based on the location of the noise region in the image to be processed, the first target image is denoised using a denoising model to obtain the third target image.

[0091] Step E1 has identified the noise-prone areas in the image to be processed. The electronic device can use the location parameters of these noise areas and the first target image as input to a denoising model. This model then performs targeted denoising on the first target image, resulting in a denoised image, or third target image, output by the denoising model. This third target image can be output to the electronic device's display screen for user viewing.

[0092] In some examples, similar to the de-virtualization model, the network structure used in the denoising model can be a generative adversarial network or other networks, which is not limited here.

[0093] It is understood that before putting the noise detection model and the noise removal model into application, they should be trained in advance using their respective training sets; that is, in this embodiment, the noise detection model and the noise removal model used are specifically pre-trained models.

[0094] In some embodiments, considering that motion blur, blurring, and noise often occur together in video streams used in security monitoring, to better improve the quality of the video stream, the motion blur removal process, the blur removal process, and the noise reduction process can be combined, and the motion blur removal process, blur removal process, and noise reduction process can be performed sequentially on the image to be processed. Considering that motion blur is the primary problem, and blurring and noise are secondary problems that accompany motion blur, the execution order can be set as follows: motion blur removal process first, and the order of blur removal process and noise reduction process is not limited. Therefore, the motion blur removal method proposed in this application embodiment may further include:

[0095] F1. Determine the blurred regions in the image to be processed using the blur detection model.

[0096] F2. Determine the noise region in the image to be processed using a noise detection model.

[0097] It is understood that steps F1 and F2 can be executed in parallel with step 101, or they can be executed before or after step 101. This application embodiment does not limit the execution order of steps F1 and F2.

[0098] F3. Based on the position of the blurred region in the image to be processed, the first target image is deblurred using a deblurring model to obtain a first intermediate image. Then, based on the position of the noise region in the image to be processed, the first intermediate image is denoised using a denoising model to obtain a fourth target image. This fourth target image can be output to the display screen of an electronic device for user viewing.

[0099] Alternatively, based on the location of the noise region in the image to be processed, the first target image is denoised using a denoising model to obtain a second intermediate image. Then, based on the location of the blurred region in the image to be processed, the second intermediate image is deblurred using a deblurring model to obtain a fifth target image. This fifth target image can be output to the display screen of an electronic device for user viewing.

[0100] For example only, please refer to Figure 3 , Figure 3 An example of the data flow is given when an image is processed in the order of de-ghosting, de-blurring, and de-noising.

[0101] It is understood that the network structures of the neural networks constituting the ghosting detection model, the blurring detection model, and the noise detection model can be the same or different, and no restrictions are imposed here.

[0102] In one application scenario, a corresponding detection model can be generated for each detection task; that is, the ghosting detection model, the blurring detection model, and the noise detection model are independent of each other. In this application scenario, even if the network structures of the neural networks constituting the ghosting detection model, the blurring detection model, and the noise detection model are the same, the specific model parameters of the ghosting detection model, the blurring detection model, and the noise detection model will still be different.

[0103] It is understandable that, for this application scenario, the training dataset should be divided into three categories to train the ghosting detection model, the blurring detection model, and the noise detection model respectively.

[0104] In another application scenario, a multi-task detection model can be generated, capable of performing motion blur detection, bokeh detection, and noise detection tasks; that is, the motion blur detection model, bokeh detection model, and noise detection model are actually integrated into the same detection model. In this application scenario, the specific model parameters of the detection model are unique, but it can simultaneously output the detection results of each task.

[0105] Understandably, for this application scenario, training datasets can be merged (by labeling the training data with the corresponding tasks), and multi-task detection models can be trained directly using the merged training datasets.

[0106] In some embodiments, the motion blur detection model can output not only a motion blur region detection box for determining the motion blur region, but also the motion blur level of the image to be processed. Similarly, when the electronic device introduces de-blurring processing, the bokeh detection model can output not only a bokeh region detection box for determining the bokeh region, but also the bokeh level of the image to be processed. Similarly, when the electronic device introduces denoising processing, the noise detection model can output not only a noise region detection box for determining the noise region, but also the noise level of the image to be processed. The motion blur level, bokeh level, and noise level are explained below:

[0107] Regarding the level of motion blur:

[0108] In this embodiment, the length of the moving object's shadow in the image is quantified using the width of the moving object as a scale. This shadow length represents the shadow level. Specifically, the length of the moving object's shadow in the image refers to the width of the shadow region. Let the width of the moving object in the image be W0, and assuming the width of the shadow region in the image is W1, then the shadow level of the image can be calculated using the following formula:

[0109]

[0110] Where C is used for the motion blur level; symbol Used to indicate rounding down.

[0111] Considering that in the field of security monitoring, a motion blur level of 5 is already quite severe, this application embodiment limits the upper limit of the motion blur level to 5. That is, when the width of the motion blur area in the image is more than five times the width of the moving object, the motion blur level of the image is uniformly set to 5.

[0112] Please see Figure 4 , Figure 4 A quantization example image of the motion blur level is given, where the small black dots represent the target motion blur pixels and the small black squares represent the background objects. Figure 4 In the image, the width of the trailing shadow area is more than three times the width of the moving object, therefore... Figure 4 The image shown has a motion blur level of 3.

[0113] It is understood that the motion blur level proposed in this application comprehensively considers the correlation between motion blur length and the width of moving objects, as well as the visual perception of motion blur at different distances. It is closer to human eye perception than the method of absolute data quantification and can more conveniently, intuitively and reliably represent the degree of motion blur in an image.

[0114] Regarding the level of blurring:

[0115] In this embodiment of the application, the blurring of moving objects in the image can be divided into 5 levels based on the classification method in Table 1 below:

[0116] 1 Almost no blurring 2 Around 20% blurring 3 Approximately 50% blurred 4 About 80% blurred 5 The target object is completely invisible

[0117] Table 1

[0118] Regarding noise levels:

[0119] In this embodiment of the application, noise related to moving objects in the image can be divided into 5 levels based on the classification method in Table 2 below:

[0120] 1 Some noise 2 There is some noise, which slightly affects the viewing experience. 3 The noise level is too high, affecting the viewing experience. 4 Mostly covered by noise 5 The moving object was completely covered by noise.

[0121] Table 2

[0122] It is understandable that before training the motion blur detection model, it is necessary to first collect a dataset of real low-light scenes and manually or otherwise label the motion blur regions and levels of each image in the dataset to construct the corresponding training set for the motion blur detection model. Similarly, the corresponding training sets for the blur detection model and the noise detection model can be constructed using the same process, only the labeling content differs, which will not be elaborated here. Using the training sets corresponding to each detection model, the models are trained so that the motion blur detection model can output the motion blur level of the image to be processed, the blur detection model can output the blur level of the image to be processed, and the noise detection model can output the noise level of the image to be processed.

[0123] When the electronic device is not a video stream acquisition device (e.g., a surveillance camera), it can feed back the motion blur level of the image to be processed from the motion blur detection model, the blur level of the image to be processed from the blur detection model, and / or the noise level of the image to be processed from the noise detection model to the acquisition device. The acquisition device can then adjust its parameters based on the received (or self-determined) motion blur level, blur level, and / or noise level, resulting in higher image quality for subsequently acquired video streams.

[0124] In some embodiments, to help obtain a detection model with better performance and effectiveness (which may be any of the following models: a motion blur detection model, a bokeh detection model, and a noise detection model), the electronic device may adjust and optimize the neural network constituting the detection model, including:

[0125] Replace the last fully connected layer of the neural network with a global pooling layer; that is, perform a global average on each channel of the last n feature maps individually, thereby obtaining an n-dimensional vector P.

[0126] The vector P is concatenated with a softmax function for classification, resulting in P. i The weights Wi between the specified category and the target category. The neural network performs a binary classification task. As an example, in the case of a motion blur detection model, the binary classification task is to determine whether each pixel belongs to the motion blur category or not; correspondingly, the specified category is the motion blur category.

[0127] The dimensions of the n feature maps are adjusted to match the dimensions of the original input image. Based on the resized n feature maps and vector P, the weight distribution of the original image can be obtained. Visualizing this weight distribution yields a heatmap representing the training results.

[0128] The heatmap allows users to determine the training performance of the detection model: if the heatmap shows a higher weight for areas outside the marked regions (i.e., background areas), it indicates that the detection model has incorrectly identified the background area of ​​the original image as the target area (i.e., a trailing area, a blurred area, or a noise area, the category of which is determined by the detection model). The training performance is poor, and the original image can be identified as a negative sample.

[0129] The detection model can be trained multiple times based on a set of identified negative samples to learn the characteristics of the negative samples.

[0130] It is understandable that, through the above process, electronic devices can filter out negative samples during the training of the detection model, thereby achieving data augmentation. Furthermore, electronic devices can repeatedly train the detection model to be trained based on the filtered negative samples, thereby improving the accuracy and recall of the final trained detection model.

[0131] In some embodiments, when the motion blur detection model also outputs a motion blur level, in order to accurately locate target motion blur pixels that are severely affected by motion blur, before determining the target motion blur pixels through steps B1-B3, the electronic device may consider setting a second pixel value difference threshold t2 based on the motion blur level of the image to be processed. Specifically, the setting strategy for the second pixel value difference threshold t2 may be: negatively correlated with the motion blur level.

[0132] In some examples, for a ghosting region with a ghosting level no higher than a preset level, the second pixel value difference threshold t2 can be set to a first default value; for a ghosting region with a ghosting level higher than a preset level, the second pixel value difference threshold t2 can be set to a second default value, and the second default value is less than the first default value, thereby allowing more target ghosting pixels to be exposed.

[0133] As can be seen from the above, in this embodiment, on the one hand, the entire ghosting region is not directly subjected to ghosting removal processing. Instead, target ghosting regions that are indeed necessary for ghosting removal are further selected, and ghosting removal processing is performed only on these target ghosting regions to ensure that the first target image obtained after ghosting removal does not exhibit blocky phenomena. On the other hand, considering the adaptability and consistency of human visual perception, the real scene is fully considered during ghosting removal processing. Specifically, the target ghosting regions are replaced with the corresponding regions of the reference image. In summary, this embodiment can reduce the damage to the original image background during ghosting removal while ensuring the ghosting removal effect, and avoid excessive differences between the ghosting-removed image and the original image.

[0134] Corresponding to the ghosting removal method provided above, this application also provides a ghosting removal device. For example... Figure 5 As shown, the ghosting removal device 5 includes:

[0135] The first determining module 501 is used to determine the motion shadow region in the image to be processed by the motion shadow detection model;

[0136] The second determining module 502 is used to determine the target trailing pixel points in each trailing shadow area according to the reference image, wherein the reference image and the image to be processed come from the same video stream, the trailing shadow area is obtained by dividing the trailing shadow area, and the number of trailing shadow areas is two or more.

[0137] The filtering module 503 is used to filter the target shadow regions based on the number of target shadow pixels in each shadow region.

[0138] The replacement module 504 is used to replace each target shadow region in the image to be processed with the corresponding region in the reference image to obtain the first target image.

[0139] In some embodiments, the second determining module 502 includes:

[0140] The frame difference image generation unit is used to generate a frame difference image based on the inter-frame differences between the reference image and the image to be processed.

[0141] The binarization unit is used to binarize the frame difference image to obtain a binary image;

[0142] The target trailing pixel determination unit is used to determine the pixels corresponding to the target binarized pixels in the binarized image within each trailing region as target trailing pixels. The target binarized pixels are pixels with a preset pixel value.

[0143] In some embodiments, the filtering module includes:

[0144] The trailing density determination unit is used to determine the trailing density of each trailing region based on the number of target trailing pixels in each trailing region.

[0145] The target trailing shadow region determination unit is used to determine the trailing shadow region with a trailing shadow density greater than a preset density threshold as the target trailing shadow region.

[0146] In some embodiments, the ghosting removal device 5 further includes:

[0147] The acquisition module is used to acquire multiple consecutive frames of images from a video stream;

[0148] The detection module is used to detect the inter-frame differences between adjacent images in a series of consecutive images;

[0149] The third determining module is used to determine any frame image in a series of consecutive frames as a reference image if the inter-frame differences of all adjacent images meet the preset difference conditions.

[0150] In some embodiments, the ghosting removal device 5 further includes:

[0151] The fourth determination module is used to determine the blurred regions in the image to be processed through the blur detection model;

[0152] The de-blurring module is used to perform de-blurring processing on the first target image based on the position of the blurred region in the image to be processed, and obtain the second target image.

[0153] In some embodiments, the ghosting removal device 5 further includes:

[0154] The fifth determination module is used to determine the noise region in the image to be processed through a noise detection model;

[0155] The noise reduction module is used to perform noise reduction processing on the first target image based on the location of the noise region in the image to be processed, and obtain the third target image.

[0156] In some embodiments, the motion blur detection model is further configured to output the motion blur level of the image to be processed, the motion blur level being used to describe the degree of motion blur in the image to be processed; the motion blur removal device 5 further includes:

[0157] The feedback module is used to feed back the motion blur level to the video stream acquisition device so that the acquisition device can adjust its parameters according to the motion blur level.

[0158] As can be seen from the above, in this embodiment, on the one hand, the entire ghosting region is not directly subjected to ghosting removal processing. Instead, target ghosting regions that are indeed necessary for ghosting removal are further selected, and ghosting removal processing is performed only on these target ghosting regions to ensure that the first target image obtained after ghosting removal does not exhibit blocky phenomena. On the other hand, considering the adaptability and consistency of human visual perception, the real scene is fully considered during ghosting removal processing. Specifically, the target ghosting regions are replaced with the corresponding regions of the reference image. In summary, this embodiment can reduce the damage to the original image background during ghosting removal while ensuring the ghosting removal effect, and avoid excessive differences between the ghosting-removed image and the original image.

[0159] Corresponding to the ghosting removal method provided above, this application also provides an electronic device. This is merely an example; the electronic device may be a personal computer, smartphone, tablet, surveillance camera, or server, etc., and is not limited thereto. Please refer to... Figure 6 The electronic device 6 in this embodiment includes: a memory 601, and one or more processors 602. Figure 6 (Only one is shown) and a computer program stored in memory 601 and executable on the processor. Memory 601 stores software programs and units. The processor 602 executes various functional applications and data processing by running the software programs and units stored in memory 601 to obtain resources corresponding to the aforementioned preset events. Specifically, the processor 602 performs the following steps when running the aforementioned computer program stored in memory 601:

[0160] The motion blur region in the image to be processed is determined by the motion blur detection model;

[0161] Based on the reference image, the target trailing pixels within each trailing region are determined. The reference image and the image to be processed come from the same video stream. The trailing regions are obtained by dividing the trailing regions, and the number of trailing regions is more than two.

[0162] The target trailing regions are obtained by filtering based on the number of target trailing pixels in each trailing region.

[0163] The shadow regions of each target in the image to be processed are replaced with the corresponding regions in the reference image to obtain the first target image.

[0164] Assuming the above is the first possible implementation, in the second possible implementation provided based on the first possible implementation, the target trailing pixel points within each trailing shadow region are determined according to the reference image, including:

[0165] A frame difference image is generated based on the inter-frame differences between the reference image and the image to be processed;

[0166] Binarize the frame difference image to obtain a binary image;

[0167] The pixels corresponding to the target binarized pixels in the binarized image within each shadow region are determined as target shadow pixels. The target binarized pixels are pixels with a preset pixel value.

[0168] In a third possible implementation based on the first possible implementation described above, the target trailing shadow regions are selected by filtering according to the number of target trailing shadow pixels within each trailing shadow region, including:

[0169] The trailing density of each trailing region is determined based on the number of target trailing pixels within each trailing region.

[0170] The region with a trailing density greater than a preset density threshold is defined as the target trailing region.

[0171] In the fourth possible implementation provided based on the first possible implementation described above, the reference image is determined through the following steps:

[0172] In a video stream, capture multiple consecutive frames of images;

[0173] In a series of consecutive images, detect the inter-frame differences between adjacent images;

[0174] If the inter-frame differences of all adjacent images meet the preset difference conditions, then any frame in a series of consecutive frames will be determined as the reference image.

[0175] In a fifth possible implementation provided based on the first, second, third, or fourth possible implementation described above, the processor 602 further performs the following steps when running the computer program stored in the memory 601:

[0176] The blurred regions in the image to be processed are determined using a blur detection model;

[0177] Based on the location of the blurred region in the image to be processed, the first target image is processed by a blur removal model to obtain the second target image.

[0178] In a sixth possible implementation provided based on the first, second, third, or fourth possible implementation described above, the processor 602 further performs the following steps when running the computer program stored in the memory 601:

[0179] The noise region in the image to be processed is determined by a noise detection model;

[0180] Based on the location of the noise region in the image to be processed, the first target image is denoised using a denoising model to obtain the third target image.

[0181] In a seventh possible implementation provided based on the first, second, third, or fourth possible implementations described above, the motion blur detection model is further used to output the motion blur level of the image to be processed, the motion blur level being used to describe the degree of motion blur in the image to be processed; the processor 602 further performs the following steps when running the computer program stored in the memory 601:

[0182] The motion blur level is fed back to the video stream acquisition device so that the acquisition device can adjust its parameters according to the motion blur level.

[0183] It should be understood that, in the embodiments of this application, the processor 602 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0184] Memory 601 may include read-only memory and random access memory, and provides instructions and data to processor 602. Some or all of memory 601 may also include non-volatile random access memory. For example, memory 601 may also store device category information.

[0185] As can be seen from the above, in this embodiment, on the one hand, the entire ghosting region is not directly subjected to ghosting removal processing. Instead, target ghosting regions that are indeed necessary for ghosting removal are further selected, and ghosting removal processing is performed only on these target ghosting regions to ensure that the first target image obtained after ghosting removal does not exhibit blocky phenomena. On the other hand, considering the adaptability and consistency of human visual perception, the real scene is fully considered during ghosting removal processing. Specifically, the target ghosting regions are replaced with the corresponding regions of the reference image. In summary, this embodiment can reduce the damage to the original image background during ghosting removal while ensuring the ghosting removal effect, and avoid excessive differences between the ghosting-removed image and the original image.

[0186] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0187] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0188] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of external device software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0189] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0190] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing associated hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer-readable storage device, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the contents of the aforementioned computer-readable storage media may be appropriately added to or subtracted from the contents according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media may not include electrical carrier signals and telecommunication signals.

[0192] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for removing ghosting, characterized in that, include: The motion blur region in the image to be processed is determined by a motion blur detection model. The motion blur detection model is constructed by a one-stage convolutional neural network containing convolutional layers, pooling layers and fully connected layers. The number of layers in the convolutional neural network does not exceed 24. The image to be processed is a non-first frame image in the received video stream that was captured during a preset nighttime period or whose image brightness is lower than a preset brightness threshold. Based on a reference image, target trailing pixels within each trailing region are determined. The reference image and the image to be processed originate from the same video stream, and the reference image is determined through the following steps: acquiring multiple consecutive frames preceding the image to be processed within the video stream, wherein the interval between the first frame of the multiple consecutive frames and the image to be processed is within a preset interval threshold; detecting inter-frame differences between adjacent frames within the multiple consecutive frames; if all inter-frame differences between adjacent frames satisfy a preset difference condition, then any frame from the multiple consecutive frames is determined as the reference image; the trailing regions are obtained by dividing the trailing regions, and the number of trailing regions is two or more. The target trailing regions are obtained by filtering based on the number of target trailing pixels in each of the aforementioned trailing regions; The target shadow regions in the image to be processed are replaced with the corresponding regions in the reference image to obtain the first target image; The step of determining the target trailing pixel points within each trailing shadow region based on the reference image includes: A frame difference image is generated based on the inter-frame differences between the reference image and the image to be processed; The frame difference image is binarized to obtain a binarized image; The pixels within each shadow region that correspond to the target binarized pixels in the binarized image are determined as target shadow pixels. The target binarized pixels are pixels with a preset pixel value.

2. The method for removing ghosting as described in claim 1, characterized in that, The step of filtering target ghosting regions based on the number of target ghosting pixels in each of the ghosting regions includes: The trailing density of each of the trailing shadow regions is determined based on the number of target trailing pixels within each of the trailing shadow regions. The region of the trailing shadow with a density greater than a preset density threshold is defined as the target trailing shadow region.

3. The method for removing ghosting as described in any one of claims 1 to 2, characterized in that, The method for removing ghosting also includes: The blurred regions in the image to be processed are determined using a blur detection model; Based on the position of the blurred region in the image to be processed, the first target image is processed by a blur removal model to obtain the second target image.

4. The method for removing ghosting as described in any one of claims 1 to 2, characterized in that, The method for removing ghosting also includes: The noise region in the image to be processed is determined by a noise detection model; Based on the location of the noise region in the image to be processed, the first target image is denoised using a denoising model to obtain the third target image.

5. The method for removing ghosting as described in any one of claims 1 to 2, characterized in that, The motion blur detection model is also used to output the motion blur level of the image to be processed, and the motion blur level is used to describe the degree of motion blur of the image to be processed; The method for removing ghosting also includes: The motion blur level is fed back to the video stream acquisition device so that the acquisition device adjusts its parameters according to the motion blur level.

6. A device for removing motion blur, characterized in that, include: The first determining module is used to determine the motion blur region in the image to be processed by a motion blur detection model. The motion blur detection model is constructed by a one-stage convolutional neural network that includes convolutional layers, pooling layers and fully connected layers, and the number of layers in the convolutional neural network does not exceed 24. The image to be processed is a non-first frame image in the received video stream that was captured during a preset nighttime period or whose image brightness is lower than a preset brightness threshold. The second determining module is used to determine the target trailing pixel points in each trailing shadow region based on the reference image, wherein the reference image and the image to be processed come from the same video stream, the trailing shadow region is obtained by dividing the trailing shadow region, and the number of the trailing shadow region is two or more. The filtering module is used to filter and obtain the target trailing shadow region based on the number of target trailing shadow pixels in each of the trailing shadow regions. The replacement module is used to replace each of the target shadow regions in the image to be processed with the corresponding regions of the reference image to obtain the first target image; The second determining module includes: A frame difference image generation unit is used to generate a frame difference image based on the inter-frame difference between the reference image and the image to be processed; A binarization unit is used to binarize the frame difference image to obtain a binarized image; The target trailing pixel determination unit is used to determine the pixels in each trailing region that correspond to the target binarized pixels in the binarized image as target trailing pixels. The target binarized pixels are pixels with a preset pixel value. The ghosting removal device further includes: The acquisition module is used to acquire, in the video stream, a series of consecutive frames of images preceding the image to be processed, wherein the interval between the first frame of the series of consecutive frames and the image to be processed is within a preset interval threshold. The detection module is used to detect inter-frame differences between adjacent images in the continuous multi-frame images; The third determining module is used to determine any frame image in the consecutive multi-frame images as the reference image if the inter-frame differences of all the adjacent images meet the preset difference conditions.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images