A video processing method, apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure disclose a video processing method and device, electronic equipment and storage medium, the method comprising: determining a blurred video frame in an original video; deleting the blurred video frame from the original video to obtain an intermediate video not including the blurred video frame; determining a to-be-inserted video frame based on a video frame adjacent to a target timestamp in the intermediate video, the target timestamp being a timestamp of the blurred video frame; inserting the to-be-inserted video frame at a position corresponding to the target timestamp in the intermediate video to obtain a target video. The video processing method provided by the present disclosure achieves the purposes of removing blurred frames in a video and improving video quality.

Description

Technical Field

[0001] This disclosure relates to the field of information technology, and in particular to a video processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, with the rapid development of electronic device technology, many electronic devices support video recording, and video recording quality has become one of the important indicators for evaluating electronic devices. Many factors affect video recording quality, such as resolution, saturation, and sharpness, among which sharpness is a very important factor.

[0003] When shooting video using electronic devices, if the subject moves or the electronic device shakes during the exposure time, motion blur will appear in the video, which will manifest as blurry, diffused, or trailing images.

[0004] Therefore, in order to obtain a better quality video, it is necessary to process the blurry images in the video. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a video processing method, apparatus, electronic device, and storage medium, which achieve the purpose of removing blurry frames from videos and improving video quality.

[0006] In a first aspect, embodiments of this disclosure provide a video processing method, the method comprising:

[0007] Identify blurry video frames in the original video;

[0008] The blurred video frames are removed from the original video to obtain an intermediate video that does not include the blurred video frames;

[0009] The video frame to be inserted is determined based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, where the target timestamp is the timestamp of the blurred video frame.

[0010] The video frame to be inserted is inserted into the intermediate video at the position corresponding to the target timestamp to obtain the target video.

[0011] Secondly, embodiments of this disclosure also provide a video processing apparatus, the apparatus comprising:

[0012] The first determining module is used to determine the blurred video frames in the original video;

[0013] The deletion module is used to delete the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame;

[0014] The second determining module is used to determine the video frame to be inserted based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, wherein the target timestamp is the timestamp of the blurred video frame.

[0015] The frame interpolation module is used to insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp, so as to obtain the target video.

[0016] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0017] One or more processors;

[0018] Storage device for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the video processing method as described above.

[0020] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the video processing method described above.

[0021] The technical solution provided in this disclosure has at least the following advantages compared with the prior art:

[0022] The video processing method provided in this disclosure first identifies blurry video frames in the original video, then deletes the blurry video frames from the original video, and inserts clear video frames at the positions of the deleted video frames by frame interpolation, thereby improving video quality. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 This is a flowchart of a video processing method according to an embodiment of the present disclosure;

[0025] Figure 2 This is a flowchart of a video processing method according to an embodiment of the present disclosure;

[0026] Figure 3 This is a schematic diagram of the edge position of an image in an embodiment of this disclosure;

[0027] Figure 4This is a schematic diagram illustrating the change in the sum of pixel values ​​of all pixels in the edge image corresponding to each video frame in an original video according to an embodiment of this disclosure.

[0028] Figure 5 This is a flowchart of a video processing method according to an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of a video processing device according to an embodiment of the present disclosure;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0033] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0034] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0035] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0036] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0037] Figure 1 This is a flowchart illustrating a video processing method according to an embodiment of the present disclosure. The method can be executed by a video processing device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a display terminal, specifically including but not limited to smartphones, PDAs, tablets, portable wearable devices, smart home devices (e.g., table lamps), and other electronic devices with a display screen.

[0038] like Figure 1 As shown, the method may specifically include the following steps:

[0039] Step 110: Identify the blurred video frames in the original video.

[0040] Specifically, the edge image of each video frame in the original video can be determined by a preset algorithm, and then the video frames with blurred images and motion blur can be determined based on the edge images. Such video frames are called blurred video frames. In this embodiment of the disclosure, the blurred video frames are marked as the first video frames.

[0041] Step 120: Delete the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame.

[0042] Step 130: Determine the video frame to be inserted based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, wherein the target timestamp is the timestamp of the blurred video frame.

[0043] Assuming the timestamp of the blurred frame is 2s, meaning the first video frame is located at position 2s in the original video playback, the intermediate video obtained after deleting the first video frame from the original video will be missing a video frame at position 2s on the playback timeline. In order to obtain a better video effect and ensure the continuity of the video, a video frame (called the video frame to be inserted) is inserted at position 2s to ensure the continuity of the video frame.

[0044] Optionally, the video frame to be inserted is determined based on video frames at positions 1s and 3s in the intermediate video. Specifically, it can be determined by a preset frame interpolation algorithm, which predicts the video frame located between two adjacent video frames; for example, it can be determined based on motion estimation methods or through a neural network. This disclosure does not limit the method of determining the video frame to be inserted based on video frames in the intermediate video whose timestamps are adjacent to the target timestamp.

[0045] Step 140: Insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp to obtain the target video.

[0046] For example, the first video frame has one unit and a corresponding timestamp of 2 seconds. This means the first video frame is located at the 2-second mark of the original video playback. After deleting the first video frame from the original video, the 2-second mark on the original video's playback timeline will be missing a video frame. To achieve better video quality and ensure video continuity, a video frame is inserted at the 2-second mark to maintain the continuity of the video frame. In this embodiment, the inserted video frame is marked as the second video frame.

[0047] In some implementations, the number of second video frames inserted is the same as the number of first video frames deleted. For example, if two first video frames are deleted from the original video, then two second video frames are inserted into the original video after the deletion.

[0048] In some implementations, the number of inserted second video frames is greater than the number of deleted first video frames. For example, two first video frames are deleted from the original video, and then three second video frames are inserted into the original video. Inserting more second video frames can increase the frame rate of the original video, thereby further improving the playback quality of the original video.

[0049] The video processing method provided in this disclosure first identifies video frames with poor image quality in the original video, then deletes the video frames with poor image quality from the original video, and inserts video frames with better image quality at the positions of the deleted video frames by frame interpolation. This can remove blurry frames in the video and insert clear video frames to replace the removed blurry frames, thereby achieving the purpose of improving video quality.

[0050] Based on the above embodiments, Figure 2 This is a flowchart illustrating a video processing method. Based on the above embodiments, this embodiment provides a specific implementation method for step 110, "determining blurred video frames in the original video".

[0051] like Figure 2 As shown, the video processing method includes the following steps:

[0052] Step 210: Determine the first edge image at a preset position of the current video frame based on the Sobel operator, wherein the current video frame is any video frame in the original video.

[0053] Step 220: Determine the blurred video frame based on the edge image of each video frame at a preset position in the original video.

[0054] Here, the preset position refers to the edge position of the image, such as... Figure 3The diagram illustrates the edge positions of an image, where the white lines delineate the image's edge positions. The image consisting only of the white lines is the edge image, i.e., the image composed of edge positions. The Sobel operator is primarily used for image edge detection. It detects edges based on the weighted difference of the gray levels of a pixel's upper, lower, left, and right neighbors, identifying features that reach extreme values ​​at the edges. Applying this operator to any point in the image will generate a corresponding gray-level vector or its normal vector. In other words, for each video frame in the original video, its edge image is calculated separately. Based on the edge images of each video frame, video frames whose image quality does not meet preset conditions are identified, i.e., blurred video frames are determined based on the edge images. These blurred video frames are the first video frame. The number of blurred video frames in the original image may be multiple (multiple means two or more) or one.

[0055] Furthermore, in some embodiments, determining the blurred video frame based on the edge image at a preset position of each video frame in the original video includes:

[0056] Determine a first cumulative sum of pixel values ​​for each pixel in the first edge image; determine a second cumulative sum of pixel values ​​for each pixel in the second edge image at a location of a neighboring video frame, wherein the neighboring video frame is a video frame in the original video adjacent to the current video frame; determine the absolute value of a first difference between the first cumulative sum and the second cumulative sum; and determine the blurred video frame based on the absolute value of the first difference.

[0057] Optionally, in some embodiments, if the absolute value of the first difference is greater than a first preset threshold, then the current video frame is determined to be the blurred video frame. For example, suppose the sum of the pixel values ​​of all pixels in the edge image of the i-th (i≥1) video frame in the original video is edge_sum(i), and the sum of the pixel values ​​of all pixels in the edge image of the (i-1)-th video frame is edge_sum(i-1), and cond1=abs(edge_sum(i)- edge_sum(i-1)), where abs() represents a function that takes the absolute value; if cond1 is greater than the first preset threshold, then the i-th video frame in the original video is considered to be the first video frame.

[0058] More specifically, suppose the edge image of the i-th video frame consists of 4 pixels with pixel values ​​of 5, 15, 2, and 0 respectively. The sum of these pixel values ​​is edge_sum(i) = 5 + 15 + 2 + 0 = 22. The edge image of the (i-1)-th video frame consists of 4 pixels with pixel values ​​of 4, 10, 1, and 3 respectively. The sum of these pixel values ​​is edge_sum(i-1) = 4 + 10 + 1 + 3 = 18. Therefore, cond1(i) = abs(edge_sum(i) - edge_sum(i-1)) = abs(22 - 18) = 4. (See reference...) Figure 4 The diagram shows the cumulative change of pixel values ​​of all pixels in the edge image corresponding to each video frame in the original video. The location of the abrupt change is the possible time point where blurring occurs.

[0059] In some embodiments, to further improve the accuracy of determining blurred frames, determining the blurred video frame based on the absolute value of the first difference includes: determining the maximum value of the absolute value of the difference between the total number of pixels in the first edge image and the second edge image at each pixel value based on the histogram of the first edge image and the histogram of the second edge image.

[0060] Specifically, based on the histogram of the first edge image, the total number of first pixels in the first edge image whose pixel value is a target value is determined, where the target value is any pixel value in the first edge image; based on the histogram of the second edge image, the total number of second pixels in the second edge image whose pixel value is the target value is determined; the absolute value of a second difference between the total number of first pixels and the total number of second pixels is determined; the maximum value among the absolute values ​​of the second difference corresponding to each pixel value in the first edge image is determined; and the blurred video frame is determined based on the absolute value of the first difference and the maximum value. For example, assuming that the pixel values ​​of all pixels in the first edge image range from 0 to 15, totaling 16 values, the target value is any one of these 16 values; the pixel values ​​of all pixels in the second edge image range from 0 to 15; then the maximum value among the absolute values ​​of the second difference corresponding to each pixel value in the first edge image can be expressed as max{abs(hist(i)(k)-hist(i-1)(k))}, where abs() represents the function of taking the absolute value, hist(i)(k) represents the total number of first pixels with pixel value k in the i-th edge image (which can be understood as the first edge image), and hist(i-1)(k) represents the total number of second pixels with pixel value k in the (i-1)-th edge image (which can be understood as the second edge image), that is, the target value is k, 0≤k<16. More specifically, suppose that in the edge image of the i-th video frame, the total number of pixels with a value of 0 is 5, the total number of pixels with a value of 1 is 15, the total number of pixels with a value of 2 is 2, and the total number of pixels with a value of 3 is 0; in the edge image of the (i-1)-th video frame, the total number of pixels with a value of 0 is 4, the total number of pixels with a value of 1 is 10, the total number of pixels with a value of 2 is 1, and the total number of pixels with a value of 3 is 3; then cond2(i) = max{abs(hist(i)(k)-hist(i-1)(k))} = max{abs(5-4), abs(15-10), abs(2-1), abs(0-3)} = 5.

[0061] In some embodiments, determining the blurred video frame based on the absolute value of the first difference and the maximum value includes: performing a weighted summation of the absolute value of the first difference and the maximum value to obtain the blur degree; if the blur degree is greater than a second preset threshold, then determining the current video frame as the blurred video frame. For example, the blur degree cond(i) = c1 * cond1(i) + c2 * cond2(i), where c1 and c2 are preset constants. When cond(i) > thr1, the i-th video frame is determined to be a blurred frame, i.e., the first video frame whose image quality does not meet the preset condition, where thr1 is the second preset threshold.

[0062] In some embodiments, to reduce computational load and improve computational speed, before determining the first cumulative sum of pixel values ​​of each pixel in the first edge image, the step of determining the blurred video frame based on the edge images at preset positions of each video frame in the original video further includes: normalizing the edge images at preset positions of each video frame in the original video to map the pixel values ​​of the pixels in the edge images to a preset range; wherein, the edge image is a single-channel image. Initially, the pixel value of each pixel in the edge image ranges from 0 to 255. When determining the blurred frame, it is necessary to calculate the total number of pixels corresponding to each pixel value and the cumulative sum of pixel values ​​of each pixel, thus the computational load is large. In order to reduce the computational load and improve computational efficiency, before determining the first cumulative sum of pixel values ​​of each pixel, the edge images are normalized to map the pixel values ​​of the pixels in the edge images to a preset range, for example, from the range of 0-255 to 0-16.

[0063] In some embodiments, to ensure video continuity, deletion of a large number of first video frames is not permitted. Therefore, if the number of blurred video frames exceeds a third preset threshold, the method further includes: filtering multiple first video frames and retaining only a limited number of them. For example, if the third preset threshold is 8, then a maximum of 8 first video frames can be retained. If the number of first video frames determined by the method described in the above embodiments is 10, then 2 need to be filtered out, and 8 need to be retained. That is, a maximum of 8 first video frames can be deleted from the original video. Specifically, the maximum of the absolute values ​​of the first differences or the maximum of the blur levels is determined as the extreme point; the blurred video frames are filtered based on the timestamp of the video frame corresponding to the extreme point to obtain blurred video frames with a frame count equal to the third preset threshold. Further, filtering the blurred video frames based on the timestamp of the video frame corresponding to the extreme point to obtain blurred video frames with a frame count equal to the third preset threshold includes:

[0064] Centered on the timestamp of the video frame corresponding to the extreme point, a set number of video frames are taken forward and backward respectively as the retained blurred video frames after filtering; the set number of frames is determined according to the third preset threshold.

[0065] For example, by calculating and determining that the blur level of the first video frame in the original video is 0, the blur level of the second video frame in the original video is 0, the blur level of the third video frame in the original video is 6, the blur level of the fourth video frame in the original video is 7, the blur level of the fifth video frame in the original video is 6, and the blur level of the sixth video frame in the original video is 6, and assuming that the second preset threshold is 5, then the third, fourth, fifth, and sixth video frames are determined to be the aforementioned first video frames. The number of first video frames is 4, that is, the number of blurred frames is 4. Assuming that the third preset threshold is 3, then 1 needs to be filtered out from the above 4 first video frames, and 3 need to be retained. Specifically, since the maximum blur level is 7, 7 is determined as the extreme point. The video frame corresponding to this extreme point is the fourth video frame, and the timestamp of the video frame corresponding to this extreme point is the timestamp of the fourth video frame in the original video. Taking this timestamp as the center, a set number of video frames are taken forward and backward respectively as the blurred video frames to be retained after filtering. For example, the video frame whose timestamp is closest to the timestamp of the fourth video frame is taken forward, which is the third video frame, and the video frame whose timestamp is closest to the timestamp of the fourth video frame is taken backward, which is the fifth video frame. Therefore, the third, fourth, and fifth video frames are determined as the final first video frame, and the first, second, and sixth video frames are filtered out.

[0066] Step 230: Delete the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame.

[0067] Step 240: Determine the video frame to be inserted based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, where the target timestamp is the timestamp of the blurred video frame.

[0068] Step 250: Insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp to obtain the target video.

[0069] General overview, for reference, etc. Figure 5 The flowchart of a video processing method shown includes: performing blur frame detection on the original video, obtaining a blur frame sequence and deleting the blur frame sequence from the original video, and then inserting the deleted video frames back into the original video to obtain the target video after blur repair, thereby achieving the purpose of improving video quality.

[0070] Figure 6 This is a schematic diagram of the structure of a video processing apparatus according to an embodiment of this disclosure. Figure 6As shown, the video processing device specifically includes: a first determining module 610, a deletion module 620, a second determining module 630, and a frame interpolation module 640.

[0071] The system comprises: a first determining module 610 for determining blurred video frames in the original video; a deletion module 620 for deleting the blurred video frames from the original video to obtain an intermediate video excluding the blurred video frames; a second determining module 630 for determining a video frame to be inserted based on video frames in the intermediate video whose timestamps are adjacent to a target timestamp, wherein the target timestamp is the timestamp of the blurred video frame; and a frame insertion module 640 for inserting the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp to obtain a target video.

[0072] Optionally, the first determining module 610 specifically includes: a first determining unit, used to determine a first edge image at a preset position of the current video frame based on the Sobel operator, wherein the current video frame is any video frame in the original video; and a second determining unit, used to determine the blurred video frame based on the edge images at preset positions of each video frame in the original video.

[0073] Optionally, the second determining unit specifically includes: a first determining subunit, configured to determine a first accumulated sum of pixel values ​​of each pixel in the first edge image; determine a second accumulated sum of pixel values ​​of each pixel in the second edge image at the location of a neighboring video frame, wherein the neighboring video frame is a video frame in the original video adjacent to the current video frame; determine the absolute value of a first difference between the first accumulated sum and the second accumulated sum; and a second determining subunit, configured to determine the blurred video frame based on the absolute value of the first difference.

[0074] Optionally, the second determining subunit is specifically used to: if the absolute value of the first difference is greater than the first preset threshold, then determine the current video frame as the blurred video frame.

[0075] Optionally, the second determining subunit is specifically configured to: determine the total number of first pixels in the first edge image whose pixel value is a target value based on the histogram of the first edge image, wherein the target value is any one of the pixel values ​​in the first edge image; determine the total number of second pixels in the second edge image whose pixel value is the target value based on the histogram of the second edge image; determine the absolute value of a second difference between the total number of first pixels and the total number of second pixels; determine the maximum value among the absolute values ​​of the second difference corresponding to each pixel value in the first edge image; and determine the blurred video frame based on the absolute value of the first difference and the maximum value.

[0076] Optionally, the second determining subunit is specifically used to: perform a weighted summation of the absolute value of the first difference and the maximum value to obtain the degree of ambiguity; if the degree of ambiguity is greater than a second preset threshold, then determine the current video frame as the ambiguous video frame.

[0077] Optionally, it further includes: a normalization module, used to perform normalization processing on the edge images at preset positions of each video frame in the original video before determining the first sum of pixel values ​​of each pixel in the first edge image, so as to map the pixel values ​​of the pixels in the edge image to a preset range; wherein, the edge image is a single-channel image.

[0078] Optionally, it also includes: a filtering module, used to determine the maximum of the absolute values ​​of the first differences or the maximum of the blur levels as an extreme point when the number of the first video frames exceeds a third preset threshold; and to filter the blurred video frames based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames with a frame count equal to the third preset threshold.

[0079] Optionally, the filtering module is specifically used to: take the timestamp of the video frame corresponding to the extreme point as the center, and take a set number of video frames forward and backward respectively as the blurred video frames to be retained after filtering; the set number of frames is determined according to the third preset threshold.

[0080] The video processing apparatus provided in this embodiment first identifies video frames with poor image quality in the original video, then deletes the video frames with poor image quality from the original video, and inserts video frames with better image quality at the positions of the deleted video frames by frame interpolation. This can remove blurry frames in the video and insert clear video frames to replace the removed blurry frames, thereby achieving the purpose of improving video quality.

[0081] The video processing apparatus provided in this disclosure can execute the steps in the video processing method provided in this disclosure, and has the execution steps and beneficial effects, which will not be described in detail here.

[0082] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device 700 in the embodiments of this disclosure. The electronic device 700 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 7The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0083] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing device 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0084] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0085] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the methods as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0086] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0087] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0088] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0089] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0090] Identify blurry video frames in the original video; delete the blurry video frames from the original video to obtain an intermediate video that does not include the blurry video frames; determine the video frame to be inserted based on the video frames in the intermediate video whose timestamps are adjacent to a target timestamp, where the target timestamp is the timestamp of the blurry video frame; insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp to obtain the target video.

[0091] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0092] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0095] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0096] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] According to one or more embodiments of this disclosure, a video processing method is provided, comprising: determining a blurred video frame in an original video; deleting the blurred video frame from the original video to obtain an intermediate video excluding the blurred video frame; determining a video frame to be inserted based on video frames in the intermediate video whose timestamps are adjacent to a target timestamp, wherein the target timestamp is the timestamp of the blurred video frame; and inserting the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0098] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, determining the blurred video frame in the original video includes: determining a first edge image at a preset position of the current video frame based on the Sobel operator, wherein the current video frame is any video frame in the original video; and determining the blurred video frame based on the edge images at preset positions of each video frame in the original video.

[0099] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, determining the blurred video frame based on edge images of preset positions of each video frame in the original video includes: determining a first sum of pixel values ​​of each pixel in the first edge image; determining a second sum of pixel values ​​of each pixel in a second edge image of a neighboring video frame at a preset position, wherein the neighboring video frame is a video frame in the original video adjacent to the current video frame; determining the absolute value of a first difference between the first sum and the second sum; and determining the blurred video frame based on the absolute value of the first difference.

[0100] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, determining the blurred video frame based on the absolute value of the first difference includes: if the absolute value of the first difference is greater than a first preset threshold, then determining the current video frame as the blurred video frame.

[0101] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, determining the blurred video frame based on the absolute value of the first difference includes:

[0102] Based on the histogram of the first edge image, determine the total number of first pixels in the first edge image whose pixel value is a target value, where the target value is any pixel value in the first edge image; based on the histogram of the second edge image, determine the total number of second pixels in the second edge image whose pixel value is the target value; determine the absolute value of a second difference between the total number of first pixels and the total number of second pixels; determine the maximum value among the absolute values ​​of the second difference corresponding to each pixel value in the first edge image; determine the blurred video frame based on the absolute value of the first difference and the maximum value.

[0103] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, determining the blurred video frame based on the absolute value of the first difference and the maximum value includes: performing a weighted summation on the absolute value of the first difference and the maximum value to obtain the blur degree; if the blur degree is greater than a second preset threshold, then determining the current video frame as the blurred video frame.

[0104] According to one or more embodiments of this disclosure, in the video processing method provided by this disclosure, optionally, before determining the first sum of pixel values ​​of each pixel in the first edge image, the step of determining the blurred video frame based on the edge images of each video frame at a preset position in the original video further includes: performing normalization processing on the edge images of each video frame at a preset position in the original video respectively, so as to map the pixel values ​​of the pixels in the edge images to a preset range.

[0105] According to one or more embodiments of this disclosure, in the video processing method provided in this disclosure, optionally, if the number of blurred video frames exceeds a third preset threshold, the method further includes: determining the maximum of the absolute values ​​of the first differences or the maximum of the blur levels as an extreme point; filtering the blurred video frames based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames with a frame count equal to the third preset threshold.

[0106] According to one or more embodiments of this disclosure, in the video processing method provided by this disclosure, optionally, the step of filtering the blurred video frames based on the timestamp of the video frame corresponding to the extreme point to obtain blurred video frames with a frame number equal to the third preset threshold includes: taking a set number of video frames forward and backward as the center, respectively, as the blurred video frames retained after filtering, with the timestamp of the video frame corresponding to the extreme point as the center; the set number of frames is determined according to the third preset threshold.

[0107] According to one or more embodiments of this disclosure, a video processing apparatus is provided, comprising: a first determining module, configured to determine a blurred video frame in an original video; a deletion module, configured to delete the blurred video frame from the original video to obtain an intermediate video excluding the blurred video frame; a second determining module, configured to determine a video frame to be inserted based on video frames in the intermediate video whose timestamps are adjacent to a target timestamp, wherein the target timestamp is the timestamp of the blurred video frame; and a frame insertion module, configured to insert the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0108] According to one or more embodiments of this disclosure, in the video processing apparatus provided in this disclosure, optionally, the first determining module specifically includes: a first determining unit, configured to determine a first edge image at a preset position of the current video frame based on the Sobel operator, wherein the current video frame is any video frame in the original video; and a second determining unit, configured to determine the blurred video frame based on the edge images at preset positions of each video frame in the original video.

[0109] According to one or more embodiments of this disclosure, in the video processing apparatus provided in this disclosure, optionally, the second determining unit specifically includes: a first determining subunit, configured to determine a first accumulated sum of pixel values ​​of each pixel in the first edge image; determine a second accumulated sum of pixel values ​​of each pixel in a second edge image at a neighboring video frame location, wherein the neighboring video frame is a video frame in the original video adjacent to the current video frame; determine the absolute value of a first difference between the first accumulated sum and the second accumulated sum; and a second determining subunit, configured to determine the blurred video frame based on the absolute value of the first difference.

[0110] According to one or more embodiments of this disclosure, in the video processing apparatus provided in this disclosure, optionally, the second determining subunit is specifically used to: if the absolute value of the first difference is greater than a first preset threshold, then determine the current video frame as the blurred video frame.

[0111] According to one or more embodiments of this disclosure, in the video processing apparatus provided by this disclosure, optionally, the second determining subunit is specifically configured to: determine the total number of first pixels in the first edge image whose pixel value is a target value based on the histogram of the first edge image, wherein the target value is any one of the pixel values ​​in the first edge image; determine the total number of second pixels in the second edge image whose pixel value is the target value based on the histogram of the second edge image; determine the absolute value of a second difference between the total number of first pixels and the total number of second pixels; determine the maximum value among the absolute values ​​of the second differences corresponding to each pixel value in the first edge image; and determine the blurred video frame based on the absolute value of the first difference and the maximum value.

[0112] According to one or more embodiments of this disclosure, in the video processing apparatus provided in this disclosure, optionally, the second determining subunit is specifically used to: perform a weighted summation of the absolute value of the first difference and the maximum value to obtain the blur degree; if the blur degree is greater than a second preset threshold, then determine the current video frame as the blurry video frame.

[0113] According to one or more embodiments of this disclosure, the video processing apparatus provided in this disclosure may optionally further include: a normalization module, used to perform normalization processing on the edge images at preset positions of each video frame in the original video before determining the first sum of pixel values ​​of each pixel in the first edge image, so as to map the pixel values ​​of the pixel points in the edge image to a preset range; wherein the edge image is a single-channel image.

[0114] According to one or more embodiments of this disclosure, the video processing apparatus provided in this disclosure may optionally further include: a filtering module, configured to determine the maximum of the absolute values ​​of the first differences or the maximum of the blur levels as an extreme point when the number of the first video frames exceeds a third preset threshold; and to filter the blurred video frames based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames with a frame count equal to the third preset threshold.

[0115] According to one or more embodiments of this disclosure, in the video processing apparatus provided in this disclosure, optionally, the filtering module is specifically used to: take a set number of video frames forward and backward as the center, respectively, based on the timestamp of the video frame corresponding to the extreme point, as the blurred video frames retained after filtering; the set number of frames is determined according to the third preset threshold.

[0116] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0117] One or more processors;

[0118] Memory, used to store one or more programs;

[0119] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the video processing methods provided in this disclosure.

[0120] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a video processing method as described in any of the present disclosure.

[0121] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the video processing method described above.

[0122] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0123] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0124] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video processing method, characterized in that, The method includes: Identify blurry video frames in the original video; The blurred video frames are removed from the original video to obtain an intermediate video that does not include the blurred video frames; The video frame to be inserted is determined based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, where the target timestamp is the timestamp of the blurred video frame. Insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp to obtain the target video; The process of determining the blurred video frames in the original video includes: Determine the first edge image at the preset position of the current video frame; Determine the absolute value of the first difference between the first sum of pixel values ​​of each pixel in the first edge image and the second sum of pixel values ​​of each pixel in the second edge image at a preset position of a neighboring video frame adjacent to the current video frame; Based on the histogram of the first edge image, determine the total number of first pixels in the first edge image whose pixel value is a target value, wherein the target value is any one of the pixel values ​​in the first edge image; Based on the histogram of the second edge image, determine the total number of second pixels in the second edge image whose pixel values ​​are the target values; Determine the absolute value of the second difference between the total number of the first pixel and the total number of the second pixel; Determine the maximum value among the absolute values ​​of the second differences corresponding to each pixel value in the first edge image; The blurred video frame is determined based on the absolute value of the first difference and the maximum value.

2. The method according to claim 1, characterized in that, The process of determining the blurred video frames in the original video includes: The first edge image at a preset position of the current video frame is determined based on the Sobel operator, wherein the current video frame is any video frame in the original video; The blurred video frame is determined based on the edge image of each video frame at a preset position in the original video.

3. The method according to claim 1, characterized in that, Determining the blurred video frame based on the absolute value of the first difference and the maximum value includes: The degree of ambiguity is obtained by weighted summation of the absolute value of the first difference and the maximum value. If the degree of blurriness is greater than the second preset threshold, then the current video frame is determined to be the blurry video frame.

4. The method according to claim 3, characterized in that, If the number of blurred video frames exceeds a third preset threshold, the method further includes: The maximum of the absolute values ​​of the first difference or the maximum of the degrees of ambiguity is determined as the extreme point; The blurred video frames are filtered based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames with a frame count equal to the third preset threshold.

5. The method according to claim 4, characterized in that, The step of filtering the blurred video frames based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames with a frame count equal to the third preset threshold includes: Using the timestamp of the video frame corresponding to the extreme point as the center, a set number of video frames are taken forward and backward respectively as the fuzzy video frames that are retained after filtering. The set number of frames is determined based on the third preset threshold.

6. The method according to any one of claims 2-5, characterized in that, Before determining the first sum of pixel values ​​of each pixel in the first edge image, determining the blurred video frame based on the edge image of each video frame at a preset position in the original video includes: The edge images at preset positions in each video frame of the original video are normalized to map the pixel values ​​of the pixels in the edge images to a preset range.

7. A video processing apparatus, characterized in that, include: The first determining module is used to determine the blurred video frames in the original video; The deletion module is used to delete the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame; The second determining module is used to determine the video frame to be inserted based on the video frames in the intermediate video whose timestamps are adjacent to the target timestamp, wherein the target timestamp is the timestamp of the blurred video frame. The frame interpolation module is used to insert the video frame to be inserted into the intermediate video at the position corresponding to the target timestamp, so as to obtain the target video; The process of determining the blurred video frames in the original video includes: Determine the first edge image at the preset position of the current video frame; Determine the absolute value of the first difference between the first sum of pixel values ​​of each pixel in the first edge image and the second sum of pixel values ​​of each pixel in the second edge image at a preset position of a neighboring video frame adjacent to the current video frame; Based on the histogram of the first edge image, determine the total number of first pixels in the first edge image whose pixel value is a target value, wherein the target value is any one of the pixel values ​​in the first edge image; Based on the histogram of the second edge image, determine the total number of second pixels in the second edge image whose pixel values ​​are the target values; Determine the absolute value of the second difference between the total number of the first pixel and the total number of the second pixel; Determine the maximum value among the absolute values ​​of the second differences corresponding to each pixel value in the first edge image; The blurred video frame is determined based on the absolute value of the first difference and the maximum value.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images