Video processing method, device, electronic device, and storage medium

Abstract

The embodiments of the present disclosure disclose a video processing method, an apparatus, an electronic device, and a storage medium, which includes the steps of: determining a blurred video frame in an original video; deleting the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame; determining a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp that is the timestamp of the blurred video frame; and inserting the video frame to be inserted at a position corresponding to the target timestamp in the intermediate video to obtain a target video. The video processing method according to the present disclosure achieves the purpose of removing the blurred frame in the video and improving the quality of the video.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to a Chinese patent application filed on October 27, 2021, bearing application number 202111257788.0 and entitled "Video Processing Method, Apparatus, Electronic Device, and Storage Medium," the entire contents of which are incorporated herein by reference.

[0002] [Technical field] The present disclosure relates to the field of information technology, and in particular to video processing methods, devices, electronic devices, and storage media. [Background technology]

[0003] With the rapid development of electronic technology in recent years, many electronic devices are now capable of video recording, and video recording quality has become an important indicator for evaluating electronic devices. There are many factors that affect video recording quality, such as resolution, color saturation, and clarity, but clarity is the most important factor.

[0004] When capturing video using electronic devices, if the subject moves within the exposure time or the electronic device shakes, the captured video may have motion blur, specifically, blurred, diffused, or smeared images.

[0005] Therefore, to obtain a good quality video, it is necessary to process the blurred images in the video. Summary of the Invention [Problem to be solved by the invention]

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a video processing method, an apparatus, an electronic device, and a storage medium, which achieve the purpose of removing blur frames in a video and improving the quality of the video. [Means for solving the problem]

[0007] In a first aspect, embodiments of the present disclosure include: determining blurred video frames in the original video; removing the blurred video frames from the original video to obtain an intermediate video that does not include the blurred video frames; determining a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, which is the timestamp of the blurred video frame; inserting the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0008] In a second aspect, embodiments of the present disclosure also include: a first determining module for determining blurred video frames in the original video; a removal module for removing the blurred video frames from the original video to obtain an intermediate video that does not include the blurred video frames; a second determination module for determining a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, the target timestamp being the timestamp of the blurred video frame; a frame insertion module for inserting the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0009] In a third aspect, embodiments of the present disclosure also include: one or more processors; a storage device for storing one or more programs; The one or more programs, when executed by the one or more processors, cause the one or more processors to perform the above-described video processing method.

[0010] In a fourth aspect, embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the above video processing method.

[0011] Compared with the prior art, the technical solutions according to the embodiments of the present disclosure have at least the following advantages:

[0012] According to the video processing method of the embodiment of the present disclosure, firstly, blurred video frames in the original video are determined, then the blurred video frames are deleted from the original video, and clear video frames are inserted in the deleted video frames in a frame insertion manner, thereby achieving the purpose of improving the video quality.

[0013] These and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following specific embodiments, with reference to the drawings. Identical or similar reference numerals represent identical or similar elements throughout the drawings. It should be understood that the drawings are schematic, and that objects and elements are not necessarily drawn to scale. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. [Figure 3] FIG. 10 is a schematic diagram of edge positions of an image in an embodiment of the present disclosure. [Figure 4] FIG. 10 is a schematic diagram illustrating the change in the cumulative sum of image values of all image points in an edge image corresponding to each video frame in an original video in an embodiment of the present disclosure. [Figure 5] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a structural schematic diagram of a video processing device according to an embodiment of the present disclosure. [Figure 7] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings. Although several embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms and should not be construed as being limited to the embodiments described herein, but rather, these embodiments are provided to provide a more complete and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are used for illustrative purposes only and are not used to limit the protection scope of the present disclosure.

[0016] It should be understood that the steps described in the method embodiments of the present disclosure may be performed in a different order or in parallel. Furthermore, method embodiments may include additional steps and / or omit the performance of steps shown. The scope of the present disclosure is not limited in this respect.

[0017] As used herein, the term "comprises" and variations thereof are the open "includes," i.e., "including, but not limited to." The term "based on" means "based at least in part on." The term "in one embodiment" means "at least one embodiment." The term "in another embodiment" means "at least one other embodiment." The term "in some embodiments" means "at least some embodiments." Relevant definitions of other terms are provided in the description below.

[0018] It should be noted that the concepts of "first," "second," etc. referred to in this disclosure are used only to distinguish between different devices, modules, or units, and do not define the order or interdependence of functions performed by these devices, modules, or units.

[0019] It should be noted that those skilled in the art will appreciate that the modifications "one" and "multiple" referred to in this disclosure are general rather than limiting and should be understood as "one or more" unless the context clearly indicates otherwise.

[0020] The names of messages or information interacted between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only and are not used to limit the scope of these messages or information.

[0021] 1 is a flowchart of a video processing method according to an embodiment of the present disclosure. The method can be performed by a video processing device, which may be implemented in software and / or hardware, and which may be located in electronic devices such as display terminals, including but not limited to electronic devices with display screens, such as smartphones, pocket computers, tablets, portable wearable devices, smart home devices (e.g., desk lamps), among others.

[0022] As shown in FIG. 1, the method may specifically include steps 110 to 140.

[0023] Step 110: Determine blurred video frames in the original video.

[0024] Specifically, a preset algorithm is used to determine an edge image of each video frame in the original video, and then, based on the edge image, a video frame in which the image is blurred or smeared is determined, and such a video frame is designated as a blurred video frame, and in the embodiment of the present disclosure, the blurred video frame is designated as the first video frame.

[0025] Step 120: Remove the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame.

[0026] Step 130: Determine a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, which is the timestamp of the blurred video frame.

[0027] If the timestamp of the blurred frame is 2s, the position where the original video is played for 2s is the position of the first video frame. In the intermediate video obtained by deleting the first video frame from the original video, one video frame is missing at the position of 2 seconds on the playback time axis. In order to achieve a good video effect and ensure the continuity of the video, one video frame (the video frame in question is called the video frame to be inserted) is inserted at the position of 2 seconds, thereby ensuring the continuity of the video screen.

[0028] Optionally, the video frame to be inserted is determined based on video frames at positions 1 second and 3 second in the intermediate video, specifically, a preset frame insertion algorithm predicts a video frame at the center of two adjacent video frames based on the two adjacent video frames, and determines the video frame to be inserted based on, for example, a motion estimation method or a neural network. The embodiments of the present disclosure do not limit the manner of determining the video frame to be inserted based on video frames in the intermediate video whose timestamps are adjacent to the target timestamp.

[0029] Step 140: The video frame to be inserted is inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0030] For example, the first video frame is 1 and its corresponding timestamp is 2s, that is, the position where the original video is played up to 2s becomes the position of the first video frame. After deleting the first video frame from the original video, one video frame is missing at the position of 2 seconds on the playback time axis of the original video. In order to achieve a good video effect and ensure the continuity of the video, one video frame is inserted at the position of 2 seconds, thereby ensuring the continuity of the video screen. In the embodiment of the present disclosure, the inserted video frame is the second video frame.

[0031] In some embodiments, the number of second video frames inserted is the same as the number of first video frames deleted, for example, if two first video frames are deleted from the original video, then two second video frames are inserted into the original video after the deletion.

[0032] In some embodiments, the number of inserted second video frames is greater than the number of deleted first video frames, for example, two first video frames are deleted from the original video, and three second video frames are inserted into the original video after the deletion. Inserting more second video frames improves the frame rate of the original video, further enhancing the playback effect of the original video.

[0033] In the video processing method according to the embodiment of the present disclosure, first, video frames with poor image quality in the original video are determined, then the video frames with poor image quality are deleted from the original video, and video frames with high image quality are inserted in the deleted video frames using a frame insertion method, so that blurred frames in the video can be removed and clear video frames can be inserted in place of the removed blurred frames, thereby achieving the purpose of improving the quality of the video.

[0034] Based on the above embodiment, Fig. 2 is a flow diagram of a video processing method. Based on the above embodiment, this embodiment describes a specific embodiment of the above step 110 "determine blurred video frames in the original video".

[0035] As shown in FIG. 2, the video processing method includes steps 210 to 250.

[0036] Step 210: Determine a first edge image at a preset position of a current video frame, which is any video frame in the original video, based on a Sobel operator sobel.

[0037] Step 220: Determine the blurred video frame based on an edge image at a preset position of each video frame in the original video.

[0038] Here, the "predetermined position" refers to the edge position of the image. For example, in the schematic diagram of the edge position of an image shown in FIG. 3, the part drawn with a white line is the edge position of the image, and an image containing only the white line is the edge image, i.e., the image composed of the edge positions. The Sobel operator (sobel) is mainly used for edge detection in images. Edge detection is performed by utilizing the characteristic that the weighted grayscale difference between adjacent points above, below, left, and right of an image point becomes an extreme value at an edge. When this operator is applied to any point in the image, a corresponding grayscale vector and its normal vector are generated. In other words, for each video frame in the original video, its edge image is calculated, and from the edge images of each video frame, video frames whose image quality does not meet the predetermined condition are determined, i.e., blurred video frames, i.e., the first video frames, are determined from the edge images. The number of blurred video frames in the original image may be multiple (multiple means two or more) or one.

[0039] Furthermore, in some embodiments, the step of determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video comprises: The method includes the steps of determining a first cumulative sum of image values of each image point in the first edge image, determining a second cumulative sum of image values of each image point in a second edge image at a predetermined position of an adjacent video frame, which is a video frame adjacent to the current video frame in the original video, determining an absolute value of a first difference between the first cumulative sum and the second cumulative sum, and determining the blurred video frame based on the absolute value of the first difference.

[0040] Optionally, in some embodiments, if the absolute value of the first difference is greater than a first preset threshold, the current video frame is determined to be the blurred video frame. For example, if the cumulative sum of image values of all image points in an edge image of the i (i≧1)th video frame in the original video is edge_sum(i), and the cumulative sum of image values of all image points in the edge image of the (i−1)th video frame is edge_sum(i−1), then cond1=abs(edge_sum(i)−edge_sum(i−1)), where abs() represents a function that takes an absolute value, and if cond1 is greater than a first preset threshold, the i-th video frame in the original video is the first video frame.

[0041] More specifically, if the edge image of the i-th video frame includes four image points, and the image values corresponding to these four image points are 5, 15, 2, and 0, respectively, the cumulative sum of the image values corresponding to these four image points is edge_sum(i)=5+15+2+0=22. If the edge image of the (i-1)-th video frame includes four image points, and the image values corresponding to these four image points are 4, 10, 1, and 3, the cumulative sum of the image values corresponding to these four image points is edge_sum(i-1)=4+10+1+3=18. In this case, cond1(i)=abs(edge_sum(i)-edge_sum(i-1))=abs(22-18)=4. Referring to FIG. 4, which is a schematic diagram showing changes in the cumulative sum of the image values of all image points in the edge image corresponding to each video frame in the original video, the positions where there is a sudden change indicate points where blurring may occur.

[0042] In some embodiments, to further improve the accuracy of determining the blurred video frame, the step of determining the blurred video frame based on the absolute value of the first difference includes a step of determining the maximum absolute value of the difference between the total number of image points at each image value of the first edge image and the second edge image based on a histogram of the first edge image and a histogram of the second edge image.

[0043] Specifically, based on a histogram of the first edge image, the total number of first image points whose image values in the first edge image are a target value that is one of the image values in the first edge image is determined; based on a histogram of the second edge image, the total number of second image points whose image values in the second edge image are the target value is determined; the absolute value of a second difference between the total number of first image points and the total number of second image points is determined; the maximum value of the absolute values of the second differences corresponding to each image value in the first edge image is determined; and the blurred video frame is determined based on the absolute value of the first difference and the maximum value. For example, if the image values of all image points in a first edge image range from 0 to 15, totaling 16 values, i.e., the target value is one of the 16 values, and the image values of all image points in a second edge image range from 0 to 15, the maximum of the absolute values of the second differences corresponding to each image value in the first edge image may be expressed as max{abs(hist(i)(k)-hist(i-1)(k))}, where abs() represents a function that takes the absolute value, hist(i)(k) represents the total number of first image points with image value k in the i-th edge image (which may also be understood as the first edge image), and hist(i-1)(k) represents the total number of second image points with image value k in the (i-1)-th edge image (which may also be understood as the second edge image), i.e., the target value is k, where 0≦k<16. More specifically, if in the edge image of the i-th video frame, the total number of image points with image value 0 is 5, the total number of image points with image value 1 is 15, the total number of image points with image value 2 is 2, and the total number of image points with image value 3 is 0, and in the edge image of the (i-1)-th video frame, the total number of image points with image value 0 is 4, the total number of image points with image value 1 is 10, the total number of image points with image value 2 is 1, and the total number of image points with image value 3 is 3, then cond2(i) = max{abs(hist(i)(k)-hist(i-1)(k))} = max{abs(5-4), abs(15-10), abs(2-1), abs(0-3} = 5.

[0044] In some embodiments, determining the blurred video frame based on the absolute value of the first difference and the maximum value includes: obtaining a blur degree by weighting the absolute value of the first difference and the maximum value; and determining the current video frame as the blurred video frame if the blur degree is greater than a second preset threshold. Illustratively, the blur degree cond(i)=c1*cond1(i)+c2*cond2(i), where c1 and c2 are preset constants. If cond(i)>thr1, the i-th video frame is determined as a blurred frame, i.e., the first video frame whose image quality does not satisfy a preset condition, where thr1 is the second preset threshold.

[0045] In some embodiments, to reduce the amount of calculation and increase the calculation speed, before determining a first cumulative sum of image values of each image point in the first edge image, determining the blurred video frame based on an edge image at a predetermined position in each video frame in the original video further includes normalizing the edge image at the predetermined position in each video frame in the original video to map the image values of the image points in the edge image into a predetermined space, where the edge image is a single-channel image. Initially, the image value of each image point in the edge image ranges from 0 to 255. Since determining the blurred video frame requires calculating the total number of image points corresponding to each image value and the cumulative sum of the image values of each image point, the amount of calculation is enormous. To reduce the amount of calculation and increase the calculation efficiency, before determining the first cumulative sum of image values of each image point, each edge image is normalized to map the image values of the image points in the edge image into a predetermined space, for example, from a space of 0 to 255 to a space of 0 to 16.

[0046] In some embodiments, to ensure video continuity, many first video frames are prohibited from being deleted. Therefore, when the number of blurred video frames exceeds a third preset threshold, the method further includes a step of filtering a plurality of first video frames and leaving only a limited number of first video frames. For example, if the third preset threshold is 8, at most 8 first video frames are left; if the number of first video frames determined according to the determination method in the above embodiment is 10, two first video frames are filtered and 8 are left, i.e., at most 8 first video frames are deleted from the original video. Specifically, the maximum of the absolute values of the first differences or the maximum of the blur degrees is determined as an extreme point, and the blurred video frames are screened based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames whose frame number is equal to the third preset threshold. Furthermore, the step of screening the blurred video frames based on the timestamps of the video frames corresponding to the extreme points to obtain blurred video frames whose frame number is equal to the third preset threshold further includes: The method includes a step of sequentially taking a set number of video frames in a forward and backward direction around the timestamp of the video frame corresponding to the extreme point as the blurred video frames retained after screening, wherein the set number of frames is determined by the third preset threshold.

[0047] For example, if the calculation results show that the blur degree corresponding to the first video frame in the original video is 0, the blur degree corresponding to the second video frame in the original video is 0, the blur degree corresponding to the third video frame in the original video is 6, the blur degree corresponding to the fourth video frame in the original video is 7, the blur degree corresponding to the fifth video frame in the original video is 6, and the blur degree corresponding to the sixth video frame in the original video is 6, and if the second preset threshold is 5, the third, fourth, fifth, and sixth video frames are determined as the first video frames, and the number of first video frames is 4, i.e., the number of blur frames is 4. If the third preset threshold is 3, one of the four first video frames is filtered and three remain. Specifically, since the maximum blur degree is 7, 7 is determined as an extreme point, and the video frame corresponding to this extreme point is the fourth video frame, and the timestamp of the video frame corresponding to this extreme point is the timestamp of the fourth video frame in the original video. With the timestamp as the center, a set number of video frames are taken in the forward and backward directions as blurred video frames reserved after screening. For example, in the forward direction, the video frame whose timestamp is closest to the timestamp of the fourth video frame, i.e., the third video frame, is taken, and in the backward direction, the video frame whose timestamp is closest to the timestamp of the video frame numbered 4, i.e., the fifth video frame, is taken. Thus, the third video frame, the fourth video frame, and the fifth video frame are determined as the final first video frame, and the first video frame, the second video frame, and the sixth video frame are filtered.

[0048] Step 230: Remove the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame.

[0049] Step 240: Determine a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, which is the timestamp of the blurred video frame.

[0050] Step 250: The video frame to be inserted is inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video.

[0051] Generally, referring to the schematic flow diagram of the video processing method shown in Figure 5, specifically, the method includes performing blur frame detection on the original video, obtaining a blur frame sequence, deleting the blur frame sequence from the original video, and then interpolating the deleted video frames into the original video using a frame insertion method to obtain a target video with the blur restored, thereby achieving the purpose of improving the video quality.

[0052] 6 is a structural schematic diagram of a video processing device in an embodiment of the present disclosure. As shown in FIG. 6, the video processing device specifically includes a first determination module 610, a deletion module 620, a second determination module 630, and a frame insertion module 640.

[0053] Wherein, the first determination module 610 is used to determine blurred video frames in the original video, the deletion module 620 is used to delete the blurred video frames from the original video to obtain an intermediate video that does not include the blurred video frames, the second determination module 630 is used to determine a video frame to be inserted based on a video frame in the intermediate video whose timestamp is adjacent to a target timestamp that is the timestamp of the blurred video frame, and the frame insertion module 640 is used to insert the video frame to be inserted at a position corresponding to the target timestamp in the intermediate video to obtain a target video.

[0054] Optionally, the first determination module 610 specifically includes: a first determination unit for determining a first edge image at a predetermined position of a current video frame, which is any video frame in the original video, based on a Sobel operator sobel; and a second determination unit for determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video.

[0055] Optionally, the second determination unit is specifically used for determining a first cumulative sum of image values of each image point in the first edge image, determining a second cumulative sum of image values of each image point in a second edge image at a predetermined position of an adjacent video frame, which is a video frame adjacent to the current video frame in the original video, and determining an absolute value of a first difference between the first cumulative sum and the second cumulative sum, and the second determiner unit is used for determining the blurred video frame based on the absolute value of the first difference.

[0056] Optionally, the second determiner unit is specifically used for determining the current video frame as the blurred video frame when the absolute value of the first difference is greater than a first preset threshold.

[0057] Optionally, the second determiner unit is specifically used for determining, based on a histogram of the first edge image, a total number of first image points whose image values in the first edge image are a target value that is any of the image values in the first edge image; determining, based on a histogram of the second edge image, a total number of second image points whose image values in the second edge image are the target value; determining an absolute value of a second difference between the total number of first image points and the total number of second image points; determining a maximum value among the absolute values of the second differences corresponding to each image value in the first edge image; and determining the blurred video frame based on the absolute value of the first difference and the maximum value.

[0058] Optionally, the second determiner unit is specifically used for: weighting the absolute value of the first difference and the maximum value to obtain a blur degree; and determining the current video frame as the blur video frame if the blur degree is greater than a second preset threshold.

[0059] Optionally, before determining a first cumulative sum of image values of each image point in the first edge image, the method further includes a normalization module for mapping image values of image points in the edge image to a predetermined space by normalizing the edge images at predetermined positions of each video frame in the original video, respectively, wherein the edge image is a single-channel image.

[0060] Optionally, the method further includes a screening module for determining the maximum value among the absolute values of the first differences or the maximum value among the blur degrees as an extreme point when the number of the first video frames exceeds a third preset threshold, and screening the blurred video frames based on timestamps of video frames corresponding to the extreme points to obtain blurred video frames whose frame number is the third preset threshold.

[0061] Optionally, the screening module is specifically used to sequentially take a set number of video frames in forward and backward directions around the timestamp of the video frame corresponding to the extreme point as the blurred video frames retained after screening, and the set number of frames is determined by the third preset threshold.

[0062] A video processing device according to an embodiment of the present disclosure first determines video frames with poor image quality in an original video, then deletes the video frames with poor image quality from the original video, and inserts video frames with high image quality in the deleted video frames using a frame insertion method, thereby removing blurred frames in the video and inserting clear video frames in place of the deleted blurred frames, thereby achieving the purpose of improving video quality.

[0063] The video processing device according to the embodiments of the present disclosure can perform the steps of the video processing method according to the method embodiments of the present disclosure, and the steps performed and the beneficial effects will not be described in detail here.

[0064] FIG. 7 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Reference is now made to FIG. 7 , which specifically illustrates a structural schematic diagram suitable for implementing an electronic device 700 according to an embodiment of the present disclosure. The electronic device 700 according to an embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablets (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., car navigation terminals), and wearable electronic devices, as well as fixed terminals such as digital TVs, desktop computers, and smart home devices. The electronic device illustrated in FIG. 7 is merely an example and does not limit the functionality and scope of use of the embodiment of the present disclosure.

[0065] 7, electronic device 700 may include a processing unit (e.g., a central processor, a graphics processor, etc.) 701 that can perform various appropriate operations and processes to implement methods of embodiments of the present disclosure in accordance with a program stored in read-only memory (ROM) 702 or a program loaded from storage device 708 into random access memory (RAM) 703. RAM 703 also stores various programs and data necessary for the operation of electronic device 700. Processing unit 701, ROM 702, and RAM 703 are connected to one another via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0066] Typically, input devices 706 including a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 707 including a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 708 including magnetic tape, hard disk, etc., and communication devices 709 may be connected to the I / O interface 705. The communication devices 709 may enable the electronic device 700 to communicate wirelessly or via wires with other devices to exchange data. While FIG. 7 illustrates the electronic device 700 with various devices, it should be understood that it is not necessary to implement or include all of the devices shown. Alternatively, more or fewer devices may be implemented or included.

[0067] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program embodied on a non-transitory computer-readable medium, the computer program including program code for executing the methods illustrated in the flowcharts, thereby realizing the methods. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 709, installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the functions defined in the methods according to the embodiments of the present disclosure are performed.

[0068] It should be noted that the computer-readable medium referred to above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical memory device, a magnetic memory device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device. In contrast, in this disclosure, a computer-readable signal medium may include a propagating data signal, in baseband or as part of a carrier, that carries computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied in a computer-readable medium may be transmitted over any suitable medium, including, but not limited to, electrical cable, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0069] In some embodiments, clients and servers may communicate using any network protocol now known or later developed, such as HyperText Transfer Protocol (HTTP), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), interconnected networks (e.g., the Internet), end-to-end networks (e.g., ad-hoc end-to-end networks), and networks now known or later developed.

[0070] The computer-readable medium may be included in the electronic device, or may exist separately from the electronic device.

[0071] The computer-readable medium includes one or more programs, which, when executed by the electronic device,

[0072] The electronic device is caused to perform the steps of determining a blurred video frame in an original video, deleting the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame, determining a video frame to be inserted based on a video frame in the intermediate video whose timestamp is adjacent to a target timestamp that is the timestamp of the blurred video frame, and inserting the video frame to be inserted at a position in the intermediate video that corresponds to the target timestamp to obtain a target video.

[0073] Optionally, when the one or more programs are executed by an electronic device, the electronic device may perform other steps described in the above embodiments.

[0074] Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and the like, and conventional procedural programming languages such as the "C" language, or a combination thereof. The program code may execute entirely on the user computer, partially on the user computer, as a separate software package, partially on the user computer, partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0075] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program segment, or portion of code, including one or more executable instructions for implementing a given logical function. It should be noted that in some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may actually be executed substantially in parallel or may be executed in the reverse order depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated hardware-based system that performs a given function or operation, or in a combination of dedicated hardware and computer instructions.

[0076] The units described in connection with the embodiments of the present disclosure may be implemented in software or hardware, and the names of the units, in some cases, are not limitations on the units themselves.

[0077] The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0078] In the context of this disclosure, a machine-readable medium may be any tangible medium that contains or stores a program usable by or in combination with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of a machine-readable storage medium may include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical memory device, a magnetic memory device, or any suitable combination of the above.

[0079] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing method, including the steps of determining a blurred video frame in an original video, deleting the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame, determining a video frame to be inserted based on a video frame in the intermediate video whose timestamp is adjacent to a target timestamp that is the timestamp of the blurred video frame, and inserting the video frame to be inserted at a position in the intermediate video that corresponds to the target timestamp, to obtain a target video.

[0080] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, the step of determining a blurred video frame in the original video includes a step of determining a first edge image at a predetermined position of a current video frame, which is any video frame in the original video, based on a Sobel operator sobel, and a step of determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video.

[0081] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, the step of determining the blurred video frame based on an edge image at a predetermined position of each video frame in the original video includes the steps of determining a first cumulative sum of image values of each image point in the first edge image, determining a second cumulative sum of image values of each image point in a second edge image at a predetermined position of an adjacent video frame, which is a video frame adjacent to the current video frame in the original video, determining an absolute value of a first difference between the first cumulative sum and the second cumulative sum, and determining the blurred video frame based on the absolute value of the first difference.

[0082] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, the step of determining the blurred video frame based on the absolute value of the first difference includes a step of determining the current video frame as the blurred video frame if the absolute value of the first difference is greater than a first predetermined threshold.

[0083] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, the step of determining the blurred video frame based on the absolute value of the first difference comprises: The method includes determining, based on a histogram of the first edge image, a total number of first image points whose image values in the first edge image are a target value that is any of the image values in the first edge image; determining, based on a histogram of the second edge image, a total number of second image points whose image values in the second edge image are the target value; determining an absolute value of a second difference between the total number of first image points and the total number of second image points; determining a maximum value among the absolute values of the second differences corresponding to each image value in the first edge image; and determining the blurred video frame based on the absolute value of the first difference and the maximum value.

[0084] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, the step of determining the blurred video frame based on the absolute value of the first difference and the maximum value includes the steps of: weighting the absolute value of the first difference and the maximum value to obtain a blur degree; and determining the current video frame as the blurred video frame if the blur degree is greater than a second preset threshold.

[0085] According to one or more embodiments of the present disclosure, in the video processing method according to the present disclosure, optionally, before determining a first cumulative sum of image values of each image point in the first edge image, the step of determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video further includes a step of mapping the image values of image points in the edge images to a predetermined space by respectively normalizing the edge images at predetermined positions of each video frame in the original video.

[0086] According to one or more embodiments of the present disclosure, in the video processing method of the present disclosure, optionally, if the number of blurred video frames exceeds a third preset threshold, the method further includes a step of determining the maximum value among the absolute values of the first differences or the maximum value among the blur degrees as an extreme point, and a step of screening the blurred video frames based on the timestamps of the video frames corresponding to the extreme points, and obtaining blurred video frames whose frame number is the third preset threshold.

[0087] According to one or more embodiments of the present disclosure, in the video processing method of the present disclosure, optionally, the step of screening the blurred video frames based on the timestamps of the video frames corresponding to the extreme points and obtaining blurred video frames with a frame number equal to the third preset threshold includes steps of sequentially taking a set number of video frames in forward and backward directions, centered on the timestamps of the video frames corresponding to the extreme points, as blurred video frames retained after screening, wherein the set number of frames is determined by the third preset threshold.

[0088] According to one or more embodiments of the present disclosure, the present disclosure provides a video processing device, including: a first determination module for determining a blurred video frame in an original video; a deletion module for deleting the blurred video frame from the original video to obtain an intermediate video that does not include the blurred video frame; a second determination module for determining a video frame to be inserted based on a video frame in the intermediate video whose timestamp is adjacent to a target timestamp that is the timestamp of the blurred video frame; and a frame insertion module for inserting the video frame to be inserted at a position in the intermediate video that corresponds to the target timestamp to obtain a target video.

[0089] According to one or more embodiments of the present disclosure, in a video processing device according to the present disclosure, optionally, the first determination module specifically includes: a first determination unit for determining a first edge image at a predetermined position of a current video frame, which is any video frame in the original video, based on a Sobel operator sobel; and a second determination unit for determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video.

[0090] According to one or more embodiments of the present disclosure, in the video processing device according to the present disclosure, optionally, the second determination unit specifically includes a first determiner unit for determining a first cumulative sum of image values of each image point in the first edge image, determining a second cumulative sum of image values of each image point in the second edge image at a predetermined position of an adjacent video frame, which is a video frame adjacent to the current video frame in the original video, and determining an absolute value of a first difference between the first cumulative sum and the second cumulative sum, and a second determiner unit for determining the blurred video frame based on the absolute value of the first difference.

[0091] According to one or more embodiments of the present disclosure, in a video processing device according to the present disclosure, optionally, the second determiner unit is specifically used to determine the current video frame as the blurred video frame when the absolute value of the first difference is greater than a first preset threshold.

[0092] According to one or more embodiments of the present disclosure, in the video processing device according to the present disclosure, optionally, the second determiner unit is specifically used for determining, based on a histogram of the first edge image, a total number of first image points whose image values in the first edge image are a target value that is any of the image values in the first edge image; determining, based on a histogram of the second edge image, a total number of second image points whose image values in the second edge image are the target value; determining an absolute value of a second difference between the total number of first image points and the total number of second image points; determining a maximum value among the absolute values of the second differences corresponding to each image value in the first edge image; and determining the blurred video frame based on the absolute value of the first difference and the maximum value.

[0093] According to one or more embodiments of the present disclosure, in the video processing device according to the present disclosure, optionally, the second determiner unit is specifically used for weighted summing the absolute value of the first difference and the maximum value to obtain a blur degree, and if the blur degree is greater than a second preset threshold, determining the current video frame as the blurred video frame.

[0094] According to one or more embodiments of the present disclosure, the video processing device according to the present disclosure optionally further includes a normalization module for mapping image values of image points in the edge image to a predetermined space by normalizing the edge images at predetermined positions of each video frame in the original video before determining a first cumulative sum of image values of each image point in the first edge image, wherein the edge image is a single-channel image.

[0095] According to one or more embodiments of the present disclosure, the video processing device according to the present disclosure optionally further includes a screening module for determining, when the number of the first video frames exceeds a third preset threshold, the maximum value among the absolute values of the first differences or the maximum value among the blur degrees as an extreme point, and screening the blurred video frames based on the timestamps of the video frames corresponding to the extreme points, to obtain blurred video frames whose frame number is the third preset threshold. According to one or more embodiments of the present disclosure, in the video processing device according to the present disclosure, optionally, the screening module is specifically used to take a set number of video frames in forward and backward order, centered on the timestamp of the video frame corresponding to the extreme point, as blurred video frames retained after screening, and the set number of frames is determined by the third preset threshold.

[0096] According to one or more embodiments of the present disclosure, the present disclosure provides a method for manufacturing a semiconductor device, comprising: one or more processors; a memory for storing one or more programs; The one or more programs, when executed by the one or more processors, cause the one or more processors to perform any of the video processing methods according to the present disclosure. According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements any of the video processing methods according to the present disclosure.

[0097] An embodiment of the present disclosure also provides a computer program product including a computer program or instructions that, when executed by a processor, implements the above video processing method.

[0098] The above description merely describes the preferred embodiments and applied technical principles of the present disclosure. Those skilled in the art will understand that the scope of the present disclosure is not limited to the technical solution consisting of a specific combination of the above technical features, but should also cover other technical solutions consisting of any combination of the above technical features or their equivalent features without departing from the idea of the above disclosure. For example, a technical solution formed by mutually replacing the above features with technical features having similar functions (but not limited to) disclosed in the present disclosure.

[0099] Additionally, although operations are shown in a particular order, this should not be understood as requiring that these operations be performed in the particular order shown, or sequentially. In some environments, multitasking or parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Some features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.

[0100] Although the present subject matter has been described in language specific to structural features and / or methodological logical operations, it should be understood that the subject matter defined in the claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. 1. A video processing method comprising: determining blurred video frames in the original video; removing the blurred video frames from the original video to obtain an intermediate video that does not include the blurred video frames; determining a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, which is the timestamp of the blurred video frame; inserting the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video; The step of determining a blurred video frame in the original video includes the steps of: determining a first edge image at a predetermined position of a current video frame, the current video frame being any video frame in the original video, based on a Sobel operator (sobel); and determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video; a first cumulative sum of image values of each image point in a second edge image at a predetermined position of an adjacent video frame in the original video, the adjacent video frame being a video frame adjacent to the current video frame; an absolute value of a first difference between the first cumulative sum and the second cumulative sum; and a step of determining the blurred video frame based on the absolute value of the first difference.

2. determining the blurred video frame based on the absolute value of the first difference; 2. The method of claim 1, further comprising determining the current video frame as the blurred video frame if the absolute value of the first difference is greater than a first preset threshold.

3. determining the blurred video frame based on the absolute value of the first difference; determining a total number of first image points in the first edge image whose image value is a target value based on a histogram of the first edge image; determining a total number of second image points in the second edge image whose image value is the target value based on a histogram of the second edge image; determining an absolute value of a second difference between the first total number of image points and the second total number of image points; determining the maximum of the absolute values of the second differences corresponding to each image value in the first edge image; and determining the blurred video frame based on the absolute value of the first difference and the maximum value.

4. determining the blurred video frame based on the absolute value of the first difference and the maximum value, calculating a weighted sum of the absolute value of the first difference and the maximum value to obtain a blur degree; and determining the current video frame as the blurred video frame if the blur degree is greater than a second preset threshold.

5. If the number of blurred video frames exceeds a third preset threshold, determining a maximum value among the absolute values of the first differences or a maximum value among the blur degrees as an extreme point; 5. The method of claim 4, further comprising: screening the blurred video frames based on timestamps of video frames corresponding to the extreme points to obtain blurred video frames whose frame number is the third preset threshold.

6. The step of screening the blurred video frames according to the timestamps of the video frames corresponding to the extreme points to obtain the blurred video frames whose frame number is the third preset threshold value, includes: The method includes sequentially taking a predetermined number of video frames in a forward direction and a backward direction around the time stamp of the video frame corresponding to the extremum point as the blurred video frames reserved after screening; The method of claim 5 , wherein the set number of frames is determined by the third preset threshold.

7. determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video before determining a first cumulative sum of image values of each image point in the first edge image; 2. The method of claim 1, further comprising the step of mapping image values of image points in the edge images to a predetermined space by normalizing the edge images at predetermined positions of each video frame in the original video, respectively.

8. 1. A video processing device comprising: a first determining module for determining blurred video frames in the original video; a removal module for removing the blurred video frames from the original video to obtain an intermediate video that does not include the blurred video frames; a second determination module for determining a video frame to be inserted based on a video frame whose timestamp in the intermediate video is adjacent to a target timestamp, the target timestamp being the timestamp of the blurred video frame; a frame insertion module for inserting the video frame to be inserted into the intermediate video at a position corresponding to the target timestamp to obtain a target video; The first determination module includes: a first determination unit for determining a first edge image at a predetermined position of a current video frame, which is any video frame in the original video, based on a Sobel operator (sobel); and a second determination unit for determining the blurred video frame based on edge images at predetermined positions of each video frame in the original video; The video processing device is characterized in that the second determination unit is further used for determining a first cumulative sum of image values of each image point in the first edge image, determining a second cumulative sum of image values of each image point in a second edge image at a predetermined position of an adjacent video frame, which is a video frame adjacent to the current video frame in the original video, determining an absolute value of a first difference between the first cumulative sum and the second cumulative sum, and determining the blurred video frame based on the absolute value of the first difference.

9. An electronic device, one or more processors; a storage device for storing one or more programs; The electronic device, characterized in that the one or more programs, when executed by the one or more processors, cause the one or more processors to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium on which a computer program is stored, A computer-readable storage medium, characterized in that the program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images