Method, system, device and medium for high-definition video processing based on image replacement

Abstract

This invention discloses a method, system, device, and medium for video high-definition processing based on image replacement, comprising: capturing video of a target object using a first camera on a user terminal to obtain first video data, and simultaneously capturing multiple first image data of the target object using a second camera on the user terminal; performing clarity detection on the first video data and identifying several blurred video frames; determining the first image data corresponding to the capture time of the blurred video frames as the image to be replaced; and replacing the blurred video frames in the first video data with the image to be replaced to obtain second video data. This invention achieves high-definition processing of captured video using multiple cameras on a user terminal, improving the efficiency of video high-definition processing and the user experience, and can be applied to the field of video processing technology.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a method, system, device and medium for high-definition video processing based on image replacement. Background Art

[0002] In the prior art, mobile phone video and photo taking are two independent functions, that is, the camera of the same mobile phone can only perform video or photo taking operations at the same time. In actual practice, the clarity of photos is often higher than that of videos, and the so-called "dual camera" and "triple camera" only change the field of view and resolution of video or photo taking by adding cameras.

[0003] Since video recording and photo taking cannot be performed at the same time, and both use the main camera of the mobile phone, the video repair work after mobile phone shooting is often repaired and optimized through other video software at a later stage to ensure the clarity and smoothness of the video. This requires users to manually repair and optimize the video, which is inefficient and affects the user experience. Summary of the invention

[0004] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0005] To this end, an object of an embodiment of the present invention is to provide a video high-definition processing method based on image replacement, which realizes high-definition processing of captured videos through multiple cameras of a user terminal, thereby improving the efficiency of video high-definition processing and user experience.

[0006] Another object of an embodiment of the present invention is to provide a video high-definition processing system based on image replacement.

[0007] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0008] In a first aspect, an embodiment of the present invention provides a method for processing video high definition based on image replacement, comprising the following steps:

[0009] The target object is photographed by a first camera of the user terminal to obtain first video data, and the target object is continuously captured by a second camera of the user terminal to obtain a plurality of first image data;

[0010] Performing clarity detection on the first video data, and determining a number of blurred video frames of the first video data;

[0011] Determine that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced;

[0012] The blurred video frame in the first video data is replaced by the image to be replaced to obtain the second video data.

[0013] Further, in one embodiment of the present invention, the step of capturing a target object through a first camera of a user terminal to obtain first video data, and simultaneously capturing the target object through a second camera of the user terminal to obtain a plurality of first image data, specifically includes:

[0014] Sending a first control instruction to the first camera through a control module of the user terminal, and simultaneously sending a second control instruction to the second camera;

[0015] In response to the first control instruction, the target object is photographed by the first camera to obtain the first video data, and first shooting time information of the first video data is determined;

[0016] In response to the second control instruction, the target object is continuously captured by the second camera at a preset time interval to obtain a plurality of first image data, and second shooting time information corresponding to each of the first image data is determined.

[0017] Furthermore, in one embodiment of the present invention, the step of performing clarity detection on the first video data and determining a number of blurred video frames of the first video data specifically includes:

[0018] Determine a definition evaluation value of each video frame of the first video data by using a Sobel operator, and determine a definition detection threshold range of the first video data;

[0019] The video frame whose clarity evaluation value is not within the clarity detection threshold range is determined as the blurred video frame.

[0020] Further, in one embodiment of the present invention, the step of determining the definition evaluation value of each video frame of the first video data by using a Sobel operator specifically includes:

[0021] Extracting a plurality of video frames according to the first video data;

[0022] Calculate the edge gradient value of each pixel in the video frame by using the Sobel operator;

[0023] The sum of squares of edge gradient values ​​of all pixels in the video frame is determined as a clarity evaluation value of the video frame.

[0024] Further, in one embodiment of the present invention, the step of determining the definition detection threshold range of the first video data specifically includes:

[0025] Performing video interception on the first video data to obtain a plurality of video clips;

[0026] Performing image sampling on the video clip to obtain a plurality of corresponding sample video frames, and determining the grayscale average of pixels at the same position in each of the sample video frames, and then determining the background image corresponding to the video clip according to the grayscale average;

[0027] Calculate the edge gradient value of each pixel in the background image by using the Sobel operator;

[0028] Determine the sum of squares of edge gradient values ​​of all pixels in the background image as the average clarity value of the corresponding video clip;

[0029] The minimum value and the maximum value of the clarity average value of each of the video clips are determined, and the clarity detection threshold range is determined according to the minimum value and the maximum value.

[0030] Further, in one embodiment of the present invention, the step of determining that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced specifically includes:

[0031] Determining, according to the first shooting time information, the shooting moment of the blurred video frame as the first shooting moment;

[0032] determining, according to the second shooting time information, a shooting time of each of the first image data as a second shooting time;

[0033] The first image data corresponding to the second shooting time closest to the first shooting time is determined as the image to be replaced.

[0034] Further, in one embodiment of the present invention, the step of replacing the blurred video frame in the first video data with the image to be replaced to obtain the second video data specifically includes:

[0035] In response to a video preview operation of the user terminal, playing the first video data through the user terminal;

[0036] When the blurred video frame is played, the playback is paused and the screen is reduced, and then the corresponding image to be replaced pops up on the playback interface, so that the user terminal displays the blurred video frame and the image to be replaced to the user;

[0037] In response to an image replacement operation of the user terminal, the blurred video frame is replaced by the image to be replaced to generate the second video data.

[0038] In a second aspect, an embodiment of the present invention provides a video high-definition processing system based on image replacement, comprising:

[0039] A shooting module, used to shoot a video of a target object through a first camera of a user terminal to obtain first video data, and simultaneously to continuously capture the target object through a second camera of the user terminal to obtain a plurality of first image data;

[0040] a clarity detection module, configured to perform clarity detection on the first video data and determine a number of blurred video frames of the first video data;

[0041] a module for determining an image to be replaced, configured to determine that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced;

[0042] The video frame replacement module is used to replace the blurred video frame in the first video data with the image to be replaced to obtain the second video data.

[0043] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein the program, when executed by the processor, realizes the video high-definition processing method based on image replacement as described in the first aspect above.

[0044] In a fourth aspect, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium for computer-readable storage, and which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the video high-definition processing method based on image replacement as described in the first aspect above.

[0045] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0046] The embodiment of the present invention obtains first video data by taking video of a target object through a first camera of a user terminal, and obtains multiple first image data by continuously capturing the target object through a second camera of the user terminal at the same time, performs clarity detection on the first video data, and determines a number of blurred video frames of the first video data, determines the first image data corresponding to the shooting moment of the blurred video frame as the image to be replaced, and replaces the blurred video frame in the first video data with the image to be replaced to obtain the second video data. The embodiment of the present invention obtains image data by taking a photo through a second camera of a user terminal during the shooting process of the first camera, and then performs clarity detection on the video data obtained by the first camera to obtain a blurred video frame, and then replaces the blurred video frame with the image data shot at the same time as the blurred video frame to improve the overall clarity of the video, thereby realizing high-definition processing of the shot video, and improving the efficiency of high-definition processing of the video and the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A flowchart of the steps of a method for processing video high definition based on image replacement provided by an embodiment of the present invention;

[0049] Figure 2 A flow chart of step S101 provided in an embodiment of the present invention;

[0050] Figure 3 A flow chart of step S102 provided in an embodiment of the present invention;

[0051] Figure 4 A flow chart of step S1021 provided in an embodiment of the present invention;

[0052] Figure 5 Another flow chart of step S1021 provided in an embodiment of the present invention;

[0053] Figure 6 A flow chart of step S103 provided in an embodiment of the present invention;

[0054] Figure 7 A flow chart of step S104 provided in an embodiment of the present invention;

[0055] Figure 8A schematic diagram of the structure of a video high-definition processing system based on image replacement provided by an embodiment of the present invention;

[0056] Fig. 9 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0059] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0060] The video high-definition processing method based on image replacement provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video high-definition processing method based on image replacement, etc., but is not limited to the above forms.

[0061] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0062] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0063] like Figure 1 FIG. 1 is a flowchart of a method for processing high definition video based on image replacement according to an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for processing video high definition based on image replacement, which specifically includes the following steps:

[0064] S101. Capture a target object through a first camera of a user terminal to obtain first video data, and simultaneously capture the target object through a second camera of the user terminal to obtain multiple first image data.

[0065] Specifically, the user terminal of an embodiment of the present invention includes a first camera (main camera) and a second camera (intercept camera), which are installed at similar positions in the same area of ​​the user terminal. The first camera and the second camera can respectively perform video shooting and photo shooting at the same time. When the user terminal controls the first camera to shoot video of the target object, the second camera will also continuously capture the target object.

[0066] like Figure 2 FIG. 1 is a flow chart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 2 As an optional implementation, the step of capturing a target object through a first camera of the user terminal to obtain first video data, and simultaneously capturing the target object through a second camera of the user terminal to obtain a plurality of first image data, specifically includes:

[0067] S1011, sending a first control instruction to a first camera through a control module of a user terminal, and simultaneously sending a second control instruction to a second camera;

[0068] S1012, in response to the first control instruction, capturing a target object through a first camera to obtain first video data, and determining first capturing time information of the first video data;

[0069] S1013. In response to the second control instruction, continuously capture the target object at a preset time interval through the second camera to obtain a plurality of first image data, and determine second shooting time information corresponding to each first image data.

[0070] Specifically, the first video data is the timing data of the video frames, which records the shooting time of each video frame, and the first shooting time information can be obtained; when the second camera continuously captures the target object, the shooting time corresponding to each first image data is also recorded to form the second shooting time information.

[0071] S102: Perform clarity detection on the first video data, and determine a number of blurred video frames of the first video data.

[0072] Specifically, the embodiment of the present invention uses the Sobel operator to perform clarity detection on the first video data, so as to determine a number of blurred video frames.

[0073] like Figure 3 FIG. 1 is a flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 3 As an optional implementation, the step of performing clarity detection on the first video data and determining a number of blurred video frames of the first video data specifically includes:

[0074] S1021, determining a definition evaluation value of each video frame of the first video data by using a Sobel operator, and determining a definition detection threshold range of the first video data;

[0075] S1022: Determine that a video frame whose clarity evaluation value is not within a clarity detection threshold range is a blurred video frame.

[0076] like Figure 4 FIG. 1 is a flow chart of step S1021 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation, the step of determining the definition evaluation value of each video frame of the first video data by using a Sobel operator specifically includes:

[0077] S10211. Extract and obtain multiple video frames according to the first video data;

[0078] S10212, calculating the edge gradient value of each pixel in the video frame by using the Sobel operator;

[0079] S10213. Determine the sum of the squares of the edge gradient values ​​of all pixels in the video frame as the clarity evaluation value of the video frame.

[0080] Specifically, the formula for calculating the edge gradient value of each pixel in the video frame using the Sobel operator is as follows:

[0081]

[0082]

[0083]

[0084] Among them, F(x,y) represents the edge gradient value of the pixel point (x,y), f(x,y) represents the gray value of the pixel point (x,y), S x represents the convolution kernel of the Sobel operator in the x direction, S y Represents the convolution kernel of the Sobel operator in the y direction.

[0085] After calculating the edge gradient value of each pixel in the video frame according to the Sobel operator, the square sum of the edge gradient values ​​of all pixels in the video frame is used as the clarity evaluation value of the video frame. The specific formula is as follows:

[0086]

[0087] Among them, value represents the clarity evaluation value of the video frame, F k (x, y) represents the edge gradient value of the kth pixel in the video frame, and n represents the total number of pixels in the video frame.

[0088] like Figure 5 FIG. 1 is another flow chart of step S1021 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, the step of determining a definition detection threshold range of the first video data specifically includes:

[0089] S10214, performing video interception on the first video data to obtain multiple video segments;

[0090] S10215, performing image sampling on the video clip to obtain a plurality of corresponding sample video frames, and determining the grayscale average value of pixels at the same position in each sample video frame, and then determining the background image corresponding to the video clip according to the grayscale average value;

[0091] S10216, calculating the edge gradient value of each pixel in the background image by using the Sobel operator;

[0092] S10217, determining that the sum of squares of edge gradient values ​​of all pixels in the background image is the average clarity value of the corresponding video clip;

[0093] S10218. Determine the minimum and maximum values ​​of the average clarity of each video segment, and determine a clarity detection threshold range according to the minimum and maximum values.

[0094] Specifically, sample video frames in the first video data are extracted in real time, for example, a video clip with a duration of 1 minute is intercepted, and sampling is performed every 5 seconds to obtain 12 sample video frames in total. The 12 sample video frames are converted from RGB space to grayscale space, and the grayscale values ​​of the pixels at the same position in the sample video frames are accumulated and averaged, and the average value is used as the grayscale value of the pixels at the corresponding position in the background image corresponding to the video clip, thereby obtaining the background image of the video clip, and then the clarity evaluation value of the background image is calculated based on the background image using the above formula as the clarity average value of the corresponding video clip. After calculating the clarity average values ​​of all video clips, the maximum and minimum values ​​are determined, and the clarity detection threshold range can be obtained.

[0095] The obtained clarity evaluation value value of each video frame is compared with the determined clarity detection threshold range. If the value falls within the clarity detection threshold range, the corresponding video frame is determined to be clear, otherwise it is determined to be a blurred video frame.

[0096] S103: Determine the first image data corresponding to the shooting moment of the blurred video frame as the image to be replaced.

[0097] like Figure 6 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, the step of determining the first image data corresponding to the shooting moment of the blurred video frame as the image to be replaced specifically includes:

[0098] S1031, determining, according to the first shooting time information, a shooting moment of the blurred video frame as a first shooting moment;

[0099] S1032, determining the shooting time of each first image data as the second shooting time according to the second shooting time information;

[0100] S1033: Determine the first image data corresponding to the second shooting moment closest to the first shooting moment as the image to be replaced.

[0101] Specifically, the shooting time corresponding to the blurred video frame is obtained, and then the first image data closest to the shooting time of the blurred video frame is selected as the image to be replaced. Finally, the extracted image to be replaced is replaced with the blurred video frame in the first video data in real time by mapping, so as to improve the clarity of the final output video.

[0102] S104: Replace the blurred video frame in the first video data with the image to be replaced to obtain second video data.

[0103] like Figure 7 FIG. 1 is a flow chart of step S104 provided in an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, the step of replacing the blurred video frame in the first video data with the image to be replaced to obtain the second video data specifically includes:

[0104] S1041. In response to a video preview operation of a user terminal, playing first video data through the user terminal;

[0105] S1042, when the blurred video frame is played, the playback is paused and the screen is reduced, and then the corresponding image to be replaced is popped up on the playback interface, so that the user terminal displays the blurred video frame and the image to be replaced to the user;

[0106] S1043. In response to an image replacement operation of the user terminal, replace the blurred video frame with the image to be replaced to generate second video data.

[0107] Specifically, after the user terminal captures and obtains the first video data, it can perform a video preview operation, and the user can watch the first video data through the user terminal. When the blurred video frame is played, the playback can be automatically paused and the screen can be reduced. At the same time, the corresponding image to be replaced will pop up on the playback interface, and the user can choose whether to replace it. If the user chooses to replace, the blurred video frame will be replaced by the image to be replaced, and the playback will continue until all the blurred video frames are replaced by the user's choice (the user can also exit the preview midway, and it is defaulted that all subsequent blurred video frames will not be replaced or will be replaced according to the user settings, and the user settings can be pre-set before the video is shot), and the second video data can be obtained.

[0108] The method steps of the embodiment of the present invention are described above. It can be understood that the embodiment of the present invention obtains image data by taking a picture with the second camera of the user terminal during the process of the first camera taking pictures, and then performs clarity detection on the video data obtained by the first camera to obtain a blurred video frame, and then replaces the blurred video frame with the image data taken at the same time as the blurred video frame to improve the overall clarity of the video, thereby realizing high-definition processing of the captured video, improving the efficiency of high-definition processing of the video and the user experience.

[0109] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0110] 1) Use the intercepting camera to take photos while the main camera is shooting video. Since the clarity of the photos taken is higher than the clarity of the video taken, the blurred video frames in the video can be replaced by photos, thereby achieving the purpose of improving the clarity of the shot video. Compared with the prior art of repairing by video software, the image replacement method in the embodiment of the present invention is more realistic and more efficient.

[0111] 2) The Sobel operator clarity detection algorithm is used to calculate the clarity of the video frame, thereby realizing real-time monitoring and real-time replacement of the video, thereby improving the output efficiency of the output video.

[0112] like Figure 8 FIG. 1 is a schematic diagram of the structure of a video face swapping system provided by an embodiment of the present invention, referring to FIG. Figure 8 The embodiment of the present invention provides a video high-definition processing system based on image replacement, comprising:

[0113] A shooting module, used to shoot a video of a target object through a first camera of a user terminal to obtain first video data, and simultaneously to continuously capture the target object through a second camera of the user terminal to obtain a plurality of first image data;

[0114] A clarity detection module, used to perform clarity detection on the first video data and determine a number of blurred video frames of the first video data;

[0115] A module for determining an image to be replaced, used to determine the first image data corresponding to the shooting moment of the blurred video frame as the image to be replaced;

[0116] The video frame replacement module is used to replace the blurred video frame in the first video data with the image to be replaced to obtain the second video data.

[0117] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0118] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the program is executed by the processor, the above-mentioned video high-definition processing method based on image replacement is realized. The electronic device may be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0119] like Fig. 9 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig. 9 , an embodiment of the present invention provides an electronic device, including:

[0120] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0121] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the video high-definition processing method based on image replacement in the embodiment of the present invention;

[0122] Input / output interface 903, used to implement information input and output;

[0123] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0124] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0125] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0126] like Fig.10 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.10 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1001, and the one or more programs 1001 can be executed by one or more processors to implement the above-mentioned video high-definition processing method based on image replacement.

[0127] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0129] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0130] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0131] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0132] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0133] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0134] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0135] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0136] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0137] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A video high-definition processing method based on image replacement, characterized in that: The following steps are involved: The target object is photographed by a first camera of the user terminal to obtain first video data, and the target object is continuously captured by a second camera of the user terminal to obtain a plurality of first image data; Performing clarity detection on the first video data, and determining a number of blurred video frames of the first video data; Determine that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced; Replacing the blurred video frame in the first video data with the image to be replaced to obtain second video data; The step of performing clarity detection on the first video data and determining a number of blurred video frames of the first video data specifically includes: Determine a definition evaluation value of each video frame of the first video data by using a Sobel operator, and determine a definition detection threshold range of the first video data; Determine that the video frame whose clarity evaluation value is not within the clarity detection threshold range is the blurred video frame; The step of determining the definition detection threshold range of the first video data specifically includes: Performing video interception on the first video data to obtain a plurality of video clips; Performing image sampling on the video clip to obtain a plurality of corresponding sample video frames, and determining the grayscale average of pixels at the same position in each of the sample video frames, and then determining the background image corresponding to the video clip according to the grayscale average; Calculate the edge gradient value of each pixel in the background image by using the Sobel operator; Determine the sum of squares of edge gradient values ​​of all pixels in the background image as the average clarity value of the corresponding video clip; The minimum value and the maximum value of the clarity average value of each of the video clips are determined, and the clarity detection threshold range is determined according to the minimum value and the maximum value.

2. The method for processing video high definition based on image replacement according to claim 1, characterized in that: The step of capturing a target object through a first camera of the user terminal to obtain first video data, and simultaneously capturing the target object through a second camera of the user terminal to obtain a plurality of first image data, specifically includes: Sending a first control instruction to the first camera through a control module of the user terminal, and simultaneously sending a second control instruction to the second camera; In response to the first control instruction, the target object is photographed by the first camera to obtain the first video data, and first shooting time information of the first video data is determined; In response to the second control instruction, the target object is continuously captured by the second camera at a preset time interval to obtain a plurality of first image data, and second shooting time information corresponding to each of the first image data is determined.

3. The method for processing video high definition based on image replacement according to claim 1, characterized in that: The step of determining the definition evaluation value of each video frame of the first video data by using the Sobel operator specifically includes: Extracting a plurality of video frames according to the first video data; Calculate the edge gradient value of each pixel in the video frame by using the Sobel operator; The sum of squares of edge gradient values ​​of all pixels in the video frame is determined as a clarity evaluation value of the video frame.

4. The method for processing video high definition based on image replacement according to claim 2, characterized in that: The step of determining that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced specifically includes: Determining, according to the first shooting time information, the shooting moment of the blurred video frame as the first shooting moment; determining, according to the second shooting time information, a shooting time of each of the first image data as a second shooting time; The first image data corresponding to the second shooting time closest to the first shooting time is determined as the image to be replaced.

5. A method for processing video high definition based on image replacement according to any one of claims 1 to 4, characterized in that: The step of replacing the blurred video frame in the first video data with the image to be replaced to obtain the second video data specifically includes: In response to a video preview operation of the user terminal, playing the first video data through the user terminal; When the blurred video frame is played, the playback is paused and the screen is reduced, and then the corresponding image to be replaced pops up on the playback interface, so that the user terminal displays the blurred video frame and the image to be replaced to the user; In response to an image replacement operation of the user terminal, the blurred video frame is replaced by the image to be replaced to generate the second video data.

6. A video high-definition processing system based on image replacement, characterized in that: include: A shooting module, used to shoot a video of a target object through a first camera of a user terminal to obtain first video data, and simultaneously to continuously capture the target object through a second camera of the user terminal to obtain a plurality of first image data; a clarity detection module, configured to perform clarity detection on the first video data and determine a number of blurred video frames of the first video data; a module for determining an image to be replaced, configured to determine that the first image data corresponding to the shooting moment of the blurred video frame is the image to be replaced; A video frame replacement module, used to replace the blurred video frame in the first video data with the image to be replaced to obtain second video data; The clarity detection module is specifically used for: Determine a definition evaluation value of each video frame of the first video data by using a Sobel operator, and determine a definition detection threshold range of the first video data; Determine that the video frame whose clarity evaluation value is not within the clarity detection threshold range is the blurred video frame; The determining of the definition detection threshold range of the first video data specifically includes: Performing video interception on the first video data to obtain a plurality of video clips; Performing image sampling on the video clip to obtain a plurality of corresponding sample video frames, and determining the grayscale average of pixels at the same position in each of the sample video frames, and then determining the background image corresponding to the video clip according to the grayscale average; Calculate the edge gradient value of each pixel in the background image by using the Sobel operator; Determine the sum of squares of edge gradient values ​​of all pixels in the background image as the average clarity value of the corresponding video clip; The minimum value and the maximum value of the clarity average value of each of the video clips are determined, and the clarity detection threshold range is determined according to the minimum value and the maximum value.

7. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the video high-definition processing method based on image replacement as described in any one of claims 1 to 5 are realized.

8. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the video high-definition processing method based on image replacement as described in any one of claims 1 to 5.

Images