Video processing method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a video processing method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: processing video frame information according to video processing type information corresponding to the video frame information, and obtaining target video frame information.

Description

Technical Field

[0001] This application relates to the field of video processing, specifically to a video processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Video consists of consecutive video frames, and video processing includes processing these video frames. Currently, the processing of video frames mainly depends on their content. Summary of the Invention

[0003] This application provides a video processing method, apparatus, electronic device, and computer-readable storage medium.

[0004] In a first aspect, embodiments of this application provide a video processing method, including:

[0005] Obtain the video information to be processed, which includes information from at least two video frames.

[0006] For any video frame in the video information to be processed, determine the video processing type information corresponding to that video frame.

[0007] The video frame information is processed using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0008] Secondly, embodiments of this application also provide a video processing apparatus, including:

[0009] The acquisition module is used to acquire video information to be processed, which includes information from at least two video frames.

[0010] The determination module is used to determine the video processing type information corresponding to any video frame information in the video information to be processed;

[0011] The processing module is used to process the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0012] Thirdly, embodiments of this application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the video processing method described above.

[0013] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the video processing method described above.

[0014] Fifthly, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of embodiments of this application. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a scene illustrating the video processing method provided in an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating the video processing method provided in an embodiment of this application;

[0018] Figure 3 This is a flowchart illustrating the determination of the first fusion global weight and the second fusion global weight in the video processing method provided in this application embodiment;

[0019] Figure 4 This is a schematic diagram of the structure of the video processing apparatus provided in the embodiments of this application;

[0020] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This application provides a video processing method, apparatus, electronic device, and computer-readable storage medium. Specifically, the electronic device includes a terminal device, which can be a desktop computer, laptop computer, mobile phone, tablet computer, or other similar devices.

[0023] Please see Figure 1 Taking the execution of video processing methods by terminal devices as an example, the specific execution process of the video processing methods is as follows:

[0024] Terminal device 10 acquires video information to be processed, which includes at least two video frame information. For any video frame information, terminal device 10 determines the video processing type information corresponding to the video frame information and processes the video frame information using the video processing type information to obtain the target video frame information corresponding to the video frame information.

[0025] It is understood that the embodiments of this application determine the corresponding video processing type information through video frame information, and process the video frame information based on the video processing type information to improve the accuracy of video processing.

[0026] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0027] Please see Figure 2 , Figure 2 This is a flowchart illustrating the video processing method provided in an embodiment of this application. The specific flow of the video processing method can be as follows:

[0028] 101. Obtain the video information to be processed, which includes at least two video frame information.

[0029] It should be noted that the video information to be processed is video information containing information from multiple video frames, including visual content, audio content, and possible metadata. Visual content refers to the images and scenes in the video, such as a video composed of a series of consecutive image frames (i.e., frames of images in the video); audio content refers to the sounds and music in the video; and metadata includes information such as the video's title, description, creation time, and encoding method.

[0030] The video can be any static or dynamic video. For example, it can be a video with constantly changing scene content or a video with unchanging scene content. Correspondingly, the video information to be processed can contain continuously static target objects, such as static buildings or plants, or it can contain dynamic target objects, such as walking people or animals. In this embodiment of the application, the video information to be processed can be captured by recording or broadcasting.

[0031] It should be noted that video frame information includes the image frames that make up the video and the depth information corresponding to each image frame. Each pixel in the image frame corresponds to a depth information, which is the depth value of that pixel. This depth value is the distance from the point of the object in the real scene corresponding to that pixel to the camera.

[0032] 102. For any video frame in the video information to be processed, determine the video processing type information corresponding to that video frame.

[0033] It should be noted that the video frame information is the video frame information determined by "any video frame information in the video information to be processed", that is, the video frame information to be processed in the current embodiment of this application.

[0034] It is understood that video processing type information refers to the processing type information that processes the video frame information. This processing type information may include smoothing, segmentation, enhancement, compression, recognition, or transformation, etc.

[0035] In particular, by determining the video processing type information corresponding to each video frame, it is helpful to process the video frame information based on the video processing type information, thereby improving the accuracy of the video frame information processing.

[0036] 103. Process the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0037] Since the video processing type information is determined based on the current video frame information, processing the video frame information based on the corresponding video processing type information can not only meet the processing requirements of the video frame information, but also improve the accuracy of the video frame information processing.

[0038] In summary, the embodiments of this application determine the corresponding video processing type information through video frame information, and process the video frame information based on the corresponding video processing type information, thereby improving the accuracy of video processing.

[0039] In this process, the processing of video frame information differs from that of a single image. For example, the processing of a single image primarily relies on its content, focusing on the accuracy of the processing. However, the processing of video frame information requires consideration of the temporal stability of each video frame. For instance, consecutive video frames should exhibit temporal consistency to minimize visual differences between individual frames and other frames. Therefore, the processing of video frame information often incorporates references or bases from other video frame information. Optionally, in some embodiments of this application, the step "For any given video frame, determine the corresponding video processing type information" includes:

[0040] For any video frame in the video information to be processed, determine the first video frame information from the video information to be processed based on the video frame information;

[0041] Identify the first content information and the second content information of the first video frame;

[0042] The video processing type information corresponding to the video frame information is determined based on the first content information and the second content information.

[0043] It should be noted that the first video frame information refers to other video frame information in the video information to be processed, excluding the video frame information itself. It is understood that, due to the strong correlation between consecutive video frames, using adjacent video frame information as a processing reference for the current video frame information has high reliability and accuracy. Therefore, in this embodiment, the first video frame information mainly refers to other video frame information that is temporally adjacent to the current video frame information (the video frame information to be processed in this embodiment). For example, the first video frame information is the video frame information of the preceding frame, or the preceding two or more frames, etc. In some scenarios, the first video frame information can also be the video frame information of the following frame, or the following two or more frames, of the video frame information to be processed.

[0044] Among them, content information refers to the content information of the image frame corresponding to the video frame information, such as the scene content contained in the image frame. Since the content of consecutive video frame information is usually related, for example, the content of the longer video frames in a video shot for a certain scene is the same or similar, the first content information and the second content information corresponding to the first video frame information can help to analyze the video processing type information corresponding to the video frame information, and realize the analysis of the video processing type information corresponding to the video frame information based on the content of the video frame information.

[0045] Since the relevance or reference value of video frame information during scene switching is low, the video processing type information of the video frame information can also be determined based on whether a scene switching occurs. That is, optionally, in some embodiments of this application, the step "determining the video processing type information corresponding to the video frame information based on the first content information and the second content information" includes:

[0046] Determine the scene switching status information based on the first content information and the second content information;

[0047] The video processing type information corresponding to the video frame information is determined based on the scene switching status information.

[0048] The scene switching status information indicates whether a scene switch has occurred, including scene switching and no scene switch. This scene switching status information can be determined using statistical histograms. For example, for the first content information corresponding to the video frame information, a first histogram is obtained by statistically analyzing pixel values; similarly, for the second content information corresponding to the first video frame information, a second histogram is obtained by statistically analyzing pixel values. The content difference is determined by comparing the first and second histograms. Based on a comparison of this content difference with a threshold, it is determined whether a scene switch has occurred between the first video frame information and the video frame information to be processed. For example, if the content difference is greater than the threshold, it is considered that a scene switch has occurred between the first video frame information and the video frame information to be processed, thus obtaining the scene switching status information containing the scene switch information.

[0049] Among them, determining the video processing type information of video frames by using scene switching state information helps improve the accuracy of video processing type information.

[0050] In this embodiment, smoothing video frame information is used as an example. Smoothing refers to processing video frame information frame by frame in the video information to be processed, in order to reduce noise, jitter, or other unnatural visual effects in the video frame information, thereby improving the viewing experience. For example, the current video frame information to be processed can be smoothed by using information from other adjacent video frames.

[0051] Therefore, in this embodiment, for the currently unprocessed video frame information, a first video frame information adjacent to the currently unprocessed video frame information is selected to process the currently unprocessed video frame information. It is understood that, generally, the more video frame information included in the first video frame information, the more accurate the smoothing processing of the currently unprocessed video frame information will be. However, since only first video frame information with consistent scene content is relevant during smoothing processing, this embodiment determines the specific type of smoothing based on whether a scene switch occurs between the first video frame information and the currently unprocessed video frame information. That is, zero, one, two, or more other video frame information are selected to perform smoothing processing on the currently unprocessed video frame information based on whether a scene switch occurs between them.

[0052] For example, taking the video frame information of the previous few frames of the video frame information to be processed (corresponding to the current frame T) as an example, if the scene of the previous frame (T-1) is inconsistent with the scene of the current video frame information to be processed (i.e., a scene change occurs), then there is no need to perform smoothing processing on the current video frame information to be processed; if the scene of the current frame (T-1) is consistent with the scene of the current video frame information to be processed, but the scene of the previous two frames (T-2) is inconsistent with the scene of the current video frame information to be processed, then the previous frame (T-1) is selected to perform smoothing processing on the current frame (T), i.e., two-frame smoothing processing; if the scene of the previous frame (T-1), the previous two frames (T-2), and the previous three frames (T-3) are all consistent with the scene of the current video frame information to be processed, then the previous frame (T-1), the previous two frames (T-2), and the previous three frames (T-3) can be selected to perform smoothing processing on the current video frame information to be processed, i.e., three-frame smoothing processing. Correspondingly, based on the requirement of smoothing accuracy, the video frame information of the previous three or more frames with the same scene can be selected to smooth the current video frame information.

[0053] That is, optionally, in some embodiments of this application, the first video frame information includes first video frame sub-information and second video frame sub-information, the scene switching state information includes first scene switching state sub-information between the first video frame sub-information and the video frame information, and second scene switching state sub-information between the second video frame sub-information and the video frame information, and the step "determining the video processing type information corresponding to the video frame information based on the scene switching state information" includes:

[0054] If the first scene switching state sub-information meets the first preset scene condition, then the first processing type information is used as the video processing type information; and / or,

[0055] If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the first preset scene condition, then the second processing type information is used as the video processing type information; and / or,

[0056] If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information also satisfies the second preset scene condition, then the third processing type information will be used as the video processing type information.

[0057] Specifically, the first video frame sub-information is the video frame information of the frame preceding the current video frame information (T-1), and the second video frame sub-information is the video frame information of the two frames preceding the current video frame information (T-2). The specific type of smoothing processing for the current video frame information varies depending on the different scene switching results.

[0058] For example, in the embodiments of this application, the first preset scene condition includes scene switching status information as scene switching. Then, "the first scene switching status sub-information satisfies the first preset scene condition" can be understood as a scene switching has occurred between the first video frame sub-information and the video frame information to be processed. Therefore, the first processing type information is no smoothing processing.

[0059] For example, if the first preset scene condition includes scene switching status information as scene switching and the second preset scene condition includes scene switching status information as scene not switching, then "the first scene switching status sub-information satisfies the second preset scene condition, and the second scene switching status sub-information satisfies the first preset scene condition" can be understood as no scene switching has occurred between the first video frame sub-information and the video frame information to be processed, while a scene switching has occurred between the second video frame sub-information and the video frame information. Therefore, the second processing type information is two-frame smoothing processing, that is, smoothing processing is performed on the video frame information to be processed based on the first video sub-information.

[0060] For example, if the first preset scene condition includes scene switching status information as scene switching and the second preset scene condition includes scene switching status information as scene not switching, then "the first scene switching status sub-information satisfies the second preset scene condition, and the second scene switching status sub-information satisfies the second preset scene condition" can be understood as no scene switching has occurred between the first video frame sub-information and the video frame information to be processed, and no scene switching has occurred between the second video frame sub-information and the video frame information. Therefore, the third processing type information is three-frame smoothing processing, that is, smoothing the video frame information to be processed is performed together based on the first video sub-information and the second video sub-information.

[0061] During video playback, the depth information for the same scene should be consistent. That is, if the object is static and there is no depth stretching during video shooting, the visual depth of the same object in multiple consecutive video frames should be consistent.

[0062] However, for depth information calculated through methods such as depth estimation, it is difficult to maintain temporal stability between consecutive video frames. For example, differences in depth information between adjacent video frames can cause flickering during playback. To address this issue, this embodiment selects adjacent video frames from the video information to be processed to perform depth smoothing on the current video frame information, thereby maintaining consistency in visual depth across consecutive video frames during playback.

[0063] Therefore, in this embodiment of the application, for the currently unprocessed video frame information, a first video frame information adjacent to the current video frame information is selected, and the depth information of the first video frame information is used to smooth the depth information of the currently unprocessed video frame information. That is, optionally, in some embodiments of this application, the step "processing the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information" includes:

[0064] The video processing type information corresponding to the video frame information is used to process the depth information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0065] For example, taking the video frame information of the previous few frames of the video frame information to be processed (corresponding to the current frame T) as an example, if the scene of the previous frame (T-1) is inconsistent with the scene of the current video frame information to be processed (i.e., a scene change occurs), then there is no need to smooth the depth information corresponding to the current video frame information to be processed; if the scene of the current frame (T-1) is consistent with the scene of the current video frame information to be processed, but the scene of the previous two frames (T-2) is inconsistent with the scene of the current video frame information to be processed, then the depth information of the previous frame (T-1) is selected to smooth the depth information of the current frame (T), i.e., two-frame smoothing; if the scene of the previous frame (T-1), the previous two frames (T-2), and the previous three frames (T-3) is consistent with the scene of the current video frame information to be processed, then the depth information of the previous frame (T-1), the previous two frames (T-2), and the previous three frames (T-3) can be selected to smooth the depth information of the current video frame information to be processed, i.e., three-frame smoothing. Correspondingly, based on the requirement for accuracy of the smoothing results, the depth information of the previous three or more video frames with the same scene can be selected to smooth the depth information of the current video frame.

[0066] It is understood that smoothing is generally obtained by fusing multiple reference objects and their corresponding weights. For example, after determining the specific type of smoothing based on video processing type information, the information of each video frame participating in smoothing is determined based on this specific type. Then, the depth information of each video frame is fused using the weights corresponding to each video frame information to obtain the final depth information. That is, optionally, in some embodiments of this application, the step "processing the depth information corresponding to the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information" includes:

[0067] The second video frame information is determined from the video information to be processed based on the video processing type information corresponding to the video frame information;

[0068] The fusion weight corresponding to the second video frame information is determined based on the video processing type information corresponding to the first video frame information.

[0069] Based on the fusion weight, the first depth information of the second video frame information and the second depth information of the video frame information are fused to obtain the fused depth information.

[0070] The target video frame information corresponding to this video frame information is generated based on the fused depth information.

[0071] In this approach, determining the fusion weights based on video processing type information improves the accuracy of smoothing the depth information corresponding to video frame information. For example, in the case of two-frame smoothing, the weights of the second video frame information and the current video frame information to be processed can be 0.5 each. In the case of three-frame smoothing, the fusion weights corresponding to each video frame information can be determined based on the magnitude of the depth information between the three video frame information.

[0072] Optionally, to further improve the accuracy of depth information smoothing, global weights between the depth information of each video frame can be configured at the overall level of video frame information, and local weights can also be configured at the pixel level corresponding to the depth information of the video frame information during smoothing. That is, optionally, in some embodiments of this application, the step "processing the depth information corresponding to the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information" includes:

[0073] The second video frame information is determined from the video information to be processed based on the video processing type information corresponding to the video frame information;

[0074] The first fusion global weight and the second fusion global weight of the second video frame information are determined based on the video processing type information corresponding to the video frame information.

[0075] The first fusion local weight of the second video frame information is determined based on the first depth information and the second depth information of the second video frame information, and a preset fixed value is set as the second fusion local weight of the video frame information.

[0076] According to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight, the first depth information and the second depth information are subjected to deep fusion processing to obtain the fused depth information.

[0077] The target video frame information corresponding to this video frame information is generated based on the fused depth information.

[0078] Specifically, for different video processing type information, the second video frame information participating in the smoothing process is determined according to the specific smoothing type corresponding to it. That is, optionally, in some embodiments of this application, the step "determining the second video frame information from the video information to be processed according to the video processing type information corresponding to the video frame information" includes:

[0079] If the video processing type information corresponding to the video frame information includes the first processing type information, then the second video frame information is set to empty;

[0080] If the video processing type information corresponding to the video frame information includes the second processing type information, then the video frame information preceding the video frame information in the video information to be processed will be used as the second video frame information.

[0081] If the video processing type information corresponding to the video frame information includes a third processing type information, then the first two video frames of the video frame information in the video information to be processed will be used as the second video frame information.

[0082] For example, when the video processing type information includes the first processing type information, i.e. no smoothing processing is performed, then the second video frame information is not needed, i.e. the second video frame information is empty; when the video processing type information includes the second processing type information, i.e. two-frame smoothing processing is performed, then the previous video frame information of the current video frame information to be processed is used as the second video frame information; when the video processing type information includes the third processing type information, i.e. three-frame smoothing processing is performed, then the first two (including T-1 and T-2) video frames of the current video frame information to be processed are both used as the first video frame information.

[0083] Understandably, using both global and local weights to comprehensively control the smoothing of depth information helps improve the accuracy of the smoothing results.

[0084] Among them, for local weights at the pixel level, the greater the difference in depth between the front and back of a pixel at the same location, the greater the probability of motion of that pixel. Therefore, determining local weights at the pixel level based on depth information helps to improve the accuracy of local weights.

[0085] Specifically, the following sections will describe the global weights and local weights separately.

[0086] Regarding the explanation of global weights, in detail:

[0087] When only two frames are involved in smoothing, the global weights corresponding to the depth information of the two video frames can be 0.5 each. For three-frame or multi-frame smoothing, the differences in depth information between each video frame need to be considered. For example, in the case of three-frame smoothing, if the depth information of the current (T) video frame is consistent with the depth information of the previous (T-1) video frame, and the depth information of the current (T) video frame is consistent with the depth information of the two previous (T-2) video frames, then the weights of the depth information of the three video frames can be the same, for example, 1 / 3 respectively. If the depth information of the current (T) video frame to be processed differs significantly from the depth information of the previous (T-1) video frame, and the depth information of the current (T) video frame to be processed differs significantly from the depth information of the two previous (T-2) video frames, then the depth information of (T-1) and (T-2) is mainly used as the adjustment direction, that is, the weights of the depth information of (T-1) and (T-2) video frames are increased, and the weights of the current video frame to be processed are decreased.

[0088] For example, in this embodiment of the application, if the video processing type information corresponding to the video frame information meets the preset type conditions, then the second video frame information includes the third video frame sub-information and the fourth video frame sub-information, the first fusion global weight includes the first sub-weight of the third video frame sub-information and the second sub-weight of the fourth video frame information, and the first depth information includes the first depth sub-information corresponding to the third video frame sub-information and the second depth sub-information corresponding to the fourth video frame information.

[0089] The first fusion global weight and the second fusion global weight of the second video frame information are determined based on the video processing type information corresponding to the video frame information, including:

[0090] The scene type information corresponding to the video frame information is determined based on the first depth sub-information corresponding to the third video frame sub-information, the second depth sub-information corresponding to the fourth video frame sub-information, and the second depth information corresponding to the video frame information.

[0091] If the scene type information meets the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the first preset strategy; and / or,

[0092] If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information satisfies the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the second preset strategy; and / or,

[0093] If the scene type information meets the second scene type condition, and the scene type information corresponding to the third video frame sub-information does not meet the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the third preset strategy.

[0094] In this embodiment of the application, the preset type condition refers to the type of three-frame smoothing processing. That is, "the video processing type information meets the preset type condition" can be understood as the video processing type information being three-frame smoothing processing.

[0095] It should be noted that the scene type information is based on the scene type reflected by the depth information corresponding to three video frames. Specifically, the scene type information corresponding to the video frame information is determined with video frame information (T) as the current frame and the two preceding frames (T-1 and T-2) as the second video frame information. The scene type information corresponding to the third video frame sub-information (T-1) is determined with the third video frame sub-information as the current frame and the two preceding frames (T-2 and T-3) as the second video frame information. For example, in this embodiment, the scene type information includes static flickering scenes and non-static flickering scenes. A static flickering scene refers to a scene where the scene content does not change, but the depth changes significantly. A non-static flickering scene refers to a scene other than a static flickering scene.

[0096] Correspondingly, based on the scene type information during three-frame smoothing, the global weights corresponding to each video frame can be determined according to the corresponding preset strategy.

[0097] For example, if the first scene type condition includes a static flickering scene, then "scene type information satisfies the first scene type condition" means "the scene type information corresponding to the video frame information satisfies the first scene type condition". This can be understood as the case where the video frame information to be processed forms a static flickering scene with its two preceding video frames. Accordingly, the first preset strategy is to reduce the second fusion weight information corresponding to the current video frame information, and increase the first sub-weight and the second sub-weight. For example, the second fusion weight information, the first sub-weight, and the second sub-weight can be 2 / 30, 19 / 30, and 9 / 30, respectively.

[0098] In cases where a non-static flickering scene is formed between the video frame to be processed and its two preceding video frames, the scene type information corresponding to the preceding video frame can be used to comprehensively analyze the overall scene of the video frame to be processed. For example, in this embodiment, if the scene type information corresponding to the video frame to be processed is a non-static flickering scene, and the scene type information corresponding to the preceding video frame is a static flickering scene, it indicates that the current video frame to be processed is likely to contain a moving target object. Therefore, the proportion of the global weight corresponding to the current video frame to be processed can be slightly larger, while the proportion of the global weight corresponding to its two preceding video frames can be slightly smaller. The second preset strategy is to adjust the second fusion weight information to be slightly larger and the first sub-weight and the second sub-weight to be slightly smaller. For example, the second fusion weight information, the first sub-weight, and the second sub-weight can be set to 13 / 30, 9 / 30, and 8 / 30, respectively.

[0099] Similarly, in this embodiment, if the scene type information corresponding to the video frame information to be processed is a non-static flickering scene, and if the scene type information corresponding to the previous video frame information to be processed is also a non-static flickering scene, it indicates that the current video frame information to be processed is likely to contain a moving target object. In this case, the second fusion weight information, the first sub-weight, and the second sub-weight can be basically close. That is, the third preset strategy is to adjust the second fusion weight information, the first sub-weight, and the second sub-weight on average. For example, the second fusion weight information, the first sub-weight, and the second sub-weight are set to 1 / 3, 1 / 3, and 1 / 3, respectively.

[0100] Specifically, the analysis for both the existence of a target object that has begun to move and the existence of a target object that is already in motion includes:

[0101] First, there is no scene switching between the current video frame information to be processed and the previous two video frame information;

[0102] Secondly, if the scene type information corresponding to the current video frame information to be processed is a non-static flickering scene, and the scene type of the previous video frame information corresponding to the current video frame information to be processed is a static flickering scene, it can be understood that the situation has changed from static to dynamic. Therefore, it belongs to the scene that has started to move, that is, there is a target object that has started to move in the scene.

[0103] Similarly, if the scene type information corresponding to the current video frame information to be processed is a non-static flickering scene, and the scene type of the previous video frame information corresponding to the current video frame information to be processed is also a non-static flickering scene, it can be understood that the situation is changing from dynamic to dynamic. Therefore, it belongs to a continuously moving scene, that is, there is a moving or continuously moving target object in the scene.

[0104] In this embodiment of the application, in order to more accurately determine the scene type information corresponding to the current video frame information to be processed, a judgment condition for whether it belongs to a static flickering scene is also provided. For example, the judgment criteria for a static flickering scene are:

[0105] 1) Calculate the depth difference between the current video frame information to be processed and the third video frame sub-information to obtain diff1, and calculate the depth difference between the current video frame information to be processed and the fourth video frame sub-information to obtain diff2.

[0106] 2) If the following three conditions are met simultaneously, it is considered a static flickering scene.

[0107] a) diff1>4 accounts for more than 30 / 255 of the entire graph;

[0108] b) The proportion of diff2>4 in the entire image exceeds 39 / 255;

[0109] c) The proportion of diff1>20 in the entire image is less than 11 / 255.

[0110] In condition c, diff1 > 20 indicates a very large difference in estimated depth values, which helps determine the presence of large motion. A ratio less than 11 / 255 indicates a very small proportion of large motion. Combined, condition c means that the proportion of large motion in the image is very small, at which point the image content is considered essentially unchanged, i.e., the image is a static scene.

[0111] Furthermore, in condition a, diff1>4 indicates a slight difference in depth values, and a full-image proportion exceeding 30 / 255 indicates a relatively large area. Combined, these represent a large proportion of regions exhibiting slight depth differences, thus indicating a flickering scene. Similarly, condition b indicates a flickering scene, meaning that the frame image and the first image, as well as the frame image and the second image, are both in flickering scenes.

[0112] Combining conditions a, b, and c, we find that the current video frame information, the third video frame sub-information, and the fourth video frame sub-information are in a static flickering scenario.

[0113] In summary, for the global weights, the global weights corresponding to each video frame participating in the smoothing process are determined based on the video processing type information corresponding to the video frame information to be processed, thereby improving the accuracy of the global weights.

[0114] Regarding the explanation of local weights, in detail:

[0115] Since the depth difference of pixels at the same location can reflect the local scene change, the local weight of a pixel can be determined based on the difference in depth information corresponding to pixels at the same location. That is, optionally, in some embodiments of this application, the step "determining the first fused local weight of the second video frame information based on the first depth information of the second video frame information and the second depth information of the video frame information" includes:

[0116] Calculate the pixel depth difference between corresponding pixels in the first depth information and the second depth information;

[0117] The pixel weights corresponding to each pixel in the first depth information are determined based on the depth difference of each pixel and the preset depth value.

[0118] The first fusion local weights of the second video frame information are generated by using the weights of each pixel.

[0119] The preset depth value is a pre-set depth value; for example, the default preset depth value is 38.

[0120] It is understandable that a large difference in pixel depth indicates a significant change in depth between frames, which also means that there is likely motion in this area. Therefore, if the absolute value of the pixel depth difference is greater than or equal to the preset depth value, the pixel weight of the pixel in the first depth information can be reduced; if the absolute value of the pixel depth difference is less than the preset depth value, the pixel weight of the pixel in the first depth information can be increased.

[0121] Understandably, such adjustments are practical. For example, a large pixel depth difference indicates a high probability of motion in that area, so depth changes are normal, meaning the depth in that area should not be smoothed out. Therefore, the pixel weight corresponding to the current depth information to be processed should be larger, while the pixel weights of the previous frames should be smaller. Conversely, a small pixel depth difference indicates minimal depth change between frames, suggesting minimal motion. In this case, the depth of the previous frames can be used for smoothing, by appropriately increasing the pixel weights of those frames. The second fusion local weight of the pixels in the current video frame information is set to a fixed value, for example, 1. This primarily adjusts the overall fusion local weight by regulating the first fusion local weight of the pixels in the second video frame information.

[0122] In general, the role of local weights (including the first and second fused local weights) is to smooth out areas with little motion and smooth out areas with some motion. This is because the depth should remain relatively constant in areas with little motion, while the depth should naturally vary in areas with motion, thus requiring less smoothing.

[0123] For example, the pixel weight Mask at the pixel level can be calculated using the following formula:

[0124] Mask T-i =Clip(thr-|depth) T-i -depth T |+8,0,thr) / thr

[0125] thr=30

[0126] Here, different indices of Mask correspond to the pixel weights of different image pixels, and Clip in the formula represents the truncation operation, i.e., the matrix thr-|depth. T-i -depth T Values ​​less than 0 in |+8 are uniformly assigned the value 0; values ​​greater than thr in the matrix are uniformly assigned the value thr. Here, T represents the current timestamp, i.e., the current video frame information to be processed. It can be understood that substituting i=0 into the formula yields the pixel weight of the pixel point for the current video frame information to be processed, which is 1.

[0127] It is understandable that after calculating the global weights and local weights, the current video frame information to be processed can be smoothed based on the global weights and local weights. That is, optionally, in some embodiments of this application, the step "performing depth fusion processing on the first depth information and the second depth information according to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight to obtain fused depth information" includes:

[0128] For the corresponding first pixel and second pixel in the first depth information and the second depth information;

[0129] The first local depth value is obtained by multiplying the first depth value of the first pixel with the first global fusion weight and the first local fusion weight; the second local depth value is obtained by multiplying the second depth value of the second pixel with the second global fusion weight and the second local fusion weight.

[0130] The pixel fusion depth value corresponding to the first pixel and the second pixel is obtained by adding the first local depth value and the second local depth value;

[0131] The fused depth information is generated based on the fusion depth value of each pixel.

[0132] For example, the depth value of a pixel in the smoothing process in the embodiments of this application. smooth The calculation formula can be expressed as:

[0133]

[0134] Here, Weight represents the global weight, and different indices correspond to different subweights of the image. For example, taking the current (T) video frame information to be processed as an example, Weight T The second fusion global weight corresponding to the current (T) video frame information to be processed, Weight T-1 It is the first fusion global weight corresponding to the previous video frame information of the current video frame information to be processed.

[0135] It is understandable that the above calculation formula, after simplification or modification, can be expressed as:

[0136]

[0137] That is, after determining the first global fusion weight, the second global fusion weight, the first local fusion weight, and the second local fusion weight of each pixel in the video frame information in the aforementioned manner, the pixel fusion depth value of the pixel after depth smoothing is obtained by substituting it into the above formula. The smoothed fused depth information is then obtained by using the pixel fusion depth value of each pixel after depth smoothing.

[0138] In summary, this application's embodiments utilize multi-frame depth information for smoothing to improve the temporal stability of video depth. By configuring global weights at the image level and local weights at the pixel level, the rationality and accuracy of depth smoothing are enhanced. This avoids the problem of temporal flickering during video playback and improves the temporal stability of video depth.

[0139] To facilitate understanding of the embodiments of this application, the determination process of the first and second fusion global weights of this scheme is described below with a more specific processing flow. The first fusion global weight includes a first sub-weight corresponding to the video frame information of the previous frame and a second sub-weight corresponding to the information of the previous two video frames. For details, please refer to... Figure 3 As shown, Figure 3 This is a flowchart illustrating the determination of the first fusion global weight and the second fusion global weight in the video processing method provided in this application embodiment, wherein the process includes:

[0140] 201. Determine whether the current video frame information has changed scene from the previous video frame information. If it has changed, proceed to step 202. If it has not changed, proceed to step 203.

[0141] 202. Keep the depth information of the current video frame unchanged;

[0142] 203. Determine whether the current video frame information has changed scene compared with the previous two video frame information. If it has changed, proceed to step 204; otherwise, proceed to step 205.

[0143] The first two video frame information are the video frame information corresponding to the previous video frame information. For example, if the timestamp corresponding to the current video frame information is T, then the timestamp corresponding to the previous video frame information is T-1, and the timestamp corresponding to the first two video frame information is T-2.

[0144] 204. Generate a second fusion global weight for the current video frame information and a first fusion global weight for the previous video frame information, wherein the first fusion global weight and the second fusion global weight are the same and each accounts for half of the weight.

[0145] At this time, the first fusion global weight includes the first sub-weight and the second sub-weight. For example, the first sub-weight is 15 / 30 and the second fusion global weight is 15 / 30. At this time, the second sub-weight of the first two video frame information is 0 by default.

[0146] 205. Determine whether the current video frame information, the previous video frame information, and the two previous video frame information belong to a static flickering scene. If yes, proceed to step 206; otherwise, proceed to step 207.

[0147] 206. Generate a second fusion global weight for the current video frame information, a first sub-weight for the previous video frame information, and a second sub-weight for the previous two video frame information according to the first preset strategy. At this time, the first preset strategy includes a higher first sub-weight, a moderate second sub-weight, and a lower second fusion global weight.

[0148] At this point, the first fusion global weight includes the first sub-weight and the second sub-weight. For example, the first sub-weight is 19 / 30, the second sub-weight is 9 / 30, and the second fusion global weight is 2 / 30.

[0149] 207. Determine if the previous scene was a static flickering scene. If yes, proceed to step 208; otherwise, proceed to step 209.

[0150] 208. Generate a second fusion global weight for the current video frame information, a first sub-weight for the previous video frame information, and a second sub-weight for the previous two video frame information according to the second preset strategy. At this time, the second preset strategy includes a higher second fusion global weight and slightly smaller first and second sub-weights.

[0151] At this point, the first fusion global weight includes the first sub-weight and the second sub-weight. For example, the first sub-weight is 9 / 30, the second sub-weight is 8 / 30, and the second fusion global weight is 13 / 30.

[0152] 209. Generate a second fusion global weight for the current video frame information, a first sub-weight for the previous video frame information, and a second sub-weight for the two previous video frame information according to the third preset strategy. At this time, the third preset strategy includes the same first sub-weight, second sub-weight, and second fusion global weight.

[0153] At this point, the first fusion global weight includes the first sub-weight and the second sub-weight. For example, the first sub-weight is 10 / 30, the second sub-weight is 10 / 30, and the second fusion global weight is 10 / 30.

[0154] In summary, the embodiments of this application improve the accuracy of the global weights used for image depth fusion by analyzing scene changes and depth changes to configure the first and second global fusion weights.

[0155] To facilitate better implementation of the video processing method of this application, this application also provides a video processing apparatus based on the above-described video processing method. The meaning of the third target term is the same as in the above-described video processing method; specific implementation details can be found in the description of the method embodiments.

[0156] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of the video processing apparatus provided in this application, wherein the video processing apparatus may include:

[0157] The acquisition module 301 is used to acquire video information to be processed, which includes at least two video frame information.

[0158] The determining module 302 is used to determine the video processing type information corresponding to any video frame information in the video information to be processed;

[0159] Processing module 303 is used to process the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0160] Preferably, for any video frame in the video information to be processed, the determining module 302 determines the video processing type information corresponding to that video frame, including:

[0161] For any video frame in the video information to be processed, determine the first video frame information from the video information to be processed based on the video frame information;

[0162] Identify the first content information and the second content information of the first video frame;

[0163] The video processing type information corresponding to the video frame information is determined based on the first content information and the second content information.

[0164] Preferably, the determining module 302 determines the video processing type information corresponding to the video frame information based on the first content information and the second content information, including:

[0165] Determine the scene switching status information based on the first content information and the second content information;

[0166] Determine the video processing type information corresponding to the video frame information based on the scene switching status information;

[0167] Preferably, the determining module 302 determines the video processing type information corresponding to the video frame information based on the scene switching status information, including:

[0168] The first video frame information includes first video frame sub-information and second video frame sub-information. The scene switching state information includes first scene switching state sub-information between the first video frame sub-information and the video frame information, and second scene switching state sub-information between the second video frame sub-information and the video frame information.

[0169] If the first scene switching state sub-information meets the first preset scene condition, then the first processing type information is used as the video processing type information; and / or,

[0170] If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the first preset scene condition, then the second processing type information is used as the video processing type information; and / or,

[0171] If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the second preset scene condition, then the third processing type information will be used as the video processing type information.

[0172] Preferably, the processing module 303 processes the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information, including:

[0173] The video processing type information corresponding to the video frame information is used to process the depth information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0174] Preferably, the processing module 303 processes the depth information corresponding to the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information, including:

[0175] The second video frame information is determined from the video information to be processed based on the video processing type information corresponding to the video frame information;

[0176] The first fusion global weight and the second fusion global weight of the second video frame information are determined based on the video processing type information corresponding to the video frame information.

[0177] The first fusion local weight of the second video frame information is determined based on the first depth information and the second depth information of the second video frame information, and a preset fixed value is set as the second fusion local weight of the video frame information.

[0178] According to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight, the first depth information and the second depth information are subjected to deep fusion processing to obtain the fused depth information.

[0179] Generate the target video frame information corresponding to the video frame information based on the fused depth information;

[0180] Preferably, the processing module 303 determines the first fusion global weight and the second fusion global weight of the second video frame information based on the video processing type information corresponding to the video frame information, including:

[0181] If the video processing type information corresponding to the video frame information meets the preset type conditions, then the second video frame information includes the third video frame sub-information and the fourth video frame sub-information, the first fusion global weight includes the first sub-weight of the third video frame sub-information and the second sub-weight of the fourth video frame information, and the first depth information includes the first depth sub-information corresponding to the third video frame sub-information and the second depth sub-information corresponding to the fourth video frame information.

[0182] The scene type information corresponding to the video frame information is determined based on the first depth sub-information corresponding to the third video frame sub-information, the second depth sub-information corresponding to the fourth video frame sub-information, and the second depth information corresponding to the video frame information.

[0183] If the scene type information meets the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the first preset strategy; and / or,

[0184] If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information satisfies the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the second preset strategy; and / or,

[0185] If the scene type information meets the second scene type condition, and the scene type information corresponding to the third video frame sub-information does not meet the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the third preset strategy.

[0186] Preferably, the processing module 303 determines the first fusion local weight of the second video frame information based on the first depth information and the second depth information of the second video frame information, including:

[0187] Calculate the pixel depth difference between corresponding pixels in the first depth information and the second depth information;

[0188] The pixel weights corresponding to each pixel in the first depth information are determined based on the depth difference of each pixel and the preset depth value.

[0189] The first fusion local weights are generated by using the weights of each pixel to generate the information of the second video frame;

[0190] Preferably, the processing module 303 determines the second video frame information from the video information to be processed based on the video processing type information corresponding to the video frame information, including:

[0191] If the video processing type information corresponding to the video frame information includes the first processing type information, then the second video frame information is set to empty;

[0192] If the video processing type information corresponding to the video frame information includes the second processing type information, then the video frame information preceding the video frame information in the video information to be processed will be used as the second video frame information.

[0193] If the video processing type information corresponding to the video frame information includes the third processing type information, then the first two video frames of the video frame information in the video information to be processed will be used as the second video frame information.

[0194] Preferably, the processing module 303 performs depth fusion processing on the first depth information and the second depth information according to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight to obtain fused depth information, including:

[0195] For the corresponding first pixel and second pixel in the first depth information and the second depth information;

[0196] The first local depth value is obtained by multiplying the first depth value of the first pixel with the first global fusion weight and the first local fusion weight; the second local depth value is obtained by multiplying the second depth value of the second pixel with the second global fusion weight and the second local fusion weight.

[0197] The pixel fusion depth value corresponding to the first pixel and the second pixel is obtained by adding the first local depth value and the second local depth value;

[0198] The fused depth information is generated based on the fusion depth value of each pixel.

[0199] In this embodiment, the acquisition module 301 first acquires the video information to be processed, which includes at least two video frame information. Then, the determination module 302 determines the video processing type information corresponding to any video frame information in the video information to be processed. Subsequently, the processing module 303 processes the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0200] In summary, the embodiments of this application determine the corresponding video processing type information through video frame information, and process the video frame information based on the corresponding video processing type information, thereby improving the accuracy of video processing.

[0201] In this embodiment, multi-frame depth information is used for smoothing to improve the temporal stability of video depth. By configuring global weights at the image level and local weights at the pixel level, the rationality and accuracy of depth smoothing are enhanced. This avoids the problem of temporal flickering during video playback and improves the temporal stability of video depth.

[0202] In addition, this application also provides an electronic device, such as Figure 5 As shown, it illustrates the structural diagram of the electronic device involved in this application, specifically:

[0203] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0204] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 402, and calls data stored in the memory 402, to perform various functions and process data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0205] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0206] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0207] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0208] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402, thereby implementing the steps in any of the video processing methods provided in this application, specifically:

[0209] This application embodiment obtains video information to be processed, which includes at least two video frame information. For any video frame information in the video information to be processed, the video processing type information corresponding to the video frame information is determined, and the video frame information is processed using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

[0210] In summary, the embodiments of this application determine the corresponding video processing type information through video frame information, and process the video frame information based on the corresponding video processing type information, thereby improving the accuracy of video processing.

[0211] In this embodiment, multi-frame depth information is used for smoothing to improve the temporal stability of video depth. By configuring global weights at the image level and local weights at the pixel level, the rationality and accuracy of depth smoothing are enhanced. This avoids the problem of temporal flickering during video playback and improves the temporal stability of video depth.

[0212] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0213] Therefore, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the video processing methods provided in this application.

[0214] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0215] Since the instructions stored in the computer-readable storage medium can execute any of the steps in the video processing provided in this application, the beneficial effects that any of the video processing methods provided in this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0216] The foregoing has provided a detailed description of a video processing method, apparatus, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0217] It should be noted that, in the specific embodiments of this application, data related to scene content information, depth information, and timing information corresponding to video frame information, etc., are required to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A video processing method, characterized in that, include: Obtain video information to be processed, which includes at least two video frame information; For any video frame in the video information to be processed, determine the video processing type information corresponding to that video frame; The video frame information is processed using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

2. The video processing method according to claim 1, characterized in that, For any video frame in the video information to be processed, determining the video processing type information corresponding to that video frame includes: For any of the video frame information in the video information to be processed, determine the first video frame information from the video information to be processed based on the video frame information; Identify the first content information and the second content information of the first video frame information; The video processing type information corresponding to the video frame information is determined based on the first content information and the second content information.

3. The video processing method according to claim 2, characterized in that, The step of determining the video processing type information corresponding to the video frame information based on the first content information and the second content information includes: Determine scene switching status information based on the first content information and the second content information; The video processing type information corresponding to the video frame information is determined based on the scene switching status information.

4. The video processing method according to claim 3, characterized in that, The first video frame information includes first video frame sub-information and second video frame sub-information, and the scene switching state information includes first scene switching state sub-information between the first video frame sub-information and the video frame information, and second scene switching state sub-information between the second video frame sub-information and the video frame information. The step of determining the video processing type information corresponding to the video frame information based on the scene switching state information includes: If the first scene switching state sub-information satisfies the first preset scene condition, then the first processing type information is used as the video processing type information; and / or, If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the first preset scene condition, then the second processing type information is used as the video processing type information; and / or, If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the second preset scene condition, then the third processing type information is used as the video processing type information.

5. The video processing method according to claim 1, characterized in that, The step of processing the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information includes: The video processing type information corresponding to the video frame information is used to process the depth information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information.

6. The video processing method according to claim 5, characterized in that, The step of processing the depth information corresponding to the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information includes: The second video frame information is determined from the video information to be processed based on the video processing type information corresponding to the video frame information; The first fusion global weight and the second fusion global weight of the second video frame information are determined based on the video processing type information corresponding to the video frame information. The first fusion local weight of the second video frame information is determined based on the first depth information and the second depth information of the second video frame information, and a preset fixed value is set as the second fusion local weight of the video frame information. According to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight, the first depth information and the second depth information are subjected to deep fusion processing to obtain fused depth information; The target video frame information corresponding to the video frame information is generated based on the fused depth information.

7. The video processing method according to claim 6, characterized in that, If the video processing type information corresponding to the video frame information meets the preset type conditions, then the second video frame information includes the third video frame sub-information and the fourth video frame sub-information, the first fusion global weight includes the first sub-weight of the third video frame sub-information and the second sub-weight of the fourth video frame information, and the first depth information includes the first depth sub-information corresponding to the third video frame sub-information and the second depth sub-information corresponding to the fourth video frame information. The step of determining the first fusion global weight and the second fusion global weight of the second video frame information based on the video processing type information corresponding to the video frame information includes: The scene type information corresponding to the video frame information is determined based on the first depth sub-information corresponding to the third video frame sub-information, the second depth sub-information corresponding to the fourth video frame sub-information, and the second depth information corresponding to the video frame information. If the scene type information meets the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the first preset strategy; and / or, If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information satisfies the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the second preset strategy; and / or, If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information does not satisfy the first scene type condition, then the first sub-weight, the second sub-weight, and the second fusion global weight are determined according to the third preset strategy.

8. The video processing method according to claim 6, characterized in that, The step of determining the first fusion local weight of the second video frame information based on the first depth information and the second depth information of the second video frame information includes: Calculate the pixel depth difference between each corresponding pixel in the first depth information and the second depth information; The pixel weights corresponding to each pixel in the first depth information are determined based on the pixel depth differences and preset depth values. The first fusion local weights of the second video frame information are generated by the weights of each pixel.

9. The video processing method according to claim 6, characterized in that, The step of determining the second video frame information from the video information to be processed based on the video processing type information corresponding to the video frame information includes: If the video processing type information corresponding to the video frame information includes the first processing type information, then the second video frame information is set to empty; If the video processing type information corresponding to the video frame information includes second processing type information, then the video frame information preceding the video frame information in the video information to be processed is taken as the second video frame information. If the video processing type information corresponding to the video frame information includes a third processing type information, then the first two video frames of the video frame information in the video information to be processed are taken as the second video frame information.

10. The video processing method according to claim 6, characterized in that, The step of performing depth fusion processing on the first depth information and the second depth information according to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight to obtain fused depth information includes: For the corresponding first pixel and second pixel in the first depth information and the second depth information; Calculate the product of the first depth value of the first pixel with the first fusion global weight and the first fusion local weight to obtain the first local depth value; and calculate the product of the second depth value of the second pixel with the second fusion global weight and the second fusion local weight to obtain the second local depth value. The first local depth value and the second local depth value are added together to obtain the pixel fusion depth value corresponding to the first pixel and the second pixel; The fused depth information is generated based on the fusion depth values ​​of each pixel.

11. A video processing apparatus, characterized in that, include: The acquisition module is used to acquire video information to be processed, which includes at least two video frame information. The determining module is used to determine the video processing type information corresponding to any video frame information in the video information to be processed; The processing module is used to process the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information. Preferably, for any video frame in the video information to be processed, the determining module determines the video processing type information corresponding to that video frame, including: For any of the video frame information in the video information to be processed, determine the first video frame information from the video information to be processed based on the video frame information; Identify the first content information and the second content information of the first video frame information; The video processing type information corresponding to the video frame information is determined based on the first content information and the second content information; Preferably, the determining module determines the video processing type information corresponding to the video frame information based on the first content information and the second content information, including: Determine scene switching status information based on the first content information and the second content information; The video processing type information corresponding to the video frame information is determined based on the scene switching status information. Preferably, the determining module determines the video processing type information corresponding to the video frame information based on the scene switching status information, including: The first video frame information includes first video frame sub-information and second video frame sub-information, and the scene switching state information includes first scene switching state sub-information between the first video frame sub-information and the video frame information, and second scene switching state sub-information between the second video frame sub-information and the video frame information. If the first scene switching state sub-information satisfies the first preset scene condition, then the first processing type information is used as the video processing type information; and / or, If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the first preset scene condition, then the second processing type information is used as the video processing type information; and / or, If the first scene switching state sub-information satisfies the second preset scene condition, and the second scene switching state sub-information satisfies the second preset scene condition, then the third processing type information is used as the video processing type information. Preferably, the processing module processes the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information, including: The video processing type information corresponding to the video frame information is used to process the depth information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information. Preferably, the processing module processes the depth information corresponding to the video frame information using the video processing type information corresponding to the video frame information to obtain the target video frame information corresponding to the video frame information, including: The second video frame information is determined from the video information to be processed based on the video processing type information corresponding to the video frame information; The first fusion global weight and the second fusion global weight of the second video frame information are determined based on the video processing type information corresponding to the video frame information. The first fusion local weight of the second video frame information is determined based on the first depth information and the second depth information of the second video frame information, and a preset fixed value is set as the second fusion local weight of the video frame information. According to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight, the first depth information and the second depth information are subjected to deep fusion processing to obtain fused depth information; Based on the fused depth information, the target video frame information corresponding to the video frame information is generated; Preferably, the processing module determines the first fusion global weight and the second fusion global weight of the second video frame information based on the video processing type information corresponding to the video frame information, including: If the video processing type information corresponding to the video frame information meets the preset type conditions, then the second video frame information includes the third video frame sub-information and the fourth video frame sub-information, the first fusion global weight includes the first sub-weight of the third video frame sub-information and the second sub-weight of the fourth video frame information, and the first depth information includes the first depth sub-information corresponding to the third video frame sub-information and the second depth sub-information corresponding to the fourth video frame information. The scene type information corresponding to the video frame information is determined based on the first depth sub-information corresponding to the third video frame sub-information, the second depth sub-information corresponding to the fourth video frame sub-information, and the second depth information corresponding to the video frame information. If the scene type information meets the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the first preset strategy; and / or, If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information satisfies the first scene type condition, then the first sub-weight, the second sub-weight, and the second fused global weight are determined according to the second preset strategy; and / or, If the scene type information satisfies the second scene type condition, and the scene type information corresponding to the third video frame sub-information does not satisfy the first scene type condition, then the first sub-weight, the second sub-weight, and the second fusion global weight are determined according to the third preset strategy. Preferably, the processing module determines the first fusion local weight of the second video frame information based on the first depth information and the second depth information of the second video frame information, including: Calculate the pixel depth difference between each corresponding pixel in the first depth information and the second depth information; The pixel weights corresponding to each pixel in the first depth information are determined based on the pixel depth differences and preset depth values. The first fusion local weights of the second video frame information are generated by the weights of each pixel; Preferably, the processing module determines the second video frame information from the video information to be processed based on the video processing type information corresponding to the video frame information, including: If the video processing type information corresponding to the video frame information includes the first processing type information, then the second video frame information is set to empty; If the video processing type information corresponding to the video frame information includes second processing type information, then the video frame information preceding the video frame information in the video information to be processed is taken as the second video frame information. If the video processing type information corresponding to the video frame information includes a third processing type information, then the first two video frame information of the video frame information in the video information to be processed shall be used as the second video frame information. Preferably, the processing module performs depth fusion processing on the first depth information and the second depth information according to the first fusion global weight, the second fusion global weight, the first fusion local weight, and the second fusion local weight to obtain fused depth information, including: For the corresponding first pixel and second pixel in the first depth information and the second depth information; Calculate the product of the first depth value of the first pixel with the first fusion global weight and the first fusion local weight to obtain the first local depth value; and calculate the product of the second depth value of the second pixel with the second fusion global weight and the second fusion local weight to obtain the second local depth value. The first local depth value and the second local depth value are added together to obtain the pixel fusion depth value corresponding to the first pixel and the second pixel; The fused depth information is generated based on the fusion depth values ​​of each pixel.

12. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the video processing method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the video processing method as described in any one of claims 1-10.

Images