Video transition identification method and device based on monocular depth field, equipment and medium

Abstract

The invention relates to the technical field of computer vision, solves the problem of how to improve the stability of inter-frame information and the accuracy of video transition identification in the prior art, and provides a video transition identification method and device based on a monocular depth field, equipment and a medium. The method comprises the following steps: preprocessing a video stream to be subjected to transition identification to obtain a plurality of frames of first video images; performing monocular depth field calculation on each first video image to obtain a plurality of frames of second depth field images; and according to each second depth field image, determining whether video transition occurs, and obtaining a video transition determination result. The advanced monocular vision algorithm is adopted to evaluate the depth map of each frame, and the transition position in the video is automatically calibrated by comparing the depth field differences of continuous frames, so that the requirement of manual inspection is reduced, the automation level and the editing efficiency of video editing are improved, and the video editing efficiency is improved. And powerful technical support is provided for professional and amateur video producers.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a video transition recognition method and device based on monocular depth field, equipment and medium. BACKGROUND

[0002] With the development of mobile Internet, users increasingly use videos to record and share their lives. In order to improve the quality of videos, it is often necessary to edit the videos. When connecting multiple video segments, it is often necessary to add transition effects to enhance the coherence of the videos. At present, the selection of transition effects usually depends on the analysis of the pixel differences of the first and last frames of the video segments, but this method is complex and resource-consuming. The recognition of video transitions can provide a basis for video processing such as segmentation and reorganization, and thus video transition recognition is of great significance in the field of video processing.

[0003] The commonly used video transition recognition methods in the prior art include scene change methods and motion information methods. The scene change methods include color histogram difference and edge detection. The color histogram difference uses the color histogram of the frame to detect the change of the scene, thereby recognizing the transition. The edge detection identifies the scene change by analyzing the change of the edge features in the frame. However, the scene change method often has difficulty in capturing gradual transition, because the change in such transition is relatively smooth and may not result in significant scene change. The motion information methods include optical flow analysis and motion feature change. The optical flow analysis detects whether there is significant motion change by analyzing the optical flow information between frames, thereby recognizing the transition. The motion feature change calculates the change of motion features in the frame, such as the motion vector field. In complex scenes, the motion information may be affected by multiple factors, thereby affecting the recognition accuracy. The monocular depth field technology can provide depth information for each pixel, which helps to more comprehensively understand the scene structure and the spatial relationship of objects, thereby more accurately detecting the transition, especially in the case of complex background or scene change. At the same time, the monocular depth field can better capture the gradual features of soft transition, because the depth information helps to identify the subtle spatial changes between frames. On the other hand, by using depth information, the monocular depth field can reduce the misjudgment caused by factors such as light changes and camera shaking, thereby improving the accuracy of transition recognition. Although the monocular depth field overcomes the shortcomings of the above technologies, it still has some defects. The monocular depth field is a depth estimation for a single frame, which will result in inconsistent depth information between frames, unstable inter-frame information, and thus affect the video transition recognition.

[0004] Therefore, how to improve the stability of inter-frame information and the accuracy of video transition recognition is a problem to be solved. SUMMARY

[0005] Therefore, the embodiment of the present application provides a monocular depth field-based video transition identification method, device, equipment and medium to solve the problem of how to improve the stability of inter-frame information and the accuracy of video transition identification in the prior art.

[0006] In a first aspect, the embodiment of the present application provides a monocular depth field-based virtual fence setting method, which comprises:

[0007] Preprocessing a video stream to be identified for transition to obtain a plurality of first video images;

[0008] Monocular depth field calculation is performed on each of the first video images to obtain a plurality of second depth field images;

[0009] According to each of the second depth field images, it is determined whether a video transition has occurred to obtain a video transition determination result.

[0010] Preferably, the preprocessing of the video stream to be identified for transition to obtain a plurality of first video images comprises:

[0011] The video stream is decomposed to obtain a plurality of original video images;

[0012] Corner extraction and corner number statistics are performed on each of the original video images to obtain the Harris corner number of each original video image;

[0013] According to a preset image frame number, mean value calculation is performed on each of the Harris corner numbers to obtain a Harris corner number average value;

[0014] According to the Harris corner number of each original video image, in combination with the product of the Harris corner number average value and a preset blur threshold, it is determined whether each of the original video images is blurred to obtain a determination result;

[0015] According to the determination result, all blurred images in the original video images are removed to obtain the first video images.

[0016] Preferably, the depth field calculation according to the first video images to obtain a plurality of second depth field images comprises:

[0017] Monocular depth detection is performed on the first video images to obtain a plurality of first depth field images;

[0018] Median filtering is performed on the first depth field images to obtain a plurality of second depth field images.

[0019] Preferably, the determination according to each of the second depth field images whether a video transition has occurred to obtain a video transition determination result comprises:

[0020] According to the second depth field image, a plurality of depth field pixel coordinates are obtained;

[0021] The depth field pixel coordinates of adjacent second depth field images are subjected to difference calculation to obtain a depth field difference image;

[0022] According to a preset binary threshold, the depth field difference image is subjected to binary processing to obtain a binary image;

[0023] The binary image is subjected to connected domain analysis to obtain a connected domain analysis result;

[0024] According to the connected domain analysis result and each second depth field image, it is determined whether a video transition has occurred to obtain a video transition determination result.

[0025] Preferably, the determination according to the connected domain analysis result and each second depth field image whether a video transition has occurred to obtain a video transition determination result comprises:

[0026] According to the connected domain analysis result, a plurality of connected domains are obtained, and the area and the center of gravity of each connected domain are obtained;

[0027] According to the area of each connected domain and a preset area threshold, each connected domain is screened to obtain a plurality of significant change regions;

[0028] The number of regions of the significant change regions is obtained, and the center of gravity change average of the significant change regions is calculated according to the center of gravity of each connected domain;

[0029] According to the number of regions of the significant change regions and the center of gravity change average, it is determined according to each second depth field image whether a video transition has occurred to obtain a video transition determination result.

[0030] Preferably, the obtaining of the number of regions of the significant change regions and the center of gravity change average of the significant change regions comprises:

[0031] According to the center of gravity of each connected domain, a target center of gravity of each significant change region is obtained;

[0032] According to the target center of gravity of the significant change region in the binary image of the adjacent frame, a difference calculation is performed to obtain a center of gravity change value corresponding to each significant change region;

[0033] According to the number of regions of the significant change regions and the center of gravity change value, the center of gravity change average is calculated.

[0034] Preferably, the video transition judgment result is obtained according to whether a video transition occurs or not according to the region quantity of the significant change region and the gravity center change average value, and the video transition judgment result comprises:

[0035] A preset significant change region quantity threshold value and a gravity center change average value threshold value are obtained.

[0036] The region quantity of the significant change region is compared with the quantity threshold value, and a first comparison result is obtained.

[0037] The gravity center change average value is compared with the average value threshold value, and a second comparison result is obtained.

[0038] According to the first comparison result and the second comparison result, when the region quantity is greater than the quantity threshold value and the gravity center change average value is greater than the average value threshold value, the video transition judgment result is that a video transition occurs.

[0039] In a second aspect, an embodiment of the present application provides a video transition recognition device based on monocular depth field, and the device comprises:

[0040] A preprocessing module pre-processes a video stream to be recognized to obtain a plurality of first video images.

[0041] A depth field calculation module calculates monocular depth field of each of the first video images to obtain a plurality of second depth field images.

[0042] A transition judgment module judges whether a video transition occurs or not according to each of the second depth field images to obtain a video transition judgment result.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, which comprises at least one processor, at least one memory, and computer program instructions stored in the memory, and when the computer program instructions are executed by the processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0044] In a fourth aspect, an embodiment of the present application provides a storage medium, and computer program instructions are stored in the storage medium, and when the computer program instructions are executed by the processor, the method of the first aspect in the above-mentioned embodiment is implemented.

[0045] In summary, the beneficial effects of the present application are as follows:

[0046] The video transition recognition method, apparatus, device, and medium based on monocular depth field provided in this invention preprocesses the video stream to be recognized to obtain several frames of first video images. By performing blurry image preprocessing, blurry images are discarded to ensure that only clear images are used for subsequent processing. Monocular depth field calculation is performed on each of the first video images to obtain several frames of second depth field images. The depth field map provides distance information of various points in the environment and accurately describes the three-dimensional structure of the environment. Monocular depth field algorithms estimate depth information from two-dimensional images, reducing the need for multi-camera systems or other expensive sensors, thus lowering hardware costs. Simultaneously, the algorithm can perform real-time calculations while processing depth information, avoiding complex hardware configurations and data processing requirements. Based on each second depth field image, it determines whether a video transition has occurred, obtaining the transition judgment result. It calculates the depth difference between adjacent frames and performs binarization processing, using the average change in centroid and the number of significantly changing regions as judgment conditions to accurately identify transitions. This helps ensure smooth and natural video, improves the final video viewing experience, and enhances the accuracy of transition point identification and processing, improving the video transition effect. It effectively judges video transition situations, reduces the need for manual inspection, improves the automation level and efficiency of video editing, and provides reliable technical support for video editing and processing. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0048] Figure 1 This is a schematic diagram of the overall process of the video transition recognition method based on monocular depth field in Embodiment 1 of the present invention;

[0049] Figure 2 This is a schematic diagram of the video transition recognition device based on monocular depth field in Embodiment 2 of the present invention;

[0050] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0051] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the drawings and embodiments, in order to make the purposes, technical solutions and advantages of the present application clearer. It should be understood that the specific embodiments described herein are configured only to explain the present application and are not configured to limit the present application. The present application can be implemented without some of these specific details for those skilled in the art. The following description of the embodiments is merely to provide a better understanding of the present application by showing examples of the present application.

[0052] It should be noted that, in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or equipment including the elements.

[0053] It should be noted that all actions of acquiring signals, information or data in the present application are carried out in compliance with the corresponding data protection regulations and policies of the place, and with the authorization given by the owner of the corresponding device.

[0054] Embodiment 1

[0055] Please refer to Figure 1 The embodiment of the present application provides a video transition identification method and device based on monocular depth field, equipment and medium, the method comprises:

[0056] S1, pre-processing the video stream to be identified for transition to obtain a plurality of first video images;

[0057] Blur images can cause depth prediction errors and data confusion, and data enhancement methods can enhance the robustness of the model by generating more training data, such as simulating blur, and reducing sensitivity to abnormal data. Noise filtering methods use spatial filtering techniques to smooth depth data by post-processing depth maps, reducing image prediction errors and data confusion. In contrast to the above two methods, the method of rejecting blurred images can directly avoid using low-quality data in depth prediction, fundamentally reducing depth estimation errors caused by blur and dependence on subsequent noise filtering processing, improving data quality, ensuring that only clear images are used for subsequent processing, and thus improving the prediction accuracy of the model.

[0058] Preferably, the video stream to be identified for scene transition is preprocessed to obtain a plurality of first video images, which includes:

[0059] S11, decompose the video stream to obtain a plurality of original video images;

[0060] The original video image can be used as an input for various purposes, such as training a deep learning model or performing detailed video content analysis, providing basic visual information of the video, and being suitable for various subsequent analysis and processing

[0061] Specifically, the obtained video stream is disassembled into image frames to obtain a plurality of original video images.

[0062] S12, corner point extraction and corner point number statistics are performed on each original video image to obtain the Harris corner point number of each original video image;

[0063] Harris corner point detection is an algorithm for detecting feature points in an image, which has a simple implementation method and high stability for feature points. The corner point is a point in the image with significant changes. The core idea is to analyze the gray scale changes of the image using local windows to detect the position of the corner point. The specific process is as follows: first, calculate the gray gradient of each pixel point in the image; then, calculate the covariance matrix of each pixel point based on the image gradient to describe the local changes of the image; then, calculate the corner point response function R to determine whether the pixel point is a corner point. The corner point response function R is expressed as follows:

[0064] R = det(M) - k · (trace(M)) 2

[0065] Where det(M) is the determinant of the matrix, trace(M) is the sum of the diagonal elements of the matrix, and k is an empirical constant, usually taking a value between 0.04 and 0.06; finally, the result of the corner point response function is thresholded to obtain the final corner point position.

[0066] Specifically, the Harris corner point detection algorithm is used to extract the Harris corner points in each original video image and count the number of Harris corner points.

[0067] S13, according to a preset image frame number, the average value of the Harris corner point number is calculated to obtain the average value of the Harris corner point number;

[0068] According to the sliding window with a sliding step of 7 frames, the average value H_avg of the Harris corner point number of the continuous 7 frames is calculated, which is convenient for subsequent processing.

[0069] S14, judging whether each of the original video images is blurred according to the Harris corner point number of each of the original video images, in combination with a product of the Harris corner point number average value and a preset blur threshold, to obtain a judgment result;

[0070] The number of corner points of a blurred image is usually only half of that of a normal image, and it is difficult to extract Harris corner points. Too few corner points affect the overall recognition result, and therefore, the part of the image needs to be removed for accurate transition judgment in the subsequent process.

[0071] Specifically, if the Harris corner point number H_new of the original video image of the current frame is less than Thresh*H_avg, it is considered that the new original video image of the current frame is a blurred image, wherein Thresh is a blur ratio threshold, and is generally less than 0.5.

[0072] S15, removing all blurred images in the original video images according to the judgment result to obtain the first video images;

[0073] The original video images are screened to ensure that only high-quality image data is used for further analysis. Clear images provide more detailed and feature information, which can more accurately capture information in the video scene and help improve the performance of the model.

[0074] Specifically, according to the judgment result, all blurred images in the original video images are removed to obtain the first video images.

[0075] S2, monocular depth field calculation is performed on each of the first video images to obtain a plurality of second depth field images;

[0076] The depth field image provides distance information of each point in the environment and accurately describes the three-dimensional structure of the environment. The monocular depth field algorithm estimates depth information from a two-dimensional image, which can reduce the need for a multi-camera system or other expensive sensors, thereby reducing hardware costs. At the same time, the algorithm can process depth information in real time, avoiding complex hardware configuration and data processing requirements.

[0077] Traditional depth estimation algorithms mainly include geometric methods for recovering depth information from two-dimensional images. Geometric methods mainly include structured light and stereo vision. The structured light method obtains depth information by projecting a light pattern. The stereo vision method estimates depth by using images captured by two cameras. The problem of traditional depth estimation methods is that they have low universality, usually need to be adjusted for specific scenes or methods, and may need more training and adjustment in complex scenes.

[0078] Preferably, the depth field calculation according to the first video images to obtain a plurality of second depth field images comprises:

[0079] S21, monocular depth detection is performed on the first video image to obtain a plurality of frames of first depth field images; depth information allows a detailed environment model to be created, so that the system can understand the object and spatial relationship in the scene, which is very important for complex environment analysis and virtual reality applications. Accurate depth field images enable the cut detection system to better identify and calibrate the cut point, efficiently process a large amount of video data, and improve editing efficiency and automation level.

[0080] The monocular depth field algorithm based on deep neural network MobileDepth overcomes the shortcomings of traditional depth estimation algorithms, has low hardware cost, fast prediction speed, and good performance in the field of depth estimation. MobileDepth proposes a fully convolutional network, uses a regularized network (RegNet) as an encoder, a split-connection block as a decoder, and additional skip connections between the encoder and the decoder, so that the decoder can obtain high-level features. The structure of the split-connection block can reduce time delay and obtain higher performance in evaluation indicators compared with ordinary depth separable convolution blocks. The algorithm evaluation is evaluated by the following performance indicators:

[0081] Root Mean Squared error (RMSE)

[0082]

[0083] Average Relative Error (REL)

[0084]

[0085] where y i is the i-th actual depth value, is the i-th predicted depth value, and n is the total number of actual depth values.

[0086] Specifically, the first video image is input into the MobileDepth algorithm model, and the model will predict the depth field of each frame of the first video image based on visual clues (such as texture gradient, shape, and occlusion relationship, etc.) in the input image to obtain a plurality of frames of first depth field images.

[0087] S22, median filtering is performed on the first depth field image to obtain a plurality of frames of second depth field images.

[0088] The median filter can effectively remove random noise in the image while preserving edge information, avoiding blurring of image details. The filtered depth image is smooth and clear, which helps to improve the accuracy of subsequent processing. By reducing noise, the obtained second depth field image more accurately reflects the actual depth information of the scene, which helps to improve the reliability of the depth data. While denoising, the edge details of the image are preserved, which is crucial for subsequent image analysis or calculation.

[0089] Specifically, the size of the filter window is determined, and the window size is determined according to the actual situation, and the window slides on the image; for each pixel in the window, the median of all pixel values in the window is calculated; the calculated median is used to replace the value of the center pixel of the window; the above operation is applied to each pixel in the image to obtain a plurality of second depth field images.

[0090] S3, determining whether a video transition occurs according to each of the second depth field images, to obtain a video transition determination result;

[0091] The commonly used video transition recognition methods in the prior art include scene change method and motion information method. The scene change method includes color histogram difference and edge detection. The color histogram difference uses the color histogram of the frame to detect the change of the scene, so as to recognize the transition. The edge detection identifies the scene change by analyzing the change of the edge features in the frame. However, the scene change method is often difficult to capture gradual transition, because the change in such transition is relatively smooth, and may not cause significant scene change. The motion information method includes optical flow analysis and motion feature change. The optical flow analysis detects whether there is significant motion change by analyzing the optical flow information between frames, and then recognizes the transition. The motion feature change calculates the change of the motion features in the frame, such as the motion vector field. In a complex scene, the motion information may be affected by multiple factors, thereby affecting the recognition accuracy.

[0092] Accurate recognition of the transition point helps to ensure smooth and natural video, and improves the viewing experience of the final video. The accuracy of recognizing and processing the transition point improves the transition effect of the video, so that the viewing experience of the audience is more coherent and natural. The depth information helps to identify the transition in a complex scene, and is more suitable for processing scene changes with multiple levels and depths compared with the traditional method.

[0093] Preferably, the determining whether a video transition occurs according to each of the second depth field images, to obtain a video transition determination result comprises:

[0094] S31, obtaining a plurality of depth field pixel coordinates according to the second depth field image;

[0095] Specifically, by extracting the depth value of the pixel coordinates, the depth information and details in the image can be accurately obtained, which helps further analysis or processing. Obtaining the pixel coordinates of these regions can help locate and identify the target region. For the second depth field image, calculate its depth field Dt(x, y), where t represents time, (x, y) represents the pixel position in the image, and obtain a number of depth field pixel coordinates.

[0096] S32, difference calculation is performed on the depth field pixel coordinates of the adjacent second depth field image to obtain a depth field difference image;

[0097] Specifically, by calculating the depth field difference, the significant change region in the scene, such as the video transition point or object movement, can be better identified and located. According to the depth field pixel coordinates, the depth field difference image between adjacent frames is calculated, and the calculation formula is as follows:

[0098] △D t (x,y)=|D t (x,y)-D t-1 (x,y)|

[0099] Where t is the current frame second depth field image, t-1 is the previous frame second depth field image, and the depth field difference image of adjacent frames is obtained.

[0100] S33, according to a preset binary threshold, the depth field difference image is binarized to obtain a binary image;

[0101] Specifically, the binary image marks the region with significant depth change as 1, so that these regions are more prominent in the image, facilitating further analysis and processing, reducing the complexity of data processing, and accurately locating the transition point or significant change region in the scene, thereby improving the accuracy of transition detection. In addition, the binary image also provides an intuitive way to show the depth change region, making visualization and understanding of these changes more intuitive, and the marking of significant change regions in the image facilitates further data labeling and analysis. The depth field difference image ΔDt(x, y) is binarized, a binary threshold τ is set, and a binary image Bt(x, y) is generated. The detailed setting expression is as follows:

[0102]

[0103] If the depth field difference image pixel value is greater than the binary threshold τ, the region is marked as 1, and if the depth field difference image pixel value is less than or equal to the binary threshold τ, the region is marked as 0, and the binary image is obtained.

[0104] S34, the binary image is subjected to connected component analysis to obtain a connected component analysis result;

[0105] Specifically, the binary image Bt(x, y) is subjected to connected component analysis, and the connected regions are labeled. Each pixel in the image is analyzed, and an 8-neighborhood is usually selected for analysis. If any point in the 8-neighborhood has the same pixel value as the center point, a connected component is formed with the center point. If none of the points has the same pixel value, the point is an isolated point and is discarded. The 8-neighborhood analysis is then performed again on the newly added pixel points, thereby labeling the connected regions in the entire image, denoted as {Ri}, where i represents the number of the connected region, and the area of each connected region |Ri| is calculated to obtain a plurality of first areas.

[0106] S35, according to the connected component analysis result, judging whether a video transition occurs according to each second depth field image, to obtain a video transition judgment result.

[0107] Specifically, connected component analysis refers to grouping pixel points with similar features (such as significant changes in depth value) and adjacent in space into the same connected region in the image processing process, and is commonly used for detecting object boundaries, motion regions, or significant change regions. In this scheme, the connected component is a significant change region extracted from the depth difference map, and the second depth field image is a depth information map in the video consecutive frames, which is used to analyze the foreground and background changes of objects in the video. The purpose of this step is to determine whether a transition occurs between the current frame and the previous frame by using the change characteristics in the depth information. Video transition usually represents the overall switching of two scenes, which can cause the depth values of a large range of regions in the depth map to change dramatically, thereby forming multiple or large-area connected components. Therefore, by further combining the analysis result of the connected components with the original depth image, it can be effectively determined whether it is a video transition or a local object movement. In the implementation process, the number, area, shape feature, etc. of all connected regions are obtained according to the connected component extraction result of the previous step; then, the overall depth distribution and change trend of the current frame (i.e. the second depth image) are combined, such as whether there are multiple large-area significant change regions, whether the change regions cover the main part of the picture, to determine whether a transition occurs. Rule judgment (such as the number of connected components exceeding a threshold, or the distribution of the center of mass of the connected regions being extensive) or through a trained classification model can be used for discrimination. By fusing the spatial structure analysis based on connected components and the overall scene perception ability of the original depth image, the accuracy of video transition detection is improved. Compared with the traditional method based on RGB pixel change, it is less susceptible to local object motion, illumination change, etc., and has stronger robustness and ability to be suitable for weak texture or night environment.

[0108] Preferably, the S35 comprises:

[0109] According to the connected component analysis result, a plurality of connected regions are obtained, and the area and center of mass of each connected region are obtained.

[0110] Specifically, the connected region refers to a region in the depth difference map in which adjacent pixel values exceed a certain change threshold and are continuous in space; the area represents the number of pixels in the region, and the center of gravity is the weighted average coordinate of the pixel position in the region, used to describe the geometric center of the region. For example, if the disappearance or appearance of an object between a frame and the previous frame causes a significant depth mutation, a change will form a connected region with a larger area and a clear position. By extracting the structural features of the connected domain, criteria are provided for subsequent judgment of whether it is a scene change; the transition is often accompanied by a dramatic change in the overall structure of the picture, and the corresponding number of connected regions is larger and the area distribution is larger, so obtaining the geometric features of these regions is an important basis for identifying transitions. In specific implementation, first, all connected domains are obtained based on the depth difference map using common region extraction algorithms such as Flood Fill or the connectedComponents function in OpenCV; then, the number of pixels in each connected region is calculated one by one to obtain its area, and the average of the row and column coordinates of all pixels in the region is taken as the center of gravity coordinate. The area and center of gravity data of all connected domains are used as intermediate variables for subsequent processing. Extracting the spatial feature information of the connected region converts the depth change from the pixel level to the structural level, greatly reducing the data dimension and enhancing the tolerance to local disturbances, thereby improving the reliability of subsequent transition recognition and judgment.

[0111] According to the area of each of the connected regions and a preset area threshold, the connected regions are screened to obtain a plurality of significant change regions;

[0112] Specifically, the significant change region refers to a connected region whose area exceeds a set threshold and is sufficient to reflect a significant change in the scene. The area threshold can be set according to the resolution and change amplitude of the actual video content, such as 1% of the total number of image pixels, to filter out false connected regions caused by small noise or local object jitter, with the purpose of excluding meaningless small regions caused by minor disturbances, edge errors or unstable depth prediction, and only retaining significant change regions closely related to transitions. This step is crucial to improving the accuracy of transition judgment, especially in complex dynamic backgrounds or low-texture environments. In implementation, the area of each connected region obtained in the previous step is compared with the set area threshold, and if the area of a certain region is below the threshold, it is excluded, otherwise it is included in the set of significant change regions. Through the area-based region filtering mechanism, the number and quality of regions participating in the subsequent judgment are effectively controlled, the probability of false judgment is reduced, and the algorithm calculation efficiency is guaranteed, laying a foundation for large-scale video scene processing.

[0113] The number of regions of the significant change regions is obtained, and the center of gravity change average of the significant change regions is calculated according to the center of gravity of each of the connected regions;

[0114] Specifically, the number of regions is the total number of significant change regions left after the previous step of screening, denoted as Nt. The mean change of gravity center refers to the average displacement of the geometric gravity center of these significant change regions between consecutive frames, which is used to quantify the overall change amplitude. This mean value reflects the overall shift degree of the picture structure in space, and the purpose is to comprehensively evaluate the number of significant change regions and the intensity of spatial position change as two key criteria for judging whether a video transition occurs. Scene switching usually shows that multiple regions change simultaneously and the position is widely distributed, so these two indicators have strong discrimination. In implementation, first, the number of significant change regions in the current frame Nt is counted; then, the Euclidean distance between the gravity center positions of each region in the current frame and the previous frame is calculated, and the gravity center displacement values of all regions are averaged to obtain the mean change of gravity center. The corresponding pairing can be achieved by buffering the gravity center coordinates of the previous frame and the current gravity center list. The introduction of dynamic evaluation of spatial change can identify not only changes but also changes in multiple locations simultaneously and on a large scale, thereby effectively distinguishing between local object motion and global scene switching, significantly improving the accuracy and practicality of transition detection.

[0115] Preferably, the acquiring the number of regions of the significant change regions, and calculating the mean change of gravity center of the significant change regions according to the gravity center of each of the connected regions comprises:

[0116] According to the gravity center of each of the connected regions, acquiring a target gravity center of each of the significant change regions;

[0117] Specifically, the target gravity center refers to the geometric center position of the significant change region in the image, usually represented by pixel coordinates (x, y). Since the significant change region is an effective connected domain selected by the area threshold, its gravity center has strong structural representativeness. The extraction of the target gravity center is the basis for subsequent calculation of the gravity center change. For each connected region identified as a significant change, a spatial position identifier is established, thereby providing data support for subsequent time series change analysis. In particular, when the scene undergoes large-scale structural changes, the gravity center positions of each region will also undergo significant displacement, so the reference positions of these regions in the current frame need to be clearly recorded. In the implementation process, all significant change regions are traversed, and the x coordinates and y coordinates of all pixel points in the region are averaged based on the x coordinates and y coordinates of all pixel points in the region to obtain the target gravity center of the region. For example, a region contains 100 pixels, the sum of the x coordinates is 13000, and the x gravity center of the region is 130. The target gravity center results of all regions are stored in an array or a mapping structure in the form of coordinates for cross-frame comparison. The geometric center that can be repeatedly positioned is calculated using the pixel distribution in the connected domain, which provides an accurate reference for subsequent region matching and displacement measurement, and is beneficial to eliminate jitter interference and enhance the structural stability of transition judgment.

[0118] According to the target barycenter of the significant change region in the binarized image of the adjacent frame, a difference value is calculated to obtain a barycenter change value corresponding to each significant change region;

[0119] Specifically, the barycenter change value is a measure of the spatial displacement of each significant change region between adjacent frames. Difference value calculation generally refers to the calculation process of Euclidean distance, which is used to determine whether the region has moved significantly between consecutive frames. By quantifying the spatial displacement, the dynamic behavior of the region is evaluated. When the scene transitions, the barycenter of each region tends to shift as a whole, rather than just a slight local shake, so the barycenter change value is an important basis for distinguishing between local motion and global transformation. In implementation, for each significant change region of the current frame, according to its position number or geometric approximation relationship, the corresponding significant change region in the previous frame is found, and then the barycenter coordinates of the region in the two frames are extracted, and the Euclidean distance (√[(x2-x1) 2 +(y2-y1)2]) is calculated as the barycenter change value of the region. To improve the robustness of matching, a matching distance threshold or region label identifier can also be introduced. The introduction of region-level spatial motion analysis avoids the sensitivity of pure pixel difference method to noise. Through structure matching and displacement measurement, the recognition ability of cross-frame dramatic change behavior is improved, providing key support for accurate detection of video transitions.

[0120] According to the number of regions of the significant change region and the barycenter change value, the barycenter change mean value is calculated.

[0121] Specifically, the barycenter change mean value refers to the average result of the barycenter change values of all significant change regions, which is a global quantification of the change intensity. Its unit is usually still in pixels or distance units, indicating that on average each significant region moves between adjacent frames. The displacement results of multiple regions are integrated into a single indicator to assist in determining whether a video transition has occurred. In actual scenarios, if the number of significant regions is large and the barycenter change mean value is large, it indicates that the overall structure of the picture has changed dramatically, and it is more likely to be a video transition behavior. The implementation process usually adds the barycenter change values of all significant change regions and then divides by the number of regions to obtain the mean value. For example, if the barycenter change values of 5 regions are 10, 12, 15, 8 and 14, the barycenter change mean value is (10+12+15+8+14) / 5=11.8. This mean value can be compared with a pre-set threshold as a quantitative basis for transition judgment, constructing a comprehensive indicator that takes into account change intensity and change range, reflecting not only whether there is dramatic displacement, but also whether it is the synchronous change of multiple regions, thereby improving the stability and discrimination accuracy of transition detection.

[0122] According to the number of regions of the significant change region and the barycenter change mean value, according to each second depth field image, it is determined whether a video transition has occurred, to obtain a video transition judgment result.

[0123] Specifically, multiple key indicators are integrated to determine whether a video transition occurs, the core basis of which is the number of significant change regions and the mean of the center of gravity change, and the second depth field image extracted from the time series, that is, the image frame used to identify changes after preprocessing. The video transition here refers to the case where the video content changes from one scene to another in time, which is common at the junction of two shots in film editing. Determining whether a transition occurs is a key basis for video structure understanding and content analysis, especially in intelligent video processing scenarios such as ad insertion, video summarization, and content review. On the basis of excluding non-transition factors such as background micro-changes and slight occlusions, the transition event is accurately identified through quantitative criteria; generally, if the number of significant change regions reaches a certain threshold, and the center of gravity of these regions has a large-scale overall shift in adjacent frames, it is likely that the picture has switched as a whole. In the implementation process, first, the number of significant change regions Nt and the mean of the center of gravity change Dt calculated in the previous step are obtained from the current frame, and then they are compared with the preset determination thresholds Nth and Dth. On this basis, the global histogram change degree or the regional difference form of the inter-frame depth map can also be introduced as auxiliary features to further enhance robustness. Through multi-dimensional numerical fusion and non-direct image comparison, structured transition detection is realized, avoiding the misjudgment problem of traditional RGB frame difference method in scenes such as occlusion and light change, and having good universality and anti-interference. It can be widely applied to video processing systems that require high-accuracy transition recognition.

[0124] Preferably, the video transition is determined according to the number of significant change regions and the mean of the center of gravity change, according to each of the second depth field images, to obtain a video transition determination result, which comprises:

[0125] Obtaining a preset number threshold of significant change regions and a mean threshold of center of gravity change;

[0126] Specifically, two thresholds are obtained: the significant change region number threshold Nth and the center of gravity change mean threshold Dth, which are usually determined by experiments or training samples and can reflect the change range in general scene switching. Then, the number of significant change regions Nt obtained by analyzing the current frame is compared with Nth to determine whether the number of change regions is sufficient; at the same time, the mean of the center of gravity change Dt is compared with Dth to determine whether there is a significant overall position shift. The two judgment processes output the first comparison result and the second comparison result, respectively.

[0127] Comparing the number of significant change regions with the number threshold to obtain a first comparison result;

[0128] comparing the center of gravity change mean value and the mean value threshold value, to obtain a second comparison result;

[0129] According to the first comparison result and the second comparison result, when the number of regions is greater than the number threshold value and the center of gravity change mean value is greater than the mean value threshold value, a video transition judgment result is obtained that a video transition has occurred.

[0130] Specifically, based on a decision strategy similar to a logical AND gate: only when Nt>Nth and Dt>DthD_t>D_{th} is it considered that the current frame has a video transition; otherwise, even if a certain index exceeds the threshold value, the system still considers it as a local dynamic, occlusion, micro-motion, and the like, non-transition behavior. This double-criterion mechanism effectively enhances the accuracy and robustness of the system, avoids misjudgment caused by relying on a single feature, and improves the reliability of transition recognition. The threshold value can be dynamically adjusted to adapt to different types of videos.

[0131] Embodiment 2

[0132] Please refer to Figure 2 The embodiment of the application provides a video transition recognition device based on a monocular depth field, which comprises:

[0133] A preprocessing module pre-processes a video stream to be transition-recognized to obtain a plurality of first video images;

[0134] A depth field calculation module calculates a depth field according to the first video images to obtain a plurality of second depth field images;

[0135] A transition judgment module judges whether a video transition has occurred according to each of the second depth field images to obtain a video transition judgment result.

[0136] Specifically, the device comprises a preprocessing module, a depth field calculation module, and a transition judgment module. The preprocessing module preprocesses a video stream to be identified to obtain a plurality of first video images. The depth field calculation module calculates a plurality of second depth field images according to the first video images. The transition judgment module judges whether a video transition has occurred according to the second depth field images to obtain a video transition judgment result. The device can discard blurred images by preprocessing the blurred images, ensuring that only clear images are used for subsequent processing. The depth field image provides distance information of each point in the environment, accurately describing the three-dimensional structure of the environment. The monocular depth field algorithm estimates depth information from a two-dimensional image, which can reduce the need for a multi-camera system or other expensive sensors, thereby reducing hardware costs. The algorithm can calculate in real time while processing depth information, avoiding complex hardware configuration and data processing requirements. The algorithm calculates the depth difference between adjacent frames and performs binary processing, and uses the barycenter change mean and the number of significant change regions as judgment conditions to accurately identify the transition. Accurate identification of the transition point helps to ensure smooth and natural video, improve the viewing experience of the final video, and improve the accuracy of identifying and processing the transition point, thereby improving the transition effect of the video, effectively judging the video transition, reducing the need for manual inspection, improving the automation level and editing efficiency of video editing, and providing reliable technical support for video editing and processing.

[0137] It should be noted that the modules and units in the device are one-to-one corresponding to the steps in the method, and therefore the specific embodiments of the device can refer to the embodiments of the method, which will not be described here.

[0138] Embodiment 3

[0139] In addition, the device can be combined with the method. Figure 1 The method can be implemented by an electronic device, and the method is characterized in that the method is implemented by the electronic device. Figure 3 The hardware structure of the electronic device is shown in the schematic diagram.

[0140] The electronic device can include a processor and a memory storing computer program instructions.

[0141] In particular, the processor can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or can be configured as one or more integrated circuits that implement embodiments of the present application.

[0142] The memory can include mass storage for data or instructions. By way of example, and not limitation, the memory can include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The memory can be removable and / or built-in (or fixed) in suitable cases. The memory can be internal or external in suitable cases. In particular embodiments, the memory is non-volatile semiconductor memory. In particular embodiments, the memory includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. The memory can have a different cost and / or performance profile than the storage.

[0143] The processor implements any one of the above-mentioned monocular depth field based video cut identification methods by reading and executing the computer program instructions stored in the memory.

[0144] In one example, the electronic device can further include a communication interface and a bus. Wherein, as shown in Figure 3 The processor 401, the memory 402, and the communication interface 403 are connected through the bus 410 and complete communication among each other.

[0145] The communication interface is mainly used to realize the communication among the modules, devices, units and / or equipment in the embodiments of the present application.

[0146] The bus includes hardware, software, or both, to couple components of the electronic device to each other and to couple components to other components in communication couplings. By way of example, and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, the bus can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0147] Embodiment 4

[0148] In addition, in combination with the video transition recognition method based on monocular depth field in one of the above embodiments, the present embodiment can provide a computer readable storage medium for implementation. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement any of the video transition recognition methods based on monocular depth field in the above embodiments.

[0149] In summary, the present embodiment provides a video transition recognition method based on monocular depth field, device, equipment and medium.

[0150] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0151] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0152] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from that in the embodiments, or several steps can be performed simultaneously.

[0153] The above description is merely a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the above-described system, modules and units for the convenience and brevity of description, and the corresponding processes in the foregoing method embodiments can be referred to, which will not be described here again. It should be understood that the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A video transition recognition method based on monocular depth field, characterized in that, The method includes: The video stream to be identified is preprocessed to obtain several frames of the first video image; Perform monocular depth field calculation on each of the first video images to obtain several frames of second depth field images; Based on each of the second depth field images, it is determined whether a video transition has occurred, and the video transition determination result is obtained.

2. The video transition recognition method based on monocular depth field according to claim 1, characterized in that, The preprocessing of the video stream to be identified to obtain several frames of the first video image includes: The video stream is decomposed to obtain several frames of original video images; Corner points are extracted and the number of corner points is counted for each of the original video images to obtain the Harris corner count for each original video image; The average value of the Harris corner count is calculated based on the preset number of image frames. Based on the Harris corner count of each original video image, and combined with the product of the average Harris corner count and a preset blur threshold, it is determined whether each original video image is blurred, and the determination result is obtained. Based on the judgment result, all blurred images in the original video image are removed to obtain the first video image.

3. The video transition recognition method based on monocular depth field according to claim 1, characterized in that, The step of performing monocular depth field calculation on each of the first video images to obtain several frames of second depth field images includes: Perform monocular depth detection on the first video image to obtain several frames of the first depth field image; The first depth field image is subjected to median filtering to obtain several frames of the second depth field image.

4. The video transition recognition method based on monocular depth field according to any one of claims 1 to 3, characterized in that, The step of determining whether a video transition has occurred based on each of the second depth field images, and obtaining the video transition determination result, includes: Based on the second depth field image, obtain several depth field pixel coordinates; The depth field pixel coordinates of adjacent second depth field images are compared to obtain a depth field difference image. The depth field difference image is binarized according to a preset binarization threshold to obtain a binarized image. Perform connected component analysis on the binarized image to obtain the connected component analysis results; Based on the connected component analysis results, and based on each of the second depth field images, it is determined whether a video transition has occurred, and a video transition determination result is obtained.

5. The video transition recognition method based on monocular depth field according to claim 4, characterized in that, The step of determining whether a video transition has occurred based on the connected component analysis results and each of the second depth field images, to obtain the video transition determination result, includes: Based on the connected component analysis results, multiple connected regions are obtained, and the area and centroid of each connected region are obtained. Based on the area of ​​each connected region and a preset area threshold, each connected region is filtered to obtain several significantly changing regions; Obtain the number of regions with significant changes, and calculate the average change in the centroid of each of the connected regions based on the centroid of the significant changes. Based on the number of regions with significant changes and the average change in the center of gravity, and based on each of the second depth field images, it is determined whether a video transition has occurred, and a video transition determination result is obtained.

6. The video transition recognition method based on monocular depth field according to claim 5, characterized in that, The step of obtaining the number of regions with significant changes and calculating the average change in the centroid of the regions with significant changes based on the centroid of each connected region includes: Based on the centroid of each of the connected regions, obtain the target centroid of each of the significantly changed regions; The difference is calculated based on the target centroid of the significantly changed regions in the binarized images of adjacent frames to obtain the centroid change value corresponding to each significantly changed region. The mean change in the center of gravity is calculated based on the number of regions with significant changes and the change in the center of gravity.

7. The video transition recognition method based on monocular depth field according to claim 5, characterized in that, The step of determining whether a video transition has occurred based on the number of regions with significant changes and the average change in the center of gravity, and based on each of the second depth field images, to obtain the video transition determination result includes: Obtain the preset threshold for the number of significantly changing regions and the threshold for the mean value of the center of gravity change; The number of regions with significant changes is compared with the number threshold to obtain a first comparison result; The mean value of the center of gravity change is compared with the mean threshold value to obtain a second comparison result; Based on the first comparison result and the second comparison result, when the number of regions is greater than the number threshold and the average value of the center of gravity change is greater than the average threshold, the video transition judgment result is that a video transition has occurred.

8. A video transition recognition device based on monocular depth field, characterized in that, The device includes: The preprocessing module preprocesses the video stream to be identified for transition recognition, obtaining several frames of the first video image; The depth field calculation module performs monocular depth field calculation on each of the first video images to obtain several frames of second depth field images. The transition judgment module determines whether a video transition has occurred based on each of the second depth field images, and obtains the video transition judgment result.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-7.

10. A storage medium storing computer program instructions thereon, characterized in that, The method as described in any one of claims 1-7 is implemented when the computer program instructions are executed by the processor.

Images