Split-screen video detection method, device, computer equipment and storage medium

Abstract

The present application relates to a split-screen video detection method, device, computer device, and storage medium. The method includes: obtaining a video to be detected, performing frame extraction on the video to be detected to obtain an image sequence, performing edge detection on each frame in the image sequence, identifying edge pixel points in the frame, and marking the coordinates of the edge pixel points. According to the coordinates of the edge pixel points in the same frame, detecting whether the edge pixel points form a horizontal line or a vertical line in the frame to obtain a line detection result, and aggregating the line detection results of each frame in the image sequence to obtain a split-screen detection result of the video to be detected. By using this method, it is possible to accurately identify whether a video is split into multiple screens, and furthermore, based on the split-screen detection result, it is possible to effectively detect problems such as a small proportion of effective video frames and low video quality caused by split-screening.

Description

Technical Field

[0001] This application relates to the technical field of video processing, and in particular to a split-screen video detection method, device, computer device, and storage medium. Background Art

[0002] With the development of multimedia technology, videos, as a type of multimedia material, have been widely spread on major video platforms. Major video platforms also need to review videos, such as video content review and video quality detection.

[0003] As the video display methods become more and more diverse, various forms of videos have emerged. For example, there are multiple screens playing simultaneously in a video, or a landscape video is re-produced into a portrait video suitable for mobile phone display. Since there may be the same screen among the multiple simultaneously played screens, when a landscape video is converted to a portrait video, the upper and lower parts of the portrait video need to be filled with meaningless screens.

[0004] In traditional technologies, the review of videos is generally based on the detection and analysis of complete videos, making it difficult to accurately identify whether a video is split into multiple screens, which may lead to problems such as a small proportion of effective screens and low video quality due to split screens, and these problems are difficult to be detected. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a split-screen video detection method, device, computer device, and storage medium that can achieve accurate split-screen detection.

[0006] A split-screen video detection method, the method includes:

[0007] Obtain a video to be detected, perform frame extraction on the video to be detected to obtain an image sequence;

[0008] Perform edge detection on each frame in the image sequence, identify the edge pixel points in the frame, and mark the coordinates of the edge pixel points;

[0009] According to the coordinates of the edge pixel points in the same frame, detect whether the edge pixel points form a horizontal line or a vertical line in the frame to obtain a line detection result;

[0010] Collect the line detection results of each frame in the image sequence to obtain the split-screen detection result of the video to be detected.

[0011] A split-screen video detection device, the device includes:

[0012] An image sequence acquisition module, configured to obtain a video to be detected, perform frame extraction on the video to be detected to obtain an image sequence;

[0013] An edge detection module, which is used to perform edge detection on each frame of an image sequence, identify edge pixel points in the image, and mark the coordinates of the edge pixel points;

[0014] A straight line detection module, which is used to detect whether the edge pixel points in the same image form a horizontal straight line or a vertical straight line according to the coordinates of the edge pixel points in the image, and obtain a straight line detection result;

[0015] A split screen detection module, which is used to collect the straight line detection results of each frame of the image sequence to obtain the split screen detection result of the video to be detected.

[0016] A computer device, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0017] Obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence;

[0018] Perform edge detection on each frame of the image sequence, identify edge pixel points in the image, and mark the coordinates of the edge pixel points;

[0019] According to the coordinates of the edge pixel points in the same image, detect whether the edge pixel points form a horizontal straight line or a vertical straight line in the image, and obtain a straight line detection result;

[0020] Collect the straight line detection results of each frame of the image sequence to obtain the split screen detection result of the video to be detected.

[0021] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0022] Obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence;

[0023] Perform edge detection on each frame of the image sequence, identify edge pixel points in the image, and mark the coordinates of the edge pixel points;

[0024] According to the coordinates of the edge pixel points in the same image, detect whether the edge pixel points form a horizontal straight line or a vertical straight line in the image, and obtain a straight line detection result;

[0025] Collect the straight line detection results of each frame of the image sequence to obtain the split screen detection result of the video to be detected.

[0026] The above split-screen video detection method, device, computer device, and storage medium perform edge detection on each image in the video frame extraction image sequence, accurately identify the edge pixel points in the image, and based on the coordinates of each edge pixel point, detect whether there are horizontal or vertical lines formed by the edge pixel points in the image as the basis for multi-screen analysis of each image. By aggregating the line detection results of each frame in the image sequence and based on the line detection results in each image, an accurate split-screen detection result of the video to be detected is obtained, which can accurately identify whether the video is divided into multiple screens. Furthermore, based on the split-screen detection result, problems such as a small proportion of effective video frames and low video quality caused by split-screen can be effectively detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 FIG. 1 is an application environment diagram of the split-screen video detection method in an embodiment;

[0028] Figure 2 FIG. 2 is a schematic flowchart of the split-screen video detection method in an embodiment;

[0029] Figure 3 FIG. 3 is a schematic flowchart of the split-screen video detection method in another embodiment;

[0030] Figure 4 FIG. 4 is a schematic flowchart of the split-screen video detection method in yet another embodiment;

[0031] FIG. 5(a) is an original image in the image sequence in the split-screen video detection method in an embodiment;

[0032] FIG. 5(b) is an image in the image sequence in the split-screen video detection method in an embodiment with edge pixel points marked;

[0033] FIG. 5(c) is an image in the image sequence in the split-screen video detection method in an embodiment including line detection results;

[0034] Figure 6 FIG. 6 is a schematic flowchart of the split-screen video detection method in still another embodiment;

[0035] Figure 7 FIG. 7 is a schematic flowchart of the split-screen video detection method in yet another embodiment;

[0036] Figure 8 FIG. 8 is a schematic diagram of a left-right split-screen video in the split-screen video detection method in an embodiment;

[0037] Figure 9 FIG. 9 is a schematic diagram of an up-down three-screen (with the upper and lower screens static) video in the split-screen video detection method in an embodiment;

[0038] Figure 10It is a video schematic diagram of upper and lower three - frequency division (upper and lower pictures are blurred) in the split - screen video detection method in an embodiment;

[0039] Figure 11 It is a video schematic diagram of repeated nine - frequency division in the split - screen video detection method in an embodiment;

[0040] Figure 12 It is a flowchart of the split - screen video detection method in another embodiment;

[0041] Figure 13 It is a flowchart of the split - screen video detection method in one of the embodiments;

[0042] Figure 14 It is a structural block diagram of the split - screen video detection device in an embodiment;

[0043] Figure 15 It is an internal structure diagram of a computer device in an embodiment. Specific embodiments

[0044] In order to make the purpose, technical solutions and advantages of this application clearer, the following further details this application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit this application.

[0045] The split - screen video detection method provided by this application can be applied to an application environment as shown in Figure 1 In the figure. Among them, the terminal 102 communicates with the server 104 through the network. The terminal 102 sends the video to be detected to the server 104. The server 104 performs frame extraction on the video to be detected to obtain an image sequence, performs edge detection on each frame in the image sequence, identifies the edge pixel points in the picture, and marks the coordinates of the edge pixel points. According to the coordinates of the edge pixel points in the same picture, it detects whether the edge pixel points form a horizontal line or a vertical line in the picture to obtain a line detection result, and aggregates the line detection results of each frame in the image sequence to obtain a split - screen detection result of the video to be detected. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0046] In other embodiments, the split - screen video detection method provided by this application can also be applied to the terminal or the server, and can be specifically deployed according to actual needs.

[0047] In one embodiment, as shown in Figure 2 In the figure, a split - screen video detection method is provided, and this method is applied to Figure 1Taking the server in [description] as an example, the method includes the following steps:

[0048] Step 210: Obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence.

[0049] The video to be detected can be a video that needs to be subjected to split-screen detection, or a video that needs to perform quality analysis through split-screen detection. For example, the effective picture in the video is identified through split-screen processing for video quality analysis.

[0050] A video is obtained by arranging multiple consecutive images at a certain time interval. When the change of consecutive images exceeds 24 frames per second, according to the principle of persistence of vision, the human eye cannot distinguish a single static picture, and it looks like a smooth and continuous visual effect, that is, the video we see. Video frame extraction refers to the process of performing frame-by-frame processing on video images and extracting the frame-divided video images according to a certain rule. In the embodiment, frame extraction is performed at a certain time interval, for example, extraction is performed at a frame rate of 1 s / frame. In other embodiments, a non-uniform time interval frame extraction method can also be used, such as using the method of extracting key frames of the video for frame extraction. The specific frame extraction method is not limited here. By arranging the extracted video image frames in chronological order, an image sequence corresponding to the video can be obtained. Using the image sequence as the object of split-screen analysis can effectively reduce the amount of data to be analyzed and improve the data processing efficiency while ensuring the accuracy of the detection results.

[0051] Step 220: Perform edge detection on each frame in the image sequence, identify the edge pixel points in the frame, and mark the coordinates of the edge pixel points.

[0052] An edge refers to a set of pixel points whose surrounding pixel brightness changes sharply. It is a basic feature of an image, and edges exist between objects, backgrounds, and regions. The purpose of edge detection is to identify the pixel points with obvious brightness changes in the image. By performing edge detection on the image, the amount of data in the image can be greatly reduced, and the information that can be considered irrelevant can be eliminated, while retaining the important structural attributes of the image. If an edge is considered to be a place where the brightness of a certain number of points changes, then edge detection is generally to calculate the derivative of this brightness change. In the embodiment, the edge detection methods can be divided into two categories: namely, search-based and zero-crossing-based. Among them, the search-based edge detection method first calculates the edge strength, usually represented by the first derivative, such as the gradient modulus. Then, it estimates the local direction of the edge, usually using the direction of the gradient, and uses this direction to find the maximum value of the local gradient modulus. The zero-crossing-based method finds the zero-crossing points of the second derivative obtained from the image to locate the edges. Usually, the Laplacian operator or the zero-crossing points of non-linear differential equations are used.

[0053] In an embodiment, edge detection can be implemented using an edge detection algorithm. Specifically, such as the Canny edge detection algorithm, or using operators such as Laplacian and Sobel as the basic methods for edge and contour detection, etc.

[0054] For each image in the image sequence, its size is a fixed value. The size of the image can be represented based on the pixel distribution, i.e., the resolution. For example, the size can be represented as w×h, such as 800×600, 640×480, etc. Among them, 800×600 can represent that the number of pixel points in the width direction is 800, and the number of pixel points in the length direction is 600. Based on the pixel distribution of the image, a corresponding coordinate system can be constructed. In an embodiment, generally, the upper left corner of the image is used as the coordinate origin, and the upper edge and left edge of the image are used as the coordinate axes to construct the coordinate system. Among them, the coordinate data on the coordinate axes corresponds to the size. For example, for an image of 800×600, the coordinates of the center point of the image are (400, 300), the coordinates of the lower left corner are (0, 600), the coordinates of the lower right corner are (800, 600), the coordinates of the upper left corner are (0, 0), and the coordinates of the upper right corner are (800, 0). In other embodiments, the size of the image can also be identified in other ways, such as the length and width of the image, such as 8×6 inches, etc.

[0055] Based on the coordinate system constructed according to the pixel distribution of the image, the coordinates of each pixel point in the image can be obtained. After identifying the edge pixel points in the image, the edge pixel points can be coordinate-identified to determine the positions of the edge pixel points in the image.

[0056] Step 230: According to the coordinates of each edge pixel point in the same image, detect whether the edge pixel points form a horizontal line or a vertical line in the image, and obtain a line detection result.

[0057] Among them, a horizontal straight line refers to a straight line that exists in the image, has a certain length, and is perpendicular to the ordinate of the coordinate axis. A vertical straight line refers to a straight line that exists in the image, has a certain length, and is perpendicular to the abscissa of the coordinate axis. The horizontal straight line and the vertical straight line are perpendicular to each other. It can be understood that the horizontal straight line and the vertical straight line here are relative to the coordinate axis of the image (or the edge line of the image), rather than the lines that are strictly parallel or perpendicular to the horizontal plane. For the case where there are multiple frames in the video, the edge lines of each frame are all in the horizontal direction or the vertical direction. Therefore, when performing split-screen detection, it is necessary to focus on detecting the horizontal and vertical straight lines in each frame of the image sequence. Specifically, straight line detection can be performed first. After the straight line detection, the horizontal and vertical straight lines among the detected straight lines can be selected according to the angle. It is also possible to directly use the horizontal and vertical directions as the detection directions to directly detect the horizontal and vertical straight lines, and then based on the judgment of the lengths of the horizontal and vertical straight lines, determine whether the horizontal or vertical straight line meets the detection requirements of the horizontal or vertical straight line.

[0058] In an embodiment, the straight line detection in the image can be performed by means of point clustering. Taking the direct detection of horizontal and vertical straight lines as an example, based on the coordinates of each edge pixel point in the same image, point clustering processing is performed, and the number of edge pixel points detected for each straight line in the horizontal and vertical directions is voted to determine whether it meets the threshold requirement, so as to determine whether there is a horizontal or vertical straight line that meets the detection requirements. When there is none, the output is empty. Among them, voting refers to the data processing method of summarizing the obtained results and retaining the results with the number of votes (the number of occurrences) exceeding the threshold as the final result.

[0059] Step 240, collect the straight line detection results of each frame of the image sequence to obtain the split-screen detection result of the video to be detected.

[0060] The straight line detection result of each frame of the image includes the coordinate information of each straight line. In the embodiment, the split-screen detection result includes the specific coordinate positions of the split screen, that is, the specific positions of each sub-screen in the video after splitting. After performing straight line detection on each frame of the image, the straight line detection result of each frame of the image will be obtained. By collecting the straight line detection results of each frame of the image, determine the number of times each detected straight line appears repeatedly in the image sequence. If the number of occurrences is greater than the set number threshold, determine that the detected straight line is the multi-screen dividing line in the video, and thus obtain the split-screen detection result of the video based on the dividing line.

[0061] In one embodiment, as Figure 3 shown, collecting the straight line detection results of each frame of the image sequence to obtain the split-screen detection result of the video to be detected includes steps 310 to 340.

[0062] Step 310: Obtain the straight line detection results in the horizontal direction and the vertical direction in the image sequence respectively by summarization.

[0063] Step 320: Count the occurrence times of the straight lines with the same coordinates in the straight line detection results in the horizontal direction and the vertical direction respectively.

[0064] Step 330: When the occurrence times are greater than the times threshold, mark the straight line as a boundary line.

[0065] Among them, the times threshold is associated with the number of images in the image sequence. For example, the times threshold can be half of the number of images in the image sequence.

[0066] Step 340: Obtain the split screen detection result carrying the split screen position information according to the coordinate data of the boundary line.

[0067] By performing straight line detection on each of the n frames of images in the image sequence, the per-frame horizontal / vertical straight line detection results are obtained. If the vertical straight line detection result of image i is [y i,0 ,y i,1 ,…], and the horizontal straight line detection result is [x i,0 ,x i,1 ,…], then the set of horizontal straight line detection results of the image sequence 1 to n is [[y 1,0 ,y 1,1 ,…],[y 2,0 ,y 2,1 ,…],…,[y n,0 ,y n,1 ,…]], and the set of vertical straight line detection results is [[x 1,0 ,x 1,1 ,…],[x 2,0 ,x 2,1 ,…],…,[x n,0 ,x n,1 ,…]]; by counting the occurrence times count(y i,j ) of each straight line in the horizontal straight line detection results one by one, if count(y i,j )>n / 2, then record y i,j as the horizontal boundary; similarly, the vertical boundary is obtained. It can be understood that the threshold n / 2 here can be adjusted according to the overall clarity of the video, the video type and style, and is not limited here. Based on the coordinate positions of the horizontal boundary and the vertical boundary in the figure, the split screen detection result of the video to be detected can be obtained. In the embodiment, the split screen detection result includes the split screen position information corresponding to the coordinate data of the boundary line for performing picture segmentation processing.

[0068] The above split-screen video detection method performs edge detection on each image in the video frame extraction image sequence, accurately identifies the edge pixel points in the image, and based on the coordinates of each edge pixel point, detects whether there are horizontal or vertical lines formed by the edge pixel points in the image, so as to serve as the basis for multi-screen analysis of each image. By collecting the line detection results of each frame of the image sequence and based on the line detection results of each image, an accurate split-screen detection result of the video to be detected is obtained, which can accurately identify whether the video is divided into multiple screens. Furthermore, based on the split-screen detection result, problems such as a small proportion of effective video frames and low video quality caused by split-screen can be effectively detected.

[0069] In one embodiment, as Figure 4 shown, performing edge detection on each frame of the image sequence, identifying the edge pixel points in the image, and marking the coordinates of the edge pixel points includes steps 410 to 420.

[0070] Step 410, grayscale each frame of color image in the image sequence to obtain a grayscale image sequence.

[0071] Step 420, perform edge detection on each frame of grayscale image in the grayscale image sequence, identify the edge pixel points in the grayscale image whose brightness difference from adjacent pixel points is greater than a preset threshold, and mark the coordinates of the edge pixel points.

[0072] Generally speaking, in an unprocessed video, its consecutive frames are all color images. Specifically, each frame of RGB image in the image sequence is converted into a grayscale image one by one, and the grayscale value range of each pixel is 0 - 255. By performing grayscale processing on the image, the standardization of the image can be achieved, and each pixel in the image is represented by a brightness value. Using the grayscale processed image for edge detection can reduce interference factors. As shown in Figures 5(a) and 5(b), they are the images before and after edge detection processing respectively. Through edge detection, identify the edge pixel points in the grayscale image whose brightness difference from adjacent pixel points is greater than a preset threshold to obtain a more accurate edge detection result. Among them, the preset threshold can be specifically set according to actual requirements. The lower the threshold, the more edge lines can be detected, and the result is more easily affected by image noise and more likely to pick out irrelevant features from the image. On the contrary, a high threshold will miss thin or short line segments. In the embodiment, the preset threshold can be set to a relatively high value to reduce the interference of irrelevant features and facilitate subsequent line detection.

[0073] In one embodiment, as Figure 6 shown, before detecting whether the edge pixel points in the same image form a horizontal or vertical line based on the coordinates of each edge pixel point in the image to obtain the line detection result, it further includes step 610.

[0074] Step 610: Obtain the pixel size of the images in the image sequence, and based on the pixel size, construct a coordinate system.

[0075] According to the coordinates of the edge pixel points in the same image, detect whether the edge pixel points form a horizontal line or a vertical line in the image, and the line detection result includes steps 620 to 640.

[0076] Step 620: Randomly select a parallel line of any coordinate axis, determine the coordinates of the parallel line in the coordinate system, and based on the pixel size, determine the number of reference pixel points of the image in the direction indicated by the parallel line.

[0077] Step 630: According to the coordinates of the edge pixel points in the same image and the coordinates of the parallel line, identify the target edge pixel points among the edge pixel points whose distance from the parallel line does not exceed one pixel point.

[0078] Step 640: When the number of target edge pixel points exceeds the corresponding quantity threshold of the reference pixel points, mark the parallel line to obtain the line detection result.

[0079] The pixel size (w, h) of the image can specifically be data used to characterize the pixel arrangement composition of the image. For example, the number of pixel points in the width direction is 600, and the number of pixel points in the length direction is 480, and the pixel size of the image can be expressed as 600×480. Randomly selecting a parallel line of the coordinate axis includes parallel lines parallel to the X-axis and parallel to the Y-axis. For any randomly selected parallel line of the coordinate axis, by translating the parallel line by a distance of one pixel point, two adjacent lines can be obtained, and the edge pixel points whose distance from the parallel line does not exceed one pixel point include the edge pixel points located on the parallel line and the adjacent lines of the parallel line.

[0080] In a specific embodiment, extract the edge pixel points in the same image, determine the coordinates of the edge pixel points in the coordinate system, and the unified format is (x, y).

[0081] For the horizontal direction, randomly select a parallel line y = y0 parallel to the X-axis in the coordinate axis, and perform point clustering based on the coordinate y: If the number of edge pixel points satisfying y - y0 ≤ 1 is greater than w / 2, then record the selected parallel line y = y0 as the horizontal boundary line meeting the line detection requirements. When there is no horizontal boundary line satisfying the above conditions, the output is empty.

[0082] For the vertical direction, randomly select a parallel line x = x0 parallel to the Y-axis in the vertical direction and the coordinate axis, and perform point clustering based on the coordinate x: If the number of edge pixel points satisfying x - x0 ≤ 1 is greater than h / 2, record the selected parallel line x = x0 as the vertical boundary line that meets the line detection requirements. When there is no output boundary line that meets the above conditions, the output is empty.

[0083] When there are no boundary lines that meet the line detection requirements in both the vertical and horizontal directions, output the split screen detection result indicating that the video to be detected has not been split.

[0084] For example, the line detection result of the image shown in Figure 5(c) is a horizontal boundary line [](the output is empty), and a vertical boundary line [0.495*w].

[0085] In one embodiment, as Figure 7 shown, after collecting the line detection results of each frame in the image sequence to obtain the split screen detection result of the video to be detected, it further includes steps 710 to 720.

[0086] Step 710: According to the split screen position information in the split screen detection result, split the video into multiple sub-pictures.

[0087] Step 720: Perform validity detection on each sub-picture to determine the valid pictures of the video.

[0088] The split screen position information refers to the straight line determined as the boundary line of the video after split screen detection. According to the coordinates of the straight line in the coordinate system, the split screen position information can be obtained. Based on the coordinates of this straight line, the video can be split into multiple sub-pictures. The validity detection specifically includes static picture detection and blurred picture detection. When the clarity of the sub-picture is not less than the preset clarity threshold and the content of the sub-pictures in adjacent frames is different, the sub-picture is a valid picture. In addition, when there are multiple identical pictures, one of them is selected as the valid picture.

[0089] As Figures 8 - 11 shown, Figure 8 the video shown in Figure 9 and Figure 10 the video shown in Figure 9 can be divided into left and right two sub-pictures. Among them, Figure 10 is a schematic diagram of the interface where the upper and lower pictures are static, such as the upper and lower pictures display fixed text. Figure 11 is a schematic diagram of the interface where the upper and lower pictures are blurred.

[0090] In one embodiment, as Figure 12As shown in the figure, perform validity detection on each sub - screen. Determining the valid screens of the video includes performing clarity recognition on the sub - screens, and performing inter - frame difference detection on adjacent frames of the same sub - screen, comparing the sub - screen contents of adjacent frames. When the clarity of the sub - screen is not less than the preset clarity threshold and the sub - screen contents of adjacent frames are different, determine the sub - screen as a valid screen. Specifically, the screen validity detection of the video includes steps 1210 to 1250.

[0091] Step 1210, perform clarity recognition on the sub - screen.

[0092] Step 1220, when the clarity of the sub - screen is less than the preset clarity threshold, determine the sub - screen as an invalid blurred screen.

[0093] Step 1230, perform inter - frame difference detection on adjacent frames of the same sub - screen, and compare the sub - screen contents of adjacent frames.

[0094] Step 1240, when the sub - screen contents of adjacent frames are the same, determine the sub - screen as an invalid static screen.

[0095] Step 1250, when the clarity of the sub - screen is not less than the preset clarity threshold and the sub - screen contents of adjacent frames are different, determine the sub - screen as a valid screen.

[0096] In an embodiment, judge the clarity of the sub - screen through a clarity recognition algorithm. If the clarity of the sub - screen is less than the preset clarity threshold, then judge that the sub - screen is invalid, where the clarity threshold can be set according to the video quality requirements.

[0097] Inter - frame difference refers to the data - processing process of taking the difference by matching the same pixel points in two adjacent frames of a video image sequence. Through inter - frame difference detection, judge whether the sub - screen contents of adjacent frames are the same, so as to judge whether the sub - screen is a static screen. If the sub - screen is a static screen, then judge that the sub - screen is invalid.

[0098] If the clarity of the sub - screen is not less than the preset clarity threshold and the sub - screen contents of adjacent frames are different, that is, when the sub - screen is neither a static screen nor a blurred screen, then the sub - screen is a valid screen.

[0099] In one embodiment, after splitting the video into multiple sub - screens according to the split - screen position information in the split - screen detection result, it further includes: obtaining the similarity of each sub - screen. When it is determined according to the similarity that all sub - screens are of the same content, select any one sub - screen as the valid screen.

[0100] By calculating the similarity of multiple sub - pictures, according to the similarity calculation results, it is determined whether the multiple sub - pictures are of the same content. If the picture contents are the same or similar, for example, the similarity reaches the set similarity threshold, then any one of the sub - pictures is taken as the result. For example, the first sub - picture is selected as the valid picture.

[0101] In other embodiments, the detection of the valid picture can also be performed by detecting whether the sub - picture is a secondary production of an animation, and whether there are valid information such as characters and landscapes.

[0102] In the embodiment, taking Figures 8 - 11 the video shown as an example, the final output result of the split - screen video detection is Figure 8 a left - right split - screen with a vertical boundary line of 0.495*w, Figure 9 a middle screen with horizontal boundary lines of 0.34*h and 0.695*h (the upper and lower sub - pictures are static and judged as invalid pictures), Figure 10 a middle screen with horizontal boundary lines of 0.34*h and 0.695*h (the upper and lower sub - pictures are blurred and judged as invalid pictures), Figure 11 a repeated nine - split - screen with horizontal boundary lines of 0.33*h and 0.66*h and vertical boundary lines of 0.33*w and 0.66*w, and the accurate coordinate range of the sub - pictures can be given by the above - mentioned method.

[0103] This application also provides an application scenario. Specifically, the application scenario may include video detection scenarios such as identifying self - made videos of "landscape screen to portrait screen", video review systems, or video low - quality detection. The above - mentioned split - screen video detection method can be applied to this video detection scenario. Specifically, the application of the split - screen video detection method in this application scenario is as follows:

[0104] The split - screen video detection method mainly includes six steps: video frame extraction, image normalization, image edge detection, image horizontal / vertical line detection, video frame - by - frame result output, and video split - screen result analysis.

[0105] The first step, video frame extraction: The target video is frame - extracted at a certain time interval (usually the frame rate is 1s / frame) to form an image sequence arranged in chronological order.

[0106] The second step, image normalization: The images obtained by frame extraction are RGB images, which are converted into grayscale images one by one. The grayscale value range of each pixel is 0 - 255, and the image size (w, h) is obtained.

[0107] The third step, image edge detection: Using the edge detection algorithm to identify the points with obvious brightness changes in the image and mark the pixel point coordinates.

[0108] Step 4, Detection of horizontal / vertical lines in the image based on point clustering: Based on the results of image edge detection in Step 3, use the voting method to detect whether there are lines in the horizontal and vertical directions respectively. When there are none, the output is empty. For example, in Figure 5(c), the detection results of horizontal / vertical lines in the image are: horizontal line [], vertical line [0.495*w]. The specific method includes extracting the coordinates of all marked pixel points, with the unified format of (x, y). For the horizontal direction, randomly select a parallel line y = y0 parallel to the X-axis in the coordinate axis, and perform point clustering based on the coordinate y: if the number of edge pixel points satisfying y - y0 ≤ 1 is greater than w / 2, record the selected parallel line y = y0 as the horizontal boundary line that meets the line detection requirements. When there is no horizontal boundary line that meets the above conditions, the output is empty. For the vertical direction, randomly select a parallel line x = x0 parallel to the Y-axis in the coordinate axis, and perform point clustering based on the coordinate x: if the number of edge pixel points satisfying x - x0 ≤ 1 is greater than h / 2, record the selected parallel line x = x0 as the vertical boundary line that meets the line detection requirements. When there is no output boundary line that meets the above conditions, the output is empty.

[0109] Step 5, Output of video frame-by-frame results: Execute Step 4 for each of the n frames in the image sequence to obtain the frame-by-frame horizontal / vertical line detection results; if the vertical line detection result of image i is [y i,0 , y i,1 , …], and the horizontal line detection result is [x i,0 , x i,1 , …], then the set of horizontal line detection results for image sequences 1 to n is [[y 1,0 , y 1,1 , …], [y 2,0 , y 2,1 , …], …, [y n,0 , y n,1 , …]], and the set of vertical line detection results is [[x 1,0 , x 1,1 , …], [x 2,0 , x 2,1 , …], …, [x n,0 , x n,1 , …]]; count the number of times the coordinates appear in the horizontal line detection results one by one, count(y i,j ). If count(y i,j ) > n / 2, record y i,j as the horizontal boundary line; similarly, obtain the vertical boundary line.

[0110] Step 6, Effective screen judgment: Using the horizontal and vertical boundary lines of the video obtained in Step 5, the video can be divided into m sub - screens. The content validity of the sub - screens is judged through inter - frame difference detection and fuzzy recognition. Specifically: Inter - frame difference detection means performing subtraction on the matching of the same pixel points in two adjacent frames of the video image sequence to detect the inter - frame difference, and judging whether the content of the sub - screens in adjacent frames is the same, so as to judge whether it is a still screen. If the sub - screen is a still screen, it is judged that the sub - screen is an invalid screen. Clarity recognition means judging the clarity of the sub - screen through a clarity recognition algorithm. If the sub - screen is a blurred screen with insufficient clarity, it is judged that the sub - screen is an invalid screen. If the clarity of the sub - screen meets the requirements and it is not a still screen, it is judged that the sub - screen is an effective screen. For the case of multiple screens, it is necessary to calculate the similarity between sub - screens: By calculating the similarity between multiple sub - screens, it is obtained whether multiple sub - screens are of the same content. If the screen content is the same or similar, the first sub - screen is taken as the effective screen.

[0111] Through the above - mentioned processing process, not only can multi - screen videos be split, but also for videos with black edges, white edges, background canvases, etc., the proportion of the area of the invalid canvas can be judged, and the background of the invalid canvas around it can be cut according to the coordinates; at the same time, in the current short - video field, a large number of landscape videos are re - produced into portrait videos suitable for mobile phone display. Since the upper and lower parts of the portrait video need to be filled with meaningless pictures, the proportion of the effective screen of the video is small and the video quality is low. Through the above - mentioned processing process, the video quality can be effectively detected. In addition, it can also detect "landscape video converted to portrait video".

[0112] In one embodiment, as Figure 13 shown, a split - screen video detection method is provided, which specifically includes Step 1302 to Step 1336.

[0113] Step 1302, Obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence arranged in chronological order.

[0114] Step 1304, Grayscale each color image in the image sequence to obtain a grayscale image sequence.

[0115] Step 1306, Perform edge detection on each grayscale image in the grayscale image sequence, identify the edge pixel points in the grayscale image whose brightness difference from adjacent pixel points is greater than a preset threshold, and mark the coordinates of the edge pixel points.

[0116] Step 1308, Obtain the pixel size of the images in the image sequence, and construct a coordinate system based on the pixel size.

[0117] Step 1310: Randomly select a parallel line of any coordinate axis, determine the parallel line coordinates, and determine the number of reference pixel points of the image in the direction indicated by the parallel line according to the pixel size.

[0118] Step 1312: According to the coordinates of each edge pixel point in the same image and the coordinates of the parallel line in the coordinate system, identify the target edge pixel points among the edge pixel points whose distance from the parallel line does not exceed one pixel point.

[0119] Step 1314: When the number of target edge pixel points exceeds the corresponding number threshold of the reference pixel points, mark the parallel line to obtain the straight line detection result.

[0120] Step 1316: Respectively collect the straight line detection results in the horizontal and vertical directions in the image sequence.

[0121] Step 1318: Respectively count the occurrence times of the straight lines with the same coordinates in the straight line detection results in the horizontal and vertical directions.

[0122] Step 1320: When the occurrence times are greater than the times threshold, mark the straight line as the boundary line.

[0123] Step 1322: According to the coordinate data of the boundary line, obtain the split screen detection result carrying the split screen position information.

[0124] Step 1324: According to the split screen position information in the split screen detection result, split the video into multiple sub - pictures.

[0125] Step 1326: Identify the clarity of the sub - pictures.

[0126] Step 1328: When the clarity of the sub - picture is less than the preset clarity threshold, determine that the sub - picture is an invalid blurred picture.

[0127] Step 1330: Perform inter - frame difference detection on adjacent frames of the same sub - picture and compare the sub - picture contents of adjacent frames.

[0128] Step 1332: When the sub - picture contents of adjacent frames are the same, determine that the sub - picture is an invalid static picture.

[0129] Step 1334: When the clarity of the sub - picture is not less than the preset clarity threshold and the sub - picture contents of adjacent frames are different, determine that the sub - picture is a valid picture.

[0130] Step 1336: Calculate the similarity of each sub - picture. When it is determined according to the similarity that all sub - pictures are pictures with the same content, select any one of the sub - pictures as the valid picture of the video.

[0131] It should be understood that although Figures 2 - 4 、 Figures 6 - 7 、Figures 12 - 13 The steps in the flowchart are sequentially shown according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, Figures 2 - 4 and Figures 6 - 7 and Figures 12 - 13 at least some of the steps in can include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least some of the steps or steps or stages in other steps.

[0132] In one embodiment, as Figure 14 shown, a split-screen video detection device 1400 is provided. This device can be a software module, a hardware module, or a combination of both to form part of a computer device. Specifically, the device includes: an image sequence acquisition module 1410, an edge detection module 1420, a line detection module 1430, and a split-screen detection module 1440, where:

[0133] The image sequence acquisition module 1410 is configured to acquire a video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence.

[0134] The edge detection module 1420 is configured to perform edge detection on each frame in the image sequence, identify edge pixel points in the image, and mark the coordinates of the edge pixel points.

[0135] The line detection module 1430 is configured to detect whether the edge pixel points in the same image form a horizontal line or a vertical line according to the coordinates of the edge pixel points in the image, and obtain a line detection result.

[0136] The split-screen detection module 1440 is configured to collect the line detection results of each frame in the image sequence to obtain a split-screen detection result of the video to be detected.

[0137] In one embodiment, the edge detection module is further configured to grayscale each color image in the image sequence to obtain a grayscale image sequence; perform edge detection on each grayscale image in the grayscale image sequence to identify edge pixel points whose brightness difference from adjacent pixel points is greater than a preset threshold.

[0138] In one embodiment, the straight line detection module is further configured to obtain the pixel size of the images in the image sequence, construct a coordinate system based on the pixel size; randomly select a parallel line of any coordinate axis, determine the coordinates of the parallel line, and determine the number of reference pixel points of the image in the direction indicated by the parallel line according to the pixel size; identify the target edge pixel points among the edge pixel points whose distance from the parallel line does not exceed one pixel point according to the coordinates of the edge pixel points in the same image and the coordinates of the parallel line in the coordinate system; when the number of target edge pixel points exceeds the corresponding number threshold of the reference pixel points, mark the parallel line to obtain the straight line detection result.

[0139] In one embodiment, the split screen detection module is further configured to respectively collect the straight line detection results in the horizontal direction and the vertical direction in the image sequence according to the direction of the detected straight line; respectively count the occurrence times of the straight lines with the same coordinates in the straight line detection results in the horizontal direction and the vertical direction; when the occurrence times are greater than the times threshold, mark the straight line as the boundary line, where the times threshold is associated with the number of images in the image sequence; obtain the split screen detection result carrying the split screen position information according to the coordinate data of the boundary line.

[0140] In one embodiment, the split screen video detection device further includes an effective picture detection module, and the effective picture detection module is configured to split the video into multiple sub-pictures according to the split screen position information in the split screen detection result; perform validity detection on each sub-picture to determine the effective picture of the video.

[0141] In one embodiment, the effective picture detection module is further configured to identify the clarity of the sub-picture. When the clarity of the sub-picture is less than the preset clarity threshold, determine that the sub-picture is an invalid blurred picture; perform inter-frame difference detection on adjacent frames of the same sub-picture. When the content of the sub-picture in adjacent frames is the same, determine that the sub-picture is an invalid static picture; when the clarity of the sub-picture is not less than the preset clarity threshold and the content of the sub-picture in adjacent frames is different, determine that the sub-picture is an effective picture.

[0142] In one embodiment, the effective picture detection module is further configured to obtain the similarity of each sub-picture. When it is determined according to the similarity that all sub-pictures are pictures with the same content, select any one of the sub-pictures as the effective picture.

[0143] The above split-screen video detection device accurately identifies the edge pixel points in the graph by performing edge detection on each graph in the video frame extraction image sequence. Based on the coordinates of each edge pixel point, it detects whether there are horizontal or vertical lines formed by the edge pixel points in the graph, which serves as the basis for multi-screen analysis of each graph. By collecting the line detection results of each frame in the image sequence and based on the line detection results of each graph, it obtains an accurate split-screen detection result for the video to be detected, can accurately identify whether the video is divided into multiple screens, and further, based on the split-screen detection result, can effectively detect problems such as a small proportion of effective video frames and low video quality caused by split-screen.

[0144] For the specific limitations of the split-screen video detection device, reference can be made to the limitations of the split-screen video detection method in the above text, which will not be elaborated here. Each module in the above split-screen video detection device can be implemented in whole or in part by software, hardware, and their combinations. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form for the processor to call and execute the operations corresponding to the above modules.

[0145] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 15 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store split-screen video detection data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a split-screen video detection method.

[0146] Those skilled in the art can understand that Figure 15 the structure shown in

[0147] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0148] In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps in the above-mentioned method embodiments.

[0149] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0150] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0151] The above embodiments only represent several implementation manners of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A split-screen video detection method, characterized in that, Applied to the video detection scenario of switching from landscape to portrait, the method includes: Obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence; Perform edge detection on each frame of the image sequence, identify the edge pixel points in the image, and mark the coordinates of the edge pixel points; Obtain the pixel size of the images in the image sequence, and construct a coordinate system based on the pixel size; Randomly select a parallel line of any coordinate axis, determine the coordinates of the parallel line, and determine the number of reference pixel points of the image in the direction indicated by the parallel line according to the pixel size; According to the coordinates of the edge pixel points in the same image and the coordinates of the parallel line in the coordinate system, identify the target edge pixel points among the edge pixel points whose distance from the parallel line does not exceed one pixel point; When the number of the target edge pixel points exceeds the corresponding quantity threshold of the reference pixel points, mark the parallel line to obtain a straight line detection result; Collect the straight line detection results of each frame of the image sequence to obtain the split screen detection result of the video to be detected; The collecting the straight line detection results of each frame of the image sequence to obtain the split screen detection result of the video to be detected includes: respectively collecting the straight line detection results in the horizontal and vertical directions of the image sequence; respectively counting the occurrence times of the straight lines with the same coordinates in the straight line detection results in the horizontal and vertical directions; when the occurrence times are greater than the times threshold, mark the straight line as a boundary line, where the times threshold is associated with the number of images in the image sequence; according to the coordinate data of the boundary line, obtain a split screen detection result carrying split screen position information; According to the split screen position information in the split screen detection result, split the video into multiple sub - pictures; Perform validity detection on each of the sub - pictures to determine the valid pictures of the video.

2. The method according to claim 1, characterized in that, The performing edge detection on each frame of the image sequence to identify the edge pixel points in the image includes: Perform gray - scale processing on each frame of color image in the image sequence to obtain a gray - scale image sequence; Perform edge detection on each frame of gray - scale image in the gray - scale image sequence, and identify the edge pixel points in the gray - scale image whose brightness difference from adjacent pixel points is greater than a preset threshold.

3. The method according to claim 1, characterized in that, The performing validity detection on each of the sub - pictures to determine the valid pictures of the video includes: Perform clarity recognition on the sub - pictures, and perform inter - frame difference detection on adjacent frames of the same sub - picture to compare the sub - picture content of adjacent frames; When the clarity of the sub - picture is not less than the preset clarity threshold and the sub - picture content of adjacent frames is different, determine that the sub - picture is a valid picture.

4. The method according to claim 1 or 3, characterized in that, After splitting the video into multiple sub - pictures according to the split screen position information in the split screen detection result, it further includes: Obtain the similarity of each of the sub - pictures, and when it is determined according to the similarity that each of the sub - pictures is a picture with the same content, select any one of the sub - pictures as a valid picture.

5. A split-screen video detection device, characterized in that, Applied to the video detection scenario of switching from landscape to portrait, the device includes: An image sequence obtaining module, configured to obtain the video to be detected, perform frame extraction on the video to be detected, and obtain an image sequence; An edge detection module, configured to perform edge detection on each frame of the image sequence, identify edge pixel points in the image, and mark the coordinates of the edge pixel points; A line detection module, configured to obtain the pixel size of the image in the image sequence, construct a coordinate system based on the pixel size; randomly select a parallel line of any coordinate axis, determine the coordinates of the parallel line, and determine the number of reference pixel points of the image in the direction indicated by the parallel line according to the pixel size; identify target edge pixel points among the edge pixel points whose distance from the parallel line does not exceed one pixel point according to the coordinates of each edge pixel point in the same image and the coordinates of the parallel line in the coordinate system; when the number of the target edge pixel points exceeds the corresponding number threshold of the reference pixel points, mark the parallel line to obtain a line detection result; A split-screen detection module, configured to collect the line detection results of each frame of the image sequence to obtain the split-screen detection result of the video to be detected; the split-screen detection module is further configured to respectively collect the line detection results in the horizontal direction and the vertical direction in the image sequence according to the direction of the detected line; respectively count the occurrence times of the lines with the same coordinates in the line detection results in the horizontal direction and the vertical direction; when the occurrence times are greater than the times threshold, mark the line as a boundary line, where the times threshold is associated with the number of images in the image sequence; obtain a split-screen detection result carrying split-screen position information according to the coordinate data of the boundary line; An effective picture detection module, configured to split the video into multiple sub-pictures according to the split-screen position information in the split-screen detection result; perform effectiveness detection on each sub-picture to determine the effective picture of the video.

6. The device according to claim 5, characterized in that, The edge detection module is further configured to perform grayscale processing on each frame of color image in the image sequence to obtain a grayscale image sequence; perform edge detection on each frame of grayscale image in the grayscale image sequence, and identify edge pixel points whose brightness difference from adjacent pixel points is greater than a preset threshold in the grayscale image.

7. The device according to claim 5, characterized in that, The effective picture detection module is further configured to perform clarity identification on the sub-pictures. When the clarity of a sub-picture is less than a preset clarity threshold, determine that the sub-picture is an invalid blurred picture; perform inter-frame difference detection on adjacent frames of the same sub-picture. When the content of the sub-picture in adjacent frames is the same, determine that the sub-picture is an invalid static picture; when the clarity of the sub-picture is not less than the preset clarity threshold and the content of the sub-picture in adjacent frames is different, determine that the sub-picture is an effective picture.

8. The device according to claim 5 or 7, characterized in that, The effective picture detection module is further configured to obtain the similarity of each sub-picture. When it is determined that all sub-pictures are pictures with the same content according to the similarity, select any one of the sub-pictures as the effective picture.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images