Server and multi-screen short video screening method

Abstract

The embodiment of the present application provides a server and a method for screening multi-screen short videos, wherein the server is configured to: obtain pixel points of an image frame in a short video according to the short video; grayscale the pixel points to obtain a grayscale image frame; determine a split-screen boundary according to the split-screen type of the target short video, and determine a detection area in the grayscale image frame according to the split-screen boundary, wherein the detection area refers to an area within a preset range on both sides of the split-screen boundary; calculate the grayscale change value of the pixel value within the detection area using an image clarity algorithm, and the grayscale change value is used to characterize the clarity of the image; compare the grayscale change value with a first preset threshold value, and determine whether the split-screen type of the short video is consistent with that of the target short video based on the comparison result. The present application can quickly determine whether the short video to be detected has the same split-screen type as the target short video to be screened by performing image clarity detection on a specific detection area, thereby completing the detection and screening of the split-screen type of the short video.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a server and a method for screening multi-screen short videos. Background Art

[0002] Short videos are videos under 30 minutes long and are played on various new media platforms, suitable for viewing in a leisurely state. They incorporate topics such as skill sharing, humor, fashion trends, social hot topics, street interviews, public welfare education, advertising creativity, and commercial customization. With the rise of 5G technology, the growth of short video content has exploded.

[0003] Currently, users create short videos on various short video platforms and often add special effects for processing. One such process involves creating multi-screen short videos through split-screen effects. These typically include two-screen short videos, three-screen short videos, four-screen short videos, six-screen short videos, and nine-screen short videos. Detecting multi-screen short videos, such as three-screen short videos, involves collecting a large number of three-screen and non-three-screen short videos, labeling them, and then training a deep learning model for identification. However, the data collection and model training processes in these detection methods are both cumbersome and time-consuming. Summary of the Invention

[0004] The present application provides a server and a method for screening multi-screen short videos to solve the technical problems in the prior art of cumbersome and time-consuming process when detecting multi-screen short videos.

[0005] In a first aspect, some embodiments of the present application provide a server, wherein the server is configured to:

[0006] Obtaining pixel points of an image frame in the short video according to the short video;

[0007] Grayscale processing is performed on the pixel points to obtain a grayscale image frame;

[0008] Determining a split-screen boundary according to the split-screen type of the target short video, and determining a detection area in the grayscale image frame according to the split-screen boundary, wherein the detection area refers to an area within a preset range on both sides of the split-screen boundary;

[0009] Calculating a grayscale change value of pixel values within the detection area using an image clarity algorithm, wherein the grayscale change value is used to characterize the clarity of the image;

[0010] The grayscale change value is compared with a first preset threshold value, and whether the split-screen type of the short video is consistent with the split-screen type of the target short video is determined based on the comparison result.

[0011] In a second aspect, some embodiments of the present application provide a method for screening multi-screen short videos, characterized in that the method includes:

[0012] Obtaining pixel points of an image frame in the short video according to the short video;

[0013] Grayscale processing is performed on the pixel points to obtain a grayscale image frame;

[0014] Determining a split-screen boundary according to the split-screen type of the target short video, and determining a detection area in the grayscale image frame according to the split-screen boundary, wherein the detection area refers to an area within a preset range on both sides of the split-screen boundary;

[0015] Calculating a grayscale change value of pixel values within the detection area using an image clarity algorithm, wherein the grayscale change value is used to characterize the clarity of the image;

[0016] The grayscale change value is compared with a first preset threshold value, and whether the split-screen type of the short video is consistent with the split-screen type of the target short video is determined based on the comparison result.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] The present application provides a server and a method for screening multi-screen short videos. Before screening short videos, it is necessary to first divide the short video into continuous image frames, obtain the pixel matrix of each frame, and grayscale the pixels to obtain several grayscale image frames of the short video. According to the split-screen type of the target short video to be screened, the split-screen boundary that should be present in the image frame with the split-screen type is determined. The determined split-screen boundary is taken as the center, and areas within a preset range on both sides of the split-screen boundary are selected to obtain multiple detection areas. The server uses an image clarity algorithm to detect the grayscale change value of the pixel value in the detection area. By comparing the grayscale change value with a first preset threshold, it can be determined whether the split-screen type of the current short video is consistent with the split-screen type of the target short video to be screened. In the present application, a specific detection area is determined in the image frame to avoid image clarity calculation for the entire image. By performing image clarity detection on the specific detection area, it is possible to quickly determine whether the short video to be detected has the same split-screen type as the target short video to be screened, thereby completing the detection and screening of the short video split-screen type. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 exemplarily shows a display effect diagram of a three-screen short video in a smart TV according to some embodiments;

[0021] Figure 2 exemplarily shows a schematic diagram of the display effect of a three-screen short video in a mobile terminal according to some embodiments;

[0022] Figure 3 exemplarily shows another schematic diagram of the display effect of a three-screen short video in a mobile terminal according to some embodiments;

[0023] Figure 4 hereinafter is a flow chart showing a method for screening multi-screen short videos according to some embodiments;

[0024] Figure 5 exemplarily shows a schematic diagram of the detection area division of a three-screen short video according to some embodiments;

[0025] Figure 6 exemplarily shows a schematic diagram of the detection area division of a nine-screen short video according to some embodiments;

[0026] Figure 7 exemplarily shows a schematic diagram of the detection area division of two-screen short videos according to some embodiments;

[0027] Figure 8 exemplarily shows a schematic diagram of the detection area division of a four-screen short video according to some embodiments;

[0028] Figure 9 exemplarily shows a display effect diagram of a short video on six horizontal screens in a smart TV according to some embodiments;

[0029] Figure 10 exemplarily shows a display effect diagram of a vertical six-screen short video in a smart TV according to some embodiments;

[0030] Figure 11 exemplarily shows another schematic diagram of the display effect of a four-screen short video according to some embodiments;

[0031] Figure 12 Schematic diagram of cropping effective areas in image frames in multi-screen short videos according to some embodiments is shown in FIG. DETAILED DESCRIPTION

[0032] In order to make the purpose and implementation of this application clearer, the exemplary implementation of this application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only part of the embodiments of this application, not all of the embodiments.

[0033] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0034] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0035] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0036] With the popularity of mobile terminals and smart TVs and the acceleration of the Internet, short videos have become the most popular form of information dissemination. More and more users are browsing short videos in their spare time, and short videos effectively meet everyone's leisure entertainment needs. Currently, when users create short videos on different short video platforms, they will use some special effects for processing. One of them is to create a short video into a multi-screen short video through split-screen special effects. Here, combined with the main presentation format of short video content, multi-screen short videos can include horizontal multi-screen short videos and vertical multi-screen short videos. Here, vertical multi-screen short videos refer to video media with a screen height greater than the width, and horizontal multi-screen short videos refer to video media with a height less than the width.

[0037] Figure 1 FIG 2 shows an exemplary display effect diagram of a three-screen short video in a smart TV according to some embodiments. Figure 1 As shown, the smart TV displays a vertical three-screen short video on the display interface, among which only the content of the middle split screen is the valid content in the video, and the content displayed in the left and right split screens is a blurred part. Figure 2 FIG. 4 shows an exemplary diagram of a display effect of a three-screen short video in a mobile terminal according to some embodiments. Figure 2 As shown, the mobile terminal displays three horizontal short video screens on the display interface, wherein the content of the middle split screen is the playback content of the short video, and the content in the upper and lower split screens is detailed information related to the played short video, such as the video title, video author and video introduction. Figure 3 FIG2 exemplarily shows another schematic diagram of the display effect of a three-screen short video in a mobile terminal according to some embodiments. Figure 3 , also shows a horizontal three-screen short video, and Figure 2 The difference is, Figure 3The content displayed in the three split screens is the same.

[0038] Video content analysis (VCA) uses artificial intelligence technology to perform multi-dimensional video analysis of speech, text, faces, objects, and scenes, and outputs universal video tags for use in multimedia content management, review, search, and recommendation. The typical implementation process for video content analysis involves dividing a video into fixed-length segments and sequentially feeding each segment into a neural network model for feature extraction. The extracted features vary depending on the analysis direction. Finally, the extracted features are fed into a classifier for video tag classification. For multi-screen short videos, the content displayed within each split screen is often repetitive, or only one split screen has clear content while the others are blurred. When performing video content analysis on a multi-screen short video and feeding it into a neural network model, repetitive or blurred content within the split screens can affect feature extraction, leading to inaccurate or erroneous classification results. For example, if a video content analysis is performed on a multi-screen short video to count the number of times actor A appears in the video, the repeated scenes in the multi-screen short video will result in an inaccurate count of actor A's appearances. Therefore, before performing video content analysis on multi-screen short videos, it is necessary to screen the multi-screen short videos, perform different preprocessing on short videos of different split-screen types, and then perform video content analysis to ensure the accuracy of video content analysis.

[0039] In the related art, for the detection method of multi-screen short videos, such as the detection of three-screen short videos, a large number of data samples can be collected, namely three-screen short videos and non-three-screen short videos, and labeled respectively, and then a deep learning model is trained for identification. However, in the above detection method, the process of data collection and model training is relatively cumbersome and time-consuming. In order to solve the above problem, the present application provides a server in some embodiments, which is configured to perform a screening process for multi-screen short videos.

[0040] The following describes the multi-screen short video screening process with reference to the accompanying drawings.

[0041] Figure 4 A flowchart of a method for screening multi-screen short videos according to some embodiments is shown in FIG. Figure 4 As shown, the screening process of the multi-screen short video is as follows:

[0042] S401: Obtain pixel points of an image frame in the short video according to the short video.

[0043] In some embodiments, the short video is composed of continuous image frames, which are generally RGB three-channel color image frames. Before performing image processing, it is necessary to extract the pixel data of each corresponding group of image frames through the short video. In some embodiments, the short video can be input into a video reading tool to obtain the pixel matrix of each frame of the image. For example, a short video file in mp4 format is input into a tool such as imageio or cv2 to extract the pixel points of the corresponding image frame in the short video. For an image frame of size [W, H], a W×H pixel matrix can be obtained.

[0044] S402: grayscale the pixel points to obtain a grayscale image frame.

[0045] In some embodiments, when processing a color image frame, it is necessary to process the three RGB channels, and the amount of data required to be processed is large. In order to simplify the W×H pixel matrix and improve the computing speed, the RGB three-channel color image frame can be converted into a single-channel grayscale image frame.

[0046] In some embodiments, the pixel points can be grayscaled using an integer method, that is, for each pixel point in a color image frame of RGB three channels of size [W, H], the grayscale pixel value gray_pixle = (30*R+59*G+11*B) / 100, where R, G, and B are the pixel values of the three channels of the color image frame, respectively, and finally a W×H grayscale image matrix G is obtained. Of course, the pixel points can also be grayscaled using a floating-point method, and the grayscale pixel value gray_pixle = R*0.3+G*0.59+B*0.11. The pixel points can also be grayscaled using a shift method, and the grayscale pixel value gray_pixle = (R*77+G*151+B*28)>>8. Here, there is no limitation on the method of grayscale processing of the pixel points.

[0047] S403: Determine a split-screen boundary according to the split-screen type of the target short video, and determine a detection area in the grayscale image frame according to the split-screen boundary, wherein the detection area refers to an area within a preset range on both sides of the split-screen boundary.

[0048] In some embodiments, the location of the corresponding split-screen boundary varies depending on the split-screen type of the multi-screen short video. Here, the split-screen type may refer to the number of display windows in the display interface, i.e., the number of split screens. The dividing line between each display window is the split-screen boundary. The split-screen types of multi-screen short videos generally include two-screen short videos, three-screen short videos, four-screen short videos, six-screen short videos, and nine-screen short videos.

[0049] The following describes the process of determining the split-screen boundary and the detection area, taking the target short video to be detected as a three-screen short video as an example.

[0050] Since the number of split screens in a three-screen short video is 3, that is, the number of display windows in the display interface is 3, and since we only know that the target short video is a three-screen short video, but not whether it is a vertical three-screen short video or a horizontal three-screen short video, the server can find the points where the grayscale image frame is divided into three equal parts in the width and height directions, respectively, to determine the detection area in both the width and height directions.

[0051] In some embodiments, Figure 5 exemplarily shows a schematic diagram of the detection area division of a three-screen short video according to some embodiments. Figure 5 In the three-screen short video, the image frame has a height of H and a width of W. At the points (W / 3) dividing the width of the grayscale image frame into three equal parts, a vertical split-screen boundary is drawn in the vertical direction. The vertical split-screen boundary is used to divide the grayscale image frame into three equal parts in the width direction. The detection area determined according to the vertical split-screen boundary is the vertical detection area. At the points (H / 3) dividing the height of the grayscale image frame into three equal parts, a horizontal split-screen boundary is drawn in the horizontal direction. The horizontal split-screen boundary is used to divide the grayscale image frame into three equal parts in the height direction. The detection area determined according to the horizontal split-screen boundary is the horizontal detection area.

[0052] In some embodiments, after the screen division lines are drawn at the points dividing the grayscale image frame into three equal parts, the areas within the preset range on both sides of the screen division line are selected as the detection areas. Figure 5 For the two vertical split-screen boundaries, the left and right areas with a size of dw are selected respectively to determine the two vertical detection areas, which can be expressed as [W / 3-dw:W / 3+dw,H] and [2W / 3-dw:2W / 3+dw,H] respectively. Similarly, for the two horizontal split-screen boundaries, the left and right areas with a size of dh are selected respectively to determine the two horizontal detection areas, which can be expressed as [W,H / 3-dh:H / 3+dh] and [W,2H / 3-dh:2H / 3+dh] respectively. Among them, dw and dh are generally taken as 5, that is, only 5 pixels of width need to be considered on one side of the split-screen boundary. From the above, it can be clearly seen that when the target short video is a three-screen short video, 4 detection areas need to be determined in the grayscale image frame of the short video.

[0053] Figure 6 The figure shows a schematic diagram of the detection area division of nine-screen short videos according to some embodiments, combined with Figure 6,When the target short video is a nine-screen short video, the server also needs to find the points where the grayscale image frame is divided into three equal parts in the width and height directions, and determine four detection areas. Figure 5 It's not difficult to find that when the target short video is a three-screen short video or a nine-screen short video, the number of detection areas determined is the same, and the relative positions of the detection areas in the grayscale image frame are the same. It should be noted that the same relative positions in the grayscale image frame refer to the screen boundaries and detection areas being determined at the points dividing the grayscale image frame into three equal parts.

[0054] The following describes the process of determining the split-screen boundary and the detection area, taking the case where the split-screen type of the target short video to be detected is a two-screen short video as an example.

[0055] Since the number of split screens for a two-screen short video is 2, that is, the number of display windows in the display interface is 2, and since we only know that the target short video is a two-screen short video, and we don't know whether it is a vertical two-screen short video or a horizontal two-screen short video, the server can find the points that divide the grayscale image frame into two equal parts in the width and height directions, respectively, to determine the detection area in both the width and height directions.

[0056] Figure 7 FIG2 exemplarily shows a schematic diagram of the detection area division of two-screen short videos according to some embodiments. Figure 7 After drawing a split-screen boundary at the bisection point of the grayscale image frame, we select a preset area on both sides of the split-screen boundary as the detection area. Clearly, when the target short video is a two-screen short video, we need to determine two detection areas in the grayscale image frame of the short video.

[0057] Figure 8 FIG2 exemplarily shows a schematic diagram of the detection area division of a four-screen short video according to some embodiments. Figure 8 When the target short video is a four-screen short video, the server also needs to find the bisection points of the grayscale image frame in the width and height directions, and determine two detection areas for comparison. Figure 8 and Figure 7 When the target short video is a two-screen short video or a four-screen short video, the number of the determined detection areas is the same, and the relative positions in the grayscale image frame are the same.

[0058] The following describes the process of determining the split-screen boundary and the detection area, taking the target short video with a six-screen split-screen type as an example.

[0059] Figure 9 FIG2 exemplarily shows a display effect diagram of a short video on six horizontal screens in a smart TV according to some embodiments. Figure 10FIG2 shows an exemplary display effect diagram of a six-screen short video in a vertical orientation in a smart TV according to some embodiments. Figure 9 and Figure 10 When the target short video is a six-screen short video, the number of split screens is 6, that is, the number of display windows in the display interface is 6. In addition, it is necessary to specifically filter out whether the target short video is a vertical six-screen short video or a horizontal six-screen short video. Therefore, the server needs to cover Figure 9 and Figure 10 In both cases, we can first find the bisection and trisection points in the width direction of the grayscale image frame, and then find the bisection and trisection points in the height direction of the grayscale image frame. Based on the found bisection points, we can determine the split-screen boundaries and then determine the detection areas. When the target short video is a six-screen short video, we need to determine six detection areas in the grayscale image frames of the short video.

[0060] It should be noted that the four-screen short videos, six-screen short videos and nine-screen short videos mentioned above refer to grid-shaped multi-screen short videos with both vertical and horizontal split-screen boundaries.

[0061] In some embodiments, four-screen short videos, six-screen short videos, and nine-screen short videos may also be multi-screen short videos with only one type of split-screen boundary (vertical split-screen boundary or horizontal split-screen boundary). Figure 11 FIG2 exemplarily shows another schematic diagram of the display effect of a four-screen short video according to some embodiments. Figure 11 In this four-screen short video, the screen is divided into four parts only by the vertical split-screen boundary. For multi-screen short videos with only one type of split-screen boundary, when determining the split-screen boundary, refer to the split-screen boundary determination method of the two-screen short video and the three-screen short video mentioned above. The server can determine the number of equal division points based on the split-screen type of the target short video, and then determine the split-screen boundary at the corresponding equal division points in the width and height directions of the grayscale image frame, so as to determine the detection area in both the width and height directions.

[0062] The above specifically describes the process of how to determine the corresponding detection area based on different target short videos.

[0063] S404: Calculate the grayscale change value of the pixel values in the detection area using an image clarity algorithm, where the grayscale change value is used to represent the clarity of the image.

[0064] In some embodiments, after determining the detection area, the server can read the pixels in the vertical detection area and the horizontal detection area to obtain multiple grayscale matrices. For each of the multiple grayscale matrices, the gradient average of the pixel values in each grayscale matrix is calculated using the Brenner gradient algorithm.

[0065] In some embodiments, in the vertical detection area, the initial gradient average values between two adjacent columns are calculated sequentially, the initial gradient average values within the same vertical detection area are summed, and then normalized by comparing them with the height of the grayscale image frame to obtain a standardized gradient average value. In the horizontal detection area, the initial gradient average values between two adjacent rows are calculated sequentially, the initial gradient average values within the same horizontal detection area are summed, and then normalized by comparing them with the width of the grayscale image frame to obtain a standardized gradient average value.

[0066] For example, taking the vertical detection area as an example, for a grayscale image matrix G in a three-screen short video of size W×H, read the grayscale matrix G1 = G[W / 3-dw:W / 3+dw,H] in one of the vertical detection areas. Calculate the initial gradient averages of the i-th column and the i-1-th column in the vertical detection area, sum them, and then divide by the scale in that direction to normalize and obtain the standard gradient average B(i). The calculation formula is as follows:

[0067] B(i)=sum{(G1[i,:]-G1[i-1,:])2} / H

[0068] Wherein, “:” represents the specific number of rows, and its value range is 0 to H.

[0069] Similarly, the calculation formula of the standard gradient average value B(i) in the horizontal detection area is as follows:

[0070] B(i)=sum{(G1[:,i]-G1[:,i-1])2} / W

[0071] Wherein, “:” represents the specific number of columns, and its value range is 0 to W.

[0072] S405: Compare the grayscale change value with a first preset threshold value, and determine whether the split-screen type of the short video is consistent with the split-screen type of the target short video based on the comparison result.

[0073] In some embodiments, if all standard gradient averages corresponding to the vertical detection area are less than a first preset threshold, the short video is determined to be a vertical multi-screen short video that is consistent with the split-screen type of the target short video to be detected. If all standard gradient averages corresponding to the horizontal detection area are less than a first preset threshold, the short video is determined to be a horizontal multi-screen short video that is consistent with the split-screen type of the target short video to be detected.

[0074] For a three-screen short video, the standardized gradient averages calculated for the vertical detection area are B_h1 and B_h2, and the standardized gradient averages calculated for the horizontal detection area are B_w1 and B_w2. If both B_h1 and B_h2 are less than the first preset threshold [Brenner_Confidence], the short video is a vertical three-screen short video. If both B_w1 and B_w2 are less than the first preset threshold [Brenner_Confidence], the short video is a horizontal three-screen short video.

[0075] For example, for a certain three-screen short video, it is detected that B_h1=3.45, B_h2=3.87, B_w1=0.13, B_w2=0.15, and the first preset threshold is 1. Since B_w1 and B_w2 are both smaller than the first preset threshold, the three-screen short video is a horizontal three-screen short video.

[0076] In some embodiments, the server records, through a multi-screen value, whether the split-screen type in the grayscale image frame determined by the grayscale change value in the detection area is consistent with the split-screen type of the target short video to be detected. For example, if the multi-screen value is 1, it indicates that the split-screen type in the grayscale image frame determined by the grayscale change value in the detection area is consistent with the split-screen type of the target short video. If the multi-screen value is 0, it indicates that the split-screen type in the grayscale image frame determined by the grayscale change value in the detection area is inconsistent with the split-screen type of the target short video.

[0077] In some embodiments, if the split-screen type in the grayscale image frame is determined to be consistent with the split-screen type of the target short video through the vertical detection area or the horizontal detection area, the multi-screen value of the corresponding vertical detection area or the horizontal detection area is marked as 1. If the split-screen type in the grayscale image frame is determined to be inconsistent with the split-screen type of the target short video through the vertical detection area or the horizontal detection area, the multi-screen value of the corresponding vertical detection area or the horizontal detection area is marked as 0.

[0078] Taking the target short video as a three-screen short video as an example, the server first determines whether the current short video is a three-screen short video based on the standard gradient value calculated through the vertical detection area. If the current short video is a three-screen short video, the multi-screen value of the vertical detection area is recorded as 1; otherwise, it is recorded as 0. Then, based on the standard gradient value calculated through the horizontal detection area, the server determines whether the current short video is a three-screen short video. If the current short video is a three-screen short video, the multi-screen value of the horizontal detection area is recorded as 1; otherwise, it is recorded as 0.

[0079] In some embodiments, for a grayscale image frame in a short video to be detected, the multi-screen values of the vertical detection area and the horizontal detection area are added together to obtain the multi-screen value of the current grayscale image frame.

[0080] In some embodiments, when the multi-screen value of the current grayscale image frame is 1, the current grayscale image frame is determined to be an image frame in a vertical multi-screen short video or a horizontal multi-screen short video. For example, the multi-screen value of the vertical detection area of the current grayscale image frame is 1, and the multi-screen value of the horizontal detection area is 0. Alternatively, the multi-screen value of the vertical detection area is 0, and the multi-screen value of the horizontal detection area is 1, then the sum of the two is 1, so the current grayscale image frame is correspondingly an image frame in a vertical multi-screen short video or a horizontal multi-screen short video, and its multi-screen value of the vertical detection area is 1, and the multi-screen value of the horizontal detection area is 0.

[0081] In some embodiments, when the multi-screen value of the current grayscale image frame is 2, the current grayscale image frame is determined to be an image frame in a grid-shaped multi-screen short video with both vertical and horizontal split-screen boundaries. Figure 5 The three-screen short video and Figure 6 For a three-screen short video, only one of the multi-screen values for the horizontal and vertical detection areas is 1. However, for a nine-screen short video, both the multi-screen value for the vertical detection area and the multi-screen value for the horizontal detection area are 1. This means that both vertical and horizontal split-screen boundaries exist in the grayscale image frames of the nine-screen short video.

[0082] In some embodiments, when the multi-screen value of the current grayscale image frame is 0, it is determined that the current grayscale image frame is not an image frame in the multi-screen short video.

[0083] In some embodiments, because errors may occur during the server's calculation process, a preset number of grayscale image frames may be randomly selected from multiple consecutive grayscale image frames of the short video, and the multi-screen values of each of the selected grayscale image frames may be calculated. The server sums the multi-screen values of the selected grayscale image frames and compares them with the preset number to obtain a multi-screen ratio for the short video. If the multi-screen ratio is greater than or equal to a second preset threshold, the server determines that the split-screen type of the short video is consistent with the split-screen type of the target short video.

[0084] For example, for a short video, N grayscale image frames are randomly sampled, and the multi-screen values of the N grayscale image frames are calculated respectively. The multi-screen rate calculation formula of the short video is as follows:

[0085] Multi-screen rate = (sum of multi-screen values of N grayscale image frames) / N×100%

[0086] If the multi-screen rate is greater than or equal to a second preset threshold (eg, 50%), it indicates that the short video is a multi-screen video; otherwise, it is a normal video.

[0087] In some embodiments, after the above steps, after the multi-screen short videos are screened out, if the multi-screen short videos are analyzed for video content, they can be cropped first. Figure 5 The short video shown is detected and screened as a vertical three-screen short video. All image frames in the short video are cropped and selected. Figure 12 The effective area in the image frame of the multi-screen short video shown is the middle screen picture, and finally the middle screen picture is used to perform video content analysis.

[0088] In this application, a specific detection area is determined in the image frame to avoid image clarity calculation for the entire image. By performing image clarity detection on a specific detection area, it is possible to quickly determine whether the short video to be detected has the same split-screen type as the target short video to be screened, thereby completing the detection and screening of the short video split-screen type.

[0089] Corresponding to the above-mentioned server, some embodiments of the present application also provide a method for screening multi-screen short videos, the method comprising: the server obtains pixel points of the image frame in the short video based on the short video, grayscales the pixel points, and obtains a grayscale image frame. The server determines the split-screen boundary according to the split-screen type of the target short video, and determines the detection area in the grayscale image frame according to the split-screen boundary, wherein the detection area refers to the area within a preset range on both sides of the split-screen boundary. The server uses an image clarity algorithm to calculate the grayscale change value of the pixel value in the detection area, and the grayscale change value is used to characterize the clarity of the image. The server compares the grayscale change value with a first preset threshold, and determines whether the split-screen type of the short video is consistent with the split-screen type of the target short video based on the comparison result.

[0090] In some embodiments, when the target short video is a three-screen short video, the determination of the split-screen boundary according to the split-screen type of the target short video includes: at the points where the width of the grayscale image frame is divided into three equal parts, the server makes vertical split-screen boundaries in the vertical direction, wherein the vertical split-screen boundaries are used to divide the grayscale image frame into three equal parts in the width direction, and the detection area determined according to the vertical split-screen boundaries is the vertical detection area. At the points where the height of the grayscale image frame is divided into three equal parts, the server makes horizontal split-screen boundaries in the horizontal direction, wherein the horizontal split-screen boundaries are used to divide the grayscale image frame into three equal parts in the height direction, and the detection area determined according to the horizontal split-screen boundaries is the horizontal detection area.

[0091] Since the above embodiments are all described by reference in combination with other embodiments, different embodiments have the same parts, and the same and similar parts between the various embodiments in this specification can be referred to each other. No further detailed explanation is given here.

[0092] It should be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such circuit structure, article or device. In the absence of further restrictions, the presence of an element defined by the phrase "includes a..." does not exclude the presence of other identical elements in the circuit structure, article or device comprising the element.

[0093] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the disclosure of the invention herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0094] The above embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A server, characterized in that: The server is configured to: Obtaining pixel points of an image frame in the short video according to the short video; Grayscale processing is performed on the pixel points to obtain a grayscale image frame; Determining the number of split screens according to the split screen type of the target short video, and determining corresponding equal division points in the width direction and / or height direction of the grayscale image frame according to the number of split screens; Determine a screen splitting boundary according to the equally divided points, and take the screen splitting boundary as the center, selecting an area within a preset range on both sides of the screen splitting boundary as the detection area; Calculating a grayscale change value of pixel values within the detection area using an image clarity algorithm, wherein the grayscale change value is used to characterize the clarity of the image; Comparing the grayscale change value with a first preset threshold, and determining whether the split-screen type of the short video is consistent with the split-screen type of the target short video according to the comparison result; The step of calculating the grayscale change value of the pixel values within the detection area by using an image clarity algorithm includes: Read the pixel points in the vertical detection area and the horizontal detection area respectively to obtain multiple grayscale matrices; The Brenner gradient algorithm is used to calculate the gradient average of the pixel values in each grayscale matrix.

2. The server according to claim 1, wherein: When the target short video is a three-screen short video, in the step of determining the split-screen boundary according to the split-screen type of the target short video, the server is configured to: At points dividing the width of the grayscale image frame into three equal parts, vertical split screen boundaries are drawn in the vertical direction, wherein the vertical split screen boundaries are used to divide the grayscale image frame into three equal parts in the width direction, and the detection area determined according to the vertical split screen boundaries is the vertical detection area; At the height three-division points of the grayscale image frame, horizontal screen division lines are drawn in the horizontal direction, wherein the horizontal screen division lines are used to divide the grayscale image frame into three equal parts in the height direction, and the detection area determined according to the horizontal screen division lines is the horizontal detection area.

3. The server according to claim 1, wherein: In the step of calculating the gradient average of the pixel values in each grayscale matrix using the Brenner gradient algorithm, the server is configured to: In the vertical detection area, sequentially calculating initial gradient averages between two adjacent columns, summing the initial gradient averages in the same vertical detection area, and then normalizing the sum by comparing it with the height of the grayscale image frame to obtain a standard gradient average; In the horizontal detection area, the initial gradient average values between two adjacent rows are calculated in sequence, and the initial gradient average values in the same horizontal detection area are summed and then normalized by comparing with the width of the grayscale image frame to obtain a standard gradient average value.

4. The server according to claim 3, wherein: In the step of comparing the grayscale change value with a first preset threshold and determining whether the split-screen type of the short video is consistent with the split-screen type of the target short video according to the comparison result, the server is configured to: If all standard gradient averages corresponding to the vertical detection area are less than a first preset threshold, determining that the short video is a vertical multi-screen short video consistent with the split-screen type of the target short video; If all standard gradient averages corresponding to the horizontal detection area are less than a first preset threshold, it is determined that the short video is a horizontal multi-screen short video that is consistent with the split-screen type of the target short video.

5. The server according to claim 1, wherein: The server is further configured to: If the split-screen type in the grayscale image frame is determined to be consistent with the split-screen type of the target short video through the vertical detection area or the horizontal detection area, the multi-screen value of the corresponding vertical detection area or the horizontal detection area is marked as 1, wherein the multi-screen value is used to indicate whether the split-screen type in the grayscale image frame determined by the grayscale change value in the detection area is consistent with the split-screen type of the target short video; If the split-screen type in the grayscale image frame is determined to be inconsistent with the split-screen type of the target short video through the vertical detection area or the horizontal detection area, the multi-screen value of the corresponding vertical detection area or the horizontal detection area is marked as 0.

6. The server according to claim 5, wherein: The server is further configured to: Adding the multi-screen values of the vertical detection area and the horizontal detection area to obtain a multi-screen value of the current grayscale image frame; When the multi-screen value of the current grayscale image frame is 1, it is determined that the current grayscale image frame is an image frame in a vertical multi-screen short video or a horizontal multi-screen short video; When the multi-screen value of the current grayscale image frame is 2, it is determined that the current grayscale image frame is an image frame in a grid-shaped multi-screen short video in which vertical split-screen boundaries and horizontal split-screen boundaries coexist; When the multi-screen value of the current grayscale image frame is 0, it is determined that the current grayscale image frame is not an image frame in the multi-screen short video.

7. The server according to claim 1, wherein: The server is further configured to: Randomly selecting a preset number of grayscale image frames from a plurality of continuous grayscale image frames of the short video; Calculating the multi-screen value of each selected grayscale image frame, wherein the multi-screen value is used to indicate whether the split-screen type in the grayscale image frame determined by the grayscale change value in the detection area is consistent with the split-screen type of the target short video; Summing the multi-screen values of the selected grayscale image frames and comparing the sum with the preset number to obtain the multi-screen rate of the short video; When the multi-screen rate is greater than or equal to a second preset threshold, it is determined that the split-screen type of the short video is consistent with the split-screen type of the target short video.

8. A method for screening multi-screen short videos, characterized in that: The method comprises: Obtaining pixel points of an image frame in the short video according to the short video; Grayscale processing is performed on the pixel points to obtain a grayscale image frame; Determining the number of split screens according to the split screen type of the target short video, and determining corresponding equal division points in the width direction and / or height direction of the grayscale image frame according to the number of split screens; Determine a screen splitting boundary according to the equally divided points, and take the screen splitting boundary as the center, selecting an area within a preset range on both sides of the screen splitting boundary as the detection area; Calculating a grayscale change value of pixel values within the detection area using an image clarity algorithm, wherein the grayscale change value is used to characterize the clarity of the image; Comparing the grayscale change value with a first preset threshold, and determining whether the split-screen type of the short video is consistent with the split-screen type of the target short video according to the comparison result; The step of calculating the grayscale change value of the pixel values within the detection area by using an image clarity algorithm includes: Read the pixel points in the vertical detection area and the horizontal detection area respectively to obtain multiple grayscale matrices; The Brenner gradient algorithm is used to calculate the gradient average of the pixel values in each grayscale matrix.

9. The method for screening multi-screen short videos according to claim 8, characterized in that: When the target short video is a three-screen short video, determining the split-screen boundary according to the split-screen type of the target short video includes: At points dividing the width of the grayscale image frame into three equal parts, vertical split screen boundaries are drawn in the vertical direction, wherein the vertical split screen boundaries are used to divide the grayscale image frame into three equal parts in the width direction, and the detection area determined according to the vertical split screen boundaries is the vertical detection area; At the height three-division points of the grayscale image frame, horizontal screen division lines are drawn in the horizontal direction, wherein the horizontal screen division lines are used to divide the grayscale image frame into three equal parts in the height direction, and the detection area determined according to the horizontal screen division lines is the horizontal detection area.

Images