Rotating background video recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for identifying a rotating background video, the method comprising: extracting two or more target video frames from a video to be reviewed; when it is determined based on the target video frames that the video to be reviewed contains a background split screen, and the background split screen meets a preset condition, selecting a reference point sequence in the background split screen of each target video frame; performing similarity matching on the reference point sequences in each adjacent two target video frames to obtain a similarity matching result between each adjacent two target video frames; and determining whether the video to be reviewed is a rotating background video based on each similarity matching result. The above method selects a reference point sequence in the background split screen of the extracted video frame, and based on the result of similarity matching of the reference point sequences of the two adjacent video frames, it can be determined whether there is a similar reference point sequence in the two adjacent video frames, and can more accurately identify whether the video to be reviewed is a rotating background video.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, computer equipment and storage medium for identifying a rotating background video. Background Art

[0002] With the development of video technology, short videos have gradually become an important way of content dissemination. As the threshold for content production is lowered, the output of videos has grown rapidly. In order to ensure the legality of the published content, videos must be reviewed before they are released to identify and filter out videos that do not comply with regulations or are of low quality.

[0003] A large number of horizontal videos are re-produced into vertical videos suitable for display on certain terminals. Among them, a commonly used method of re-producing horizontal videos is to fill the blank space with background canvas on the top and bottom of the horizontal video, and the background canvas rotates with the center of the video as the rotation center. However, this type of background usually affects the user's perception and destroys the beauty of the picture. Therefore, when reviewing videos, it is necessary to identify such videos that fill the top and bottom with rotating canvas as the background. Summary of the invention

[0004] Based on this, it is necessary to provide a rotating background video recognition method, device, computer equipment and storage medium that can recognize rotating background videos in response to the above technical problems.

[0005] A rotating background video recognition method, the method comprising:

[0006] Extracting two or more target video frames from the video to be reviewed;

[0007] When it is determined according to the target video frame that the video to be reviewed contains a background split screen, and the background split screen meets a preset condition, respectively selecting a reference point sequence in the background split screen of each target video frame;

[0008] Performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames;

[0009] Determine whether the video to be reviewed is a rotating background video based on each of the similarity matching results.

[0010] A rotating background video recognition device, the device comprising:

[0011] A video frame extraction module, used to extract two or more target video frames from the video to be reviewed;

[0012] A reference point sequence selection module, for selecting reference point sequences in the background split screens of each target video frame respectively when it is determined according to the target video frame that the video to be reviewed contains a background split screen and the background split screen meets a preset condition;

[0013] A matching module, used for performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames;

[0014] The judgment module is used to determine whether the video to be reviewed is a rotating background video according to each similarity matching result.

[0015] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned rotating background video recognition method when executing the computer program.

[0016] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned rotating background video recognition method when executed by a processor.

[0017] The above-mentioned rotating background video recognition method, device, computer equipment and storage medium extract some video frames from the video to be reviewed as target video frames. When it is determined based on the target video frames that the video to be reviewed contains a background split screen and the background split screen meets the preset conditions, a reference point sequence is selected in the background split screen, and then the reference point sequences in two adjacent target video frames are matched for similarity, and whether the video to be reviewed is a rotated background video is determined based on the similarity matching result; the above-mentioned method extracts video frames from the video, selects a reference point sequence in the background split screen of the extracted video frames, and performs similarity matching based on the reference point sequences of two adjacent video frames. Based on the similarity matching result, it can be determined whether there are similar reference point sequences in the two adjacent video frames, and then based on whether there are similar reference point sequences in all adjacent video frames in the video to be reviewed, it can be more accurately identified whether the video to be reviewed is a rotating background video. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A diagram showing an application environment of a rotating background video recognition method in an embodiment;

[0019] Figure 2 It is a schematic diagram of a process of a rotating background video recognition method in one embodiment;

[0020] Figure 3 A schematic diagram of a video obtained by filling the background above and below the video in a specific embodiment of an embodiment;

[0021] Figure 4A schematic diagram of a flow chart of performing similarity matching on reference point sequences in each two adjacent target video frames to obtain similarity matching results between each two adjacent target video frames in an embodiment;

[0022] Figure 5 is a schematic diagram of a reference point sequence in the first frame of the current video frame group in a specific embodiment;

[0023] Figure 6 is a flow chart of a rotating background video recognition method in another embodiment;

[0024] Figure 7 A schematic diagram of a flow chart of a background video recognition method in a specific embodiment;

[0025] Figure 8 is a schematic diagram of a target video frame in a specific embodiment;

[0026] Fig. 9 is a structural block diagram of a rotating background video recognition device in one embodiment;

[0027] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The rotating background video recognition method provided by the present application can, in some embodiments, be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. After the server 104 obtains the video to be reviewed from the terminal, for the video to be reviewed, the server 104 extracts some video frames therein as target video frames, and when it is determined based on the target video frame that the video to be reviewed contains a background split screen, and the background split screen meets the preset conditions, a reference point sequence is selected in the background split screen, and then the reference point sequences in two adjacent target video frames are matched for similarity, and whether the video to be reviewed is a rotated background video is determined according to the similarity matching result; further, if it is determined that the video is a rotated background video, the review result of the video can be fed back to the terminal 102 as needed. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0030] In other embodiments, the rotating background video recognition method is executed in a server. Due to the huge amount of videos, the review of the videos is not performed in real time. In this embodiment, the user uploads the video to the server. When the video review module of the server reviews the video, it can directly obtain the video to be reviewed from the server and execute the steps of the above method. If it is determined that the video is a rotating background video, the review result of the video can be fed back to the terminal as needed. In other embodiments, the rotating background video recognition method can also be executed in a terminal. The method is deployed in the video review module of the terminal. When the user wants to publish a video, the video is uploaded, and the video review module in the terminal executes the above method on the video to determine whether the video is a rotating background video.

[0031] In one embodiment, Figure 2 As shown, a rotating background video recognition method is provided, and the method is applied to Figure 1 The server 104 in the example is used for explanation, including steps S210 to S240.

[0032] Step S210: extract two or more target video frames from the video to be reviewed.

[0033] The videos to be reviewed are videos that need to be reviewed and determined to be rotating background videos. They can be videos uploaded by users and expected to be released. The video content is reviewed before release, and videos that do not meet quality requirements are filtered out through review.

[0034] The video frame is extracted from the video to be reviewed, specifically, the video frame is extracted from the video to be reviewed using a preset frame rate, and the preset frame rate can be set according to actual conditions, for example, 1 frame per second, 5 frames per second; in this embodiment, the video frame extracted from the video to be reviewed is recorded as the target video frame. In one embodiment, the target video frame is usually an RGB image, and after the target video frame is obtained, the target video frame can be converted into a grayscale image.

[0035] Step S220, when it is determined according to the target video frame that the video to be reviewed contains a background split screen, and the background split screen meets a preset condition, a reference point sequence is selected in the background split screen of each target video frame.

[0036] A common method in video production is to fill the background above and below the horizontal video to convert it into a vertical video, or fill the background on the left and right of the vertical video to convert it into a horizontal video. This type of video can be called a split-screen video, in which the filled background is the background split screen of this embodiment. Figure 3The figure shows a schematic diagram of a video obtained by filling the background above and below the video in a specific embodiment. In this embodiment, sub-split screen 1 and sub-split screen 3 are both background split screens, and sub-split screen 2 is a video content split screen.

[0037] In one embodiment, the target video frame can be detected to determine whether the video to be reviewed is a split-screen video, that is, it contains a background split screen, and at the same time, it is determined whether the background split screen meets the preset conditions; the preset conditions can be set according to actual conditions. In a specific embodiment, the preset condition is that the video includes two background split screens, which are respectively distributed on both sides of the video content split screen; in a specific embodiment, determining whether the video to be reviewed contains a background split screen based on the target video frame includes the steps of: determining the horizontal boundary of the target video frame based on edge detection and straight line detection. If it is detected that there are two horizontal dividing lines in the target video frame and they are located in symmetrical positions, it is determined that the video to be reviewed is a three-screen video, including two background split screens, which are respectively distributed on both sides of the video content split screen.

[0038] After determining that the video to be reviewed contains a background split screen and the background split screen meets a preset condition, a reference point sequence is selected in each target video frame; wherein the reference point sequence is a sequence composed of a plurality of reference points.

[0039] Step S230 , performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames.

[0040] The similarity matching of the reference point sequences of each two adjacent target video frames is actually to calculate the similarity of the reference point sequences in each two adjacent target video frames; in one embodiment, after selecting the reference point sequence in each target video frame, it also includes obtaining the pixel value of each reference point in the reference point sequence, wherein the pixel value is the pixel gray value corresponding to the position of the reference point of the target video frame. Further, in this embodiment, the similarity is calculated for the reference point sequences of each two adjacent target video frames, and specifically, the similarity can be calculated by the pixel value of each reference point in the two reference point sequences to obtain the similarity matching result.

[0041] Furthermore, the similarity calculation of the reference point sequence of each two adjacent target video frames can be achieved in any manner, for example, common similarity calculation methods such as cosine distance and Euclidean distance can be used, or discrete integrals can be obtained for the curve formed by the pixel values ​​of the reference points, and the integrals are taken for the offset values ​​that are close to each other to determine the similarity of the reference point sequences in the two adjacent target video frames. In other embodiments, the similarity between the reference point sequences can also be calculated in other ways; further, for the similarity matching results calculated for all adjacent target video frames of the video to be reviewed, assuming that n target video frames are extracted, (n-1) similarity matching results will be obtained.

[0042] Step S240: determining whether the video to be reviewed is a rotating background video according to the similarity matching results.

[0043] Based on the similarity matching results, it can be determined whether there is a similar reference point sequence in two adjacent target video frames. For two adjacent target video frames, if a similar reference point sequence is detected, it can be further determined to determine the positions corresponding to the similar reference point sequences in the two target video frames, and then determine whether the similar reference point sequences have shifted in position between the two adjacent frames; if the similar reference point sequences have shifted in position in the two target video frames, it can be detected in the next target video frame whether there is a similar reference point sequence, and whether the similar reference point sequences have shifted in position, and so on; and then based on all similarity matching results in the video to be reviewed, it can be determined whether the background of the video to be reviewed has rotated.

[0044] The above-mentioned rotating background video identification method extracts some video frames from the video to be reviewed as target video frames. When it is determined based on the target video frames that the video to be reviewed contains a background split screen and the background split screen meets the preset conditions, a reference point sequence is selected in the background split screen, and then the reference point sequences in two adjacent target video frames are matched for similarity, and whether the video to be reviewed is a rotated background video is determined based on the similarity matching result; the above-mentioned method extracts video frames from the video, and selects a reference point sequence in the background split screen of the extracted video frames, and similarity matching is performed based on the reference point sequences of two adjacent video frames. Based on the similarity matching result, it can be determined whether there are similar reference point sequences in the two adjacent video frames, and then based on whether there are similar reference point sequences in all adjacent video frames in the video to be reviewed, it can be more accurately identified whether the video to be reviewed is a rotating background video.

[0045] Further, in one embodiment, a reference point sequence is selected in the background split screen of each target video frame, including: selecting a reference frame of each target video frame at the same position in the background split screen of each target video frame, wherein the reference frame does not contain text; in the reference frame, selecting a reference point on a target arc with the video center of the video to be reviewed as the center; in one embodiment, selecting a reference point on the target arc includes: selecting continuous pixel points on the target arc as reference points, which may be all pixel points on the target arc or partially continuous pixel points on the target arc; further, the reference point sequence is obtained after sorting the reference points in a preset direction.

[0046] In one embodiment, the reference frame is a quadrilateral. First, the reference frame is selected in the background split screen, and then the reference points are selected in the reference frame. Then, the reference points are sorted to obtain a reference point sequence. The reference frame satisfies certain conditions, which can ensure that the reference points that meet the requirements are selected in the reference frame. In this embodiment, since text information will affect the judgment of the rotating background, the selected reference frame does not contain text information. Furthermore, in one embodiment, the reference frame is in the same background split screen.

[0047] Further, in this embodiment, the reference points are selected from the reference frame by selecting them on an arc with the video center of the video to be reviewed as the center, that is, the reference points selected in this embodiment are all on the part of the arc that falls within the reference frame, and in this embodiment, the arc is recorded as the target arc; further, the radius of the target arc can be set according to actual conditions, such as Figure 3 As shown, the dotted line frame indicated by reference numeral 301 corresponds to the reference frame, and the solid line arc indicated by reference numeral 302 corresponds to the target arc.

[0048] In one embodiment, the preset direction may be a clockwise direction or a counterclockwise direction, and the reference point sequence is obtained by sorting the reference points selected in the target video frame.

[0049] In one embodiment, Figure 4 As shown, similarity matching is performed on the reference point sequences in each two adjacent target video frames to obtain the similarity matching result between each two adjacent target video frames, including steps S231 to S235.

[0050] Step S231 , arbitrarily selecting two adjacent video frames that have not been selected from each target video frame as a current video frame group.

[0051] Step S232: for the first frame of the current video frame group, the reference point sequence is equally divided into a first reference point subsequence and a second reference point subsequence.

[0052] Among them, each target video frame is sorted according to the time sequence of the video to be reviewed, and the first frame of the current video frame group can be a target video frame that is ranked front in the time sequence; the reference point sequence in the first frame is equally divided into two parts, and the reference points located on both sides of the dividing point are respectively used as two reference point sub-sequences. In this embodiment, the two reference point sub-sequences are recorded as the first reference point sub-sequence and the second reference point sub-sequence.

[0053] Furthermore, in an embodiment where the reference point sequence is composed of all pixel points on the target arc, the reference point sequence is divided into a first reference point subsequence and a second reference point subsequence with the midpoint of the target arc as a dividing point; Figure 5, which is a schematic diagram of a reference point sequence in the first frame of the current video frame group in a specific embodiment, wherein the dotted line indicated by reference numeral 501 corresponds to the first reference point subsequence, and the solid line indicated by reference numeral 502 corresponds to the second reference point subsequence.

[0054] Step S233 , selecting a plurality of first sequence segments having the same length as the first reference point subsequence and a plurality of second sequence segments having the same length as the second reference point subsequence from the reference point sequence of the second frame of the current video frame group.

[0055] The second frame of the current video frame group is another target video frame other than the first frame in the target video frame group; in this embodiment, a sequence segment with the same length as the first reference point subsequence obtained in the reference point sequence of the second frame is recorded as a first sequence segment, and a sequence segment with the same length as the second reference point subsequence is recorded as a second sequence segment; it can be understood that multiple sequence segments with the same length as the first reference point subsequence and the second reference point subsequence can be obtained in the reference point sequence of the second frame.

[0056] In one embodiment, in the second frame of the current video frame group, a sequence segment having the same length as the first reference point subsequence and a sequence segment having the same length as the second reference point subsequence are selected in sequence from left to right (or from right to left) in a sliding window manner; in a specific embodiment, the sliding distance of the sliding window is 1 each time.

[0057] Step S234, respectively determining the first similarity between each first sequence segment and the first reference point subsequence, and the second similarity between each second sequence segment and the second reference point subsequence, and returning to the step of arbitrarily selecting two adjacent video frames that have not been selected from each target video frame as the current video frame group, until all target video frames are selected.

[0058] The similarity between two reference point sequence segments can be calculated by the similarity of the pixel values ​​corresponding to each reference point on the reference point sequence; specifically, common similarity calculation methods such as cosine distance and Euclidean distance can be used, or discrete integrals can be obtained for the curve formed by the pixel values ​​of the reference points, and the integrals are taken to the offset values ​​close to each other to determine the similarity of the reference point sequence. In this embodiment, the similarity between each first sequence segment and the first reference point subsequence is recorded as the first similarity, and the similarity between each second sequence segment and the second reference point subsequence is recorded as the second similarity; it can be understood that the number of the first similarity and the second similarity is the same as the number of the first sequence segments and the second sequence segments, respectively.

[0059] After calculating and obtaining each first similarity and each second similarity in the current video frame group, return to reselect the current video frame group to calculate each first similarity and second similarity in all adjacent target video frames.

[0060] Step S235 : generating a similarity matching result between every two adjacent target video frames based on all the first similarities and all the second similarities.

[0061] After obtaining the first similarities and the second similarities in all adjacent target video frames, a similarity matching result between every two adjacent target video frames in the entire video to be reviewed may be generated according to these similarity results.

[0062] In the above embodiment, by sequentially performing similarity matching on the reference subsequence segments in each of two adjacent target video frames, a similarity matching result of the entire video to be reviewed is obtained.

[0063] Furthermore, in one embodiment, after respectively determining the first similarity between each first sequence segment and the first reference point subsequence, and the second similarity between each second sequence segment and the second reference point subsequence, it also includes: marking the first marked similarity with the smallest value among the first similarities in the current video frame group and less than a preset threshold, and marking the second marked similarity with the smallest value among the second similarities and less than the preset threshold.

[0064] Assuming that the current video frame group includes x first sequence segments, it corresponds to x first similarities. The first similarity with the smallest similarity and less than a preset threshold is determined from the x first similarities and recorded as the first marked similarity; the second marked similarity can be determined similarly; wherein the preset threshold can be set according to actual conditions; further, if there is no first similarity less than the preset threshold in the current video frame group, the first marked similarity of the current video frame group is recorded as none, and similarly, if there is no second similarity less than the preset threshold in the current video frame group, the second marked similarity of the current video frame group is recorded as none.

[0065] In this embodiment, generating similarity matching results between each two adjacent target video frames based on all first similarities and all second similarities includes: generating similarity matching results between each two adjacent target video frames based on each first tag similarity and each second tag similarity. Further, determining whether the video to be reviewed is a rotated background video based on each similarity matching result includes: obtaining a first tag offset corresponding to the first tag similarity and a second tag offset corresponding to the second tag similarity; determining whether the video to be reviewed is a rotated background video based on the first tag offset and the second tag offset.

[0066] The first marker offset and the second marker offset actually represent the offset of similar reference point sequences in each adjacent target video frame, that is, the position offset; in a specific embodiment of intercepting sequence segments in the second frame in a sliding window manner, the first marker similarity corresponds to two sequence segments in the first frame and the second frame, and the window sliding distance between the sequence segment in the second frame and the first reference point subsequence of the first frame is the first marker offset corresponding to the first marker similarity.

[0067] In one embodiment, determining whether the video to be reviewed is a rotating background video based on the first mark offset and the second mark offset includes: determining the target offset that appears the most times among the first mark offsets and the second mark offsets; when the number of occurrences of the target offset is greater than a preset number of times and the target offset is non-zero, determining that the video to be reviewed is a rotating background video.

[0068] In one embodiment, the preset number is half of the number of the current video frame group; the target offset refers to the value with the largest number of occurrences among all the values ​​of the first mark offset and the second mark offset; in one embodiment, all the first mark offsets are recorded as the first offset sequence, and all the second mark offsets are recorded as the second offset sequence. The first offset sequence and the second offset sequence are combined together, and the mark offset with the largest number of occurrences (which may be the first mark offset or the second mark offset) is determined as the target offset. It can be understood that, assuming that the number of target video frames is n, the lengths of the first offset sequence and the second offset sequence are both (n-1).

[0069] After determining the target offset, it is also necessary to obtain the number of occurrences of the target offset in the first mark offset and the second mark offset. When the number of occurrences is greater than or equal to (n-1) / 2, and the target offset itself is non-zero, the video to be reviewed is determined to be a rotating background video. In fact, it is determined by judging whether the number of occurrences of the target offset with the largest number of occurrences reaches half of the number of mark offsets, that is, (n-1) / 2. At the same time, the non-zero target offset itself excludes the situation where the background of the video is still.

[0070] In one embodiment, when the number of occurrences of the target offset is greater than a preset number and the target offset is non-zero, it is determined that the video to be reviewed is a rotating background video, and it can be determined that the background of the rotating background video rotates at a constant speed.

[0071] In the above embodiment, by determining whether the value of the mark offset that appears most frequently between the first mark offset and the second mark offset is zero, and whether the number of occurrences of the mark offset that appears most frequently exceeds a preset number, it is determined whether the background of the video to be reviewed is rotated, so that videos with a rotating background can be identified more accurately.

[0072] In one embodiment, Figure 6 As shown, after determining that the video to be reviewed is a rotating background video, it also includes step S610: determining the rotation direction of the rotating background video according to the position relationship between the position corresponding to the target offset and the dividing point.

[0073] From the above description, it can be seen that the target offset is an offset in the first mark offset or the second mark offset, and the positional relationship of the dividing point corresponding to the target offset actually refers to whether the target offset belongs to the first mark offset or the second mark offset; in a specific embodiment, the preset direction of the reference point sorting is clockwise. If the target offset belongs to the first mark offset, it means that the rotation direction of the rotating background video is clockwise. If the target offset belongs to the second mark offset, it means that the rotation direction of the rotating background video is counterclockwise.

[0074] In one embodiment, after determining that the video to be reviewed is a rotating background video, it also includes step S620: obtaining the time interval between two adjacent target video frames, and determining the rotation angle of the two adjacent target video frames based on the position relationship between the position corresponding to the target offset and the dividing point; step S630, determining the background rotation speed of the rotating background video based on the time interval and rotation angle between the two adjacent target video frames.

[0075] Among them, the time interval between two adjacent target video frames can be determined according to the frame rate of the target video frame extracted; in one embodiment, the rotation angle of two adjacent target video frames is determined according to the positional relationship between the position corresponding to the target offset and the dividing point, including: determining the rotation direction of the video to be reviewed according to the positional relationship between the position corresponding to the target offset and the dividing point, if it is a clockwise rotation, selecting a reference point sequence in two adjacent target video frames in the first reference point subsequence to determine the rotation angle; if it is a clockwise rotation, selecting a reference point sequence in two adjacent target video frames in the second reference point subsequence to determine the rotation angle.

[0076] Further, determining the background rotation speed of the rotating background video according to the time interval and the rotation angle between two adjacent target video frames includes: determining the ratio of the rotation angle to the time interval as the background rotation speed of the rotating background video.

[0077] In the above embodiment, after determining that the video to be reviewed is a rotating background video, the rotation direction and rotation speed of the rotating background can also be determined to have a better understanding of the rotating background video.

[0078] In one embodiment, the split screen of the video to be reviewed is determined to be upper, middle and lower thirds according to the target video frame, and the upper and lower split screens of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is the video content split screen.

[0079] The split-screen situation of the video to be reviewed refers to whether the video to be reviewed contains background split-screen, the number of background split-screens when it contains background split-screens, and the positional relationship between the background split-screen and the video content split-screen; when it is determined that the split-screen situation of the video to be reviewed is three sub-split screens, and each sub-split screen is distributed according to the relationship of top, middle and bottom, it is judged that the background split-screen meets the preset conditions.

[0080] The same position is selected in the background split screen of each target video frame as the reference frame of each target video frame, including: arbitrarily selecting a background split screen as the target background split screen; when it is recognized that the target background split screen contains text, determining the position of the text frame corresponding to the text; using the straight line where the left edge of the text frame, the straight line where the right edge is located, the adjacent edges of the target background split screen and the video content split screen, and the edge of the text frame close to the video content split screen as four edges to obtain the reference frame.

[0081] In one embodiment, identifying whether there is text in the target background split screen and determining the position of the text box when the text is recognized can be achieved through OCR (Optical Character Recognition); wherein the text box includes all the text. In one embodiment, the text recognition on the target background split screen can be performed on the first target video frame of the video to be reviewed, or it can be performed on each target video frame of the video to be reviewed; further, in the embodiment of recognizing each target video frame, the position of the text recognized in different target video frames may be different, in which case the position of the text box and other information can be obtained by taking the maximum boundary; in one embodiment, the left and right edges of the text box are parallel to the left and right edges of the video to be reviewed, and the upper and lower edges of the text box are parallel to the upper and lower edges of the video to be reviewed.

[0082] Furthermore, in a specific embodiment, the straight line where the left edge of the text box is located, the straight line where the right edge is located, the adjacent edges of the target background split screen and the video content split screen, and the edge of the text box close to the video content split screen are respectively used as four edges, and the obtained reference frame is as follows: Figure 3 The dotted frame is shown by reference number 301. It is understandable that in other embodiments, the reference frame can be selected by other methods, as long as the requirements of the reference frame are met, such as there is no text information in the reference frame.

[0083] Furthermore, in the reference frame, selecting a reference point on a target arc with the video center of the video to be reviewed as the center includes: determining a target distance between the border of the text box close to the video content split screen and the target adjacent edge, the target adjacent edge being the adjacent edge between the target background split screen and the video content split screen; determining a first numerical value corresponding to half the height of the video content sub-split screen and a second numerical value corresponding to a preset number of times the height value of the left edge of the reference frame, the preset number being less than 1; in the reference frame, selecting a reference point on a target arc with the video center of the video to be reviewed as the center and the sum of the first numerical value and the second numerical value as the radius.

[0084] In a specific embodiment, the preset number is two-thirds.

[0085] In this embodiment, since text information has a certain influence on the judgment of the rotating background, when selecting the reference frame and the reference point, if the selected background split screen includes text, the text frame is first determined in the above manner, and the reference frame is selected at a position avoiding the text frame, and then the reference point sequence is selected in the reference frame, so that the reference point sequence will not be selected in a place where there is text. In this way, the adverse effects of the text in the background on the judgment of the rotating background video can be avoided.

[0086] In another embodiment, the split screen of the video to be reviewed is determined as upper, middle and lower thirds according to the target video frame, and the upper and lower split screens of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is the video content split screen.

[0087] The same position is selected in the background split screen of each target video frame as the reference frame of each target video frame, including: arbitrarily selecting a background split screen as the target background split screen; when it is recognized that the background split screen does not contain text, the target background split screen is used as the reference frame.

[0088] Furthermore, in the reference frame, selecting a reference point on a target arc with the video center of the video to be reviewed as the center includes: determining a target distance between a corner of adjacent sides of the video content split screen and the target background split screen and the center of the video content split screen; in the reference frame, selecting a reference point on a target arc with the video center of the video to be reviewed as the center and the target distance as the radius.

[0089] In this embodiment, the video to be reviewed is also taken as an example of a video with a top-middle-bottom split screen. Since there is no text in the selected target background split screen, the target background split screen itself is used as the reference frame, and reference points are selected from the reference frame to obtain a reference point sequence. It should be noted that in other embodiments, the radius can also be selected as a radius of other lengths, and the condition to be satisfied is that the selected radius makes a part of the arc of the corresponding circle fall into the reference frame.

[0090] In another embodiment, the above-mentioned rotating background video recognition method can also be used in other three-screen situations of videos to be reviewed, wherein the selection of the reference frame still satisfies the condition that the frame does not contain text, and the reference point sequence is selected within the reference frame.

[0091] The present application also provides an application scenario, which applies the above-mentioned rotating background video recognition method. Figure 7 As shown, specifically, the application of the rotating background video recognition method in this application scenario is as follows:

[0092] 1. Extract video frames from the video to be reviewed: Extract frames from the video to be reviewed at a certain time interval (for example, the frame rate can be 1s / frame, 1s / 5 frames, etc.) to form a time-ordered image sequence N (composed of the above two or more target video frames) with a length of n. The image obtained by extracting frames is an RGB image, which is converted into a grayscale image one by one, with each pixel grayscale value ranging from 0 to 255, and the image size (w, h) is obtained.

[0093] 2. Perform split-screen detection on the video to be reviewed: Identify whether the video is composed of multiple sub-split screens through the image sequence. If the video is composed of three sub-split screens, give the split-screen position h0 and proceed to step 3. Figure 8 The figure shows a schematic diagram of a target video frame in a specific embodiment. The video is divided into three parts, upper, middle and lower, with 0.23*h and 0.77*h as the boundaries, and is marked as sub-split screen 1, sub-split screen 2 and sub-split screen 3 respectively. If there is a background canvas in the video, it usually appears in the upper and lower sub-split screens, so the subsequent analysis is performed on sub-split screen 1 or 3. The following takes the subsequent analysis of sub-split screen 1 as an example. In a specific embodiment, the video split screen detection specifically determines the horizontal boundary of the video based on edge detection and straight line detection. If there are two horizontal dividing lines and they are located in a symmetrical position, it is judged as a three-part screen (three sub-split screens are distributed in the upper, middle and lower parts), and the split screen position can be determined. In one embodiment, if the video to be reviewed is not a three-part screen video, it is determined that the video to be reviewed is not a rotating background video.

[0094] 3. Perform text recognition on the content of the video to be reviewed: Perform OCR recognition on the first image in the image sequence and mark the text coordinate position ((w1,h1),(w2,h2)); or perform OCR recognition on all images in the image sequence and take the maximum boundary of the text box located within the upper split screen range; where (w1,h1) is the coordinate of the upper left corner of the text box, and (w2,h2) is the coordinate of the lower right corner of the text box. Figure 8As shown, the result of OCR recognition is that the image contains 3 text boxes, which are marked as ①②③ in the figure respectively. Among them, only the text boxes located in the upper split screen (i.e., sub-split screen 1) of the upper, middle and lower split screens are taken for subsequent analysis, and their coordinates are ((0.11*w1,0.07*h1),(0.89*w2,0.16*h2)).

[0095] 4. Select the reference frame and reference point sequence: Select the area and points for matching in the target video frame as the reference frame and reference point sequence. The reference frame will be used as the only area for background rotation identification, and the reference point sequence is the pixel point sequence in the reference frame that is the matching object. Among them, the standards for selecting the reference frame and reference point sequence are as follows (in this embodiment, the selection of the reference frame and reference point in sub-split screen 1 is taken as an example): (1) The reference frame is in the upper split screen of the upper, middle and lower three-screen, that is, sub-split screen 1; (2) There is no text information in the reference frame; (3) The reference point is on the arc with the center of the video (0.5*w, 0.5*h) as the rotation center. The specific steps are as follows:

[0096] ① According to the boundary of the upper split screen and the position of the OCR text box, the lower edge h2 of the text box and the boundary h0 of the upper split screen are used as the upper and lower boundaries of the reference box respectively;

[0097] ② According to the position of the OCR text box, the left edge w1 and the right edge w2 of the text box are used as the left and right boundaries of the reference box respectively;

[0098] ③ Get the reference frame position ((w b1 ,h b1 ),(w b2 ,h b2 ));

[0099] ④ Select the rotation radius r as the reference point selection, r = (h b2 -h b1 )*2 / 3+(0.5*hh b2 ), where h is the height of the video to be reviewed, h b2 The height of sub-screen 1, h b1 It is the distance between the bottom edge of the text box and the top edge of sub-split screen 1.

[0100] ⑤In the reference frame, mark the coordinates and grayscale values ​​of all points with a distance r from the center point of the image (0.5*w, 0.5*h), and take 0.5*w as the reference. The point to the left of 0.5*w is the left reference point, which is used as the reference point of the clockwise rotating background. The left reference points are arranged from left to right to obtain the reference point pixel value sequence L left (the first reference point subsequence above); the point to the right of 0.5*w is the right reference point, which is used as the reference point for rotating the canvas counterclockwise. The right reference points are arranged from left to right to obtain the reference point pixel value sequence L right(the second reference point subsequence mentioned above).

[0101] In another embodiment, if there is no text frame within the sub-split screen 1, the entire sub-split screen 1 is taken as the reference frame, and the radius can be determined by: Among them, w is the width of the video to be reviewed, h is the height of the video to be reviewed, and h b2 is the height value of the sub-split screen 1; it can be understood that the radius is the distance from the left edge point (or right edge point) of the dividing line between the sub-split screen 1 and the sub-split screen 2 to the center point.

[0102] 5. Reference point matching: Match similar points of adjacent frames for the left reference point sequence and the right reference point sequence respectively. The specific steps are as follows:

[0103] ① For the next target video frame in the image sequence, select the same reference frame and rotation radius as in step 4, mark the position and pixel value of the reference point, and no longer distinguish between left and right. The reference points are arranged from left to right to obtain the reference point sequence L;

[0104] ② Using a sliding window, take the left For continuous sequences of equal length, calculate the similarity (the first similarity mentioned above) respectively, mark the sliding window offset position d corresponding to the similarity with the smallest similarity and less than the threshold (the first marked similarity mentioned above) left (the first marker offset mentioned above), if it does not exist, it is recorded as None; similarly, in L, take the value of L right Calculate the similarity of consecutive sequences of equal length (the second similarity mentioned above), mark the similarity with the smallest similarity and less than the threshold (the second marked similarity mentioned above) corresponding to the sliding window offset position d right (the above-mentioned second marker offset), if not present, is recorded as None. The similarity between the sequence segment and the first reference point subsequence or the second reference point subsequence is calculated, specifically, the similarity of the pixels of each reference point in the sequence segment is calculated.

[0105] 6. Rotation background judgment: Based on the reference point results, it is concluded whether it is a rotation background. Repeat the above steps for adjacent frames in the image sequence to obtain the results of reference point matching of all adjacent frames. The image sequence N can obtain two offset sequences of length (n-1) corresponding to the left reference point sequence and the right reference point sequence (composed of (n-1) d left The offset sequence is composed of (n-1) d right offset sequence composed of .

[0106] Take the offset value with the largest number of occurrences in the two offset sequences, denoted as d max , if d max Satisfy: (d max The number of occurrences of) ≥ (n-1) / 2, and dmax Not 0 (i.e. Figure 7 The judgment step shown in the figure shows that if the spacing between similar points of multiple groups of adjacent frames is the same and the similar spacing is non-zero), then the video to be reviewed is determined to be a rotating canvas background video; further, if d max is d left The offset value in the offset sequence composed of d is used to determine whether the rotation of the canvas is clockwise. max is d right If the offset value in the offset sequence is the same as the offset value in the offset sequence, the rotating canvas is determined to be counterclockwise rotating, otherwise it is a non-rotating canvas background video. It should be noted that the method in the above embodiment determines that the background of the rotating background video is uniformly rotating in the same direction; in other embodiments, it can be determined whether it is a non-uniformly rotating background video or a background video that does not rotate in the same direction based on two offset sequences of length (n-1), such as when the offset value in an offset sequence gradually increases (or decreases) according to a certain rule, it is determined to be a non-uniformly rotating background video, etc.

[0107] Furthermore, after determining the rotation direction of the video to be reviewed, select the same reference point of two adjacent video frames in the rotation direction. For example, if the video to be reviewed rotates clockwise, select a reference point in the left reference sequence. The rotation angle between adjacent video frames is determined based on the position of the reference point in the next video frame. Combined with the time interval between adjacent video frames (which can be determined based on the frame rate of the extracted video frames), the rotation speed of the video to be reviewed can be determined.

[0108] The rotating background video recognition method in the above embodiment detects the upper, middle and lower three-screen videos, uses the OCR recognition results and the split-screen boundaries to obtain the reference frame and reference points, and completes the recognition of the rotating canvas with the video center as the rotation center by matching the reference points of adjacent frames, and determines its rotation direction; not only can it eliminate the influence of fixed text information on the background canvas recognition, but also can more accurately determine its rotation direction and rotation speed.

[0109] It should be understood that although Figure 2 , 4 The steps in the flowcharts of and 6 are shown in sequence as indicated by arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , 4 At least part of the steps in 6 may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0110] In one embodiment, Fig. 9 As shown, a rotating background video recognition device is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a video frame extraction module 910, a reference point sequence selection module 920, a matching module 930 and a judgment module 940, wherein:

[0111] The video frame extraction module 910 is used to extract two or more target video frames from the video to be reviewed;

[0112] The reference point sequence selection module 920 is used to select reference point sequences in the background split screen of each target video frame when it is determined that the video to be reviewed contains a background split screen according to the target video frame and the background split screen meets a preset condition;

[0113] A matching module 930 is used to perform similarity matching on reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames;

[0114] The judgment module 940 is used to determine whether the video to be reviewed is a rotating background video according to each similarity matching result.

[0115] The above-mentioned rotating background video recognition device extracts some video frames from the video to be reviewed as target video frames. When it is determined based on the target video frames that the video to be reviewed contains a background split screen and the background split screen meets the preset conditions, a reference point sequence is selected in the background split screen, and then the reference point sequences in two adjacent target video frames are matched for similarity, and whether the video to be reviewed is a rotated background video is determined based on the similarity matching result; the above-mentioned device extracts video frames from the video, and selects a reference point sequence in the background split screen of the extracted video frames, and similarity matching is performed based on the reference point sequences of two adjacent video frames. Based on the similarity matching result, it can be determined whether there are similar reference point sequences in the two adjacent video frames, and then based on whether there are similar reference point sequences in all adjacent video frames in the video to be reviewed, it can be more accurately identified whether the video to be reviewed is a rotating background video.

[0116] In one embodiment, the reference point sequence selection module 920 in the above-mentioned rotating background video recognition device includes: a reference frame determination module, which is used to select a reference frame with the same position as each target video frame in the background split screen of each target video frame, wherein the reference frame does not contain text; a reference point selection module, which is used to select reference points on a target arc with the video center of the video to be reviewed as the center in the reference frame; and a sorting module, which is used to sort the reference points according to a preset direction to obtain a reference point sequence.

[0117] In one embodiment, the matching module 930 in the rotating background video recognition device includes: a current video frame group selection module, a reference point sequence division module, an equal length sequence segment determination module, a similarity determination module and a similarity matching result generation module. Among them:

[0118] The current video frame group selection module is used to arbitrarily select two adjacent video frames that have not been selected from each target video frame as the current video frame group; the reference point sequence division module is used to equally divide the reference point sequence into a first reference point subsequence and a second reference point subsequence for the first frame of the current video frame group; the equal-length sequence segment determination module is used to select a plurality of first sequence segments of equal length to the first reference point subsequence and a plurality of second sequence segments of equal length to the second reference point subsequence from the reference point sequence of the second frame of the current video frame group; the similarity determination module is used to respectively determine the first similarity between each first sequence segment and the first reference point subsequence, and the second similarity between each second sequence segment and the second reference point subsequence, and return to the step of arbitrarily selecting two adjacent video frames that have not been selected from each target video frame as the current video frame group, until all target video frames are selected; the similarity matching result generation module is used to generate the similarity matching result between each two adjacent target video frames based on all the first similarities and all the second similarities.

[0119] In one embodiment, the above-mentioned rotating background video recognition device also includes: a marking module, which is used to mark the first marking similarity with the smallest value among the first similarities in the current video frame group and less than a preset threshold, and the second marking similarity with the smallest value among the second similarities and less than the preset threshold; in this embodiment, the judgment module 940 is specifically used to obtain the first marking offset corresponding to the first marking similarity and the second marking offset corresponding to the second marking similarity; determine whether the video to be reviewed is a rotating background video based on the first marking offset and the second marking offset.

[0120] In one embodiment, the judgment module 940 in the above-mentioned rotating background video recognition device is specifically used to determine the target offset that appears the most times among the first mark offsets and the second mark offsets; when the number of occurrences of the target offset is greater than the preset number of times and the target offset is non-zero, it is determined that the video to be reviewed is a rotating background video.

[0121] In one embodiment, the rotating background video recognition device further includes: a rotation direction determination module, which is used to determine the rotation direction of the rotating background video according to the position relationship between the position corresponding to the target offset and the demarcation point.

[0122] In one embodiment, the rotating background video recognition device further includes: a rotation speed determination module, which is used to obtain the time interval between two adjacent target video frames, and determine the rotation angle of the two adjacent target video frames according to the position relationship between the position corresponding to the target offset and the dividing point; and determine the background rotation speed of the rotating background video according to the time interval and rotation angle between the two adjacent target video frames.

[0123] In one embodiment, the split screen of the video to be reviewed is determined as upper, middle and lower three-screen according to the target video frame, and the upper and lower split screens of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; in this embodiment, the reference frame determination module in the above-mentioned rotating background video recognition device includes: a background split screen selection module, which is used to arbitrarily select a background split screen as the target background split screen; a text box position determination module is used to determine the position of the text box corresponding to the text when it is recognized that the target background split screen contains text; in this embodiment, the reference frame determination module is also used to obtain the reference frame with the straight line where the left edge of the text box is located, the straight line where the right edge is located, the adjacent edges of the target background split screen and the video content split screen, and the edge of the text box close to the video content split screen as four edges.

[0124] Further, in the present embodiment, the above-mentioned reference point selection module includes: a distance determination module, used to determine the target distance between the border of the text box close to the video content split screen and the target adjacent edge, the target adjacent edge being the adjacent edge of the target background split screen and the video content split screen; determine a first numerical value corresponding to half the height of the video content sub-split screen and a second numerical value corresponding to a preset number of times the height value of the left edge of the reference frame, the preset number being less than 1; in the present embodiment, the reference point selection module is also used to select a reference point on a target arc in the reference frame with the video center of the video to be reviewed as the center and the sum of the first numerical value and the second numerical value as the radius.

[0125] In another embodiment, the split screen of the video to be reviewed is determined to be upper, middle and lower three-part screen according to the target video frame, and the upper split screen and the lower split screen of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; in this embodiment, the reference frame determination module in the above-mentioned rotating background video recognition device includes: a background split screen selection module, which is used to arbitrarily select a background split screen as a target background split screen; when it is recognized that the background split screen does not contain text, the target background split screen is used as the reference frame;

[0126] Furthermore, in the present embodiment, the above-mentioned reference point selection module includes: a distance determination module, used to determine the target distance between a corner of the adjacent sides of the video content split screen and the target background split screen and the center of the video content split screen; in the present embodiment, the reference point selection module is also used to select reference points on a target arc in a reference frame with the video center of the video to be reviewed as the center and the target distance as the radius.

[0127] The specific definition of the rotating background video recognition device can be found in the definition of the rotating background video recognition method in the above text, which will not be repeated here. The various modules in the rotating background video recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0128] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a rotating background video recognition method is implemented.

[0129] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0130] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0131] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0132] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A rotating background video recognition method, characterized in that: The method comprises: Extracting two or more target video frames from the video to be reviewed; When it is determined according to the target video frame that the video to be reviewed contains a background split screen, and the background split screen meets a preset condition, respectively selecting a reference point sequence in the background split screen of each target video frame; Performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames; A judgment result is determined based on each of the similarity matching results whether the similar points of multiple groups of adjacent target video frames have the same similarity spacing and a non-zero spacing, and based on the judgment result, it is determined whether the video to be reviewed is a rotating background video.

2. The rotating background video recognition method according to claim 1, characterized in that: The selecting of reference point sequences in the background split screen of each target video frame respectively includes: Selecting the same position in the background split screen of each target video frame as a reference frame of each target video frame, wherein the reference frame does not contain any text; In the reference frame, a reference point is selected on a target arc with the video center of the video to be reviewed as the center; The reference points are sorted according to a preset direction to obtain the reference point sequence.

3. The rotating background video recognition method according to claim 2, characterized in that: The performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames includes: Randomly selecting two adjacent video frames that have not been selected from each of the target video frames as a current video frame group; For the first frame of the current video frame group, the reference point sequence is equally divided into a first reference point subsequence and a second reference point subsequence; taking, from the reference point sequence of the second frame of the current video frame group, a plurality of first sequence segments having the same length as the first reference point subsequence and a plurality of second sequence segments having the same length as the second reference point subsequence; respectively determining a first similarity between each of the first sequence segments and the first reference point subsequence, and a second similarity between each of the second sequence segments and the second reference point subsequence, and returning to the step of arbitrarily selecting two adjacent video frames that have not been selected from each of the target video frames as the current video frame group, until all of the target video frames are selected; A similarity matching result between every two adjacent target video frames is generated based on all the first similarities and all the second similarities.

4. The rotating background video recognition method according to claim 3, characterized in that: After respectively determining the first similarity between each of the first sequence segments and the first reference point subsequence, and the second similarity between each of the second sequence segments and the second reference point subsequence, the method further includes: Marking a first marked similarity whose value is the smallest among the first similarities in the current video frame group and is smaller than a preset threshold, and a second marked similarity whose value is the smallest among the second similarities and is smaller than a preset threshold; The method of determining a judgment result of whether similar points of multiple groups of adjacent target video frames have similar positions with the same spacing and non-zero spacing based on each of the similarity matching results, and determining whether the video to be reviewed is a rotated background video based on the judgment result, includes: obtaining a first mark offset corresponding to the first mark similarity and a second mark offset corresponding to the second mark similarity; and determining whether the video to be reviewed is a rotated background video based on each of the first mark offsets and each of the second mark offsets.

5. The rotating background video recognition method according to claim 4, characterized in that: The determining whether the to-be-audited video is a rotating background video according to each of the first mark offsets and each of the second mark offsets includes: Determine a target offset that appears the most times among the first mark offsets and the second mark offsets; When the number of occurrences of the target offset is greater than a preset number and the target offset is non-zero, it is determined that the video to be reviewed is a rotating background video.

6. The rotating background video recognition method according to claim 5, characterized in that: After determining that the video to be reviewed is a rotating background video, at least one of the following is further included: The first item, Determining the rotation direction of the rotating background video according to the positional relationship between the position corresponding to the target offset and the demarcation point; The second item, Acquire the time interval between two adjacent target video frames, and determine the rotation angle of the two adjacent target video frames according to the position relationship between the position corresponding to the target offset and the dividing point; The background rotation speed of the rotating background video is determined according to the time interval between two adjacent target video frames and the rotation angle.

7. The rotating background video recognition method according to any one of claims 2 to 6, characterized in that: Determine, according to the target video frame, that the split screen of the video to be reviewed is divided into upper, middle and lower parts, and the upper split screen and the lower split screen of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; The selecting the same position in the background split screen of each target video frame as the reference frame of each target video frame comprises: arbitrarily selecting a background split screen as the target background split screen; when it is recognized that the target background split screen contains text, determining the position of the text frame corresponding to the text; using the straight line where the left edge of the text frame is located, the straight line where the right edge is located, the adjacent edge between the target background split screen and the video content split screen, and the edge of the text frame close to the video content split screen as four edges, respectively, to obtain the reference frame; The method of selecting a reference point on a target circular arc with the video center of the video to be reviewed as the center in the reference frame includes: determining a target distance between the border of the text frame close to the video content split screen and a target adjacent edge, wherein the target adjacent edge is an adjacent edge between the target background split screen and the video content split screen; determining a first value corresponding to half the height of the video content sub-split screen and a second value corresponding to a preset number of times the height value of the left edge of the reference frame, wherein the preset number is less than 1; and selecting a reference point on a target circular arc with the video center of the video to be reviewed as the center and a radius which is the sum of the first value and the second value in the reference frame.

8. The rotating background video recognition method according to any one of claims 2 to 6, characterized in that: Determine, according to the target video frame, that the split screen of the video to be reviewed is divided into upper, middle and lower parts, and the upper split screen and the lower split screen of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; The selecting the same position in the background split screen of each target video frame as the reference frame of each target video frame includes: arbitrarily selecting a background split screen as the target background split screen; when it is recognized that the background split screen does not contain text, using the target background split screen as the reference frame; The selecting of reference points on the target arc with the video center of the video to be reviewed as the center in the reference frame includes: determining the target distance between a corner of the adjacent sides of the video content split screen and the target background split screen and the center of the video content split screen; and selecting reference points on the target arc with the video center of the video to be reviewed as the center and the target distance as the radius in the reference frame.

9. A rotating background video recognition device, characterized in that: The device comprises: A video frame extraction module, used to extract two or more target video frames from the video to be reviewed; A reference point sequence selection module, for selecting reference point sequences in the background split screens of each target video frame respectively when it is determined according to the target video frame that the video to be reviewed contains a background split screen and the background split screen meets a preset condition; A matching module, used for performing similarity matching on the reference point sequences in each two adjacent target video frames to obtain a similarity matching result between each two adjacent target video frames; A judgment module is used to determine whether the similar point spacings of multiple groups of adjacent target video frames are the same and non-zero based on the similarity matching results, and determine whether the video to be reviewed is a rotating background video based on the judgment result.

10. The rotating background video recognition device according to claim 9, characterized in that: The reference point sequence selection module includes: A reference frame determination module, used for selecting a reference frame of each target video frame at the same position in the background split screen of each target video frame, wherein the reference frame does not contain any text; A reference point selection module, used for selecting a reference point on a target arc with the video center of the video to be reviewed as the center in the reference frame; The sorting module is used to sort the reference points according to a preset direction to obtain the reference point sequence.

11. The rotating background video recognition device according to claim 10, characterized in that: The matching module comprises: A previous video frame group selection module is used to arbitrarily select two adjacent video frames that have not been selected from each of the target video frames as the current video frame group; A reference point sequence division module, configured to equally divide the reference point sequence into a first reference point subsequence and a second reference point subsequence for the first frame of the current video frame group; an equal-length sequence segment determination module, configured to select a plurality of first sequence segments of equal length to the first reference point subsequence and a plurality of second sequence segments of equal length to the second reference point subsequence from the reference point sequence of the second frame of the current video frame group; a similarity determination module, configured to respectively determine a first similarity between each of the first sequence segments and the first reference point subsequence, and a second similarity between each of the second sequence segments and the second reference point subsequence, and return to the step of arbitrarily selecting two adjacent video frames that have not been selected from each of the target video frames as the current video frame group, until each of the target video frames is selected; A similarity matching result generating module is used to generate a similarity matching result between every two adjacent target video frames based on all the first similarities and all the second similarities.

12. The rotating background video recognition device according to claim 11, characterized in that: Also includes: A marking module, used for marking a first marked similarity whose value is the smallest and less than a preset threshold among the first similarities in the current video frame group, and a second marked similarity whose value is the smallest and less than a preset threshold among the second similarities; The judgment module is also used to: obtain a first mark offset corresponding to the first mark similarity and a second mark offset corresponding to the second mark similarity; and determine whether the video to be reviewed is a rotating background video according to each of the first mark offsets and each of the second mark offsets.

13. The rotating background video recognition device according to claim 12, characterized in that: The judgment module is used to determine the target offset that appears the most times among the first mark offsets and the second mark offsets; when the number of occurrences of the target offset is greater than a preset number and the target offset is non-zero, it is determined that the video to be reviewed is a rotating background video.

14. The rotating background video recognition device according to claim 13, characterized in that: Also includes at least one of a rotation direction determination module and a rotation speed determination module; The rotation direction determination module is used to determine the rotation direction of the rotating background video according to the position relationship between the position corresponding to the target offset and the demarcation point; The rotation speed determination module is used to obtain the time interval between two adjacent target video frames, determine the rotation angle of the two adjacent target video frames according to the position relationship between the position corresponding to the target offset and the dividing point; and determine the background rotation speed of the rotating background video according to the time interval between two adjacent target video frames and the rotation angle.

15. The rotating background video recognition device according to any one of claims 10 to 14, characterized in that: Determine, according to the target video frame, that the split screen of the video to be reviewed is divided into upper, middle and lower parts, and the upper split screen and the lower split screen of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; The reference frame determination module includes: A background split screen selection module, used to arbitrarily select a background split screen as a target background split screen; A text box position determination module, used for determining the position of the text box corresponding to the text when it is recognized that the target background split screen contains text; The reference frame determination module is further configured to obtain the reference frame by using a straight line where the left edge of the text frame is located, a straight line where the right edge is located, adjacent edges between the target background split screen and the video content split screen, and an edge of the text frame close to the video content split screen as four edges respectively; The reference point selection module includes: a distance determination module, configured to determine a target distance between a border of the text box close to the video content split screen and a target adjacent edge, wherein the target adjacent edge is an adjacent edge between the target background split screen and the video content split screen; determine a first value corresponding to half of the height of the video content sub-split screen, and a second value corresponding to a preset number of times the height of the left edge of the reference frame, wherein the preset number is less than 1; The reference point selection module is also used to select a reference point on a target arc in the reference frame, with the video center of the video to be reviewed as the center of the circle and the sum of the first value and the second value as the radius.

16. The rotating background video recognition device according to any one of claims 10 to 14, characterized in that: Determine, according to the target video frame, that the split screen of the video to be reviewed is divided into upper, middle and lower parts, and the upper split screen and the lower split screen of the video to be reviewed are background split screens, and the middle split screen of the video to be reviewed is a video content split screen; The reference frame determination module includes: A background split screen selection module, used to select any background split screen as a target background split screen; when it is recognized that the background split screen does not contain text, the target background split screen is used as the reference frame; The reference point selection module includes: a distance determination module, used to determine a target distance between a corner of an adjacent side of the video content split screen and the target background split screen and the center of the video content split screen; The reference point selection module is also used to select a reference point on a target arc in the reference frame with the video center of the video to be reviewed as the center and the target distance as the radius.

17. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the rotating background video recognition method according to any one of claims 1 to 8 are implemented.

18. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the rotating background video recognition method according to any one of claims 1 to 8.

19. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the steps of the rotating background video recognition method according to any one of claims 1 to 8 are implemented.

Images