A card point video recognition method and device

Abstract

The embodiment of the present application provides a kind of card point video identification method and device, it is related to video processing technical field.The method comprises: sampling to the video to be identified, obtains the sampling video frame set;The structural similarity SSIM between each pair of adjacent video frames in the sampling video frame set is calculated, and the SSIM sequence corresponding to the video to be identified is obtained;According to the SSIM sequence, determine whether the video to be identified is card point video.The embodiment of the present application is used to solve the problem that prior art cannot identify whether video is card point video.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a method and apparatus for identifying video frames. Background Technology

[0002] Beat-sync videos are a special style of video, characterized by transitions in sync with the rhythm of background music. Because beat-sync videos offer a better audiovisual experience compared to regular videos, they are very popular with users.

[0003] Many video processing scenarios require video recognition to determine if a video is a beat-sync video. For example, video platforms typically recommend videos to users based on their preferences. If a video platform can identify beat-sync videos in its library, it can provide targeted recommendations to users who like beat-sync videos. Therefore, video platforms have a need to identify beat-sync videos. However, when users create beat-sync videos and upload them to a video platform's library, they often do not label them as beat-sync videos. Therefore, video platforms cannot determine whether the videos uploaded by users in their library are indeed beat-sync videos. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and apparatus for identifying beat-sync videos, which solves the problem that the prior art cannot identify whether a video is a beat-sync video.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, embodiments of the present invention provide a method for beat-sync video recognition, comprising:

[0007] The video to be identified is sampled to obtain a set of sampled video frames;

[0008] Calculate the structural similarity (SSIM) between each pair of adjacent video frames in the sampled video frame set, and obtain the SSIM sequence corresponding to the video to be identified;

[0009] The video to be identified is determined based on the SSIM sequence to determine whether it is a beat-matching video.

[0010] As an optional implementation of this invention, determining whether the video to be identified is a beat-matching video based on the SSIM sequence includes:

[0011] Generate an SSIM curve based on the SSIM sequence;

[0012] The transition period of the video to be identified is obtained based on the SSIM curve, and the transition period is the period corresponding to the maximum autocorrelation of the SSIM curve;

[0013] Obtain the cycle matching error of the transition cycle; the cycle matching error is used to characterize the degree of dispersion of the difference between each cycle of the SSIM curve and the transition cycle;

[0014] The periodic matching error is used to determine whether the video to be identified is a beat-matching video.

[0015] As an optional implementation of this invention, obtaining the transition period of the video to be identified based on the SSIM curve includes:

[0016] Perform a Fourier transform on the SSIM curve to obtain the frequency distribution curve corresponding to the SSIM curve;

[0017] Based on the frequency distribution curve, a candidate frequency set is obtained, which includes a preset number of frequencies with the largest corresponding frequency amplitude on the frequency distribution curve.

[0018] Obtain the period corresponding to each frequency in the candidate frequency set, and generate a candidate period set;

[0019] Calculate the autocorrelation of the SSIM curve for each period in the candidate period set, wherein the candidate period set is...

[0020] The period corresponding to the maximum autocorrelation in the candidate autocorrelation set is determined as the transition period of the video to be identified.

[0021] As an optional implementation of this invention, obtaining the period matching error of the transition period includes:

[0022] Calculate the average value of each SSIM in the SSIM sequence to obtain the sequence mean;

[0023] The SSIM curve is binarized using the mean of the sequence as a threshold to obtain the binary curve corresponding to the SSIM curve.

[0024] The period set corresponding to the SSIM curve is obtained based on the binary curve. The period set includes the period formed by the time interval between each pair of adjacent rising edges in the binary curve and the period formed by the time interval between each pair of adjacent falling edges in the binary curve.

[0025] Based on the set of periods and the transition period, the period matching error of the transition period is obtained.

[0026] As an optional implementation of this invention, obtaining the period matching error of the transition period based on the period set and the transition period includes:

[0027] Calculate the difference between each cycle in the cycle set and the transition cycle to obtain a cycle difference set;

[0028] Calculate the standard deviation of each cycle difference in the set of cycle differences to obtain the cycle matching error of the transition cycle.

[0029] As an optional implementation of this invention, determining whether the video to be identified is a beat-matched video based on the periodic matching error includes:

[0030] Determine whether the periodic matching error is less than the threshold error;

[0031] If so, then the video to be identified is determined to be a beat-matching video;

[0032] If not, then the video to be identified is determined not to be a beat-matching video.

[0033] As an optional implementation of this invention, the method further includes:

[0034] Before obtaining the SSIM sequence corresponding to the video to be identified, each sampled video frame in the sampled video frame set is preprocessed into a grayscale image of a preset resolution.

[0035] As an optional implementation of this invention, the step of preprocessing each sampled video frame in the sampled video frame set into a grayscale image of a preset resolution includes:

[0036] Each sampled video frame in the sampled video frame set is converted into a grayscale image, and the grayscale image corresponding to each sampled video frame in the sampled video frame set is obtained.

[0037] The grayscale image corresponding to each sampled video frame in the sampled video frame set is cropped into an image with a preset length and width, and the cropped image corresponding to each sampled video frame in the sampled video frame set is obtained.

[0038] The cropped image corresponding to each sampled video frame in the sampled video frame set is sampled into an image of a preset resolution.

[0039] As an optional implementation of the present invention, the geometric center of the cropped image corresponding to each sampled video frame in the sampled video frame set coincides with the geometric center of each sampled video frame in the sampled video frame set.

[0040] As an optional implementation of the present invention, the preset value is the width of each sampled video frame in the sampled video frame set.

[0041] Secondly, embodiments of the present invention provide a video timing recognition device, comprising:

[0042] The sampling unit is used to sample the video to be recognized and obtain a set of sampled video frames.

[0043] The processing unit is used to calculate the structural similarity (SSIM) between each pair of adjacent video frames in the sampled video frame set, and to obtain the SSIM sequence corresponding to the video to be identified.

[0044] The identification unit is used to determine whether the video to be identified is a beat-matching video based on the SSIM sequence.

[0045] As an optional implementation of this invention, the recognition unit is specifically configured to: generate an SSIM curve based on the SSIM sequence; obtain the transition period of the video to be recognized based on the SSIM curve, wherein the transition period is the period corresponding to the maximum autocorrelation of the SSIM curve; obtain the period matching error of the transition period; the period matching error is used to characterize the degree of dispersion of the difference between each period of the SSIM curve and the transition period; and determine whether the video to be recognized is a beat-matched video based on the period matching error.

[0046] As an optional implementation of this invention, the recognition unit is specifically configured to perform a Fourier transform on the SSIM curve to obtain a frequency distribution curve corresponding to the SSIM curve; obtain a candidate frequency set based on the frequency distribution curve, the candidate frequency set including a preset number of frequencies with the largest corresponding frequency amplitude on the frequency distribution curve; obtain the period corresponding to each frequency in the candidate frequency set to generate a candidate period set; calculate the autocorrelation of the SSIM curve under each period in the candidate period set, the candidate autocorrelation set; and determine the period corresponding to the maximum autocorrelation in the candidate autocorrelation set as the transition period of the video to be recognized.

[0047] As an optional implementation of this invention, the identification unit is specifically used to calculate the average value of each SSIM in the SSIM sequence to obtain the sequence mean; to binarize the SSIM curve using the sequence mean as a threshold to obtain a binary curve corresponding to the SSIM curve; to obtain a period set corresponding to the SSIM curve based on the binary curve, the period set including the period formed by the time interval between each pair of adjacent rising edges in the binary curve and the period formed by the time interval between each pair of adjacent falling edges in the binary curve; and to obtain the period matching error of the transition period based on the period set and the transition period.

[0048] As an optional implementation of this invention, the identification unit is specifically used to calculate the difference between each period in the period set and the transition period to obtain a period difference set; calculate the standard deviation of each period difference in the period difference set to obtain the period matching error of the transition period.

[0049] As an optional implementation of this invention, the identification unit is specifically used to determine whether the periodic matching error is less than a threshold error; if yes, then the video to be identified is determined to be a beat-matching video; if no, then the video to be identified is determined to be a beat-matching video.

[0050] As an optional embodiment of the present invention, the checkpoint video recognition device further includes:

[0051] The preprocessing unit is used to preprocess each sampled video frame in the sampled video frame set into a grayscale image of a preset resolution before obtaining the SSIM sequence corresponding to the video to be identified.

[0052] As an optional implementation of this invention, the preprocessing unit is specifically configured to convert each sampled video frame in the sampled video frame set into a grayscale image, thereby obtaining a grayscale image corresponding to each sampled video frame in the sampled video frame set; crop the grayscale image corresponding to each sampled video frame in the sampled video frame set into an image with a length and width of a preset value, thereby obtaining a cropped image corresponding to each sampled video frame in the sampled video frame set; and sample the cropped image corresponding to each sampled video frame in the sampled video frame set into an image with a preset resolution.

[0053] As an optional implementation of the present invention, the preprocessing unit is specifically used to ensure that the geometric center of the cropped image corresponding to each sampled video frame in the sampled video frame set coincides with the geometric center of each sampled video frame in the sampled video frame set.

[0054] As an optional implementation of the present invention, the preset value is the width of each sampled video frame in the sampled video frame set.

[0055] Thirdly, embodiments of the present invention provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the electronic device to implement the video recognition method described in any of the above embodiments when executing the computer program.

[0056] Fourthly, embodiments of the present invention provide a computer-readable storage medium that, when executed by a computing device, causes the computing device to implement the video recognition method for key points described in any of the above embodiments.

[0057] Fifthly, embodiments of the present invention provide a computer program product that, when run on a computer, enables the computer to implement the video recognition method for key points described in any of the above embodiments.

[0058] The beat-sync video recognition method provided in this invention first samples the video to be recognized to obtain a set of sampled video frames. Then, it calculates the structural similarity between each pair of adjacent video frames in the sampled video frame set to obtain the SSIM sequence corresponding to the video to be recognized. Finally, it determines whether the video to be recognized is a beat-sync video based on the SSIM sequence. Since beat-sync videos have regular transitions, this invention obtains the structural similarity between each pair of adjacent video frames and determines whether the video to be recognized has regular transitions based on the structural similarity between each pair of adjacent video frames, thereby determining whether the video to be recognized is a beat-sync video. Therefore, this invention provides an implementation method for recognizing whether a video is a beat-sync video, which can solve the problem of being unable to identify whether a video is a beat-sync video. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings that need to be called in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is one of the flowcharts of the video recognition method for checkpoints provided in the embodiments of the present invention;

[0062] Figure 2 This is the second flowchart of the video recognition method for checkpoints provided in this embodiment of the invention;

[0063] Figure 3 A schematic diagram of a cropped image provided in an embodiment of the present invention;

[0064] Figure 4 A schematic diagram of the SSIM curve provided in an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of the leaf variation curve provided in an embodiment of the present invention;

[0066] Figure 6 A schematic diagram of the candidate frequency set provided in an embodiment of the present invention;

[0067] Figure 7 A schematic diagram of a binary curve provided in an embodiment of the present invention;

[0068] Figure 8 This is one of the structural schematic diagrams of the video recognition device for checkpoints provided in an embodiment of the present invention;

[0069] Figure 9 This is a second schematic diagram of the structure of the video recognition device for checkpoints provided in an embodiment of the present invention;

[0070] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0071] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0072] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0073] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. Furthermore, in the description of embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0074] This invention provides a method for beat-sync video recognition, referring to... Figure 1 As shown, the checkpoint video recognition method includes the following steps S11 to S13:

[0075] S11. Sample the video to be recognized and obtain a set of sampled video frames.

[0076] Optionally, the sampling method for the video to be identified may include sampling the video frames of the video to be identified one by one at preset time intervals. For example, the preset time interval may be 0.3 seconds, 0.5 seconds, 1 second, etc., and this embodiment of the invention does not limit it.

[0077] For example, when sampling video frames of the video to be identified one by one at intervals of 0.3 seconds, the video frames of the video to be sampled include: video frame F1, video frame F2, video frame F3, ..., video frame Fn-1, video frame Fn. When the time interval between adjacent video frames of the video to be sampled is 0.1 seconds, the set of sampled video frames includes: video frame F1, video frame F4, video frame F7, ..., video frame F3m-2, video frame F3m+1.

[0078] S12. Calculate the structural similarity (SSIM) between each pair of adjacent video frames in the sampled video frame set, and obtain the SSIM sequence corresponding to the video to be identified.

[0079] Specifically, in this embodiment of the invention, adjacent video frames in the sampled video frame set refer to video frame pairs composed of video frames with the closest temporal sequence in the sampled video frame set. For example, if the sampled video frame set includes: video frame F1, video frame F4, video frame F7, ..., video frame F3m-2, video frame F3m+1, then adjacent video frame pairs include: {video frame F1 and video frame F4}, {video frame F4 and video frame F7}, ..., {F3m-2, video frame F3m+1}.

[0080] Furthermore, the method for calculating the structural similarity between each pair of adjacent video frames in the sampled video frame set may include calculating the structural similarity between each pair of adjacent video frames in the sampled video frame set using the following formula:

[0081]

[0082] Where SSIM(x,y) is the structural similarity between video frame x and video frame y; μ x μ is the average value of all pixel values ​​in video frame x. y This is the average value of each pixel in video frame y; represents the variance of each pixel value in video frame x; σ is the variance of the pixel values ​​in video frame y. xy Let c1 be the covariance between the pixel values ​​in video frame x and the pixel values ​​in video frame y; c1 = (k1L) 2 c2 = (k2L) 2 L is the dynamic range of pixel values, k1 = 0.01, k2 = 0.03.

[0083] S13. Determine whether the video to be identified is a beat-matching video based on the SSIM sequence.

[0084] The beat-sync video recognition method provided in this invention first samples the video to be recognized to obtain a set of sampled video frames. Then, it calculates the structural similarity between each pair of adjacent video frames in the sampled video frame set to obtain the SSIM sequence corresponding to the video to be recognized. Finally, it determines whether the video to be recognized is a beat-sync video based on the SSIM sequence. Since beat-sync videos have regular transitions, this invention obtains the structural similarity between each pair of adjacent video frames and determines whether the video to be recognized has regular transitions based on the structural similarity between each pair of adjacent video frames, thereby determining whether the video to be recognized is a beat-sync video. Therefore, this invention provides an implementation method for recognizing whether a video is a beat-sync video, which can solve the problem of being unable to identify whether a video is a beat-sync video.

[0085] As an extension and refinement of the above embodiments, this invention provides another method for beat-sync video recognition, referring to... Figure 2 As shown, the checkpoint video recognition method includes the following steps:

[0086] S201. Sample the video to be identified and obtain a set of sampled video frames.

[0087] The implementation method of the above step S201 can be the same as Figure 1 The implementation of step S11 in the illustrated embodiment is the same, and will not be repeated here to avoid redundancy.

[0088] S202. Preprocess each sampled video frame in the sampled video frame set into a grayscale image with a preset resolution.

[0089] Optionally, the implementation of step S202 (preprocessing each sampled video frame in the sampled video frame set into a grayscale image of a preset resolution) may include the following steps a to c:

[0090] Step a: Convert each sampled video frame in the sampled video frame set into a grayscale image to obtain the grayscale image corresponding to each sampled video frame in the sampled video frame set.

[0091] For example, if each sampled video frame in the sampled video frame set is an RGB image, then step a above is to convert each sampled video frame from an RGB image to a grayscale image.

[0092] Since the above embodiments convert the sampled video frames into grayscale images, the above embodiments can reduce the amount of computation when calculating the structural similarity between each pair of adjacent video frames in the sampled video frame set.

[0093] Step b: Crop the grayscale image corresponding to each sampled video frame in the sampled video frame set into an image with a preset length and width, and obtain the cropped image corresponding to each sampled video frame in the sampled video frame set.

[0094] Optionally, the geometric center of the cropped image corresponding to each sampled video frame in the sampled video frame set coincides with the geometric center of each sampled video frame in the sampled video frame set.

[0095] Optionally, the preset value is the width of each sampled video frame in the sampled video frame set.

[0096] When the geometric center of the cropped image corresponding to each sampled video frame in the sampled video frame set coincides with the geometric center of each sampled video frame in the sampled video frame set, and the preset value is the width of each sampled video frame in the sampled video frame set, the cropping method of the grayscale image corresponding to each sampled video frame in the sampled video frame set is as follows: Figure 3 As shown in (a) or (b), the grayscale image 31 corresponding to the sampled video frame is cropped into a square cropped image 32 with equal length and width.

[0097] Since the above embodiments further crop the grayscale images corresponding to each sampled video frame in the sampled video frame set into images with preset length and width, the above embodiments can further reduce the amount of computation when calculating the structural similarity between each pair of adjacent video frames in the sampled video frame set.

[0098] Step c: Sample the cropped image corresponding to each sampled video frame in the sampled video frame set into an image of a preset resolution.

[0099] For example, the preset resolution can be 224*224.

[0100] Since the above embodiments further sample the cropped images corresponding to each sampled video frame in the sampled video frame set into images of a preset resolution, the above embodiments can further reduce the amount of computation when calculating the structural similarity between each pair of adjacent video frames in the sampled video frame set.

[0101] S203. Calculate the SSIM between each pair of adjacent video frames in the sampled video frame set to obtain the SSIM sequence corresponding to the video to be identified.

[0102] The implementation method of the above step S203 can be the same as Figure 1 The implementation of step S12 in the illustrated embodiment is the same, and will not be repeated here to avoid redundancy.

[0103] S204. Generate an SSIM curve based on the SSIM sequence.

[0104] That is, an SSIM curve is generated by using each pair of adjacent video frames as the horizontal axis and the SSIM of each pair of adjacent video frames as the vertical axis.

[0105] For example, the SSIM curve generated from the SSIM sequence can be as follows: Figure 4 As shown.

[0106] S205. Obtain the transition period of the video to be identified based on the SSIM curve.

[0107] The transition period is the period corresponding to the maximum autocorrelation of the SSIM curve.

[0108] As an optional implementation of this invention, the above-mentioned step S205 (obtaining the transition period of the video to be identified based on the SSIM curve) may include the following steps 1 to 5:

[0109] Step 1: Perform a Fourier transform on the SSIM curve to obtain the frequency distribution curve corresponding to the SSIM curve.

[0110] That is, the SSIM curve is transformed from a time-domain characteristic curve to a frequency-domain characteristic curve.

[0111] As illustrated in the previous example, when the SSIM curve is as follows: Figure 4 As shown, a Fourier transform is performed on the SSIM curve to obtain the frequency distribution curve corresponding to the SSIM curve, as shown below. Figure 5 As shown.

[0112] Step 2: Obtain the candidate frequency set based on the frequency distribution curve.

[0113] The candidate frequency set includes a preset number of frequencies with the largest corresponding frequency amplitude on the frequency distribution curve.

[0114] For example, the preset number can be 5. That is, the five frequencies with the largest frequency amplitude on the frequency distribution curve are used to form a candidate frequency set.

[0115] For example, refer to Figure 6 As shown, when the preset quantity can be 5, because Figure 5The five largest frequency amplitudes on the frequency distribution curve shown are frequency amplitude a, frequency amplitude b, frequency amplitude c, frequency amplitude d, and frequency amplitude e, respectively. The frequency corresponding to frequency amplitude a is frequency fa, the frequency corresponding to frequency amplitude b is frequency fb, the frequency corresponding to frequency amplitude c is frequency fc, the frequency corresponding to frequency amplitude d is frequency fd, and the frequency corresponding to frequency amplitude e is frequency fe. Therefore, the candidate frequency set includes: frequency fa, frequency fb, frequency fc, frequency fd, and frequency fe.

[0116] Step 3: Obtain the period corresponding to each frequency in the candidate frequency set and generate a candidate period set.

[0117] Since frequency and period are reciprocals of each other, the period corresponding to each frequency in the candidate frequency set can be obtained by calculating the reciprocal of each frequency in the candidate frequency set, and then a candidate period set can be generated.

[0118] As described in the example above, the candidate frequency set includes: frequency fa, frequency fb, frequency fc, frequency fd, and frequency fe. Therefore, the candidate period set includes: period 1 / fa, period 1 / fb, period 1 / fc, period 1 / fd, and period 1 / fe.

[0119] Step 4: Calculate the autocorrelation of the SSIM curve for each period in the candidate period set, wherein the candidate autocorrelation set is...

[0120] The autocorrelation of the SSIM curve at any given period refers to the correlation between the SSIM curve and the curve obtained by shifting the SSIM curve by that period. For example, the autocorrelation of the SSIM curve at period T is the correlation between the SSIM curve and the curve obtained by shifting the SSIM curve by that period T.

[0121] As illustrated in the example above, the candidate period set includes: period 1 / fa, period 1 / fb, period 1 / fc, period 1 / fd, and period 1 / fe. Therefore, the autocorrelation R of the SSIM curve under period 1 / fa is calculated. 1 / fa The autocorrelation R of the SSIM curve under period 1 / fb 1 / fb The autocorrelation R of the SSIM curve under period 1 / fc 1 / fc The autocorrelation R of the SSIM curve under period 1 / fd 1 / fd And the autocorrelation R of the SSIM curve under period 1 / fe 1 / fe The candidate autocorrelation set includes: autocorrelation R 1 / fa Autocorrelation R 1 / fb Autocorrelation R 1 / fc Autocorrelation R 1 / fd and autocorrelation R1 / fe .

[0122] Step 5: Determine the period corresponding to the maximum autocorrelation in the candidate autocorrelation set as the transition period of the video to be identified.

[0123] As described in the previous embodiment, the candidate autocorrelation set includes: autocorrelation R 1 / fa Autocorrelation R 1 / fb Autocorrelation R 1 / fc Autocorrelation R 1 / fd and autocorrelation R 1 / fe If the order of the autocorrelation values ​​in the candidate autocorrelation set is as follows: R 1 / fa >R 1 / fb >R 1 / fc >R 1 / fd >R 1 / fe Then, the period 1 / fa is determined as the transition period of the video to be identified.

[0124] S206. Obtain the period matching error of the transition cycle.

[0125] The period matching error is used to characterize the degree of dispersion of the difference between each period of the SSIM curve and the transition period.

[0126] As an optional implementation of this invention, the above step S206 (obtaining the cycle matching error of the transition cycle) is implemented by including the following steps ① to ④:

[0127] Step ①: Calculate the average value of each SSIM in the SSIM sequence to obtain the sequence mean.

[0128] That is, the average value of each SSIM in the SSIM sequence obtained in step S203 is calculated.

[0129] Step 2: Binarize the SSIM curve using the mean of the sequence as a threshold to obtain the binary curve corresponding to the SSIM curve.

[0130] For example, the portion of the SSIM curve greater than the sequence mean can be set to 1, the portion of the SSIM curve less than the sequence mean can be set to 0, and the portion of the SSIM curve equal to the sequence mean can be set to a rising edge or a falling edge.

[0131] As illustrated in the example above, when the SSIM curve is as follows: Figure 4 As shown, the SSIM curve is binarized using the sequence mean as a threshold, and the resulting binary curve corresponding to the SSIM curve is as follows. Figure 7 As shown in the diagram. An upward arrow indicates a rising edge, and a downward arrow indicates a falling edge.

[0132] Step ③: Obtain the set of periods corresponding to the SSIM curve based on the binary curve.

[0133] The period set includes the period formed by the time interval between each pair of adjacent rising edges in the binary curve and the period formed by the time interval between each pair of adjacent falling edges in the binary curve.

[0134] That is, the period formed by the time interval between the first rising edge and the second rising edge in the binary curve, the period formed by the time interval between the first falling edge and the second falling edge in the binary curve, the period formed by the time interval between the second rising edge and the third rising edge in the binary curve, the period formed by the time interval between the second falling edge and the third falling edge in the binary curve, ..., the period formed by the time interval between the second-to-last rising edge and the last rising edge in the binary curve, and the period formed by the time interval between the second-to-last falling edge and the last falling edge in the binary curve are obtained, and all of the obtained periods are combined into the period set corresponding to the SSIM curve.

[0135] Step 4: Obtain the cycle matching error of the transition cycle based on the cycle set and the transition cycle.

[0136] As an optional implementation of this invention, step ④ (obtaining the period matching error of the transition period based on the period set and the transition period) is implemented by including the following steps I and II:

[0137] Step 1: Calculate the difference between each period in the period set and the transition period to obtain the period difference set.

[0138] For example, the period set includes: period t1, period t2, period t3, ..., period tn, and the transition period is T. Then the period difference set includes: t1-T, t2-T, t3-T, ..., tn-T.

[0139] Step II: Calculate the standard deviation of each cycle difference in the set of cycle differences to obtain the cycle matching error of the transition cycle.

[0140] Optionally, the method for calculating the standard deviation of each period difference in the set of period differences may include:

[0141] First, calculate the average value of each period difference in the set of period differences.

[0142] Specifically, the method for calculating the average of the period differences in the set of period differences can include: calculating the average of the period differences in the set of period differences using the following formula:

[0143]

[0144] Where mean(T) is the average value of each period difference in the set of period differences, n is the number of period differences in the set of period differences, and ti-T is the i-th period difference in the set of period differences.

[0145] Secondly, based on the average value of each period difference in the set of period differences and each period difference in the set of period differences, the standard deviation of each period difference in the set of period differences is calculated.

[0146] Specifically, the method for calculating the standard deviation of each period difference in the period difference set based on the average value of each period difference in the period difference set and the individual period differences in the period difference set can include: calculating the standard deviation of each period difference in the period difference set based on the average value of each period difference in the period difference set and the individual period differences in the period difference set using the following formula:

[0147]

[0148] Where me is the standard deviation of each period difference in the period difference set, mean(T) is the average of each period difference in the period difference set, and ti-T is the i-th period difference in the period difference set.

[0149] S207. Determine whether the periodic matching error is less than the threshold error.

[0150] It should be noted that the threshold error value is not limited in this embodiment of the invention. The smaller the threshold, the stricter the recognition standard for the beat-matching video; the larger the threshold, the more lenient the recognition standard for the beat-matching video. Therefore, the threshold error can be set according to the requirements in actual implementation.

[0151] In step S207 above, if the period matching error is less than the threshold error, then step S208 is executed; if the period matching error is greater than or equal to the threshold error, then step S209 is executed.

[0152] S208. Determine that the video to be identified is a beat-matching video.

[0153] S209. Determine that the video to be identified is not a video with a fixed beat.

[0154] The beat-sync video recognition method provided in this invention first samples the video to be recognized to obtain a set of sampled video frames. Then, it calculates the structural similarity between each pair of adjacent video frames in the sampled video frame set to obtain the SSIM sequence corresponding to the video to be recognized. Finally, it determines whether the video to be recognized is a beat-sync video based on the SSIM sequence. Since beat-sync videos have regular transitions, this invention obtains the structural similarity between each pair of adjacent video frames and determines whether the video to be recognized has regular transitions based on the structural similarity between each pair of adjacent video frames, thereby determining whether the video to be recognized is a beat-sync video. Therefore, this invention provides an implementation method for recognizing whether a video is a beat-sync video, which can solve the problem of being unable to identify whether a video is a beat-sync video.

[0155] Based on the same inventive concept, as an implementation of the above method, this embodiment of the invention also provides a checkpoint video recognition device. This embodiment corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the checkpoint video recognition device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0156] This invention provides a video recognition device for timing points. Figure 8 A schematic diagram of the structure of the checkpoint video recognition device is shown below. Figure 8 The checkpoint video recognition device 800 includes:

[0157] Sampling unit 81 is used to sample the video to be recognized and obtain a set of sampled video frames;

[0158] Processing unit 82 is used to calculate the structural similarity (SSIM) between each pair of adjacent video frames in the sampled video frame set, and to obtain the SSIM sequence corresponding to the video to be identified;

[0159] The identification unit 83 is used to determine whether the video to be identified is a beat-through video based on the SSIM sequence.

[0160] As an optional implementation of this invention, the recognition unit 83 is specifically configured to: generate an SSIM curve based on the SSIM sequence; obtain the transition period of the video to be recognized based on the SSIM curve, wherein the transition period is the period corresponding to the maximum autocorrelation of the SSIM curve; obtain the period matching error of the transition period; the period matching error is used to characterize the degree of dispersion of the difference between each period of the SSIM curve and the transition period; and determine whether the video to be recognized is a beat-matched video based on the period matching error.

[0161] As an optional implementation of this invention, the recognition unit 83 is specifically used to perform a Fourier transform on the SSIM curve to obtain a frequency distribution curve corresponding to the SSIM curve; obtain a candidate frequency set based on the frequency distribution curve, the candidate frequency set including a preset number of frequencies with the largest corresponding frequency amplitude on the frequency distribution curve; obtain the period corresponding to each frequency in the candidate frequency set to generate a candidate period set; calculate the autocorrelation of the SSIM curve under each period in the candidate period set, the candidate autocorrelation set; and determine the period corresponding to the maximum autocorrelation in the candidate autocorrelation set as the transition period of the video to be recognized.

[0162] As an optional implementation of this invention, the identification unit 83 is specifically used to calculate the average value of each SSIM in the SSIM sequence to obtain the sequence mean; to binarize the SSIM curve using the sequence mean as a threshold to obtain a binary curve corresponding to the SSIM curve; to obtain a period set corresponding to the SSIM curve based on the binary curve, the period set including the period formed by the time interval between each pair of adjacent rising edges in the binary curve and the period formed by the time interval between each pair of adjacent falling edges in the binary curve; and to obtain the period matching error of the transition period based on the period set and the transition period.

[0163] As an optional implementation of the present invention, the identification unit 83 is specifically used to calculate the difference between each period in the period set and the transition period to obtain a period difference set; calculate the standard deviation of each period difference in the period difference set to obtain the period matching error of the transition period.

[0164] As an optional implementation of this invention, the identification unit 83 is specifically used to determine whether the periodic matching error is less than the threshold error; if yes, then the video to be identified is determined to be a beat-matching video; if no, then the video to be identified is determined to be a beat-matching video.

[0165] As an optional implementation of the present invention, refer to Figure 9 As shown, the checkpoint video recognition device 800 further includes:

[0166] The preprocessing unit 84 is used to preprocess each sampled video frame in the sampled video frame set into a grayscale image of a preset resolution before acquiring the SSIM sequence corresponding to the video to be identified.

[0167] As an optional implementation of this invention, the preprocessing unit 84 is specifically configured to convert each sampled video frame in the sampled video frame set into a grayscale image, thereby obtaining a grayscale image corresponding to each sampled video frame in the sampled video frame set; crop the grayscale image corresponding to each sampled video frame in the sampled video frame set into an image with a length and width of a preset value, thereby obtaining a cropped image corresponding to each sampled video frame in the sampled video frame set; and sample the cropped image corresponding to each sampled video frame in the sampled video frame set into an image with a preset resolution.

[0168] As an optional implementation of the present invention, the preprocessing unit 84 is specifically used to ensure that the geometric center of the cropped image corresponding to each sampled video frame in the sampled video frame set coincides with the geometric center of each sampled video frame in the sampled video frame set.

[0169] As an optional implementation of the present invention, the preset value is the width of each sampled video frame in the sampled video frame set.

[0170] The checkpoint video recognition device provided in this embodiment can execute the checkpoint video recognition method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0171] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 10 As shown, the electronic device provided in this embodiment includes a memory 101 and a processor 102. The memory 101 is used to store a computer program; the processor 102 is used to execute the video recognition method provided in the above embodiment when executing the computer program.

[0172] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the computing device to implement the video recognition method for key points provided in the above embodiments.

[0173] Based on the same inventive concept, this embodiment of the invention also provides a computer program product, which, when run on a computer, enables the computing device to implement the video recognition method for key points provided in the above embodiments.

[0174] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0175] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0176] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0177] Computer-readable media include both permanent and non-permanent, removable and non-removable storage media. Storage media can store information using any method or technology; the information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A card point video recognition method, characterized in that, The method comprises: sampling a to-be-identified video to obtain a set of sampled video frames; calculating structural similarity (SSIM) between each pair of adjacent video frames in the set of sampled video frames to obtain an SSIM sequence corresponding to the to-be-identified video; determining whether the to-be-identified video is a keyframe video according to the SSIM sequence; the determining whether the to-be-identified video is a keyframe video according to the SSIM sequence comprises: generating an SSIM curve according to the SSIM sequence; obtaining a transition period of the to-be-identified video according to the SSIM curve, the transition period being a period corresponding to a maximum autocorrelation degree of the SSIM curve; obtaining a period matching error of the transition period, the period matching error being used to represent a dispersion degree of a difference between each period of the SSIM curve and the transition period; determining whether the to-be-identified video is a keyframe video according to the period matching error.

2. The method of claim 1, wherein, the obtaining the transition period of the to-be-identified video according to the SSIM curve comprises: performing Fourier transform on the SSIM curve to obtain a frequency distribution curve corresponding to the SSIM curve; obtaining a candidate frequency set according to the frequency distribution curve, the candidate frequency set comprising a preset number of frequencies with maximum frequency amplitudes on the frequency distribution curve; obtaining periods corresponding to each frequency in the candidate frequency set to generate a candidate period set; calculating autocorrelation degrees of the SSIM curve under each period in the candidate period set respectively to obtain a candidate autocorrelation degree set; determining a period corresponding to a maximum autocorrelation degree in the candidate autocorrelation degree set as the transition period of the to-be-identified video.

3. The method of claim 1, wherein, the obtaining the period matching error of the transition period comprises: calculating an average value of each SSIM in the SSIM sequence to obtain a sequence average value; binarizing the SSIM curve by taking the sequence average value as a threshold to obtain a binary curve corresponding to the SSIM curve; obtaining a period set corresponding to the SSIM curve according to the binary curve, the period set comprising periods formed by time intervals between each pair of adjacent rising edges in the binary curve and periods formed by time intervals between each pair of adjacent falling edges in the binary curve; obtaining the period matching error of the transition period according to the period set and the transition period.

4. The method of claim 3, wherein, the obtaining the period matching error of the transition period according to the period set and the transition period comprises: calculating a difference between each period in the period set and the transition period to obtain a period difference set; calculating a standard deviation of each period difference in the period difference set to obtain the period matching error of the transition period.

5. The method of claim 1, wherein, the determining whether the to-be-identified video is a keyframe video according to the period matching error comprises: determining whether the period matching error is less than a threshold error; if yes, determining that the to-be-identified video is a keyframe video; if no, determining that the to-be-identified video is not a keyframe video.

6. The method according to any one of claims 1 to 5, characterized in that, the method further comprises: Before acquiring the SSIM sequence corresponding to the to-be-identified video, each of the sampled video frames in the set of sampled video frames is preprocessed into a grayscale image of a preset resolution.

7. The method of claim 6, wherein, The preprocessing of each of the sampled video frames in the set of sampled video frames into a grayscale image of a preset resolution comprises: Converting each of the sampled video frames in the set of sampled video frames into a grayscale image to obtain a grayscale image corresponding to each of the sampled video frames in the set of sampled video frames; Cutting the grayscale image corresponding to each of the sampled video frames in the set of sampled video frames into an image with a preset length and width to obtain a cut image corresponding to each of the sampled video frames in the set of sampled video frames; Sampling the cut image corresponding to each of the sampled video frames in the set of sampled video frames into an image of a preset resolution.

8. The method of claim 7, wherein, The geometric center of the cut image corresponding to each of the sampled video frames in the set of sampled video frames coincides with the geometric center of each of the sampled video frames in the set of sampled video frames.

9. The method of claim 7, wherein, The preset value is the width of each of the sampled video frames in the set of sampled video frames.

10. A card point video recognition apparatus, characterized by comprising: The method comprises: a sampling unit configured to sample a to-be-identified video to obtain a set of sampled video frames; a processing unit configured to calculate the structural similarity (SSIM) between each pair of adjacent video frames in the set of sampled video frames to obtain an SSIM sequence corresponding to the to-be-identified video; an identification unit configured to determine whether the to-be-identified video is a card point video according to the SSIM sequence. The identification unit is specifically configured to generate an SSIM curve according to the SSIM sequence, obtain a transition period of the to-be-identified video according to the SSIM curve, the transition period being a period corresponding to the maximum autocorrelation degree of the SSIM curve, obtain a period matching error of the transition period, the period matching error being used to represent the discrete degree of the difference between each period of the SSIM curve and the transition period, and determine whether the to-be-identified video is a card point video according to the period matching error.

11. An electronic device, comprising: The method comprises: a memory and a processor, the memory being configured to store a computer program, and the processor being configured to, when executing the computer program, enable the electronic device to implement the card point video identification method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the card point video identification method according to any one of claims 1-9.

13. A computer program product, characterised in that, When the computer program product runs on a computer, the computer implements the card point video identification method according to any one of claims 1-9.

Images