Video fluency evaluation method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides a video fluency evaluation method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a to-be-evaluated first video; performing repeated frame identification and removal on the first video to obtain a second video after repeated frame removal; target time-space domain feature extraction is carried out on the second video, target time-space domain feature information corresponding to the second video is determined, and the target time-space domain feature is a predetermined video feature associated with the video fluency; and inputting the target time-space domain feature information into the target evaluation network model for fluency evaluation to obtain a fluency evaluation result corresponding to the first video. Through the technical scheme of the embodiment of the invention, the video fluency can be automatically and accurately evaluated, and the efficiency and accuracy of video fluency evaluation are improved.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more particularly to a video smoothness evaluation method, apparatus, device, medium, and program product. Background Technology

[0002] With the rapid development of computer technology, the smoothness of video playback affects user experience, necessitating the evaluation of video smoothness to optimize the user experience. Currently, video smoothness is typically evaluated manually by watching the videos. However, this manual evaluation method is time-consuming and labor-intensive, reducing the efficiency and accuracy of video smoothness assessment. Summary of the Invention

[0003] This disclosure provides a video smoothness assessment method, apparatus, device, medium, and program product to achieve automatic and accurate assessment of video smoothness, thereby improving the efficiency and accuracy of video smoothness assessment.

[0004] In a first aspect, embodiments of this disclosure provide a method for evaluating video smoothness, including:

[0005] Obtain the first video to be evaluated;

[0006] The first video is subjected to duplicate frame identification and removal to obtain a second video after duplicate frames are removed;

[0007] The target spatiotemporal domain features are extracted from the second video to determine the target spatiotemporal domain feature information corresponding to the second video, wherein the target spatiotemporal domain features are pre-determined video features that are related to the smoothness of the video;

[0008] The target spatiotemporal feature information is input into the target evaluation network model for smoothness evaluation, and the smoothness evaluation result corresponding to the first video is obtained.

[0009] Secondly, embodiments of this disclosure also provide a video smoothness evaluation device, comprising:

[0010] The first video acquisition module is used to acquire the first video to be evaluated.

[0011] The duplicate frame removal module is used to identify and remove duplicate frames from the first video to obtain a second video after removing duplicate frames.

[0012] The spatiotemporal domain feature extraction module is used to extract target spatiotemporal domain features from the second video and determine the target spatiotemporal domain feature information corresponding to the second video, wherein the target spatiotemporal domain features are pre-determined video features that are related to video smoothness;

[0013] The smoothness evaluation module is used to input the target spatiotemporal feature information into the target evaluation network model to perform smoothness evaluation and obtain the smoothness evaluation result corresponding to the first video.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0015] One or more processors;

[0016] Storage device for storing one or more programs.

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the video smoothness evaluation method as described in any embodiment of this disclosure.

[0018] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the video smoothness evaluation method as described in any of the embodiments of this disclosure.

[0019] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the video smoothness evaluation method as described in any of the embodiments of the present disclosure.

[0020] In this embodiment, by identifying and removing duplicate frames from a first video to be evaluated, a second video with duplicate frames removed is obtained. Target spatiotemporal features are extracted from the second video to determine its corresponding target spatiotemporal feature information. This target spatiotemporal feature information is then input into a target evaluation network model for smoothness evaluation. This automatically obtains the smoothness evaluation result for the first video, thus achieving automatic video smoothness evaluation without manual intervention and improving the efficiency of video smoothness evaluation. By extracting video features related to video smoothness from the second video after removing duplicate frames, more accurate target spatiotemporal feature information can be obtained. This allows the target evaluation network model to more accurately evaluate the smoothness of the first video based on the target spatiotemporal feature information, thereby improving the accuracy of video smoothness evaluation. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 This is a flowchart illustrating a video smoothness evaluation method provided in an embodiment of this disclosure;

[0023] Figure 2 This is a flowchart illustrating another video smoothness evaluation method provided in this embodiment of the disclosure;

[0024] Figure 3 This is a flowchart illustrating yet another video smoothness evaluation method provided in this disclosure embodiment;

[0025] Figure 4 This is a schematic diagram of the structure of a video smoothness evaluation device provided in an embodiment of this disclosure;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0034] Figure 1 This is a flowchart illustrating a video smoothness evaluation method provided in an embodiment of the present disclosure. This embodiment is applicable to evaluating the subjective smoothness of video playback. The method can be executed by a video smoothness evaluation device, which can be implemented in the form of software and / or hardware, or optionally by an electronic device, such as a mobile terminal, a PC, or a server.

[0035] like Figure 1 As shown, the video smoothness evaluation method specifically includes the following steps:

[0036] S110, Obtain the first video to be evaluated.

[0037] The first video can refer to any video whose playback smoothness needs to be evaluated. For example, the video to be played on the playback device can be used as the first video. In a live streaming scenario, the live video to be played can be used as the first video to directly evaluate the intuitive smoothness of the video without actually playing it for smoothness evaluation. Smoothness can be a parameter used to evaluate the experience of watching a video. For example, smoothness can include the clarity or picture quality of the video during viewing.

[0038] S120. Perform duplicate frame identification and removal on the first video to obtain the second video after removing duplicate frames.

[0039] Duplicate frames refer to consecutively repeated video frames in the first video. Duplicate frames can be introduced during video recording or distribution. For example, abnormal situations during recording may lead to repeated recording of video frames. When the server distributes the recorded video to the client, it typically undergoes frame interpolation or frame extraction encoding or secondary transcoding to obtain the video to be played, i.e., the first video. Due to frame interpolation, duplicate frames will exist in the distributed first video. The presence of duplicate frames affects the characteristic information of the first video, such as the original frame rate differing from the actual frame rate. The original frame rate refers to the streaming frame rate of the first video, i.e., the frame rate given in the video parameters. The actual frame rate is the true frame rate of the first video after removing duplicate frames. It should be noted that video smoothness is directly affected by the actual frame rate, not the original frame rate, thus requiring duplicate frame removal processing of the first video to improve the accuracy of video smoothness assessment.

[0040] Specifically, by identifying the similarity between every two adjacent video frames in the first video, two adjacent video frames that appear repeatedly in the first video can be determined based on the similarity, and one of the video frames can be removed as a duplicate frame of the other video frame, thereby obtaining the second video after removing duplicate frames.

[0041] S130. Extract the target spatiotemporal domain features from the second video to determine the target spatiotemporal domain feature information corresponding to the second video. The target spatiotemporal domain features are pre-determined video features that are related to the smoothness of the video.

[0042] Among these, the target spatiotemporal features are the temporal and spatial features of the video that are correlated with video smoothness among all video features. The correlation between the target spatiotemporal features and video smoothness is greater than a preset correlation. The correlation can be characterized using the Pearson correlation coefficient. The Pearson correlation coefficient is used to quantify the strength and direction of the linear relationship between the two variables, video features and video smoothness. Its value ranges from -1 to 1, where 1 represents a completely positive linear relationship, -1 represents a completely negative linear relationship, and 0 represents no linear relationship between the two variables. The target spatiotemporal feature information refers to the specific feature values ​​of the target spatiotemporal features possessed by the second video.

[0043] For example, the target spatiotemporal feature information may include, but is not limited to: the actual frame rate of the second video and target feature information, wherein the target feature information may include, but is not limited to, at least one of the following: temporal complexity information, spatial complexity information, Laplacian operator information, optical flow vector information, and texture information of the second video. The actual frame rate refers to the true frame rate of the first video after removing duplicate frames. Temporal Perceptual Information (TI) is used to characterize the temporal variation of a video sequence. Video sequences with higher levels of motion typically have higher TI values. Spatial Perceptual Information (SI) is used to characterize the spatial variation of a video sequence. Video scenes with greater spatial complexity have higher SI values. Laplacian operator information is an edge detection operator used to highlight regions where image intensity changes rapidly, i.e., image edges. Optical flow vector information is a vector representing the direction and distance of pixel movement between two consecutive frames, used to calculate the speed and direction of motion of objects in a dynamic image. Texture information may include at least one texture index that reflects the texture features of the video, such as the homogeneity and correlation coefficient of the gray-level co-occurrence matrix.

[0044] Specifically, target spatiotemporal features correlated with video smoothness can be pre-determined from all video features. These video features can include spatial features such as the Laplacian operator, SI, texture, luminance, and noise, as well as temporal features such as actual frame rate, TI, and optical flow vector. The covariance between each video feature and the smoothness score in the sample video can be calculated and normalized to obtain the Pearson correlation coefficient between each feature and the smoothness score. Video features with an absolute Pearson correlation coefficient greater than a preset coefficient (e.g., 0.2) are identified as target spatiotemporal features correlated with smoothness. By extracting target spatiotemporal feature information from the second video (after removing duplicate frames), instead of extracting all video feature information from the first video containing duplicate frames, more important and accurate target spatiotemporal feature information can be obtained, thereby improving the accuracy of video smoothness assessment.

[0045] For example, step S130 may include: determining the actual frame rate of the second video based on the original frame rate and original frame number of the first video and the actual frame number of the second video; and extracting target features from the second video to determine the target feature information of the second video.

[0046] In this context, the original frame rate of the first video refers to the inherent frame rate of the first video itself, i.e., the frame rate given in the video parameters. The original frame count refers to the total number of video frames in the first video. The actual frame count of the second video refers to the total number of video frames in the second video. The actual frame rate of the second video refers to the inherent frame rate of the second video itself. The actual frame rate of the second video refers to the subjectively present true frame rate when the first video is played. Because the first video contains repeated frames, the image is static when these repeated frames are played, which cannot improve smoothness, thus making the original frame rate of the first video not equal to the true frame rate.

[0047] Specifically, the ratio between the original frame rate and the actual frame rate can be determined based on the original frame rate of the first video and the actual frame rate of the second video, and the actual frame rate of the second video can be accurately determined based on this ratio. Extracting target features from the second video (with duplicate frames removed) yields more accurate target feature information, such as the time complexity, spatial complexity, Laplacian operator information, optical flow vector information, and texture information of the second video. For example, the time complexity information extraction process is as follows: Differential processing is performed on every two adjacent video frames in the second video with n frames to obtain n-1 difference images, i.e., n-1 difference matrices. The mean of each difference matrix is ​​taken to obtain the n-1 difference mean, and the n-1 difference mean is taken again. The final mean is used as the time complexity information. The extraction process for spatial complexity information is as follows: For each video frame in the second video (n frames), perform Sobel operation to obtain n spatial matrices. Take the average of each spatial matrix to obtain n spatial means. Then, take the average of these n spatial means again. The final average is used as the spatial complexity information. The extraction process for Laplacian operator information is as follows: For each video frame in the second video (n frames), perform Laplacian operation to obtain n spatial matrices. Take the average of each spatial matrix to obtain n spatial means. Then, take the average of these n spatial means again. The final average is used as the Laplacian operator information. The extraction process for optical flow vector information is as follows: Using a dense optical flow algorithm, calculate the pixel motion velocity for every two adjacent video frames in the second video (n frames), obtaining n-1 optical flow velocities. Take the average of these n-1 optical flow velocities. The average value is used as the optical flow vector information. The process of extracting texture information is as follows: For each video frame in the second video, a specific pixel neighborhood and distance are selected, each pixel in the image is traversed, the frequency of pixels corresponding to specific gray levels in the neighborhood is counted, and these frequencies are used to construct a matrix, namely the gray-level co-occurrence matrix. The co-occurrence matrix is ​​normalized to obtain a probability matrix representing the relationship features between gray levels, and the homogeneity and correlation coefficient of the probability matrix are used as texture information.

[0048] For example, determining the actual frame rate of the second video based on the original frame rate and original frame count of the first video and the actual frame count of the second video may include: determining a frame count ratio based on the original frame count of the first video and the actual frame count of the second video; and determining the actual frame rate of the second video based on the frame count ratio and the original frame rate of the first video.

[0049] Specifically, the actual frame rate of the second video is divided by the original frame rate of the first video, and the result is determined as the frame rate ratio. This frame rate ratio is then multiplied by the original frame rate of the first video, and the result is determined as the actual frame rate of the second video. By using the second video with duplicate frames removed, the true frame rate of the first video during playback can be accurately determined, thus improving the accuracy of video smoothness assessment.

[0050] S140. Input the target spatiotemporal feature information into the target evaluation network model to perform smoothness evaluation and obtain the smoothness evaluation result corresponding to the first video.

[0051] The target evaluation network model is a neural network model used to evaluate the subjective smoothness of a video. This model can be any type of regression network model, such as Support Vector Regression (SVR), decision trees, random forests, or multilayer perceptrons (MLP). The smoothness evaluation result can be represented by an evaluation score. A higher score indicates higher video smoothness. The target evaluation network model is obtained through supervised training based on sample videos and their corresponding smoothness evaluation score labels. Sample videos can be selected from an online video database based on criteria provided by the business, such as video publishing scenarios and video views. Data augmentation can also be applied to the selected videos to obtain sample videos with multiple scenarios, qualities, and frame rates, increasing the diversity and richness of the sample videos and thus improving the model training effect. The smoothness evaluation score label can be the average score obtained from multiple users rating the same sample video.

[0052] Specifically, the target spatiotemporal domain feature information is input into the pre-trained target evaluation network model. Based on the more accurate target spatiotemporal domain feature information, the target evaluation network model performs a more accurate smoothness evaluation of the first video and outputs the smoothness evaluation result, thereby obtaining the smoothness evaluation result corresponding to the first video more accurately.

[0053] The technical solution of this disclosure involves identifying and removing duplicate frames from a first video to be evaluated, resulting in a second video after removing duplicate frames. Target spatiotemporal features are extracted from the second video to determine its corresponding target spatiotemporal feature information. This information is then input into a target evaluation network model for smoothness evaluation, automatically obtaining the smoothness evaluation result for the first video. This achieves automatic video smoothness evaluation without manual intervention, improving the efficiency of video smoothness evaluation. By extracting video features related to video smoothness from the second video after removing duplicate frames, more accurate target spatiotemporal feature information can be obtained. This allows the target evaluation network model to more accurately evaluate the smoothness of the first video based on the target spatiotemporal feature information, thereby improving the accuracy of video smoothness evaluation.

[0054] Based on the above technical solution, after step S140, the method may further include: analyzing the smoothness evaluation result corresponding to the first video based on the target feature group and target spatiotemporal feature information that have an interactive relationship in the target spatiotemporal domain features, and determining the cause of the smoothness evaluation result; wherein, the first feature and the second feature in the target feature group have a mutually promoting relationship or a mutually inhibiting relationship. The mutually promoting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the smoothness also increases, and the mutually inhibiting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the smoothness decreases.

[0055] In this context, the target feature set refers to the set of first and second features that have an interactive relationship with smoothness. This interaction can be mutually reinforcing or mutually inhibiting. Both the first and second features are one or more features in the target spatiotemporal domain. For example, temporal complexity information and optical flow vectors (equivalent to the first feature) have a mutually reinforcing relationship with frame rate (equivalent to the second feature). That is, the larger the temporal complexity information and optical flow vector, the steeper the slope of the fitting curve between frame rate and smoothness, thus increasing the frame rate leads to a greater improvement in smoothness. The number of target feature sets can be one or more; that is, one or more target feature sets with different interactive relationships can exist.

[0056] Specifically, each feature combination in all target spatiotemporal features can be traversed beforehand to obtain multiple candidate feature groups. The interaction between the first and second features in each candidate feature group and fluency can be detected. For example, for each candidate feature group, the remaining features can be fixed, and the value of the first feature can be gradually increased. For each value of the first feature, a fitting curve between the second feature and fluency can be plotted, and the slope of the curve can be calculated. This yields the slope of the fitting curve between the second feature and fluency for each value of the first feature. If the slope remains constant as the first feature increases, there is no interaction between the first and second features. If the slope increases as the first feature increases, the candidate feature group is identified as the target feature group, and a mutually reinforcing relationship exists between the first and second features. If the slope decreases as the first feature increases, the candidate feature group is identified as the target feature group, and a mutually inhibiting relationship exists between the first and second features. By utilizing the changes in the curve slope, all target feature groups that interact with fluency can be obtained. After obtaining the fluency evaluation result for the first video, the interaction between target feature groups and the real spatiotemporal features of the first video can be used to interpret and analyze the fluency evaluation result. This allows us to identify the reasons for high or low fluency scores, determining which video features cause the high or low scores, thus achieving interpretability of the fluency evaluation results and overcoming the inability of network model predictions to explain them. For example, when feature b has a mutually reinforcing relationship with feature a, an increase in feature a will significantly improve fluency. In this case, a lower fluency score will also result in a lower value for feature a. Therefore, the interaction between features b and ab can be used to explain why the fluency score is low or high, facilitating analysis and attribution.

[0057] Figure 2 This is a flowchart illustrating another video smoothness evaluation method provided by an embodiment of this disclosure. Based on the embodiments disclosed above, this disclosure provides a detailed description of the duplicate frame identification and removal process. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0058] like Figure 2 As shown, the video smoothness evaluation method specifically includes the following steps:

[0059] S210, Obtain the first video to be evaluated.

[0060] S220. Perform differential processing on each two adjacent frames in the first video to determine the first difference value sequence corresponding to the first video.

[0061] In this sequence, each difference value in the first difference value sequence represents the difference between any two adjacent frames in the first video. The smaller the difference value, the smaller the difference between the two adjacent frames, and the more similar they are. The sequence length of the first difference value sequence is one less than the sequence length of the first video. For example, if the first video has n frames, meaning the sequence length of the first video is n, the sequence length of the first difference value sequence is n-1.

[0062] Specifically, each pair of adjacent frames in the first video (e.g., the current frame and the previous frame) can be differentially processed to obtain a difference image between the current frame and the previous frame. The difference image (i.e., the difference matrix) is then averaged at the pixel level, and the resulting average difference is used as the final difference value between the current frame and the previous frame. The difference values ​​corresponding to each pair of adjacent frames in the first video are arranged according to frame order to obtain the first difference value sequence.

[0063] S230. Binarize the first difference value sequence to obtain the processed second difference value sequence.

[0064] Specifically, each difference value in the first difference value sequence is compared with a preset difference value. Each difference value less than the preset difference value is replaced with a first value, such as 0, and each difference value greater than or equal to the preset difference value is replaced with a second value, such as 1. This completes the binarization of the first difference value sequence, resulting in a second difference value sequence containing only the first and second values.

[0065] S240. Perform a repeating subsequence search on the second difference value sequence to determine the repeating ratio of the target subsequence corresponding to the second difference value sequence.

[0066] Here, the repeating subsequence refers to the subsequence that appears cyclically in the second difference sequence. The target subsequence is the subsequence that appears most frequently in the second difference sequence. The target subsequence repetition ratio is used to characterize the repetition probability of the target subsequence, that is, the probability that there are repeated frames in the target subsequence.

[0067] Specifically, since frame interpolation involves periodically inserting frames into the video, such as inserting one frame at a time, the presence of duplicate frames in the first video due to frame interpolation can be determined by searching for repeating subsequences. The second difference value sequence can be traversed and searched for subsequences, and the repetition frequency of each subsequence within the second difference value sequence can be detected to determine the target subsequence with the highest repetition frequency and its repetition ratio within the second difference value sequence.

[0068] For example, step S240 may include: performing a repeating subsequence search on the second difference value sequence to determine the target subsequence with the most repetitions; determining the target repetition length corresponding to the target subsequence based on the length of the target subsequence and the target repetition count corresponding to the target subsequence; and determining the repetition ratio of the target subsequence corresponding to the second difference value sequence based on the target repetition length and the length of the second difference value sequence.

[0069] Specifically, the second difference value sequence can be traversed and searched for repeating subsequences by gradually changing the size of the sliding window. For example, the sliding window size is equal to the subsequence length. At each sliding window size, the subsequence template within the window is used for repetition matching, with the sliding window size as the step size. If a match is successful, the repetition count corresponding to that subsequence template is incremented by 1. This process is repeated until the sliding window matches the end of the sequence, thus obtaining the repetition count for each subsequence in the second difference value sequence. The subsequence with the highest repetition count is determined as the target subsequence. The length of the target subsequence is multiplied by the target repetition count, and the result is determined as the target repetition length of the target subsequence. The target repetition length is divided by the length of the second difference value sequence, and the result is determined as the repetition ratio of the target subsequence. For example, if the second difference value sequence is 0101010111, then the target subsequence with the highest repetition count is 01, and the repetition ratio of the target subsequence is 0.8.

[0070] S250, In response to the target subsequence repetition ratio being greater than or equal to a preset repetition ratio, based on the second difference value sequence, determine and remove the repetitive frames in the first video to obtain the second video after removing the repetitive frames.

[0071] Specifically, if the repetition ratio of the target subsequence is greater than or equal to a preset repetition ratio, it indicates that there are indeed duplicate frames in the first video. In this case, the position information of the duplicate frames in the first video can be determined based on the position information of the first value in the second difference value sequence, and the duplicate frames at that position in the first video frame can be deleted to obtain the second video after removing the duplicate frames. If the repetition ratio of the target subsequence is less than the preset repetition ratio, it indicates that there are no duplicate frames in the first video. In this case, there is no need to remove duplicate frames, and the first video can be directly used as the second video.

[0072] For example, step S250, "based on the second difference value sequence, determine and remove duplicate frames in the first video to obtain a second video after removing duplicate frames", may include: determining the target video frame corresponding to the first value in the second difference value sequence, and determining the target video frame as a duplicate frame, wherein the difference value between the target video frame and its adjacent video frames is less than a preset difference value; removing the target video frame in the first video to obtain a second video after removing duplicate frames.

[0073] Specifically, each first value in the second difference value sequence corresponds to two adjacent video frames in the first video, such as the current frame and the previous frame. Since these two video frames are exactly the same, either of them can be used as the target video frame for duplication. By deleting all target video frames in the first video, the second video after removing duplicate frames is obtained.

[0074] S260. Extract the target spatiotemporal domain features from the second video to determine the target spatiotemporal domain feature information corresponding to the second video.

[0075] S270. Input the target spatiotemporal feature information into the target evaluation network model to perform smoothness evaluation and obtain the smoothness evaluation result corresponding to the first video.

[0076] The technical solution of this disclosure embodiment determines a first difference value sequence corresponding to the first video by performing differential processing on every two adjacent frames in the first video, and binarizing the first difference value sequence to obtain a processed second difference value sequence; a repeating subsequence search is performed on the second difference value sequence to determine the target subsequence repeating ratio corresponding to the second difference value sequence; in response to the target subsequence repeating ratio being greater than or equal to a preset repeating ratio, repeating frames in the first video can be more accurately determined and removed based on the second difference value sequence to obtain a more accurate second video, thereby further improving the accuracy of video smoothness evaluation.

[0077] Figure 3 This is a flowchart illustrating another video smoothness evaluation method provided in this disclosure. Based on the above-disclosed embodiments, this disclosure describes in detail the process of adjusting the recording frame rate or encoding strategy based on the smoothness evaluation results. Explanations of terms that are the same as or corresponding to those in the above-disclosed embodiments are not repeated here.

[0078] like Figure 3 As shown, the video smoothness evaluation method specifically includes the following steps:

[0079] S310, Obtain the first video to be evaluated.

[0080] S320. Perform duplicate frame identification and removal on the first video to obtain the second video after removing duplicate frames.

[0081] S330. Extract the target spatiotemporal domain features from the second video to determine the target spatiotemporal domain feature information corresponding to the second video.

[0082] S340. Input the spatiotemporal feature information of the target into the target evaluation network model to perform smoothness evaluation and obtain the smoothness evaluation result corresponding to the first video.

[0083] S350, In response to the fact that the evaluation score in the smoothness evaluation result corresponding to the first video is less than the preset score, adjust the recording frame rate or the encoding strategy corresponding to the first video.

[0084] The recording frame rate refers to the frame rate of the video recorded using a camera. The recording frame rate corresponding to the first video refers to the actual frame rate of the video before encoding. Encoding strategy refers to the encoding method that affects the video frame rate during video encoding or transcoding, such as frame extraction or frame insertion strategies. Frame extraction strategies include the number of frames to be extracted and the extraction position. Frame insertion strategies include the number of frames to be inserted and the insertion position, such as inserting one frame every one frame interval, or inserting one frame every two frames interval, etc. It should be noted that frame extraction strategies do not introduce duplicate frames, thus the original frame rate of the first video is equal to the actual frame rate.

[0085] Specifically, if the smoothness evaluation score of the first video is lower than the preset score, indicating poor smoothness, the smoothness of the first video playback can be improved by increasing the recording frame rate or adjusting the encoding strategy. For example, if the actual frame rate of the first video is lower than the original frame rate, but equal to or approximately equal to the recording frame rate, it indicates poor smoothness due to poor frame interpolation. In this case, the original frame rate can be increased by increasing the recording frame rate or adjusting the frame interpolation strategy in the encoding strategy, thereby improving smoothness. For example, adjusting the frame interpolation strategy could mean reducing the number of inserted duplicate frames and increasing the number of inserted non-duplicate frames, such as using deep learning methods to construct a video temporal model, reconstructing intermediate frames between adjacent frames, and then inserting them. If the actual frame rate of the first video is equal to the original frame rate, but the original frame rate is lower than the recording frame rate, it indicates poor smoothness due to poor frame extraction. In this case, the smoothness can be improved by increasing the recording frame rate or adjusting the frame extraction strategy in the encoding strategy, such as reducing the number of extracted frames. If the original frame rate of the first video is equal to or approximately equal to the recording frequency, or if the recording frequency, original frame rate, and actual frame rate are all equal to or approximately equal, it indicates that the poor smoothness is simply due to insufficient video frame rate or excessive video motion complexity. In this case, the smoothness of the first video can be improved by increasing the recording frame rate of the first video.

[0086] For example, the step S350 of "adjusting the recording frame rate or the encoding strategy corresponding to the first video" may include: determining the target feature in the target spatiotemporal domain features that has a mutually reinforcing relationship with the frame rate, and obtaining the target feature value corresponding to the target feature from the target spatiotemporal domain feature information; in response to the target feature value being greater than or equal to a preset feature value, increasing the recording frame rate or adjusting the encoding strategy corresponding to the first video.

[0087] Specifically, target features with a mutually reinforcing relationship with video frame rate are identified from the target spatiotemporal features. In other words, as the target feature increases, the slope of the fitting curve between frame rate and smoothness also increases, meaning the improvement in video smoothness from frame rate is more significant. Here, frame rate refers to the actual frame rate of the video (i.e., the true frame rate). Since different scenes, characters, or frames in a video can lead to different target feature values, and the larger the target feature value, the more significant the improvement in smoothness from increasing the frame rate. Therefore, when the target feature value is greater than or equal to a preset feature value, the actual frame rate of the video can be increased by increasing the recording frame rate corresponding to the first video or adjusting the encoding strategy corresponding to the first video, thereby improving video smoothness more quickly. It should be noted that when the target feature value is less than the preset feature value, increasing the recording frame rate corresponding to the first video or adjusting the encoding strategy corresponding to the first video does not significantly improve smoothness. In this case, it is unnecessary to improve smoothness by adjusting the video frame rate to avoid excessively increasing video transmission costs.

[0088] The technical solution of this disclosure improves the smoothness of the first video by adjusting the recording frame rate or the encoding strategy of the first video when the smoothness evaluation score of the first video is less than a preset score, thereby enhancing the user's viewing experience.

[0089] Figure 4 This is a schematic diagram of the structure of a video smoothness evaluation device provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the device specifically includes: a first video acquisition module 410, a duplicate frame removal module 420, a spatiotemporal feature extraction module 430, and a smoothness evaluation module 440.

[0090] The system includes a first video acquisition module 410 for acquiring a first video to be evaluated; a duplicate frame removal module 420 for identifying and removing duplicate frames from the first video to obtain a second video after removing duplicate frames; a spatiotemporal feature extraction module 430 for extracting target spatiotemporal features from the second video to determine the target spatiotemporal feature information corresponding to the second video, wherein the target spatiotemporal features are pre-determined video features related to video smoothness; and a smoothness evaluation module 440 for inputting the target spatiotemporal feature information into a target evaluation network model for smoothness evaluation to obtain the smoothness evaluation result corresponding to the first video.

[0091] The technical solution provided in this disclosure obtains a second video after removing duplicate frames by identifying and removing duplicate frames from a first video to be evaluated. It then extracts target spatiotemporal features from the second video to determine the target spatiotemporal feature information corresponding to the second video. This target spatiotemporal feature information is input into a target evaluation network model for smoothness evaluation, automatically obtaining the smoothness evaluation result for the first video. This achieves automatic video smoothness evaluation without manual intervention, improving the efficiency of video smoothness evaluation. By extracting video features related to video smoothness from the second video after removing duplicate frames, more accurate target spatiotemporal feature information can be obtained. This allows the target evaluation network model to more accurately evaluate the smoothness of the first video based on the target spatiotemporal feature information, thereby improving the accuracy of video smoothness evaluation.

[0092] Based on the above technical solution, the duplicate frame removal module 420 includes:

[0093] The first difference value sequence determination unit is used to perform difference processing on each two adjacent frames in the first video to determine the first difference value sequence corresponding to the first video.

[0094] The second difference value sequence determination unit is used to perform binarization processing on the first difference value sequence to obtain the processed second difference value sequence.

[0095] The target subsequence repetition ratio determination unit is used to perform a repetition subsequence search on the second difference value sequence and determine the target subsequence repetition ratio corresponding to the second difference value sequence;

[0096] The second video determination unit is used to determine and remove duplicate frames in the first video based on the second difference value sequence in response to the target subsequence repetition ratio being greater than or equal to a preset repetition ratio, thereby obtaining a second video after removing duplicate frames.

[0097] Based on the above technical solutions, the target subsequence repetition ratio determination unit is specifically used for:

[0098] A repeating subsequence search is performed on the second difference value sequence to determine the target subsequence with the most repetitions; based on the length of the target subsequence and the target repetition count corresponding to the target subsequence, the target repetition length corresponding to the target subsequence is determined; based on the target repetition length and the length of the second difference value sequence, the repetition ratio of the target subsequence corresponding to the second difference value sequence is determined.

[0099] Based on the above technical solutions, the second video determination unit is specifically used for:

[0100] The target video frame corresponding to the first value in the second difference value sequence is determined, and the target video frame is identified as a duplicate frame, wherein the difference between the target video frame and its adjacent video frames is less than a preset difference value; the target video frame in the first video is removed to obtain the second video after removing duplicate frames.

[0101] Based on the above technical solutions, the target spatiotemporal domain feature information includes: the actual frame rate of the second video and target feature information, wherein the target feature information includes at least one of the following:

[0102] The second video contains information on time complexity, space complexity, Laplacian operator, optical flow vector, and texture.

[0103] Based on the above technical solutions, the spatiotemporal feature extraction module 430 includes:

[0104] The actual frame rate determination unit is used to determine the actual frame rate of the second video based on the original frame rate and original frame number of the first video and the actual frame number of the second video.

[0105] The target feature information determination unit is used to extract target features from the second video and determine the target feature information of the second video.

[0106] Based on the above technical solutions, the actual frame rate determination unit is specifically used for:

[0107] The frame rate ratio is determined based on the original frame rate of the first video and the actual frame rate of the second video; the actual frame rate of the second video is determined based on the frame rate ratio and the original frame rate of the first video.

[0108] Based on the above technical solutions, the device also includes:

[0109] The evaluation result analysis module is used to analyze the smoothness evaluation result corresponding to the first video based on the target feature group with interactive relationship in the target spatiotemporal domain features and the target spatiotemporal domain feature information, and to determine the cause of the smoothness evaluation result.

[0110] In the target feature group, the first feature and the second feature have a mutually promoting relationship or a mutually inhibiting relationship. The mutually promoting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the fluency also increases. The mutually inhibiting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the fluency decreases.

[0111] Based on the above technical solutions, the device also includes:

[0112] The frame rate adjustment module is used to adjust the recording frame rate of the first video or the encoding strategy of the first video in response to the evaluation score in the smoothness evaluation result of the first video being less than a preset score.

[0113] Based on the above technical solutions, the frame rate adjustment module is specifically used for:

[0114] Identify target features in the target spatiotemporal domain features that have a mutually reinforcing relationship with the frame rate, and obtain the target feature value corresponding to the target feature from the target spatiotemporal domain feature information; in response to the target feature value being greater than or equal to a preset feature value, increase the recording frame rate corresponding to the first video or adjust the encoding strategy corresponding to the first video.

[0115] The video smoothness evaluation device provided in this disclosure can execute the video smoothness evaluation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the video smoothness evaluation method.

[0116] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0117] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0118] like Figure 5As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0119] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0120] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0121] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0122] The electronic device provided in this embodiment and the video smoothness evaluation method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0123] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the video smoothness evaluation method provided in the above embodiments.

[0124] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0125] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0126] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0127] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a first video to be evaluated; identify and remove duplicate frames from the first video to obtain a second video after removing duplicate frames; extract target spatiotemporal features from the second video to determine target spatiotemporal feature information corresponding to the second video, wherein the target spatiotemporal features are pre-determined video features related to video smoothness; and input the target spatiotemporal feature information into a target evaluation network model for smoothness evaluation to obtain a smoothness evaluation result corresponding to the first video.

[0128] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the video smoothness evaluation method provided in the above embodiments.

[0130] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0133] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0134] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0136] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0137] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for evaluating video smoothness, characterized in that, include: Obtain the first video to be evaluated; The first video is subjected to duplicate frame identification and removal to obtain a second video after duplicate frames are removed; The target spatiotemporal domain features are extracted from the second video to determine the target spatiotemporal domain feature information corresponding to the second video, wherein the target spatiotemporal domain features are pre-determined video features that are related to the smoothness of the video; The target spatiotemporal feature information is input into the target evaluation network model for smoothness evaluation, and the smoothness evaluation result corresponding to the first video is obtained.

2. The video smoothness evaluation method according to claim 1, characterized in that, The step of identifying and removing duplicate frames from the first video to obtain a second video after removing duplicate frames includes: Perform differential processing on each two adjacent frames in the first video to determine the first difference value sequence corresponding to the first video; The first difference value sequence is binarized to obtain the processed second difference value sequence; A repeating subsequence search is performed on the second difference value sequence to determine the repeating ratio of the target subsequence corresponding to the second difference value sequence; In response to the target subsequence repetition ratio being greater than or equal to a preset repetition ratio, based on the second difference value sequence, duplicate frames in the first video are determined and removed to obtain a second video after removing duplicate frames.

3. The video smoothness evaluation method according to claim 2, characterized in that, The step of searching for repeating subsequences in the second difference value sequence to determine the repeating ratio of the target subsequence corresponding to the second difference value sequence includes: Perform a repeating subsequence search on the second difference value sequence to determine the target subsequence with the highest repetition frequency; The target repetition length of the target subsequence is determined based on the length of the target subsequence and the target repetition count corresponding to the target subsequence; Based on the target repetition length and the length of the second difference value sequence, the target subsequence repetition ratio corresponding to the second difference value sequence is determined.

4. The video smoothness evaluation method according to claim 2, characterized in that, The step of determining and removing duplicate frames from the first video based on the second difference value sequence to obtain a second video after removing duplicate frames includes: The target video frame corresponding to the first value in the second difference value sequence is determined, and the target video frame is determined as a repeating frame, wherein the difference value between the target video frame and its adjacent video frames is less than a preset difference value; The target video frame is removed from the first video to obtain a second video after removing duplicate frames.

5. The video smoothness evaluation method according to claim 1, characterized in that, The target spatiotemporal domain feature information includes: the actual frame rate of the second video and target feature information, wherein the target feature information includes at least one of the following: The second video contains information on time complexity, space complexity, Laplacian operator, optical flow vector, and texture.

6. The video smoothness evaluation method according to claim 5, characterized in that, The step of extracting target spatiotemporal features from the second video and determining the target spatiotemporal feature information corresponding to the second video includes: The actual frame rate of the second video is determined based on the original frame rate and original frame count of the first video and the actual frame count of the second video. The target features of the second video are extracted to determine the target feature information of the second video.

7. The video smoothness evaluation method according to claim 6, characterized in that, Determining the actual frame rate of the second video based on the original frame rate and original frame count of the first video and the actual frame count of the second video includes: The frame rate ratio is determined based on the original frame rate of the first video and the actual frame rate of the second video. The actual frame rate of the second video is determined based on the frame rate ratio and the original frame rate of the first video.

8. The video smoothness evaluation method according to claim 1, characterized in that, After obtaining the smoothness evaluation result corresponding to the first video, the process also includes: Based on the target feature groups that have interactive relationships in the target spatiotemporal domain features and the target spatiotemporal domain feature information, the smoothness evaluation result corresponding to the first video is analyzed to determine the cause of the smoothness evaluation result; In the target feature group, the first feature and the second feature have a mutually promoting relationship or a mutually inhibiting relationship. The mutually promoting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the fluency also increases. The mutually inhibiting relationship means that as the first feature increases, the slope of the fitting curve between the second feature and the fluency decreases.

9. The video smoothness evaluation method according to any one of claims 1-8, characterized in that, After obtaining the smoothness evaluation result corresponding to the first video, the process also includes: If the evaluation score in the smoothness evaluation result of the first video is less than a preset score, the recording frame rate of the first video or the encoding strategy of the first video is adjusted.

10. The video smoothness evaluation method according to claim 9, characterized in that, Adjusting the recording frame rate or encoding strategy corresponding to the first video includes: Identify target features in the target spatiotemporal domain features that have a mutually reinforcing relationship with the frame rate, and obtain the target feature values ​​corresponding to the target features from the target spatiotemporal domain feature information; In response to the target feature value being greater than or equal to a preset feature value, the recording frame rate of the first video is increased or the encoding strategy of the first video is adjusted.

11. A video fluency evaluation device, characterized in that, include: The first video acquisition module is used to acquire the first video to be evaluated. The duplicate frame removal module is used to identify and remove duplicate frames from the first video to obtain a second video after removing duplicate frames. The spatiotemporal domain feature extraction module is used to extract target spatiotemporal domain features from the second video and determine the target spatiotemporal domain feature information corresponding to the second video, wherein the target spatiotemporal domain features are pre-determined video features that are related to video smoothness; The smoothness evaluation module is used to input the target spatiotemporal feature information into the target evaluation network model to perform smoothness evaluation and obtain the smoothness evaluation result corresponding to the first video.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the video smoothness evaluation method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the video smoothness evaluation method as described in any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the video smoothness evaluation method as described in any one of claims 1-10.

Images