Video fluency detection method, detection device, electronic equipment and storage medium

Abstract

The application discloses a video fluency detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring M photos of a playing video, the M photos corresponding to N video frames of the playing video, and one video frame corresponding to at least one photo; determining a picture similarity according to every two adjacent photos in the M photos to obtain M-1 picture similarities; determining a frame playing time length of each video frame in at least part of the N video frames according to the M-1 picture similarities; and determining the fluency of the playing video according to the frame playing time length of each video frame in the at least part of the video frames. The method realizes the calculation of the fluency of the playing video based on the frame playing time length of each video frame in at least part of the video frames of the playing video, and uses a specific numerical value to represent the subjective feeling of a user about the lag of the playing video, so that the evaluation of the video fluency is more objective and effective, and the detection accuracy of the detection of the video fluency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video detection, and in particular to a video fluency detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] When a user watches a video, the user may sometimes feel that the picture or object movement is obviously stuttered. This is because the video itself has irregular frame sequences, the video frame rate does not match the screen refresh rate or GPU rendering is abnormal, frame dropping, etc. will cause the video frame playing time to be uneven, resulting in changes in the speed of picture or object movement, and thus there will be obvious stuttering in vision. Stuttering will affect the user experience, and even make the user lose interest in the video associated product.

[0003] Fluency can be used as a multi-faceted evaluation index. For example, fluency can be used as an important index for evaluating video quality, fluency can also be used as an important index for evaluating the network status of video playback, and fluency can also be used as an important index for measuring the performance of an electronic device playing a video.

[0004] At present, there is a lack of effective fluency detection methods, and electronic device manufacturers mainly test the fluency of the electronic device playing a video in a manual feeling manner. However, because each person has different feelings about the fluency of the electronic device playing a video, the detection accuracy of detecting the fluency of the video is low. SUMMARY

[0005] In view of this, the embodiments of the present application provide a video fluency detection method and device, electronic equipment and a storage medium to overcome the above problems of the prior art.

[0006] In a first aspect, the embodiments of the present application provide a video fluency detection method, which comprises: obtaining M photos of a playing video, the M photos corresponding to N video frames of the playing video, and one video frame corresponding to at least one photo; determining a picture similarity between every two adjacent photos in the M photos to obtain M-1 picture similarities; determining a frame playing duration of each video frame in at least part of the N video frames according to the M-1 picture similarities; and determining the fluency of the playing video according to the frame playing duration of each video frame in the at least part of the video frames.

[0007] The scheme provided in the application, in the detection process of detecting the video fluency, based on the picture similarity of each two adjacent pictures in the M pictures, determines the frame playing time length of each video frame in at least part of the N video frames of the playing video, and calculates the fluency of the playing video based on the frame playing time length of each video frame, which uses specific numerical values to represent the subjective feeling of the user to the playing video lag, so that the evaluation of the video fluency is more objective and effective, and the detection accuracy of detecting the video fluency is improved.

[0008] According to the frame playing time length of the video frame, the fluency of the playing video is calculated, and the method of uniform quantization of fluency is suitable for the lag feeling caused by various reasons, such as the lag feeling caused by the mismatch of video frame rate and screen refresh rate, the lag feeling caused by GPU rendering exception, the lag feeling caused by frame drop, etc., which expands the application scene of detecting the video fluency.

[0009] The picture similarity is low when the video frame is switched, and according to the low picture similarity, the switching time length of the video frame can be accurately calculated to obtain the frame playing time length of the video frame, and the calculation accuracy of calculating the frame playing time length of the video frame is improved.

[0010] In the above step of determining the frame playing time length of each video frame in at least part of the N video frames according to M-1 picture similarities, the frame playing time length of each video frame in at least part of the N video frames is determined according to the corresponding relationship between the M-1 picture similarities and the M-1 time points in the playing time length of the playing video.

[0011] The scheme provided in the embodiment, the picture similarity is low when the video frame is switched, and according to the corresponding relationship between the picture similarity and the playing time point of the playing video, the switching time length of the video frame can be accurately calculated to obtain the frame playing time length of each video frame in at least part of the video frames, and the calculation accuracy of calculating the frame playing time length of each video frame in at least part of the video frames is improved.

[0012] In the above step of determining the frame playing time length of each video frame in at least part of the N video frames according to the corresponding relationship between the M-1 picture similarities and the M-1 time points in the playing time length of the playing video, the frame playing time length of each video frame in at least part of the N video frames is determined according to the corresponding relationship between the M-1 picture similarities and the M-1 time points in the playing time length of the playing video.

[0013] The scheme provided by the embodiment corresponds the picture similarity minimum value to the video frame switching moment of the played video, and calculates the frame playing time length of each video frame in the at least part of video frames based on the time corresponding to the picture similarity minimum value, thereby improving the calculation accuracy of the frame playing time length of each video frame in the at least part of video frames.

[0014] In the step of determining the fluency of the played video according to the frame playing time length of each video frame in the at least part of video frames, the frame playing time length of each video frame in the at least part of video frames is used to determine the stuttering rate of the played video; and the fluency of the played video is determined according to the stuttering rate.

[0015] In the step of determining the stuttering rate of the played video according to the frame playing time length of each video frame in the at least part of video frames, the frame playing time length of each video frame in the at least part of video frames is used to determine the reference frame playing time length; the stuttering time length of the played video is determined according to the reference frame playing time length and the frame playing time length of each video frame in the at least part of video frames; and the stuttering rate of the played video is determined according to the total playing time length of the N video frames and the stuttering time length.

[0016] In an example, the step of determining the reference frame playing time length according to the frame playing time length of each video frame in the at least part of video frames includes: determining the frame playing time length with the most repetitions in the frame playing time length of each video frame in the at least part of video frames as the reference frame playing time length.

[0017] For example, the step of determining the stuttering time length of the played video according to the reference frame playing time length and the frame playing time length of each video frame in the at least part of video frames includes: summing the absolute values of the time length differences between each non-reference frame playing time length and the reference frame playing time length to obtain the stuttering time length of the played video.

[0018] In another example, the step of determining the reference frame playing time length according to the frame playing time length of each video frame in the at least part of video frames includes: calculating the average frame playing time length according to the frame playing time length of each video frame in the at least part of video frames; and determining the average frame playing time length as the reference frame playing time length.

[0019] In the step of determining the fluency of the played video according to the stuttering rate, the percentage of 1 minus the stuttering rate is calculated to obtain the fluency of the played video.

[0020] In the step of determining one picture similarity between any two adjacent photos in the M photos, the first hash value of one photo and the second hash value of another photo in the any two adjacent photos are calculated; the Hamming distance of the any two adjacent photos is calculated according to the first hash value and the second hash value; and the picture similarity of the any two adjacent photos is calculated according to the Hamming distance.

[0021] The scheme provided in the embodiment has the characteristics of high calculation efficiency and high robustness to image transformation, and the picture similarity of the photo is calculated based on the hash algorithm, thereby improving the calculation efficiency and calculation robustness of the calculation of the picture similarity of the photo.

[0022] In the step of obtaining the M photos of the playing video, a collection instruction is sent to the high-speed camera, so that the high-speed camera photographs the playing picture of the playing video according to the collection instruction to obtain the M photos.

[0023] The scheme provided in the embodiment has the characteristics of high calculation efficiency and high robustness to image transformation, and the picture similarity of the photo is calculated based on the hash algorithm, thereby improving the calculation efficiency and calculation robustness of the calculation of the picture similarity of the photo.

[0024] In the step of obtaining the M photos of the playing video, a collection instruction is sent to the high-speed camera, so that the high-speed camera photographs the playing picture of the playing video according to the collection instruction to obtain the M photos.

[0025] In the step of obtaining the M photos of the playing video, a collection instruction is sent to the high-speed camera, so that the high-speed camera photographs the playing picture of the playing video according to the collection instruction to obtain the M photos.

[0026] In the step of obtaining the M photos of the playing video, a collection instruction is sent to the high-speed camera, so that the high-speed camera photographs the playing picture of the playing video according to the collection instruction to obtain the M photos.

[0027] In some optional embodiments, the chip system further includes a memory, and the memory is connected to the one or more processors through a circuit or a wire.

[0028] In some optional embodiments, the chip system further includes a communication interface.

[0029] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which includes instructions, when the instructions are executed on an electronic device, cause the electronic device to perform the method for detecting video smoothness according to the first aspect.

[0030] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, causes the electronic device to perform the method for detecting video smoothness according to the first aspect.

[0031] It can be understood that the beneficial effects of the second aspect to the sixth aspect can be referred to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0033] Figure 1 A structural schematic diagram of a software system of an electronic device provided by an embodiment of the present application is shown.

[0034] Figure 2 A flow schematic diagram of the method for detecting video smoothness provided by an embodiment of the present application is shown.

[0035] Figure 3 A scene schematic diagram of photographing by a high-speed camera in the method for detecting video smoothness provided by an embodiment of the present application is shown.

[0036] Figure 4 A scene schematic diagram of the video smoothness detection system provided by an embodiment of the present application is shown.

[0037] Figure 5 A scene schematic diagram of a time-picture similarity curve in the method for detecting video smoothness provided by an embodiment of the present application is shown.

[0038] Figure 6 A flow schematic diagram of the method for detecting video smoothness provided by an embodiment of the present application in an application scenario is shown.

[0039] Figure 7 Another flow schematic diagram of the method for detecting video smoothness provided by an embodiment of the present application is shown.

[0040] Figure 8A structural schematic diagram of a video fluency detection device provided by an embodiment of the present application is shown.

[0041] Figure 9 A hardware structural schematic diagram of an electronic device provided by an embodiment of the present application is shown.

[0042] Figure 10 A functional block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which like or similar elements or components are denoted by like reference numbers throughout the various drawings. The embodiments described below are examples only, and are not intended to limit the present application.

[0044] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. For the purpose of simplicity, the elements and settings of the particular examples below are described in some detail. Of course, they are merely examples and are in no way intended to limit the present application. Furthermore, the present application can be implemented in a variety of different embodiments and of the same or in different settings, and the disclosure below does not limit the scope of the application to any particular embodiment or setting. In addition, the repeated use of reference numbers in the drawings and the description below is intended to simplify the discussion and is in no way intended to limit the scope of the application.

[0045] When a user watches a video, sometimes the user feels that the picture or object movement has a clear sense of stuttering. This is because the video itself has an irregular frame sequence, the video frame rate does not match the screen refresh rate or the GPU rendering is abnormal, frame dropping, etc. will cause the video frame playing time to be uneven, resulting in a change in the speed of picture or object movement, and there will be a clear sense of stuttering in vision. Stuttering will affect the user experience, and even make the user lose interest in the video associated product.

[0046] Fluency can be used as a multi-faceted evaluation indicator. For example, fluency can be used as an important indicator for evaluating video quality, fluency can be used as an important indicator for evaluating the network status of video playback, and fluency can be used as an important indicator for measuring the performance of an electronic device playing a video.

[0047] At present, there is a lack of effective fluency detection methods, and electronic device manufacturers mainly test the fluency of the electronic device playing a video through artificial feeling. However, because everyone has different feelings about the fluency of the electronic device playing a video, the detection accuracy of detecting the fluency of the video is low.

[0048] To solve the above problems, the video fluency detection method, the detection device, the electronic device and the storage medium provided in the embodiments of the present application are provided. The M photos of the playing video are obtained, the M photos correspond to N video frames of the playing video, one video frame corresponds to at least one photo, a picture similarity between every two adjacent photos in the M photos is determined to obtain M-1 picture similarities, the frame playing time of each video frame in at least part of the N video frames is determined according to the M-1 picture similarities, and the fluency of the playing video is determined according to the frame playing time of each video frame in at least part of the video frames. In the detection process of detecting the video fluency, the frame playing time of each video frame in at least part of the N video frames of the playing video is determined based on the picture similarity between every two adjacent photos in the M photos, and the fluency of the playing video is calculated based on the frame playing time of each video frame. The subjective feeling of the user about the playing video is expressed by a specific numerical value, so that the evaluation of the video fluency is more objective and effective, and the detection accuracy of the video fluency is improved.

[0049] The fluency of the playing video is calculated based on the frame playing time of the video frame, the method of quantifying the fluency is unified, and is suitable for the stall feeling caused by various reasons, such as the stall feeling caused by the mismatch between the video frame rate and the screen refresh rate, the stall feeling caused by GPU rendering exception, the stall feeling caused by frame dropping, etc. The application scene of detecting the video fluency is expanded.

[0050] The picture similarity is low when the video frame is switched, and the switching time of the video frame can be accurately calculated based on the low picture similarity to obtain the frame playing time of the video frame, so that the calculation accuracy of the frame playing time of the video frame is improved.

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application.

[0052] The video fluency detection method provided in the embodiments of the present application can be applied to an electronic device. The electronic device can include various terminal devices, and the terminal device can also be referred to as a terminal, a user equipment, a mobile station, a mobile terminal, etc.

[0053] The terminal device can be a mobile phone, a sweeping robot, a drone, a smart television, a wearable device, a personal digital assistant (PDA), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, and the like. The type of the terminal device is not limited herein and can be set according to actual needs.

[0054] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Embodiments of the present application take an Android system with a layered architecture as an example to illustrate the software structure of the electronic device.

[0055] Referring to Figure 1 , a structural diagram of a software system of an electronic device is shown. The software system includes several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the system library, and the kernel layer.

[0056] The application layer can include a series of applications, for example, the application layer can include a smoothness detection application, a camera application, a gallery application, a call application, a wireless local area network (WLAN) application, a video application, a media provider (MediaProvider) application, a filesystem in userspace (FUSE) application, and the like.

[0057] The smoothness detection application can be used to obtain M photos of a playing video and detect the smoothness of the playing video according to the M photos. The smoothness detection application can be separately set, or the smoothness detection application can be embedded in a video playing application (such as Youku, iQiyi, Douyin, etc.), and the like, which is not limited herein.

[0058] The media provider is used to create multimedia files in the FUSE or access multimedia files in the FUSE. Each application in the application layer can create multimedia files in the FUSE through the media provider MediaProvider or access multimedia files in the FUSE through the media provider MediaProvider.

[0059] The FUSE is used to store multimedia files created by the media provider. Of course, in other embodiments, the FUSE can also be used to store other data.

[0060] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions. For example, the application framework layer can include a window manager, a content provider, a resource manager, and a view system, etc.

[0061] The window manager is used to manage window programs. The window manager can obtain the size of a display screen, determine whether there is a status bar, lock a screen, and capture a screen, etc.

[0062] The content provider is used to store and obtain data, and make the data accessible to the applications. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, a phone book, etc.

[0063] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, etc. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0064] The resource manager provides various resources for the applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0065] The system library can include a surface manager, media libraries, an Android runtime, etc.

[0066] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library includes two parts: one part is a function function that the Java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java file of the application layer and the application framework layer into a binary file. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection and the like.

[0067] The surface manager is used to manage the display subsystem, and provides fusion of 2D and 3D layers for multiple applications.

[0068] The media library supports multiple commonly used audio, video format playback and recording, and static image files and the like. The media library can support multiple audio and video coding formats, for example, MPEG4, H.264, MP3, AAC, AMR, JPG, PNG and the like.

[0069] The kernel layer can include a camera driver, a display driver, a Wi-Fi driver, a Bluetooth driver, an audio driver and the like.

[0070] It can be understood that, Figure 1 The layers in the software structure shown and the components included in each layer do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device can include more or fewer layers than shown, and each layer can include more or fewer components, and the present application is not limited.

[0071] It should be noted that, although the embodiments of the present application are described taking the Android system as an example, the basic principles are also applicable to electronic devices based on Symbian ( ), Android ( ), Apple ( ), Blackberry ( ), Harmony and the like.

[0072] Please refer to Figure 2 , which shows a flowchart of a video smoothness detection method provided by an embodiment of the present application. In specific embodiments, the video smoothness detection method can be applied to an electronic device, or to a processor or chip in the electronic device, and the like. In the following, taking the electronic device as an example, the flow shown in Figure 2 will be described in detail, and the video smoothness detection method can include the following steps S110 to S140.

[0073] Step S110: The electronic device acquires M photos of the playing video.

[0074] In the embodiments of the present application, when the user needs to detect the video fluency, the detection instruction can be sent to the electronic device. The electronic device receives and responds to the detection instruction, and acquires M photos of the playing video.

[0075] Among them, M photos can correspond to N video frames of the playing video, one video frame corresponds to at least one photo, N is a positive integer greater than or equal to 1, and M is a positive integer greater than or equal to N.

[0076] Regarding the electronic device receiving and responding to the detection instruction, in some embodiments, the electronic device can be provided with an input panel. When the user needs to detect the video fluency, the detection instruction can be input on the input panel of the electronic device, for example, the detection instruction can be handwritten on the input panel of the electronic device, or the detection instruction can be input by pressing the keys on the input panel of the electronic device. The electronic device receives the detection instruction through the input panel.

[0077] In other embodiments, the electronic device can be provided with a voice recognition module. When the user needs to detect the video fluency, voice information can be sent within the voice collection range of the voice recognition module. The voice recognition module collects the voice information sent by the user, and performs voice recognition on the collected voice information to obtain a voice recognition result. When it is determined that the voice recognition result contains a keyword for indicating the detection of the video fluency, for example, the keyword is "video fluency detection", and for example, the keyword is "video fluency" and "detection", etc., it is determined that the detection instruction is received.

[0078] As an example, the voice information sent by the user is: detecting the video fluency of the playing video. Then, the voice recognition result of the voice recognition contains the keywords "video fluency" and "detection". It is determined that the detection instruction is received.

[0079] In the embodiments of the present application, after the electronic device receives and responds to the detection instruction, the electronic device can acquire M photos of the playing video in various ways.

[0080] In some embodiments, the electronic device can send a collection instruction to the high-speed camera. The high-speed camera receives and responds to the collection instruction, takes a picture of the playing picture of the playing video, obtains M photos, and returns the M photos to the electronic device. The electronic device receives the M photos returned by the high-speed camera. The M photos of the playing video are taken by the high-speed camera, which are closer to the continuous video frames of the playing video, which is conducive to improving the fluency of the playing video, and further conducive to improving the detection accuracy of detecting the video fluency.

[0081] The high-speed camera can be installed at a position of a playing screen of a player capable of playing a video, and the high-speed camera can be used to capture a playing screen of the player playing the video. Figure 3 As shown in FIG. 1, a schematic diagram of a scene of a photo captured by the high-speed camera is shown. The high-speed camera is connected to the electronic device through a network, and performs data interaction with the electronic device through the network.

[0082] The high-speed camera is a device capable of capturing a moving image with an exposure time less than 1 / 1000 second. The high-speed camera is mainly used to capture the moving track of a high-speed moving object, and helps to capture images and moving processes that cannot be seen by naked eyes. At present, the shooting speed of the high-speed camera can reach 1000-10000 photos / second. Since the shooting time of each photo captured by the high-speed camera is less than the playing time of a video frame in a normal video, one or more photos of the same video frame can be captured by the high-speed camera.

[0083] The player can be an electronic device or other electronic product with a video playing function, and the type of the player is not limited herein and can be set according to actual needs.

[0084] The network can be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN), or a Wireless Personal Area Network (WPAN), and the type of the network is not limited herein and can be set according to actual needs.

[0085] As an example, as shown in FIG. 2, a schematic diagram of a scene of a photo captured by the high-speed camera is shown. Figure 4As shown, the video fluency detection system can include an electronic device 100 and a high-speed camera 200, the electronic device 100 is connected to the high-speed camera 200 through a network and performs data interaction with the high-speed camera 200 through the network. The player is the electronic device 100, and the high-speed camera 200 can be installed at a position where the playing picture of the electronic device 100 is collected. The high-speed camera 200 can be used to capture the playing picture of the video played by the electronic device 100 to obtain M photos, and send the M photos to the electronic device 100 through the network.

[0086] In some other embodiments, the electronic device is configured with a camera, and the player is another electronic product with a video playing function. The electronic device can capture the playing picture of the video played by the player through the camera to obtain M photos.

[0087] Among them, the camera can be any one of a wide-angle camera, a macro camera, an ultra-wide-angle camera, or a panoramic camera, and the type of camera is not limited here and can be set according to actual needs.

[0088] Step S120: The electronic device determines a picture similarity according to each two adjacent photos in the M photos to obtain M-1 picture similarities.

[0089] In the embodiments of the present application, after the electronic device obtains the M photos of the played video, a picture similarity can be determined according to each two adjacent photos in the M photos to obtain M-1 picture similarities.

[0090] It can be understood that in the M photos, starting from the first photo, two adjacent photos as a group of photos can form M-1 groups of photos, and a picture similarity can be determined according to two adjacent photos in each group of photos, and then M-1 picture similarities can be obtained. For example, M=8, starting from the first photo, two adjacent photos as a group of photos can form 7 groups of photos, and a picture similarity can be determined according to two adjacent photos in each group of photos, and then 7 picture similarities can be obtained. The value of M is not limited here and can be set according to actual needs.

[0091] In the embodiments of the present application, the electronic device can determine M-1 picture similarities according to each two adjacent photos in the M photos in multiple ways.

[0092] In some embodiments, after the electronic device obtains the M photos of the playing video, the electronic device can calculate a first hash value of one photo and a second hash value of another photo in any two adjacent photos, calculate a Hamming distance between the any two adjacent photos according to the first hash value and the second hash value, and calculate a picture similarity between the any two adjacent photos according to the Hamming distance, to obtain M-1 picture similarities. The hash algorithm has the characteristics of high calculation efficiency and high robustness to image transformation. The picture similarity of the photos is calculated based on the hash algorithm, thereby improving the calculation efficiency and the calculation robustness of the picture similarity calculation of the photos.

[0093] The electronic device can calculate the first hash value of one photo and the second hash value of another photo in any two adjacent photos based on an Average Hash algorithm or a Perceptual Hash algorithm.

[0094] The Hamming distance is inversely proportional to the picture similarity. The picture similarity between the any two adjacent photos can be calculated by calculating the inverse ratio of the Hamming distance. The smaller the Hamming distance is, the higher the picture similarity is. The larger the Hamming distance is, the lower the picture similarity is.

[0095] The Average Hash algorithm refers to an algorithm of reducing a photo to a fixed size (for example, 8x8 pixels), then converting the reduced photo into a grayscale image, calculating an average grayscale value of the grayscale image, comparing the grayscale value of each pixel of the grayscale image with the average grayscale value, marking the pixels in the grayscale image that are larger than the average grayscale value as 1, marking the pixels in the grayscale image that are smaller than the average grayscale value as 0, obtaining a binary result, and combining the binary result into a hash value of a fixed length.

[0096] The Perceptual Hash algorithm refers to an algorithm of converting a photo into a grayscale image, adjusting the size of the grayscale image to a fixed size (for example, 32x32 pixels), applying Discrete Cosine Transform (DCT) to the adjusted grayscale image, retaining low-frequency components, and converting the adjusted grayscale image into a binary hash value according to the relative size of the DCT coefficients.

[0097] In some embodiments, after the electronic device obtains the M photos of the played video, the electronic device can extract a first feature vector of one photo and a second feature vector of another photo in any two adjacent photos through the convolutional neural network, calculate a cosine similarity of the any two adjacent photos according to the first feature vector and the second feature vector, and determine a frame similarity of the any two adjacent photos according to the cosine similarity, to obtain M-1 frame similarities. The cosine similarity algorithm has certain stability to noise and changes in data, and calculating the frame similarity of the photos based on the cosine similarity algorithm improves the calculation accuracy of the frame similarity of the photos.

[0098] The calculation formula of the cosine similarity is as follows: similarity = (A B) / (||A|| * ||B||); wherein, A represents the first feature vector, B represents the second feature vector, A B represents the dot product of the first feature vector and the second feature vector, ||A|| represents the norm (i.e. the length) of the first feature vector, ||B|| represents the norm (i.e. the length) of the second feature vector, and similarity represents the cosine similarity of the first feature vector and the second feature vector.

[0099] The value range of similarity is between -1 and 1, the closer the value of similarity is to 1, the more similar the first feature vector and the second feature vector are, the closer the value of similarity is to -1, the less similar the first feature vector and the second feature vector are, and the value of similarity close to 0 indicates that there is no obvious similarity or difference between the first feature vector and the second feature vector.

[0100] The feature vector is obtained by feature extraction of the photos based on the convolutional neural network, and the cosine similarity of the first feature vector and the second feature vector can be determined as the frame similarity of the corresponding any two adjacent photos.

[0101] In some embodiments, after the electronic device obtains the M photos of the played video, the electronic device can convert one photo in any two adjacent photos into a first grayscale image and the other photo into a second grayscale image, perform histogram calculation on the first grayscale image based on a histogram algorithm to obtain a first grayscale histogram, perform histogram calculation on the second grayscale image based on the histogram algorithm to obtain a second grayscale histogram, calculate the Euclidean distance of the any two adjacent photos according to the first grayscale histogram and the second grayscale histogram, and determine the frame similarity of the any two adjacent photos according to the Euclidean distance, to obtain M-1 frame similarities.

[0102] The Euclidean distance is inversely proportional to the picture similarity, and the inverse of the Euclidean distance can be calculated to obtain the picture similarity of any two adjacent photos. The smaller the Euclidean distance is, the higher the picture similarity is, and the larger the Euclidean distance is, the lower the picture similarity is.

[0103] In some embodiments, after the electronic device obtains the M photos of the playing video, any two adjacent photos can be sequentially selected, and the sequentially selected any two adjacent photos are input into the pre-trained picture similarity extraction model. The picture similarity extraction model receives and responds to the sequentially input any two adjacent photos, and sequentially outputs the picture similarity of the any two adjacent photos to obtain M-1 picture similarities.

[0104] The picture similarity extraction model can be used to extract the picture similarity of any two photos. The picture similarity extraction model can be a Convolutional Neural Networks (CNN) model, a Deep Belief Networks (DBN) model, a Stacked Auto Encoder Networks (SAE) model, a Recurrent Neural Networks (RNN) model, a Deep Neural Networks (DNN) model, a Long Short-Term Memory (LSTM) network model, or a Gated Recurring Units (GRU) model, etc. The type of the picture similarity extraction model is not limited herein, and can be set according to actual needs.

[0105] Step S130: The electronic device determines the frame playing duration of each video frame in at least part of the N video frames according to the M-1 picture similarities.

[0106] In the embodiments of the present application, after the electronic device determines one picture similarity according to each two adjacent photos in the M photos to obtain M-1 picture similarities, the frame playing duration of each video frame in at least part of the N video frames can be determined according to the M-1 picture similarities.

[0107] The at least part of the video frames can be part of the continuous video frames in the N video frames, or can be all of the video frames in the N video frames, and the like. For example, N=5, the five video frames are video frame 1, video frame 2, video frame 3, video frame 4 and video frame 5, the at least part of the video frames can be video frame 1 and video frame 2, the at least part of the video frames can be video frame 1, video frame 2 and video frame 3, the at least part of the video frames can be video frame 3, video frame 4 and video frame 5, the at least part of the video frames can be video frame 1, video frame 2, video frame 3, video frame 4 and video frame 5, and the like, which is not limited herein.

[0108] In some embodiments, the electronic device determines one picture similarity according to each two adjacent pictures in the M pictures, and obtains M-1 picture similarities. Then, according to the corresponding relationship between the M-1 picture similarities and M-1 time points in the playing time length of the playing video, the frame playing time length of each video frame in the at least part of the video frames is determined. When the picture similarity is low, the frame playing time length of the video frame can be accurately calculated according to the corresponding relationship between the picture similarity and the playing time point of the playing video, and the frame playing time length of each video frame in the at least part of the video frames is obtained, thereby improving the calculation accuracy of the frame playing time length of each video frame in the at least part of the video frames.

[0109] Specifically, the electronic device determines one picture similarity according to each two adjacent pictures in the M pictures, and obtains M-1 picture similarities. Then, according to the corresponding relationship between the M-1 picture similarities and M-1 time points in the playing time length of the playing video, the frame playing time length of each video frame in the at least part of the video frames is determined. When the picture similarity is low, the frame playing time length of the video frame can be accurately calculated according to the corresponding relationship between the picture similarity and the playing time point of the playing video, and the frame playing time length of each video frame in the at least part of the video frames is obtained, thereby improving the calculation accuracy of the frame playing time length of each video frame in the at least part of the video frames.

[0110] In an application scenario, the corresponding relationship between the M-1 picture similarities and M-1 time points in the playing time length of the playing video can be represented by a time-picture similarity curve. If the frame rate (Frames Per Second, fps) of the playing video is 24, a picture similarity curve changing with time can be drawn according to the M-1 picture similarities, and a time-picture similarity curve is obtained, as shown in Figure 5As shown, the time difference between adjacent troughs in the time-image similarity curve can be calculated. Each trough corresponds to a minimum image similarity value. That is, the duration between two time points corresponding to the minimum similarity values ​​of two adjacent images is calculated, and each time difference is determined as the frame playback duration of a video frame, thus obtaining the frame playback duration of each video frame in at least some video frames.

[0111] For example, as shown in the figure, starting from left to right, the time difference between the first and second troughs can be used to determine the frame playback duration of video frame 1, the time difference between the second and third troughs can be used to determine the frame playback duration of video frame 2, the time difference between the third and fourth troughs can be used to determine the frame playback duration of video frame 3, the time difference between the fourth and fifth troughs can be used to determine the frame playback duration of video frame 4, the time difference between the fifth and sixth troughs can be used to determine the frame playback duration of video frame 5, and the time difference between the sixth and seventh troughs can be used to determine the frame playback duration of video frame 6. Video frames 1 to 6 are consecutive video frames.

[0112] Here, video frame 1 can be any video frame of the video being played. For example, video frame 1 can be the first video frame of the video being played, or it can be the second video frame of the video being played, etc. There is no limitation here.

[0113] Step S140: The electronic device determines the smoothness of video playback based on the frame playback duration of each video frame in at least some of the video frames.

[0114] In this embodiment, after the electronic device determines the frame playback duration of each video frame in at least a portion of the N video frames based on the similarity of M-1 images, it can determine the smoothness of the video playback based on the frame playback duration of each video frame in at least a portion of the video frames. This achieves the following: during the detection process of video smoothness detection, based on the image similarity of every two adjacent photos in the M photos, the frame playback duration of each video frame in at least a portion of the N video frames is determined, and the smoothness of the video playback is calculated based on the frame playback duration of each video frame. This represents the user's subjective feeling of video stuttering with specific numerical values, making the evaluation of video smoothness more objective and effective, and improving the detection accuracy of video smoothness detection.

[0115] The smoothness of video playback is calculated based on the frame playback duration of video frames. This unified method of quantifying smoothness is applicable to various causes of stuttering, such as stuttering caused by a mismatch between the video frame rate and the screen refresh rate, stuttering caused by GPU rendering anomalies, and stuttering caused by dropped frames. This expands the application scenarios for detecting video smoothness.

[0116] The picture similarity is low when the video frames are switched, and according to the low picture similarity, the switching duration of the video frames can be accurately calculated, the frame playing duration of the video frames is obtained, and the calculation accuracy of calculating the frame playing duration of the video frames is improved.

[0117] In some implementations, after the electronic device determines the frame playing duration of each video frame in at least part of the N video frames according to the M-1 picture similarities, the electronic device can determine the stuttering rate of playing the video according to the frame playing duration of each video frame in the at least part of the video frames, and determine the fluency of playing the video according to the stuttering rate.

[0118] In the process of determining the stuttering rate, the electronic device can determine a reference frame playing duration according to the frame playing duration of each video frame in the at least part of the video frames, and determine the stuttering duration of playing the video according to the reference frame playing duration and the frame playing duration of each video frame in the at least part of the video frames, and determine the stuttering rate of playing the video according to the total playing duration of the N video frames and the stuttering duration.

[0119] The reference frame playing duration can be used to represent the standard playing duration of each video frame of the playing video, and the stuttering rate can be calculated according to the total playing duration of the N video frames and the stuttering duration according to Formula One, and Formula One is: stuttering rate = stuttering duration / total playing duration of the N video frames.

[0120] Regarding the process that the electronic device determines the reference frame playing duration according to the frame playing duration of each video frame in the at least part of the video frames, as an implementation, the electronic device can count the number of repetitions of the frame playing duration of each video frame in the at least part of the video frames, and determine the frame playing duration with the most repetitions in the frame playing duration of each video frame in the at least part of the video frames as the reference frame playing duration.

[0121] As another implementation, the electronic device can calculate the average frame playing duration of each video frame according to the frame playing duration of each video frame in the at least part of the video frames, and determine the average frame playing duration as the reference frame playing duration.

[0122] Regarding the process that the electronic device determines the stuttering duration of playing the video according to the reference frame playing duration and the frame playing duration of each video frame in the at least part of the video frames, as an implementation, the electronic device can sum the absolute values of the time duration differences between each non-reference frame playing duration and the reference frame playing duration to obtain the stuttering duration of playing the video.

[0123] After obtaining the stuttering duration of playing the video, the electronic device can calculate the quotient of the stuttering duration and the total playing duration of the N video frames according to Formula One to obtain the stuttering rate of playing the video.

[0124] Regarding the process that the electronic device determines the fluency of playing the video according to the stuttering rate, in some embodiments, the electronic device can calculate 1 minus the percentage of the stuttering rate to obtain the fluency of playing the video.

[0125] In an application scenario, when the video frame rate of the playing video is 24fps and the screen refresh rate of the playing video is 30Hz, each cycle frame of the playing video includes video frame 1, video frame 2, video frame 3 and video frame 4, and the frame playing time of video frame 1, the frame playing time of video frame 2, the frame playing time of video frame 3 and the frame playing time of video frame 4 can be determined according to the picture similarity, respectively.

[0126] The repetition number of the frame playing time 1 / 30s is 3, that is, the repetition number of the frame playing time 1 / 30s is the maximum, and the frame playing time 1 / 30s is determined as the reference frame playing time.

[0127] The absolute value of the time difference between the non-reference frame playing time 2 / 30s and the reference frame playing time 1 / 30s is calculated to obtain the stuttering time = |2 / 30s-1 / 30s| = 1 / 30s.

[0128] The total playing time of each cycle frame = (1 / 30s+1 / 30s+1 / 30s+2 / 30s) = 1 / 6s, the stuttering rate of the playing video = stuttering time / total playing time of each cycle frame = (1 / 30s) / (1 / 6s) = 0.2, and the fluency of the playing video = 1-0.2*100% = 80%.

[0129] Wherein, the cycle frame is used to represent the number of frames matched with the screen refresh rate, and the cycle frame is associated with the player of the playing video.

[0130] In an application scenario, the corresponding relationship among the video frame rate of the playing video, the screen refresh rate, the refresh number of the video frame sequence, the frame playing time of the video frame sequence, the reference frame playing time, the stuttering rate and the fluency can be as shown in Table 1.

[0131] Table 1

[0132]

[0133] In an application scenario, as shown in Figure 6 The video fluency detection method can include steps S210 to S270. It should be noted that Figure 6 is a combination of the video fluency detection system shown in Figure 4 , and the video fluency detection method in an application scenario is schematically described.

[0134] Step S210: The electronic device photographs the playing picture of the playing video by the high-speed camera to obtain M photos.

[0135] The M photos correspond to N video frames of the playing video, and one video frame corresponds to at least one photo.

[0136] Step S220: The electronic device calculates the picture similarity of each two adjacent photos in the M photos by an image similarity algorithm to obtain M-1 picture similarities.

[0137] The image similarity algorithm can include any one of a mean hash algorithm, a perceptual hash algorithm, a cosine similarity algorithm, a histogram algorithm, and a picture similarity model detection algorithm, and the type of the image similarity algorithm is not limited here and can be set according to actual needs.

[0138] Step S230: The electronic device draws a time-picture similarity curve according to the M-1 picture similarities.

[0139] Step S240: The electronic device determines the frame playing time length of each video frame in at least part of the N video frames based on the time-picture similarity curve.

[0140] Step S250: The electronic device determines a reference frame playing time length according to the frame playing time length of each video frame in at least part of the video frames.

[0141] Specifically, the frame playing time length with the most repetitions in the frame playing time lengths of each video frame in at least part of the video frames is determined as the reference frame playing time length.

[0142] Step S260: The electronic device determines the stuttering time length of the playing video based on the reference frame playing time length.

[0143] Specifically, the absolute values of the time length differences between each non-reference frame playing time length and the reference frame playing time length are summed to obtain the stuttering time length of the playing video.

[0144] Step S270: The electronic device calculates the fluency of the playing video based on the stuttering time length.

[0145] Specifically, the stuttering rate of the playing video is calculated according to the total playing time length of the N video frames and the stuttering time length, and the fluency of the playing video is calculated according to the stuttering rate.

[0146] In some embodiments, after determining the smoothness of the playing video according to the frame playing duration of each video frame in the at least partial video frames, the electronic device can determine the smoothness as a first drawing vector and the playing time corresponding to the smoothness as a second drawing vector, and draw and display a smoothness change graph using the first drawing vector and the second drawing vector, so that the user can more intuitively and simply view the change of the video smoothness through the smoothness change graph, thereby improving the detection experience of the user in detecting the video smoothness.

[0147] In some embodiments, after determining the smoothness of the playing video according to the frame playing duration of each video frame in the at least partial video frames, the electronic device can generate a detection result containing the smoothness and send the detection result to a server for storage, so that the user can trace the detection process of the video smoothness according to the detection result stored in the server, thereby improving the detection experience of the user in detecting the video smoothness.

[0148] The server is connected to the electronic device through a network and performs data interaction with the electronic device through the network. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), big data, or artificial intelligence platform, etc. The type of the server is not limited here and can be set according to actual needs.

[0149] The scheme provided in this embodiment obtains M photos of a playing video, the M photos correspond to N video frames of the playing video, one video frame corresponds to at least one photo, determines a frame similarity between each two adjacent photos in the M photos to obtain M-1 frame similarities, determines the frame playing duration of each video frame in the at least partial video frames in the N video frames according to the M-1 frame similarities, and determines the smoothness of the playing video according to the frame playing duration of each video frame in the at least partial video frames, thereby realizing, in the detection process of detecting the video smoothness, determining the frame playing duration of each video frame in the at least partial video frames in the N video frames of the playing video based on the frame similarity between each two adjacent photos in the M photos obtained, and calculating the smoothness of the playing video based on the frame playing duration of each video frame, which uses the subjective feeling of the user on the lag of the playing video to express specific numerical values, makes the evaluation of the video smoothness more objective and effective, and improves the detection accuracy of detecting the video smoothness.

[0150] The frame playing duration of the video frame is calculated according to the smoothness of playing the video, and the method for uniformly quantifying the smoothness is suitable for the stall feeling caused by various reasons, such as the stall feeling caused by the mismatch between the video frame rate and the screen refresh rate, the stall feeling caused by GPU rendering abnormalities, the stall feeling caused by frame dropping, and the like, thereby expanding the application scenarios of detecting the smoothness of the video.

[0151] The picture similarity is low when the video frame is switched, and the switching duration of the video frame can be accurately calculated according to the low picture similarity, so as to obtain the frame playing duration of the video frame, thereby improving the calculation accuracy of calculating the frame playing duration of the video frame.

[0152] Please refer to Figure 7 , which shows a flowchart of a method for detecting the smoothness of a video according to another embodiment of the present application. In specific embodiments, the method for detecting the smoothness of the video can be applied to an electronic device, a processor or a chip in the electronic device, and the like. In the following, the flowchart shown in Figure 7 will be described in detail, and the method for detecting the smoothness of the video can include the following steps S310 to S350.

[0153] Step S310: An electronic device obtains M photos of a playing video.

[0154] Step S320: The electronic device determines a picture similarity between each two adjacent photos in the M photos, so as to obtain M-1 picture similarities.

[0155] Step S330: The electronic device determines the frame playing duration of each video frame in at least part of the N video frames according to the M-1 picture similarities.

[0156] Step S340: The electronic device determines the smoothness of playing the video according to the frame playing duration of each video frame in at least part of the video frames.

[0157] In the present embodiment, the steps S310, S320, S330 and S340 can refer to the contents of the corresponding steps in the foregoing embodiments, which will not be described herein again.

[0158] Step S350: When the smoothness is less than or equal to a smoothness threshold, the electronic device generates a playing mode switching prompt information.

[0159] In the present embodiment, after the electronic device determines the smoothness of playing the video according to the frame playing duration of each video frame in at least part of the video frames, and when the smoothness is less than or equal to the smoothness threshold, the playing mode switching prompt information is generated, so that a user switches the playing mode of playing the video to a standard definition playing mode according to the playing mode switching prompt information, which is conducive to improving the smoothness of playing the video and improving the detection experience of the user in detecting the smoothness of the video.

[0160] The playing mode switching prompt information can include at least one of text prompt information, sound prompt information, and light prompt information, and the type of the playing mode switching prompt information is not limited here and can be set according to actual needs.

[0161] The playing mode can be any one of a standard definition playing mode, a high definition playing mode, an ultra-high definition playing mode, or a blue-ray playing mode, and the type of the playing mode is not limited here and can be set according to actual needs.

[0162] The smoothness threshold value can be 80%, 95%, 70%, or the like. The smoothness threshold value can be a smoothness value preset by a user or a smoothness value automatically generated by the electronic device based on a detection process of detecting the smoothness of the video multiple times, and the setting manner of the smoothness threshold value is not limited here and can be set according to actual needs.

[0163] In some embodiments, after the electronic device determines the smoothness of the playing video according to the frame playing time length of each video frame in the at least part of the video frames, and when the smoothness is less than or equal to the smoothness threshold value, the stack information of the player playing the video is obtained, and the cause of the lag of the playing video is determined according to the stack information, so that the user can process the playing video according to the cause of the lag to improve the smoothness of the playing video.

[0164] The general interface display and rendering thread are both main threads, and the blockage of the main thread will cause the lag of the playing video, so the stack information of the main thread can be viewed to determine the cause of the lag of the playing video.

[0165] The scheme provided in this embodiment obtains M photos of the playing video, determines a picture similarity between each two adjacent photos in the M photos to obtain M-1 picture similarities, determines the frame playing time length of each video frame in the at least part of the video frames according to the M-1 picture similarities, determines the smoothness of the playing video according to the frame playing time length of each video frame in the at least part of the video frames, generates a playing mode switching prompt information when the smoothness is less than or equal to the smoothness threshold value, and realizes that, in the detection process of detecting the smoothness of the video, the frame playing time length of each video frame in the at least part of the N video frames of the playing video is determined based on the picture similarity between each two adjacent photos in the M photos obtained, and the smoothness of the playing video is calculated based on the frame playing time length of each video frame, so that the subjective feeling of the user to the lag of the playing video is expressed by a specific numerical value, the evaluation of the smoothness of the video is more objective and effective, and the detection accuracy of the detection of the smoothness of the video is improved.

[0166] Further, when the fluency is less than or equal to the fluency threshold, a playing mode switching prompt information is generated so that the user switches the playing mode of the playing video to the standard definition playing mode according to the playing mode switching prompt information, which is beneficial to improve the fluency of the playing video and improve the detection experience of the user in detecting the video fluency.

[0167] Please refer to Figure 8 which shows a video fluency detection device 500 provided by an embodiment of the application. The video fluency detection device 500 can be applied to an electronic device, a processor or a chip in the electronic device, and the like. In the following, the video fluency detection device 500 is described in detail with the electronic device as an example. Figure 8 The video fluency detection device 500 can include an acquisition module 510, a first determination module 520, a second determination module 530, and a third determination module 540.

[0168] The acquisition module 510 can be configured to acquire M photos of a playing video. The M photos can correspond to N video frames of the playing video, and one video frame can correspond to at least one photo. The first determination module 520 can be configured to determine a picture similarity between every two adjacent photos in the M photos to obtain M-1 picture similarities. The second determination module 530 can be configured to determine a frame playing time length of each video frame in at least part of the N video frames according to the M-1 picture similarities. The third determination module 540 can be configured to determine a fluency of the playing video according to the frame playing time length of each video frame in the at least part of the N video frames.

[0169] In some embodiments, the second determination module 530 can include a first determination sub-module.

[0170] The first determination sub-module can be configured to determine the frame playing time length of each video frame in the at least part of the N video frames according to a corresponding relationship between the M-1 picture similarities and M-1 time points in a playing time length of the playing video.

[0171] In some embodiments, the first determination sub-module can include a first determination unit.

[0172] The first determination unit can be configured to determine a time length between two time points corresponding to each adjacent two picture similarity minimum values as a frame playing time length of a video frame according to the corresponding relationship between the M-1 picture similarities and the M-1 time points in the playing time length of the playing video, to obtain the frame playing time length of each video frame in the at least part of the N video frames.

[0173] In some embodiments, the third determination module 540 can include a second determination sub-module and a third determination sub-module.

[0174] The second determining submodule can be configured to determine a stalling rate of the playing video according to the frame playing time length of each of the at least partial video frames; and the third determining submodule can be configured to determine the fluency of the playing video according to the stalling rate.

[0175] In some embodiments, the second determining submodule can include a second determining unit, a third determining unit, and a fourth determining unit.

[0176] The second determining unit can be configured to determine a reference frame playing time length according to the frame playing time length of each of the at least partial video frames; the third determining unit can be configured to determine a stalling time length of the playing video according to the reference frame playing time length and the frame playing time length of each of the at least partial video frames; and the fourth determining unit can be configured to determine a stalling rate of the playing video according to the total playing time length of the N video frames and the stalling time length.

[0177] In some embodiments, the second determining unit can include a first determining subunit.

[0178] The first determining subunit can be configured to determine, as the reference frame playing time length, the frame playing time length with the most repetitions in the frame playing time length of each of the at least partial video frames.

[0179] In some embodiments, the third determining unit can include a summing subunit.

[0180] The summing subunit can be configured to sum the absolute values of the time length differences between each non-reference frame playing time length and the reference frame playing time length to obtain the stalling time length of the playing video.

[0181] In some embodiments, the second determining unit can further include a calculating subunit and a second determining subunit.

[0182] The calculating subunit can be configured to calculate an average frame playing time length according to the frame playing time length of each of the at least partial video frames; and the second determining subunit can be configured to determine the average frame playing time length as the reference frame playing time length.

[0183] In some embodiments, the third determining submodule can include a calculating unit.

[0184] The calculating unit can be configured to calculate the fluency of the playing video by subtracting the percentage of the stalling rate from 1.

[0185] In some embodiments, the first determining module 520 can include a first calculating submodule, a second calculating submodule, and a third calculating submodule.

[0186] The first calculation sub-module can be configured to calculate a first hash value of one of any two adjacent photos and a second hash value of the other photo; the second calculation sub-module can be configured to calculate a Hamming distance of any two adjacent photos according to the first hash value and the second hash value; and the third calculation sub-module can be configured to calculate a frame similarity of any two adjacent photos according to the Hamming distance.

[0187] In some embodiments, the acquisition module 510 can include a sending sub-module and a receiving sub-module.

[0188] The sending sub-module can be configured to send a collection instruction to the high-speed camera, so that the high-speed camera photographs a playing frame of the playing video according to the collection instruction to obtain M photos; and the receiving sub-module can be configured to receive the M photos returned by the high-speed camera.

[0189] The scheme provided in this embodiment can obtain M photos of a playing video, the M photos correspond to N video frames of the playing video, one video frame corresponds to at least one photo, determine a frame similarity of each two adjacent photos in the M photos to obtain M-1 frame similarities, determine a frame playing time length of each video frame in at least part of the N video frames according to the M-1 frame similarities, and determine the smoothness of the playing video according to the frame playing time length of each video frame in the at least part of the N video frames. In the detection process of detecting the smoothness of the video, the frame playing time length of each video frame in at least part of the N video frames of the playing video is determined based on the frame similarity of each two adjacent photos in the M photos, and the smoothness of the playing video is calculated based on the frame playing time length of each video frame. The subjective feeling of the user about the lag of the playing video is expressed by a specific numerical value, so that the evaluation of the smoothness of the video is more objective and effective, and the detection accuracy of detecting the smoothness of the video is improved.

[0190] The smoothness of the playing video is calculated according to the frame playing time length of the video frame, the method of uniformly quantifying the smoothness is suitable for the lag feeling caused by various reasons, such as the lag feeling caused by the mismatch between the video frame rate and the screen refresh rate, the lag feeling caused by GPU rendering exception, the lag feeling caused by frame dropping, etc., and the application scene of detecting the smoothness of the video is expanded.

[0191] The frame similarity is low when the video frame is switched, and the switching time length of the video frame can be accurately calculated according to the low frame similarity to obtain the frame playing time length of the video frame, so that the calculation accuracy of calculating the frame playing time length of the video frame is improved.

[0192] It should be noted that the various embodiments of the present specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be mutually referred to. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments. For any processing manner described in the method embodiments, it can be realized by a corresponding processing module in the device embodiments, and the device embodiments will not be repeated.

[0193] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0194] Please refer to Figure 9 , which shows a hardware structure schematic diagram of an electronic device 100 provided by an embodiment of the present application. As Figure 9 indicated, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0195] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be realized in hardware, software, or a combination of software and hardware.

[0196] Exemplarily,Figure 9 The processor 110 shown can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0197] Among them, the AP can be used to control and manage the smoothness detection application program, for example, the AP can control the smoothness detection application program to perform smoothness detection on the played video.

[0198] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0199] The memory in the processor 110 can also be provided to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. Avoiding repeated access, reducing the waiting time of the processor 110, thus improving the efficiency of the system.

[0200] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0201] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and the GPU is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0202] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a Liquid Crystal Display (LCD), an Organic Light Emitting Diode (OLED), an active-matrix organic light-emitting diode, an Active-matrix Organic Light Emitting Diode (AMOLED), a Flex Light Emitting Diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, a Quantum Dot Light Emitting Diodes (QLED), etc. In some embodiments, the electronic device 100 can include 1 or N display screens 194, and N is a positive integer greater than 1.

[0203] The electronic device 100 can implement a photographing function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc.

[0204] ISP is used to process the data feedback by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, and the optical signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into a visible image. The ISP can also optimize the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be arranged in the camera 193.

[0205] The camera 193 is used to capture still images or videos. Objects generate optical images through lenses and project them onto photosensitive elements. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into a standard RGB, YUV, or other format image signal. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0206] The external memory interface 120 can be used to connect an external memory card, such as a secure digital (SD) card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve data storage functions. For example, the electronic device 100 can save files such as captured images and videos in the external memory card.

[0207] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functions of the electronic device 100 and data processing by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required for a function (such as an audio acquisition function, an image capture function, etc.), and the like. The data storage area can store data created during the use of the electronic device 100 (such as audio data, image data), and the like. In addition, the internal memory 121 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

[0208] Please refer to Figure 10This illustrates a functional block diagram of an electronic device 300 according to an embodiment of this application. Figure 10 As shown, the electronic device 300 includes: one or more processors 310 ( Figure 10 (Only one processor is shown in the diagram) and memory 320, which is coupled to one or more processors 310. The memory 320 is used to store computer program code 330, which includes computer instructions. One or more processors 310 call the computer instructions to cause the electronic device 300 to perform the steps in any of the above methods.

[0209] Those skilled in the art will understand that Figure 10 This is merely an example of electronic device 300 and does not constitute a limitation on electronic device 300. In practice, electronic device 300 may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc. Electronic device 300 may also be the same device as electronic device 100 described in the above embodiments.

[0210] The processor 310 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0211] In some embodiments, memory 320 may be an internal storage unit of electronic device 300, such as a hard disk or memory of electronic device 300. In other embodiments, memory 320 may be an external storage device of electronic device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on electronic device 300. Optionally, memory 320 may include both internal and external storage units of electronic device 300. Memory 320 is used to store operating system, application programs, bootloaders, data, and other programs, such as program code of computer programs. Memory 320 may also be used to temporarily store data that has been output or will be output.

[0212] It should be noted that the information interaction, execution process and the like between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and specific functions and technical effects brought by the same can be referred to the method embodiments part, which will not be repeated here.

[0213] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0214] The embodiment of the present application further provides a chip system, which is applied to an electronic device, and the chip system comprises one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to realize the steps in any one of the methods.

[0215] In some embodiments, the chip system further comprises a memory, and the memory is connected to the one or more processors through a circuit or a wire.

[0216] In some embodiments, the chip system further comprises a communication interface.

[0217] The embodiment of the present application further provides a computer readable storage medium, which comprises instructions, and when the instructions run on an electronic device, enable the electronic device to realize the method described in each method embodiment.

[0218] The embodiment of the present application further provides a computer program product, which, when running on an electronic device, enables the electronic device to perform the above-mentioned related steps to realize the method described in each method embodiment.

[0219] The integrated units described above, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can at least include any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk and the like. In some jurisdictions, according to legislation and patent practice, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0220] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0221] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0222] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there can be another division in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutually can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0223] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to achieve the purpose of the embodiment according to actual needs.

[0224] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0225] It should also be understood that the term "and / or" used in the description of the present application and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0226] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0227] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0228] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features thereof; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting video smoothness, the method comprising: The method comprises: acquiring M photos of a playing video, the M photos corresponding to N video frames of the playing video, one video frame corresponding to at least one photo, wherein N is a positive integer greater than 1, and M is a positive integer greater than N; determining a picture similarity between every two adjacent photos in the M photos to obtain M-1 picture similarities; determining a frame playing time length between two time points corresponding to each of two adjacent picture similarity minimum values as a frame playing time length of a video frame according to a corresponding relationship between the M-1 picture similarities and M-1 time points in a playing time length of the playing video to obtain the frame playing time length of each of at least part of the N video frames; determining a smoothness of the playing video according to the frame playing time length of each of at least part of the video frames.

2. The method of claim 1, wherein, The determination of the smoothness of the playing video according to the frame playing time length of each of at least part of the video frames comprises: determining a stuttering rate of the playing video according to the frame playing time length of each of at least part of the video frames; determining the smoothness of the playing video according to the stuttering rate.

3. The method of claim 2, wherein, The determination of the stuttering rate of the playing video according to the frame playing time length of each of at least part of the video frames comprises: determining a reference frame playing time length according to the frame playing time length of each of at least part of the video frames; determining a stuttering time length of the playing video according to the reference frame playing time length and the frame playing time length of each of at least part of the video frames; determining the stuttering rate of the playing video according to a total playing time length of the N video frames and the stuttering time length.

4. The method of claim 3, wherein, The determination of the reference frame playing time length according to the frame playing time length of each of at least part of the video frames comprises: determining a frame playing time length with a maximum repetition number in the frame playing time length of each of at least part of the video frames as the reference frame playing time length.

5. The method of claim 3, wherein, The determination of the stuttering time length of the playing video according to the reference frame playing time length and the frame playing time length of each of at least part of the video frames comprises: summing up absolute values of time length differences between each non-reference frame playing time length and the reference frame playing time length to obtain the stuttering time length of the playing video.

6. The method of claim 3, wherein, The determination of the reference frame playing time length according to the frame playing time length of each of at least part of the video frames comprises: calculating an average frame playing time length according to the frame playing time length of each of at least part of the video frames; determining the average frame playing time length as the reference frame playing time length.

7. The method according to any one of claims 2 to 6, characterized in that, The determination of the smoothness of the playing video according to the stuttering rate comprises: calculating a percentage of 1 minus the stuttering rate to obtain the smoothness of the playing video.

8. The method according to any one of claims 1 to 6, characterized in that, The determination of a picture similarity between every two adjacent photos in the M photos comprises: calculating a first hash value of one photo and a second hash value of another photo in any two adjacent photos; calculating a Hamming distance of the any two adjacent photos according to the first hash value and the second hash value; calculating a picture similarity of the any two adjacent photos according to the Hamming distance.

9. The method according to any one of claims 1 to 6, characterized in that, The acquisition of the M photos of the playing video comprises: sending a collection instruction to the high-speed camera, so that the high-speed camera photographs a playing picture of the playing video according to the collection instruction, and obtains M photos; receiving the M photos returned by the high-speed camera.

10. A device for detecting video smoothness, characterized in that, The device comprises: an acquisition module, configured to acquire M photos of a playing video, the M photos corresponding to N video frames of the playing video, one video frame corresponding to at least one photo, wherein N is a positive integer greater than 1, and M is a positive integer greater than N; a first determination module, configured to determine a picture similarity according to each two adjacent photos in the M photos, so as to obtain M-1 picture similarities; a second determination module, configured to determine, according to a corresponding relationship between the M-1 picture similarities and M-1 time points in a playing time length of the playing video, a time length between two time points corresponding to each adjacent two picture similarity minimum values as a frame playing time length of a video frame, so as to obtain the frame playing time length of each video frame in at least part of the N video frames; a third determination module, configured to determine a smoothness of the playing video according to the frame playing time length of each video frame in the at least part of the video frames.

11. An electronic device, comprising: The electronic device comprises one or more processors and a memory; The memory is coupled with the one or more processors, and the memory is configured to store computer program codes, the computer program codes comprising computer instructions, and the one or more processors are configured to invoke the computer instructions to enable the electronic device to perform the method in any one of claims 1 to 9.

12. A computer readable storage medium, characterized in that, The computer readable storage medium comprises instructions, when the instructions run on the electronic device, enable the electronic device to perform the method in any one of claims 1 to 9.

Images