Video quality assessment method, device, electronic device and storage medium
By acquiring and combining the perceived frame rate, average frame rate and normalized momentum information of video, the video quality is evaluated, and the problem of low accuracy of video evaluation in the prior art is solved, and multi-dimensional measurement and accuracy of video fluency is achieved. Evaluation improves the accuracy and objectivity of video quality evaluation.
Patent Information
- Application Number
- CN202211061562.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-31
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-08-31
AI Technical Summary
The existing video quality evaluation methods have low accuracy and are easily affected by factors such as video interpolation, scene content changes or light changes, making it difficult to comprehensively measure video fluency from multiple dimensions.
By obtaining the perceived frame rate information, average frame rate information and normalized momentum information of the target video, combining these parameters to evaluate the video quality, eliminate the subjective influence of frames and frame skipping in the time dimension, and cover the frame drop and frame intervals are discontinuous. and playback exception caused by multi-frame results.
It improves the accuracy of video quality evaluation, can objectively and accurately measure video quality in different scenarios, quickly filter out lag abnormalities perceived by the human eye, provide objective quantitative results, and evaluate the video processing algorithm and shooting performance of electronic devices.
Smart Images

Figure CN115426490B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to a video quality assessment method, device, electronic equipment and storage medium. Background Art
[0002] The rapid development of multimedia technology has led to higher and higher requirements for the user experience of multimedia files (eg, videos), among which the smoothness of video playback is an issue that users are more concerned about.
[0003] In related technologies, the smoothness of a video is usually measured by comparing adjacent frames, counting the interval between video frames, or calculating the similarity of adjacent video frames. However, the above methods only evaluate the video quality from a single dimension and are easily affected by factors such as video interpolation, scene content changes, or light changes, resulting in low accuracy in video quality evaluation. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a video quality assessment method, device, electronic device and storage medium, which can solve the problem of low accuracy of existing video quality assessment methods.
[0005] In a first aspect, an embodiment of the present application provides a method for evaluating video quality, the method comprising:
[0006] Get the target video;
[0007] Determining a video quality assessment result of the target video according to the video playback parameters of the target video;
[0008] Among them, the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information; the perceived frame rate information is used to characterize the number of valid video frames perceived by the human eye in the target video; the average frame rate information is used to characterize the smoothness of the target video; the normalized momentum information is used to characterize the change amplitude of at least one video frame of the target video relative to the previous video frame.
[0009] In a second aspect, an embodiment of the present application provides a video quality assessment device, the device comprising:
[0010] A first acquisition module, used to acquire a target video;
[0011] A first determination module, used to determine a video quality evaluation result of the target video according to a video playback parameter of the target video;
[0012] Among them, the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information; the perceived frame rate information is used to characterize the number of valid video frames perceived by the human eye in the target video; the average frame rate information is used to characterize the smoothness of the target video; the normalized momentum information is used to characterize the change amplitude of at least one video frame of the target video relative to the previous video frame.
[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.
[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0015] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
[0016] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.
[0017] The video quality assessment method, device, electronic device and storage medium provided in the embodiments of the present application obtain a target video, and determine the video quality assessment result of the target video according to the video playback parameters of the target video, wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information. In this way, the video quality of the target video is assessed in combination with the perceived frame rate information and the normalized momentum information, which can eliminate the influence of different degrees of frame drops and frame skipping on the user's subjective feelings in the time dimension, and at the same time cover the playback anomalies caused by frame drops, discontinuous frame intervals, and consistent multi-frame results, thereby improving the accuracy of video quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flowchart of a video quality evaluation method provided in an embodiment of the present application;
[0019] Figure 2 is a schematic diagram of a motion momentum curve provided in an embodiment of the present application;
[0020] Figure 3is a schematic diagram of a normalized momentum curve provided in an embodiment of the present application;
[0021] Figure 4 is a schematic diagram of a perceived frame rate curve provided in an embodiment of the present application;
[0022] Figure 5 It is a flowchart of another video quality evaluation method provided in an embodiment of the present application;
[0023] Figure 6 Schematic diagram of the hardware structure of a video quality assessment device provided in an embodiment of the present application;
[0024] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;
[0025] Figure 8 It is a schematic diagram of the hardware structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0027] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0028] The video quality evaluation method provided in the present application can be applied to an application environment including an electronic device and a server. The electronic device and the server are connected via a network. The electronic device can be, but is not limited to, various smart phones, personal computers, laptops, tablet computers, portable wearable devices, etc.
[0029] In one embodiment, an electronic device obtains a target video and sends the target video to a server, and the server determines a video quality assessment result of the target video based on video playback parameters of the target video, wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information.
[0030] The video quality assessment method provided in the embodiment of the present application may be executed by the video quality assessment device provided in the embodiment of the present application, or an electronic device integrating the video quality assessment device, wherein the video quality assessment device may be implemented in hardware or software.
[0031] By using the video quality evaluation method provided in the embodiment of the present application, the playback quality of the target video is evaluated in combination with the perceived frame rate and normalized momentum. This can eliminate the impact of different degrees of frame drops and frame skipping on the user's subjective experience in the time dimension. At the same time, it covers playback anomalies caused by frame drops, discontinuous frame intervals, and consistent multi-frame results. Therefore, objective and accurate test results can be obtained for videos shot in different scenes.
[0032] For example, in a video quality assessment scenario, through the video quality assessment method provided in the embodiments of the present application, a user can obtain objective and accurate video quality assessment results.
[0033] For another example, in a video processing algorithm testing scenario of an electronic device, the video quality evaluation result of the target video can be obtained through the video quality evaluation method provided in an embodiment of the present application, so that the video processing algorithm of the electronic device can be evaluated based on the video quality evaluation result.
[0034] For another example, in a shooting performance test scenario of an electronic device, the video quality evaluation result of the target video can be obtained through the video quality evaluation method provided in the embodiment of the present application, so that the shooting performance of the electronic device can be evaluated based on the video quality evaluation result.
[0035] For another example, in the evaluation of HDR video shooting scenes, the video quality evaluation result of the target video can be obtained through the video quality evaluation method provided in the embodiment of the present application. Based on the video quality evaluation result, it is possible to measure whether the HDR algorithm will cause freezing, frame loss, etc. in the shooting picture, and output the time and severity of the problem.
[0036] The following describes in detail the video quality evaluation method provided by the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.
[0037] See also Figure 1 , which is a flow chart of a method for evaluating video quality provided by an embodiment of the present application. The method can be applied to electronic devices. Figure 1As shown, the method may include steps 1100 to 1300.
[0038] Step 1100, obtaining a target video.
[0039] The target video is the video to be evaluated. On the one hand, the target video can be a video recorded by an electronic device, such as an offline video, an online video, a live video, a short video, etc. In this regard, by evaluating the video quality of the target video, the video recording performance of the electronic device can be evaluated. On the other hand, the target video can also be an existing video, such as an offline video in a video application. In this regard, the video quality of the target video can be evaluated using this embodiment.
[0040] Step 1200, determining a video quality assessment result of the target video according to the video playback parameters of the target video; wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information.
[0041] The average frame rate information is used to characterize the smoothness of the target video. The average frame rate of the target video refers to the number of frames displayed per second in the target video. The average frame rate is a parameter to measure the smoothness of the video. The higher the average frame rate, the better the smoothness of the video, and the lower the average frame rate, the worse the smoothness of the video.
[0042] Normalized momentum information is used to characterize the amplitude of change of at least one video frame of the target video relative to the previous video frame. The normalized momentum information may include the normalized momentum of each video frame of the target video, and the normalized momentum of each video frame may characterize the amplitude of change of the video frame relative to the previous video frame. The normalized momentum information may be used to measure the degree of frame skipping of the target video. In addition, since the amplitude of change of the calculated video frames is different for videos of different scenes. For example, the amplitude of change of the obtained video frames is different for videos obtained through different shooting modes. In this regard, based on the normalized momentum information of the target video, the degree of frame skipping of videos of different scenes may be accurately measured.
[0043] The perceived frame rate information is used to characterize the number of valid video frames perceived by the human eye in the target video. The perceived frame rate information may be a perceived frame rate sequence. The perceived frame rate sequence is a time sequence of perceived frame rates. The perceived frame rate sequence includes multiple perceived frame rates. The perceived frame rate is the number of valid video frames that can be perceived by the human eye within a preset time. The valid video frame is a video frame whose normalized momentum is not zero in the target video. Exemplarily, it may be the number of valid video frames that can be perceived by the human eye per second. For example, for a target video with a static picture, the number of valid video frames that can be perceived by the human eye per second is 1 frame, that is, the perceived frame rate is 1 frame / second. For another example, for a target video with 30 frames per second of video frames that are all changing, the number of valid video frames that can be perceived by the human eye per second is 30 frames, that is, the perceived frame rate is 30 frames / second. For another example, for a video with 30 frames per second, if the pictures of two adjacent video frames do not change, the number of valid video frames that can be perceived by the human eye per second is 15 frames, that is, the perceived frame rate is 15 frames / second. The perceived frame rate can be used to measure the degree of frame drop in the target video.
[0044] Based on this, the video quality of the target video can be evaluated through the perceived frame rate information, average frame rate information and normalized momentum information of the target video.
[0045] The following is an explanation of the process of determining each video playback parameter.
[0046] In some embodiments, before determining the video quality assessment result of the target video according to the video playback parameters of the target video, the method may further include: steps 2100 to 2300.
[0047] Step 2100: Acquire the first motion momentum of each video frame of the target video to obtain a first motion momentum sequence.
[0048] The first motion momentum is the actual motion momentum of each video frame of the target video, that is, the actual change amplitude of the video frame relative to the previous video frame. The first motion momentum sequence is obtained by arranging the first motion momentum of the first video frame in the order of timestamps. More specifically, the first motion momentum sequence can be a curve of the change of the first motion momentum of the video frame over time.
[0049] Optionally, the target video includes adjacent third video frames and fourth video frames, wherein the third video frame is the previous video frame of the fourth video frame. Feature point recognition is performed on the third video frame to obtain a first feature point in the third video frame, and feature point matching is performed on the third video frame and the fourth video frame to obtain a second feature point in the fourth video frame corresponding to the first feature point. Then, based on the first feature point and the second feature point, the affine transformation matrix of the fourth video frame relative to the third video frame can be determined, and then, based on the affine transformation matrix, the first motion momentum of the fourth video frame relative to the third video frame can be determined. Repeating the aforementioned steps can obtain the first motion momentum of each video frame of the target video, and based on the first motion momentum of each video frame, the first motion momentum sequence of the target video can be obtained.
[0050] For example, obtain the first frame of the target video, determine the optical flow feature points in the first frame of the video according to the Good Features To Track (image processing technology) function, then input the current video frame and the previous video frame into the CalcOptical Flow PyrLK function to obtain the optical flow feature points of the current video frame, and then use the Estimate RigidTransform function to calculate the affine transformation matrix between the optical flow feature points of the current video frame and the optical flow feature points of the previous video frame.
[0051] The affine transformation matrix can be expressed by the following formula (1):
[0052]
[0053] Where m is the affine transformation matrix between the optical flow feature points of the current video frame and the optical flow feature points of the previous video frame; A is the rotation matrix between the optical flow feature points of the current video frame and the optical flow feature points of the previous video frame; a 11 、a 12 represents the rotation vector of the optical flow feature point of the current video frame perpendicular to the Z direction; B is the translation matrix between the optical flow feature point of the current video frame and the optical flow feature point of the previous video frame; b1 represents the translation amount of the optical flow feature point of the current video frame in the X direction; b2 represents the translation amount of the optical flow feature point of the current video frame in the Y direction.
[0054] Afterwards, the first motion momentum of the current video frame relative to the previous video frame is calculated through the affine change matrix between the optical flow feature points of the current video frame and the optical flow feature points of the previous video frame.
[0055] The first motion momentum can be calculated by the following formula (2) and formula (3):
[0056]
[0057] da =arctan(a 11 +a 12 ) (3)
[0058] Wherein, Motion is the first motion momentum of the current video frame relative to the previous video frame; d a is the rotation vector of the current video frame relative to the previous video frame.
[0059] Repeat the above steps to obtain the first motion momentum of each video frame of the target video. According to the first motion momentum of each video frame, the first motion momentum sequence of the target video can be obtained, and then the change curve of the first motion momentum of the target video over time, that is, the first motion momentum curve, can be obtained. Figure 2 , the first motion curve of the target video can be curve a.
[0060] Step 2200: Smoothing the first motion momentum sequence to obtain a second motion momentum sequence, wherein the second motion momentum sequence includes a second motion momentum corresponding to the first motion momentum of each video frame.
[0061] The second motion momentum sequence is a motion momentum time sequence obtained by prediction when the picture motion of the target video is smooth under an ideal state.
[0062] Exemplarily, the first motion momentum sequence is windowed by a preset window function to obtain a second motion momentum sequence, and then a curve of the second motion momentum of the target video changing over time, that is, a second motion momentum curve, is obtained. Figure 2 , the second motion curve of the target video may be curve b. Optionally, the preset window function may be set according to a person's visual persistence time and sensitivity to expected motion, which is not specifically limited in the present embodiment of the application.
[0063] Step 2300 , determining the normalized momentum of each video frame according to the first motion momentum and the second motion momentum of each video frame to obtain the normalized momentum information of the target video.
[0064] In this embodiment, the calculated variation range of the video frames is different for videos of different scenes. For example, the variation range of the video frames obtained by different shooting modes is different. Therefore, based on the normalized momentum of each video frame, the degree of frame skipping of videos of different scenes can be accurately measured.
[0065] Exemplarily, the normalized momentum of the i-th video frame can be calculated by the following formula (4):
[0066]
[0067] Among them, Motion_nor i Motion is the normalized momentum of the i-th frame in the target video; i is the first motion momentum (actual motion momentum) of the i-th video frame in the target video; is the second motion momentum (ideal motion momentum) of the i-th video frame in the target video.
[0068] According to the normalized momentum of each video frame, the normalized momentum sequence of the target video can be obtained, and then the curve of the change of the normalized momentum of the target video over time, that is, the normalized momentum curve, can be obtained. Figure 3 , which is a schematic diagram of a normalized momentum curve provided in an embodiment of the present application.
[0069] In this embodiment, after determining the first motion momentum of each video frame of the target video, the first motion momentum of each video frame is smoothed to obtain the second motion momentum of each video frame, and the normalized momentum of each video frame can be determined based on the first motion momentum and the second motion momentum to obtain the normalized momentum information of the target video. In this way, the video quality of the target video can be evaluated based on the normalized momentum information of the target video to improve the accuracy of the evaluation.
[0070] In some embodiments, before determining the video quality assessment result of the target video according to the video playback parameters of the target video, the method may also include: determining a first frame number of valid video frames between the first video frame and the second video frame; determining perceived frame rate information according to the first frame number and the interval time; wherein the first video frame and the second video frame are two video frames in the target video with an interval time greater than a preset time, and the valid video frame is a video frame in the target video whose first motion momentum is not zero.
[0071] In this embodiment, the valid video frame may be a video frame that has changed relative to a previous video frame, that is, the valid video frame is not a completely still frame.
[0072] The perceived frame rate information may be a perceived frame rate sequence. The perceived frame rate sequence may be a perceived frame rate time sequence. That is, according to the time sequence of the video frames of the target video, a plurality of perceived frame rates are measured in sequence based on a preset time, thereby obtaining the perceived frame rate sequence of the target video. Optionally, the preset time may be 0.2s.
[0073] Exemplarily, the perceived frame rate may be calculated by the following formula (5):
[0074]
[0075] Among them, FPS j is the perceived frame rate of the target video at time j; (ti -t j ) is the time interval between the second video frame (the next video frame) and the first video frame (the previous video frame); t i is the timestamp (i moment) corresponding to the second video frame (the next video frame); t j is the timestamp (time j) corresponding to the first video frame (previous video frame); is the first frame number of the valid video frame between the first video frame and the second video frame; Δt is the preset time.
[0076] Taking the preset time of 0.2s as an example, the process of determining the perceived frame rate sequence of the target video is described. Specifically, based on the timestamp of each video frame of the target video, starting from the first video frame, search for video frames with an interval time greater than the preset time (0.2s), and determine the number of valid video frames between the first video frame and the found video frame based on the first motion momentum (actual motion momentum) of each video frame, and determine a perceived frame rate according to the above formula (5); then, using the found video frame as the starting moment, continue to search for video frames with an interval time greater than the preset time (0.2s), and determine the number of valid video frames between the next group of video frames with an interval time greater than the preset time based on the first motion momentum (actual motion momentum) of each video frame, and determine the next perceived frame rate according to the above formula (5), until each video frame of the target video is traversed. Based on the above steps, the perceived frame rate sequence of the target video can be obtained, that is, the curve of the perceived frame rate of the target video changing over time can be obtained. Please refer to Figure 4 , which is a schematic diagram of a perceived frame rate curve provided in an embodiment of the present application.
[0077] In this embodiment, by counting the number of valid video frames between two video frames with an interval time greater than a preset time, the perceived frame rate can be determined based on the frame number and the interval time, and then the perceived frame rate sequence of the target video can be obtained, that is, the perceived frame rate information is obtained, and whether the target video has a perceived frame rate drop can be determined based on the perceived frame rate information of the target video, so as to measure the degree of frame drop of the target video.
[0078] In some embodiments, the video quality assessment result includes at least one of freeze information, perceived frame rate variance, normalized momentum variance, and smoothness.
[0079] The freeze information of the target video may include a freeze position and a freeze type. The freeze position may be a time point when the target video freezes. According to the severity of the video freeze, the freeze type may include a first type of freeze (severe freeze), a second type of freeze (relatively freeze), and a third type of freeze (slight freeze).
[0080] Generally, when a video drops frames, it will cause the video to freeze. When a video skips frames, it will also cause the video to freeze. Based on this, the perceptual frame rate variance can be used to measure the overall frame drop degree of the target video. The normalized momentum variance can be used to measure the overall frame skip degree of the target video.
[0081] The smoothness can comprehensively reflect the degree of freeze, frame drop and frame skipping of the target video.
[0082] The following is an explanation of the process of determining the jamming information of the target video.
[0083] In some embodiments, the video quality assessment result is stuttering information. The video quality assessment result of the target video is determined based on the video playback parameters of the target video, including: when the frame rate value corresponding to the average frame rate information is greater than or equal to a preset frame rate threshold, the stuttering information of the target video is determined based on the perceived frame rate information and the normalized momentum information; when the frame rate value corresponding to the average frame rate information is less than the preset frame rate threshold, it is determined that the target video has a first type of stuttering.
[0084] In this embodiment, the average frame rate information may include the average frame rate of the target video. The average frame rate of the target video refers to the number of frames displayed per second by the target video. The average frame rate is a parameter to measure the smoothness of the video. The principle of jam detection based on the average frame rate is to set a preset frame rate threshold according to the frame rate of the human eye to distinguish between pictures and videos, as a reference threshold for video jam detection. When the frame rate value corresponding to the average frame rate information (the average frame rate of the target video) is less than the preset frame rate threshold, it means that the human eye can feel that the video is played one by one, and it is considered that the target video has a first type of jam, that is, the target video is severely jammed. When the frame rate value corresponding to the average frame rate information (the average frame rate of the target video) is greater than or equal to the preset frame rate threshold, it is necessary to further determine the jam information of the target video based on the perceived frame rate information and the normalized momentum information, that is, according to the perceived frame rate sequence of the target video and the normalized momentum of each video frame, determine the jam information of the target video.
[0085] In this embodiment, when determining the jamming information of the target video, the average frame rate of the target video can be compared with the preset frame rate threshold. When the average frame rate is less than the preset frame rate threshold, it is directly determined that the target video has the first type of jamming, that is, the target video has severe jamming. In this way, when determining the jamming information of the target video, a preliminary screening can be performed based on the average frame rate of the target video, so that the target video whose average frame rate does not meet the requirements can be directly evaluated as severe jamming, thereby reducing the amount of calculation and power consumption.
[0086] In some embodiments, the freeze information of the target video is determined based on the perceived frame rate information and the normalized momentum information, including: when the perceived frame rate information indicates that a first set event has occurred in the target video, the freeze information of the target video is determined based on the perceived frame rate information; when the perceived frame rate information indicates that the first set event has not occurred in the target video, the freeze information of the target video is determined based on the normalized momentum information; wherein the first set event is a drop in the perceived frame rate.
[0087] In this embodiment, the first set event is a drop in the perceived frame rate. The first set event occurs in the target video, that is, a frame drop event occurs in the target video. Specifically, the perceived frame rate information may include a perceived frame rate sequence of the target video, and the first set event may be determined when the target video occurs based on the size of two adjacent perceived frame rates in the perceived frame rate sequence. Figure 4 For example, the target video may experience multiple instances of perceived frame rate drops. For this, the freeze information for each first set event may be determined.
[0088] In this embodiment, the jamming of the target video may be caused by frame drops or frame skipping. In this regard, when evaluating the jamming information of the target video, it is possible to determine whether the target video has a frame drop event based on the perceived frame rate information, and when the target video has frame drops, the jamming information of the target video can be determined based on the perceived frame rate information. When the target video does not have frame drops, it is further possible to determine whether the target video has a frame skip event based on the normalized momentum information of the target video, and when the target video has frame skipping, the jamming information of the target video can be determined based on the normalized momentum information of the target video. In this way, the jamming information of the target video is determined in combination with the perceived frame rate information and the normalized momentum information of the target video, which can avoid the influence of different degrees of frame drops and frame skipping on the user's subjective feelings in the time dimension, and thus objective and accurate evaluation results can be obtained for videos of different scenes and different shooting frame rates. Furthermore, the jamming information of the target video can be determined as a whole, and the jamming anomalies that can be perceived by the human eye can be quickly filtered.
[0089] In some embodiments, the freeze information includes freeze type and freeze position. Determining the freeze information of the target video based on the perceived frame rate information may further include: steps 3100 to 3300.
[0090] Step 3100: Determine the duration of the first set event according to the perceived frame rate information.
[0091] The duration of the first set event can measure the duration of a freeze caused by a frame drop in the target video.
[0092] Optionally, the perceived frame rate information includes adjacent first perceived frame rate and second perceived frame rate, and the duration of the first set event is determined based on the perceived frame rate information, including: when the second perceived frame rate is less than the first perceived frame rate, the duration of the first set event is determined based on the first perceived frame rate, the second perceived frame rate, the preset time, and the average perceived frame rate.
[0093] The average perceived frame rate may be determined based on perceived frame rate information of the target video, wherein the perceived frame rate information is a perceived frame rate sequence of the target video.
[0094] Exemplarily, taking the perceived frame rate information as a perceived frame rate sequence as an example, the duration of the first set event can be calculated by the following formula (6):
[0095] T down =Δt*(FPS j-1 -FPS j ) / average(FPS) (6)
[0096] Among them, T down The duration of a first set event; FPS j-1 First perceived frame rate; FPS j is the second perceived frame rate; average(FPS) is the average perceived frame rate of the target video; Δt is the preset time, for example, 0.2s.
[0097] In this embodiment, the duration of the first set event can be accurately obtained, so that the degree of freeze of the target video can be evaluated based on the duration of the first set event.
[0098] After step 3100, step 3200 is executed to determine that a first type of freeze occurs in the target video when the duration is greater than or equal to the first preset duration, and to determine a freeze position corresponding to the first type of freeze.
[0099] Step 3300, when the duration is less than the first preset duration and greater than or equal to the second preset duration, determine that a second set event occurs in the target video, and determine the freeze information of the target video according to the number of times the second set event occurs.
[0100] In this embodiment, the freeze position may be a position where the first set event occurs in the target video. Specifically, the freeze position may be a playback time point corresponding to the minimum perceived frame rate corresponding to the first set event.
[0101] In this embodiment, by setting the first preset duration and the second preset duration, the severity of the jamming caused by a frame drop in the target video can be measured. Specifically, if the duration of the first set event is greater than or equal to the first preset duration, it means that the jamming caused by a frame drop is relatively long, and it is considered that the target video has a first type of jamming, that is, severe jamming; if the duration of the first set event is less than the first preset duration, and the duration of the first set event is greater than or equal to the second preset time, it means that the jamming caused by a frame drop is relatively short. At this time, it is determined that the target video has a second set event, and then the jamming information of the target video is determined according to the number of occurrences of the second set event in the target video. Among them, the second set event can include the situation that the duration of the first set event is less than the first preset duration, and the duration is greater than or equal to the second preset duration, and the normalized momentum is less than the first preset amplitude, and the normalized momentum is greater than or equal to the second preset amplitude.
[0102] In one embodiment, after determining the duration of the first set event according to the perceived frame rate information, the method further includes: when the duration is less than a second preset duration, determining that the target video has no obvious freeze.
[0103] In this embodiment, by setting the first preset duration and the second preset duration to measure the jamming degree of the target video, the jamming degree of the target video can be finely quantified, and the accuracy of the video quality assessment can be improved. In addition, the jamming position and jamming type of the target video can be obtained, so that the user can intuitively view the jamming situation of the target video.
[0104] In some embodiments, the freeze information includes a freeze type and a freeze position, the normalized momentum information includes a normalized momentum of each video frame of the target video, and the freeze information of the target video is determined based on the normalized momentum information, including: when the normalized momentum is greater than or equal to a first preset amplitude, determining that a first type of freeze occurs in the target video, and determining the freeze position corresponding to the first type of freeze; when the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to a second preset amplitude, determining that a second set event occurs in the target video, and determining the freeze information of the target video based on the number of times the second set event occurs.
[0105] In this embodiment, by setting the first preset amplitude and the second preset amplitude, the severity of the target video caused by a frame skipping can be measured. Specifically, if the normalized momentum of the video frame is greater than or equal to the first preset amplitude, it means that the video frame has a large change amplitude relative to the previous video frame, and it is considered that a serious frame skipping has occurred in the target video, and it is determined that the target video has a first type of stuttering, that is, a serious stuttering. Among them, the stuttering position of this first type of stuttering is the time point corresponding to the video frame. If the normalized momentum of the video frame is less than the first preset amplitude and the normalized momentum of the video frame is greater than or equal to the second preset amplitude, it means that the video frame has a small change amplitude relative to the previous video frame, and it is considered that a frame skipping has occurred in the target video. At this time, it is determined that a second set event has occurred in the target video, and then the stuttering information of the target video is determined according to the number of times the second set event occurs in the target video. Among them, the second set event may include the situation that the duration of the first set event is less than the first preset duration and the duration is greater than or equal to the second preset duration, and the situation that the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to the second preset amplitude.
[0106] In one embodiment, the step of determining the jamming information of the target video according to the normalized momentum of each video frame of the target video further includes: determining that the target video has no obvious jamming when the normalized momentum is less than a second preset amplitude.
[0107] In this embodiment, by setting the first preset amplitude and the second preset amplitude, determining whether the target video has frame skipping, and measuring the degree of the target video's freeze according to the degree of the target video's frame skipping, the degree of the target video's freeze can be finely quantified, and the accuracy of the video quality assessment can be improved. In addition, the freeze position and freeze type of the target video can be obtained, so that the user can intuitively view the freeze situation of the target video.
[0108] In some embodiments, the freeze information of the target video is determined based on the number of times a second set event occurs, including: when the number of times the second set event occurs is greater than the first preset number of times, determining that a second type of freeze occurs in the target video, and determining the freeze position corresponding to the second type of freeze; when the number of times the second set event occurs is less than the first preset number of times, determining that a third type of freeze occurs in the target video, and determining the freeze position corresponding to the third type of freeze.
[0109] In this embodiment, the second set event may include a situation where the duration of the first set event is less than the first preset duration and the duration is greater than or equal to the second preset duration, and a situation where the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to the second preset amplitude. The second set event occurs in the target video, that is, the target video has a slight freeze. Further, according to the number of times the target video has a slight freeze, it can be evaluated whether the target video has a second type of freeze, that is, the degree of freeze in the target video is more serious.
[0110] See also Figure 5 ,The process of determining the jam information of the target video is illustrated below with a specific example.
[0111] Step 501, obtaining the average frame rate of the target video;
[0112] Step 502, determining whether the average frame rate of the target video is less than a preset frame rate threshold, if yes, executing step 503; otherwise, executing step 504;
[0113] Step 503, determining that the target video is severely stuck;
[0114] Step 504 , judging whether the target video has a first set event according to the perceived frame rate information of the target video, if yes, executing step 505 , otherwise, executing step 509 .
[0115] The following steps 505 to 508 are steps for determining the freeze information of the target video according to the duration of the first set event.
[0116] Step 505, obtaining the duration of the first set event;
[0117] Step 506, determining whether the duration of the first set event is greater than or equal to the first preset duration, if yes, executing step 507, otherwise, executing step 508;
[0118] Step 507, determining that a severe freeze occurs in the target video, and determining a freeze position corresponding to the severe freeze;
[0119] Step 508, determining whether the duration of the first set event is greater than or equal to the second preset duration, if yes, executing step 513, otherwise, executing step 517;
[0120] The above steps 505 to 508 are repeated until the perceived frame rate information (perceived frame rate sequence) of the target video is traversed.
[0121] The steps for determining the jamming information of the target video according to the normalized momentum information of the target video are as follows.
[0122] Step 509, obtaining the normalized momentum of each video frame in the target video;
[0123] Step 510, determining whether the normalized momentum is greater than or equal to a first preset amplitude, if yes, executing step 511, otherwise, executing step 512;
[0124] Step 511, determining that a severe freeze occurs in the target video, and determining a freeze position corresponding to the severe freeze;
[0125] Step 512, determining whether the normalized momentum is greater than or equal to a second preset amplitude, if yes, executing step 513, otherwise, executing step 517;
[0126] Step 513: determine whether a second set event occurs in the target video, and record the number of times the second set event occurs.
[0127] Step 514, determining whether the number of the second set event is greater than or equal to the first preset number, if yes, executing step 515, otherwise, executing step 516;
[0128] Step 515, determining that the target video is relatively stuck;
[0129] Step 516, determining that the target video is slightly stuck;
[0130] Step 517, determine that the target video has no obvious freeze.
[0131] The above steps 510 to 513 are repeated until the normalized momentum of each video frame of the target video is traversed, thereby completing the evaluation of the overall jamming degree of the target video.
[0132] In this embodiment, the degree of frame drop and frame skipping of the target video can be measured based on the perceived frame rate information and normalized momentum information of the target video. Furthermore, based on the duration of the first set event, the number of frame drops, the degree of single frame drop, the number of frame skips, and the degree of single frame skipping, the influence of different degrees of frame drop and frame skipping on the user's subjective experience in the time dimension can be avoided, thereby accurately measuring the degree of freeze in the video shot in different scenes. Furthermore, the refined evaluation of the video can be achieved.
[0133] In some embodiments, the video quality assessment result is a perceived frame rate variance. The video quality assessment result of the target video is determined based on the video playback parameters of the target video, including: determining and outputting the perceived frame rate variance based on the perceived frame rate information; wherein the perceived frame rate variance is used to evaluate the degree of frame drop of the target video.
[0134] In this embodiment, the perceived frame rate variance of the target video may also be output to achieve objective quantification of the frame drop degree of the target video.
[0135] In some embodiments, the video quality assessment result is a normalized momentum variance. The video quality assessment result of the target video is determined according to the video playback parameters of the target video, including: determining and outputting the normalized momentum variance according to the normalized momentum of each video frame of the target video, wherein the normalized momentum variance is used to evaluate the degree of frame skipping of the target video.
[0136] In this embodiment, the normalized momentum variance of the target video may also be output to achieve objective quantification of the frame skipping degree of the target video.
[0137] In some embodiments, the video quality assessment result is a degree of smoothness. The video quality assessment result of the target video is determined according to the video playback parameters of the target video, including: determining and outputting the perceived frame rate variance according to the perceived frame rate sequence; determining and outputting the normalized momentum variance according to the normalized momentum of each video frame of the target video; determining and outputting the degree of smoothness of the target video according to the perceived frame rate variance and the normalized momentum variance.
[0138] Exemplarily, the smoothness of the target video can be calculated by the following formula (7):
[0139]
[0140] Among them, Score is the smoothness of the target video; FPS variance is the perceptual frame rate variance of the target video; Motion variance is the normalized momentum variance of the target video.
[0141] In this embodiment, the obtained video quality assessment results include freeze information, perceived frame rate variance, normalized momentum variance and smoothness, so that the objective quantification of the frame drop degree, frame skipping degree and freeze degree of the target video can be achieved, which is convenient for users to evaluate the quality of the video according to the video quality assessment results, and then realize the testing of the video processing algorithm, hardware and shooting performance of the electronic device.
[0142] In some embodiments, determining a video quality assessment result of a target video according to video playback parameters of the target video includes: inputting the video playback parameters of the target video into a pre-trained video quality assessment model to obtain a video quality assessment result of the target video.
[0143] The video quality assessment model is obtained by training the first video and the video playback parameters of the first video, and the first video is a video whose video quality assessment result has been determined. The video playback parameters of the first video may include perceived frame rate information, average frame rate information, and normalized momentum information.
[0144] Exemplarily, for a video (first video) whose video quality assessment result has been determined, the perceived frame rate information and normalized momentum information of the first video are annotated, and the annotated perceived frame rate information and normalized momentum information are collected for model training to obtain a video quality assessment model. Afterwards, when performing video quality assessment on a target video, the perceived frame rate information and normalized momentum information of the target video are obtained, and the perceived frame rate information and normalized momentum information are input into the video quality assessment model to obtain a video quality assessment result of the target video.
[0145] In this embodiment, based on the pre-trained video quality assessment model, a video quality assessment result of the target video is obtained, which has better robustness to special data and can meet the needs of different scenarios.
[0146] In an embodiment of the present application, a target video is obtained, and a video quality evaluation result of the target video is determined according to the video playback parameters of the target video, wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information. In this way, by combining the perceived frame rate information and the normalized momentum information to evaluate the video quality of the target video, the influence of different degrees of frame drops and frame skipping on the user's subjective feelings in the time dimension can be eliminated, while covering the playback anomalies caused by frame drops, discontinuous frame intervals, and consistent multi-frame results, and objective and accurate video quality evaluation results can be obtained. In addition, the video quality evaluation method provided in this embodiment has high accuracy and can be used to evaluate the video quality of videos captured in different scenes. Furthermore, the quality of the video of the continuous scene can be measured as a whole, and the stuttering anomalies that can be perceived by the human eye can be quickly filtered. In addition, objective quantitative results can be obtained, which is convenient for evaluating the video processing algorithm, hardware, and shooting performance of electronic devices using the video quality evaluation results.
[0147] The video quality assessment method provided in the embodiment of the present application may be performed by a video quality assessment device. In the embodiment of the present application, a method for performing video quality assessment by a video quality assessment device is taken as an example to illustrate the video quality assessment device provided in the embodiment of the present application.
[0148] See also Figure 6 , which is a hardware structure diagram of a video quality assessment device provided in an embodiment of the present application. The video quality assessment device 600 includes a first acquisition module 601 and a first determination module 602.
[0149] The first acquisition module 601 is used to acquire a target video.
[0150] The first determination module 602 is used to determine the video quality evaluation result of the target video according to the video playback parameters of the target video.
[0151] Among them, the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information; the perceived frame rate information is used to characterize the number of valid video frames perceived by the human eye in the target video; the average frame rate information is used to characterize the smoothness of the target video; the normalized momentum information is used to characterize the change amplitude of at least one video frame of the target video relative to the previous video frame.
[0152] In some embodiments, the first determination module 602 is specifically used to input the video playback parameters of the target video into a pre-trained video quality assessment model to obtain a video quality assessment result of the target video; wherein the video quality assessment model is obtained by training a first video and the video playback parameters of the first video, and the first video is a video whose video quality assessment result has been determined.
[0153] In some embodiments, the video quality assessment result is stuttering information, and the first determination module 602 is specifically used to: when the frame rate value corresponding to the average frame rate information is greater than or equal to a preset frame rate threshold, determine the stuttering information of the target video based on the perceived frame rate information and the normalized momentum information; when the frame rate value corresponding to the average frame rate information is less than the preset frame rate threshold, determine that the target video has a first type of stuttering.
[0154] In some embodiments, when determining the stuttering information of the target video according to the perceived frame rate information and the normalized momentum information, the first determination module 602 includes: a first determination unit, used to: when the perceived frame rate information indicates that a first set event has occurred in the target video, determine the stuttering information of the target video according to the perceived frame rate information; a second determination unit, used to: when the perceived frame rate information indicates that the first set event has not occurred in the target video, determine the stuttering information of the target video according to the normalized momentum information; wherein the first set event is a drop in perceived frame rate.
[0155] In some embodiments, the freeze information includes a freeze type and a freeze position, and the first determination unit is specifically used to: determine the duration of the first set event based on the perceived frame rate information; determine that a first type of freeze has occurred in the target video when the duration is greater than or equal to a first preset duration, and determine the freeze position corresponding to the first type of freeze; determine that a second set event has occurred in the target video when the duration is less than the first preset duration and the duration is greater than or equal to a second preset duration, and determine the freeze information of the target video based on the number of times the second set event occurs.
[0156] In some embodiments, the freeze information includes a freeze type and a freeze position, the normalized momentum information includes a normalized momentum of each video frame of the target video, and the second determination unit is specifically used to: when the normalized momentum is greater than or equal to a first preset amplitude, determine that a first type of freeze occurs in the target video, and determine the freeze position corresponding to the first type of freeze; when the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to a second preset amplitude, determine that a second set event occurs in the target video, and determine the freeze information of the target video according to the number of times the second set event occurs.
[0157] In some embodiments, determining the freeze information of the target video based on the number of times the second set event occurs includes: when the number of times the second set event occurs is greater than the first preset number of times, determining that the target video has a second type of freeze, and determining the freeze position corresponding to the second type of freeze; when the number of times the second set event occurs is less than the first preset number of times, determining that the target video has a third type of freeze, and determining the freeze position corresponding to the third type of freeze.
[0158] In some embodiments, the video quality assessment result is a degree of smoothness, and the first determination module 602 is further used to: determine and output a perceived frame rate variance based on the perceived frame rate sequence; determine and output a normalized momentum variance based on the normalized momentum of each video frame of the target video; determine and output the degree of smoothness of the target video based on the perceived frame rate variance and the normalized momentum variance; wherein the perceived frame rate variance is used to evaluate the degree of frame drop of the target video; and the normalized momentum variance is used to evaluate the degree of frame skipping of the target video.
[0159] In some embodiments, the video quality assessment device 600 also includes a second acquisition module, which is used to: acquire a first motion momentum of each video frame of the target video to obtain a first motion momentum sequence; smooth the first motion momentum sequence to obtain a second motion momentum sequence, wherein the second motion momentum sequence includes a second motion momentum corresponding to the first motion momentum of each video frame; determine the normalized momentum of each video frame based on the first motion momentum and the second motion momentum of each video frame to obtain the normalized momentum information of the target video.
[0160] In some embodiments, the video quality assessment device 600 also includes a second determination module, which is used to: determine a first frame number of valid video frames between the first video frame and the second video frame; determine the perceived frame rate information based on the first frame number and the interval time; wherein the first video frame and the second video frame are two video frames in the target video whose interval time is greater than the preset time, and the valid video frame is a video frame in the target video whose first motion momentum is not zero.
[0161] In an embodiment of the present application, a video quality assessment result of a target video is obtained based on a pre-trained video quality assessment model, which has better robustness to special data and can meet the needs of different scenarios.
[0162] In an embodiment of the present application, a target video is obtained, and a video quality evaluation result of the target video is determined according to the video playback parameters of the target video, wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information. In this way, by combining the perceived frame rate information and the normalized momentum information to evaluate the video quality of the target video, the influence of different degrees of frame drops and frame skipping on the user's subjective feelings in the time dimension can be eliminated, while covering the playback anomalies caused by frame drops, discontinuous frame intervals, and consistent multi-frame results, and objective and accurate video quality evaluation results can be obtained. In addition, the video quality evaluation method provided in this embodiment has high accuracy and can be used to evaluate the video quality of videos captured in different scenes. Furthermore, the quality of the video of the continuous scene can be measured as a whole, and the stuttering anomalies that can be perceived by the human eye can be quickly filtered. In addition, objective quantitative results can be obtained, which is convenient for evaluating the video processing algorithm, hardware, and shooting performance of electronic devices using the video quality evaluation results.
[0163] The video quality assessment device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0164] The video quality assessment device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0165] The video quality evaluation device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0166] Alternatively, if Figure 7 As shown, an embodiment of the present application further provides an electronic device 700, including a processor 701 and a memory 702, wherein the memory 702 stores a program or instruction that can be executed on the processor 701, and when the program or instruction is executed by the processor 701, each step of the above-mentioned video quality assessment method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0167] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0168] The electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810 and other components.
[0169] Those skilled in the art will appreciate that the electronic device 800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0170] Among them, the processor 810 is used to: obtain a target video; determine a video quality assessment result of the target video according to the video playback parameters of the target video; wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information; the perceived frame rate information is used to characterize the number of valid video frames perceived by the human eye in the target video; the average frame rate information is used to characterize the smoothness of the target video; the normalized momentum information is used to characterize the change amplitude of at least one video frame of the target video relative to the previous video frame.
[0171] Optionally, when determining the video quality assessment result of the target video based on the video playback parameters of the target video, the processor 810 is used to: input the video playback parameters of the target video into a pre-trained video quality assessment model to obtain the video quality assessment result of the target video; wherein the video quality assessment model is trained by a first video and the video playback parameters of the first video, and the first video is a video for which the video quality assessment result has been determined.
[0172] Optionally, the video quality assessment result is stuttering information. When the processor 810 determines the video quality assessment result of the target video based on the video playback parameters of the target video, it is used to: determine the stuttering information of the target video based on the perceived frame rate information and the normalized momentum information when the frame rate value corresponding to the average frame rate information is greater than or equal to a preset frame rate threshold; and determine that the target video has a first type of stuttering when the frame rate value corresponding to the average frame rate information is less than a preset frame rate threshold.
[0173] Optionally, when the processor 810 determines the freeze information of the target video based on the perceived frame rate information and the normalized momentum information, it is used to: determine the freeze information of the target video based on the perceived frame rate information when the perceived frame rate information indicates that a first set event has occurred in the target video; and determine the freeze information of the target video based on the normalized momentum information when the perceived frame rate information indicates that the first set event has not occurred in the target video; wherein the first set event is a drop in perceived frame rate.
[0174] Optionally, the freeze information includes a freeze type and a freeze position. When the processor 810 determines the freeze information of the target video based on the perceived frame rate information, it is used to: determine the duration of the first set event based on the perceived frame rate information; when the duration is greater than or equal to a first preset duration, determine that a first type of freeze has occurred in the target video, and determine the freeze position corresponding to the first type of freeze; when the duration is less than the first preset duration and the duration is greater than or equal to a second preset duration, determine that a second set event has occurred in the target video, and determine the freeze information of the target video based on the number of times the second set event occurs.
[0175] Optionally, the freeze information includes a freeze type and a freeze position, and the normalized momentum information includes a normalized momentum of each video frame of the target video. When the processor 810 determines the freeze information of the target video based on the normalized momentum information, it is used to: when the normalized momentum is greater than or equal to a first preset amplitude, determine that a first type of freeze occurs in the target video, and determine the freeze position corresponding to the first type of freeze; when the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to a second preset amplitude, determine that a second set event occurs in the target video, and determine the freeze information of the target video based on the number of times the second set event occurs.
[0176] Optionally, when determining the freeze information of the target video based on the number of times a second set event occurs, the processor 810 is used to: when the number of times the second set event occurs is greater than a first preset number of times, determine that a second type of freeze occurs in the target video, and determine a freeze position corresponding to the second type of freeze; when the number of times the second set event occurs is less than the first preset number of times, determine that a third type of freeze occurs in the target video, and determine a freeze position corresponding to the third type of freeze.
[0177] Optionally, the video quality assessment result is the degree of smoothness. When the processor 810 determines the video quality assessment result of the target video according to the video playback parameters of the target video, it is used to: determine and output the perceived frame rate variance according to the perceived frame rate sequence; determine and output the normalized momentum variance according to the normalized momentum of each video frame of the target video; determine and output the degree of smoothness of the target video according to the perceived frame rate variance and the normalized momentum variance; wherein the perceived frame rate variance is used to evaluate the degree of frame drop of the target video; and the normalized momentum variance is used to evaluate the degree of frame skipping of the target video.
[0178] Optionally, before determining the video quality assessment result of the target video according to the video playback parameters of the target video, the processor 810 is used to: obtain a first motion momentum of each video frame of the target video to obtain a first motion momentum sequence; smooth the first motion momentum sequence to obtain a second motion momentum sequence, wherein the second motion momentum sequence includes a second motion momentum corresponding to the first motion momentum of each video frame; determine the normalized momentum of each video frame according to the first motion momentum and the second motion momentum of each video frame to obtain the normalized momentum information of the target video.
[0179] Optionally, before determining the video quality assessment result of the target video according to the video playback parameters of the target video, the processor 810 is used to: determine a first frame number of valid video frames between the first video frame and the second video frame; determine the perceived frame rate information according to the first frame number and the interval time; wherein the first video frame and the second video frame are two video frames in the target video whose interval time is greater than the preset time, and the valid video frame is a video frame in the target video whose first motion momentum is not zero.
[0180] In an embodiment of the present application, a target video is obtained, and a video quality evaluation result of the target video is determined according to the video playback parameters of the target video, wherein the video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information. In this way, by combining the perceived frame rate information and the normalized momentum information to evaluate the video quality of the target video, the influence of different degrees of frame drops and frame skipping on the user's subjective feelings in the time dimension can be eliminated, while covering the playback anomalies caused by frame drops, discontinuous frame intervals, and consistent multi-frame results, and objective and accurate video quality evaluation results can be obtained. In addition, the video quality evaluation method provided in this embodiment has high accuracy and can be used to evaluate the video quality of videos captured in different scenes. Furthermore, the quality of the video of the continuous scene can be measured as a whole, and the stuttering anomalies that can be perceived by the human eye can be quickly filtered. In addition, objective quantitative results can be obtained, which is convenient for evaluating the video processing algorithm, hardware, and shooting performance of electronic devices using the video quality evaluation results.
[0181] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and at least one of other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0182] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0183] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 810.
[0184] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned video quality assessment method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0185] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0186] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video quality assessment method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0187] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0188] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned video quality assessment method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0189] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0190] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0191] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for evaluating video quality, characterized in that: The method comprises: Get the target video; Determining a video quality assessment result of the target video according to the video playback parameters of the target video; The video playback parameters include at least one of the following: perceived frame rate information, average frame rate information, and normalized momentum information; the perceived frame rate information is used to characterize the number of valid video frames perceived by human eyes in the target video; the average frame rate information is used to characterize the smoothness of the target video; the normalized momentum information is used to characterize the change amplitude of at least one video frame of the target video relative to the previous video frame; And, in a case where the video quality assessment result is freeze information, determining the video quality assessment result of the target video according to the video playback parameter of the target video includes: When the frame rate value corresponding to the average frame rate information is greater than or equal to a preset frame rate threshold, determining the freeze information of the target video according to the perceived frame rate information and the normalized momentum information; When the frame rate value corresponding to the average frame rate information is less than a preset frame rate threshold, it is determined that the target video has a first type of freeze.
2. The method according to claim 1, characterized in that The step of determining the video quality evaluation result of the target video according to the video playback parameter of the target video includes: Inputting the video playback parameters of the target video into a pre-trained video quality assessment model to obtain a video quality assessment result of the target video; The video quality assessment model is obtained by training the first video and the video playback parameters of the first video, and the first video is a video for which the video quality assessment result has been determined.
3. The method according to claim 1, characterized in that The determining, according to the perceived frame rate information and the normalized momentum information, the jamming information of the target video includes: When the perceived frame rate information indicates that a first set event occurs in the target video, determining freeze information of the target video according to the perceived frame rate information; When the perceived frame rate information indicates that the target video does not have a first set event, determining the freeze information of the target video according to the normalized momentum information; The first setting event is a drop in the perceived frame rate.
4. The method according to claim 3, characterized in that The freeze information includes a freeze type and a freeze position, and determining the freeze information of the target video according to the perceived frame rate information includes: Determining a duration of the first set event according to the perceived frame rate information; When the duration is greater than or equal to a first preset duration, determining that a first type of freeze occurs in the target video, and determining a freeze position corresponding to the first type of freeze; When the duration is less than the first preset duration and the duration is greater than or equal to the second preset duration, it is determined that a second set event occurs in the target video, and the freeze information of the target video is determined according to the number of times the second set event occurs.
5. The method according to claim 3, characterized in that: The freeze information includes a freeze type and a freeze position, the normalized momentum information includes a normalized momentum of each video frame of the target video, and determining the freeze information of the target video according to the normalized momentum information includes: When the normalized momentum is greater than or equal to a first preset amplitude, determining that a first type of freeze occurs in the target video, and determining a freeze position corresponding to the first type of freeze; When the normalized momentum is less than the first preset amplitude and the normalized momentum is greater than or equal to a second preset amplitude, it is determined that a second set event occurs in the target video, and the freeze information of the target video is determined according to the number of times the second set event occurs.
6. The method according to claim 4 or 5, characterized in that: The determining the freeze information of the target video according to the number of occurrences of the second set event includes: When the number of occurrences of the second set event is greater than the first preset number of times, determining that a second type of freeze occurs in the target video, and determining a freeze position corresponding to the second type of freeze; When the number of times the second set event occurs is less than the first preset number of times, it is determined that a third type of freeze occurs in the target video, and a freeze position corresponding to the third type of freeze is determined.
7. The method according to claim 1, characterized in that In a case where the video quality evaluation result is a degree of fluency, determining the video quality evaluation result of the target video according to the video playback parameter of the target video includes: Determining and outputting a perceived frame rate variance according to the perceived frame rate sequence; Determining and outputting a normalized momentum variance according to the normalized momentum of each video frame of the target video; Determining and outputting the smoothness of the target video according to the perceived frame rate variance and the normalized momentum variance; The perceived frame rate variance is used to evaluate the degree of frame drop of the target video; and the normalized momentum variance is used to evaluate the degree of frame skipping of the target video.
8. The method according to claim 1, characterized in that Before determining the video quality evaluation result of the target video according to the video playback parameters of the target video, the method further includes: Acquire a first motion momentum of each video frame of the target video to obtain a first motion momentum sequence; Performing smoothing processing on the first motion momentum sequence to obtain a second motion momentum sequence, wherein the second motion momentum sequence includes a second motion momentum corresponding to the first motion momentum of each video frame; The normalized momentum of each video frame is determined according to the first motion momentum and the second motion momentum of each video frame to obtain the normalized momentum information of the target video.
9. The method according to claim 1, characterized in that: Before determining the video quality evaluation result of the target video according to the video playback parameters of the target video, the method further includes: Determine a first frame number of valid video frames between the first video frame and the second video frame; Determining the perceived frame rate information according to the first number of frames and the interval time; The first video frame and the second video frame are two video frames in the target video whose interval time is greater than a preset time, and the valid video frame is a video frame in the target video whose first motion momentum is not zero.
Citation Information
Patent Citations
Terminal side time-domain video quality evaluation method and apparatus
CN103945214A
Data processing method in video service
CN105472404A
Dynamic control method and device for video quality, storage medium and electronic equipment
CN113438468A
Video quality evaluation method and device, electronic equipment and computer storage medium
CN114584849A
Video quality monitoring
US20120206610A1
