Quality evaluation method and apparatus, and device, storage medium and computer program product

Abstract

Provided in the present disclosure are a quality evaluation method and apparatus, and a device, a storage medium and a computer program product. The method includes: determining the quality of audio and video; determining a first factor, and determining a second factor, wherein both the first factor and the second factor are related to temporal parameters of the audio and the video; and on the basis of the quality of the audio and the video, the first factor and the second factor, determining the time-domain quality of the audio and the video.

Description

Quality assessment methods, apparatus, equipment, storage media, and computer program products

[0001] Cross-references to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 202510127614.4, filed in China on January 27, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of wireless communication technology, and in particular to a quality assessment method, apparatus, device, storage medium, and computer program product. Background Technology

[0004] Currently, interactive latency quality assessment can improve the performance of the entire audio and video transmission system, meet the high-quality audio and video needs of different users in diverse scenarios, and promote the development of the audio and video industry towards a more efficient and higher-quality direction. Audio and video media synchronization quality assessment can not only ensure the user's viewing experience, but also provide strong support for the development of audio and video technology and the dissemination of content. Audio and video temporal quality includes audio and video interactive latency quality and audio and video media synchronization quality. At present, the assessment methods for audio and video temporal quality are not accurate enough, may not maximize the optimization of user experience, and the assessment process is relatively complex. Summary of the Invention

[0005] In view of this, the present disclosure aims to provide a quality assessment method, apparatus, device, storage medium, and computer program product.

[0006] The technical solution of this disclosure embodiment is implemented as follows:

[0007] This disclosure provides a quality assessment method, the method comprising:

[0008] Determine audio and video quality;

[0009] Determine the first factor and the second factor; both the first and second factors are related to the time parameters of the audio and video.

[0010] Based on the audio and video quality, as well as the first and second factors, the temporal quality of the audio and video is determined.

[0011] Furthermore, according to at least one embodiment of this disclosure, determining the first factor includes:

[0012] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0013] The first factor is determined based on the first delay parameter of the audio and the second delay parameter of the video.

[0014] Furthermore, according to at least one embodiment of this disclosure, determining the first factor based on a first delay parameter of the audio and a second delay parameter of the video includes:

[0015] A first operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain the third delay parameter;

[0016] The first factor is determined based on the third delay parameter and the first preset parameter.

[0017] Furthermore, according to at least one embodiment of this disclosure, the first factor is determined according to the following formula:

[0018] Among them, f delay p1 represents the first factor, p1 represents the first preset parameter, and T represents the first preset parameter. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0019] Furthermore, according to at least one embodiment of this disclosure, determining the second factor includes:

[0020] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0021] The second factor is determined based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality.

[0022] Furthermore, according to at least one embodiment of this disclosure, determining the second factor based on a first delay parameter of the audio, a second delay parameter of the video, and the audio / video quality includes:

[0023] If the second delay parameter of the video is greater than or equal to the first delay parameter of the audio, a second operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain a fourth delay parameter;

[0024] The second factor is determined based on the fourth delay parameter, the second preset parameter, the third preset parameter, the fourth preset parameter, and the audio / video quality.

[0025] Furthermore, according to at least one embodiment of this disclosure, determining the second factor based on a first time parameter of the audio, a second time parameter of the video, and the audio / video quality includes:

[0026] If the second delay parameter of the video is less than the first delay parameter of the audio, a third operation is performed on the first delay of the audio and the second delay of the video to obtain a fifth delay parameter;

[0027] The second factor is determined based on the fifth delay parameter, the fifth preset parameter, the sixth preset parameter, the seventh preset parameter, and the audio / video quality.

[0028] Furthermore, according to at least one embodiment of this disclosure, the second factor is determined according to the following formula:

[0029] Among them, f sync p2 represents the second factor, p3 represents the second preset parameter, p4 represents the third preset parameter, and p4 represents the fourth preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0030] Furthermore, according to at least one embodiment of this disclosure, the second factor is determined according to the following formula:

[0031] Among them, f sync p5 represents the second factor, p6 represents the fifth preset parameter, p7 represents the sixth preset parameter, and p7 represents the seventh preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0032] Furthermore, according to at least one embodiment of this disclosure, determining the temporal quality of the audio and video based on the audio and video quality, as well as the first factor and the second factor, includes:

[0033] Based on the audio and video quality and the first factor, the audio and video temporal quality includes the audio and video interaction latency quality.

[0034] Based on the audio and video quality and the second factor, the audio and video temporal quality includes the audio and video media synchronization quality.

[0035] Furthermore, according to at least one embodiment of this disclosure, the audio and video temporal quality, including the audio and video interaction latency quality, is determined according to the following formula: Q delay =(p8+p9Q) av )-p 10 f delay

[0036] Among them, Q delay p8 represents the audio / video interaction latency quality, p9 represents the eighth preset parameter, and p represents the ninth preset parameter. 10 Q represents the tenth preset parameter. av Indicates the audio / video quality, f delay This refers to the first factor.

[0037] Furthermore, according to at least one embodiment of this disclosure, the audio and video temporal quality, including the audio and video media synchronization quality, is determined according to the following formula: Q sync =(p 11 +p 12 Q av )-p 13 f sync

[0038] Among them, Q sync p represents the synchronization quality of the audio and video media. 11 p represents the eleventh preset parameter. 12 p represents the twelfth preset parameter. 13 This represents the thirteenth preset parameter, Q. av Indicates the audio / video quality, f sync This refers to the second factor.

[0039] At least one embodiment of this disclosure provides a quality assessment apparatus, comprising:

[0040] The first processing module is used to determine the audio and video quality;

[0041] The second processing module is used to determine a first factor and a second factor; both the first factor and the second factor are related to the time parameters of the audio and video; based on the audio and video quality, as well as the first factor and the second factor, the audio and video temporal quality is determined.

[0042] At least one embodiment of this disclosure provides an electronic device, including a processor and a memory for storing a computer program capable of running on the processor.

[0043] When the processor runs the computer program, it executes the steps of any one of the methods described in the above-described electronic device.

[0044] At least one embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the above-described electronic device.

[0045] At least one embodiment of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any one of the above-described electronic devices.

[0046] The quality assessment method, apparatus, device, storage medium, and computer program product provided in this disclosure include: determining audio and video quality; determining a first factor and a second factor; both the first factor and the second factor are related to the time parameters of the audio and video; and determining the audio and video temporal quality based on the audio and video quality, as well as the first factor and the second factor.

[0047] The technical solution provided by this disclosure takes into account both time and audio / video quality when evaluating audio / video temporal quality. Since it is consistent with the user's actual experience, the evaluated audio / video temporal quality can be used to maximize and optimize the user experience. Moreover, no model is required, and the evaluation process is relatively simple to implement. Attached Figure Description

[0048] Figure 1 is a schematic diagram of the implementation of joint video and audio quality evaluation in related technologies;

[0049] Figure 2 is a schematic diagram of how audio and video synchronization is achieved in related technologies;

[0050] Figure 3 is a schematic diagram of the implementation process of the quality assessment method according to an embodiment of this disclosure;

[0051] Figure 4 is a schematic diagram of the specific implementation process of the quality assessment method according to an embodiment of this disclosure;

[0052] Figure 5 is a schematic diagram of the composition structure of the quality assessment device according to an embodiment of this disclosure;

[0053] Figure 6 is a schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0054] Before introducing the technical solutions of the embodiments of this disclosure, the audio and video interaction latency quality and audio and video media synchronization quality involved in this disclosure, as well as related technologies, will be introduced first.

[0055] Audio and video interaction latency quality assessment. In real-time communication scenarios such as video calls, video conferencing, and online education, low latency ensures the instant transmission of information, enabling participants to communicate smoothly and avoiding communication breakdowns and delayed feedback caused by latency, thereby improving collaboration efficiency and the learning experience. For live video streaming, accurate latency assessment helps optimize the live streaming process, ensuring that viewers can watch exciting content such as events and performances in real time, enhancing interactivity and engagement. In the video-on-demand field, reasonable latency control can reduce buffering time, allowing users to quickly enter the viewing state and enjoy a smooth visual experience. In addition, interaction latency quality assessment can also provide data support for network optimization and codec technology improvements, helping to improve the performance of the entire audio and video transmission system, meeting the high-quality audio and video needs of different users in diverse scenarios, and driving the audio and video industry towards a more efficient and higher-quality direction.

[0056] Audio and video media synchronization quality assessment has several key benefits. First, it significantly improves user experience. For example, when watching movies or making video calls, synchronization issues can cause a mismatch between the picture and sound, leading to user discomfort. By assessing and optimizing synchronization quality, a smooth and natural audiovisual experience can be ensured. Second, the assessment results can guide technological optimization. For instance, deep learning-based audio and video quality assessment algorithms have been developed by combining subjective and objective assessment methods. These algorithms not only improve the accuracy of assessments but also provide data support for technological research and development. Furthermore, synchronization quality assessment is crucial for content providers, helping to identify and fix problems before video delivery, preventing user churn due to quality issues. Audio and video media synchronization quality assessment not only safeguards the user's viewing experience but also provides strong support for the development of audio and video technology and the dissemination of content.

[0057] Among related technologies, technical solutions for evaluating audio-visual interaction latency quality and audio-visual media synchronization quality include:

[0058] There are various technical solutions for assessing the latency and quality of audio-visual interactions, covering subjective assessment, objective assessment, and methods combining both. Subjective assessment methods include the Double Stimulus Continuous Quality Scale (DSCQS), the Double Stimulus Impairment Scale (DSIS), the Single Stimulus Continuous Quality Evaluation (SSCQE), and the Paired Comparison (PC) method. These methods assess quality by having subjects rate the original reference video and the distorted video. Objective assessment mainly relies on mathematical models, which are divided into full-reference, partial-reference, and no-reference models. Among them, full-reference assessment is the most mature and suitable for assessing the quality and performance efficiency of coding technologies. No-reference quality assessment methods, because they can be evaluated directly without additional information, are suitable for online monitoring or quality assessment where it is inconvenient to control the source input content, and are currently a research hotspot.

[0059] Technical solutions for assessing the synchronization quality of audio and video media also include two methods: subjective assessment and objective assessment. Subjective assessment relies on direct feedback from human observers, evaluating synchronization quality by having users watch and listen to audio and video content and rate it. While accurate, this method is costly, time-consuming, and difficult to implement on a large scale. Objective assessment, on the other hand, uses automated algorithms to analyze audio and video data and assess synchronization quality. Common objective assessment methods include timestamp-based synchronization error calculation and audio and video feature matching.

[0060] Currently, methods for assessing audio-visual interaction latency quality and audio-visual media synchronization quality suffer from the following problems: First, most current methods only consider the time factor when assessing latency and synchronization, neglecting audio-visual quality, which deviates from the actual user experience. Second, current methods rely heavily on neural networks or higher-order functions to calculate latency and synchronization quality, resulting in significant computational overhead and poor robustness. Third, current methods employ different models or calculation steps for latency and synchronization quality calculations, failing to adopt a unified approach and increasing the complexity of quality assessment.

[0061] Referring to Figure 1, which is a schematic diagram of video and audio joint quality evaluation in related technologies, as shown in Figure 1, a video and audio joint quality evaluation method is disclosed in patent application No. 202010171567.0, entitled "A Method and Apparatus for Video and Audio Joint Quality Evaluation." This method includes: predicting video quality using a video quality evaluation model to obtain a video predicted quality score Qv and a video predicted quality feature fv; predicting audio quality using an audio quality evaluation model to obtain an audio predicted quality score Qa and an audio predicted quality feature fa; normalizing the video predicted quality score Qv and the audio predicted quality score Qa respectively to obtain normalized video predicted quality scores and audio predicted quality scores; fusing the normalized video predicted quality scores and audio predicted quality scores, or fusing the video predicted quality feature fv and the audio predicted quality feature fa, to obtain the predicted video and audio joint quality score Qav or the feature fav. A video and audio joint quality evaluation apparatus is also provided. Using the method and apparatus provided in this disclosure, the overall experience quality of audio and video can be effectively evaluated.

[0062] Referring to Figure 2, which is a schematic diagram of audio and video synchronization in related technologies, as shown in Figure 2, a patent application with application number 202411160775.5, entitled "An Audio and Video Signal Processing System and a Video Conferencing Terminal Device Using the Same," discloses an audio and video signal processing system and a video conferencing terminal device using the same, relating to the field of Internet technology. The system includes: an audio and video processing module, a signal preprocessing module, a signal delay measurement module, a signal synchronization processing module, and a main control module. The audio and video processing module separates the audio and video signals in the input audio and video data signals. The signal preprocessing module preprocesses the audio signal. The signal delay measurement module measures the relative delay between the audio and video signals. The signal synchronization processing module performs delay compensation based on the relative delay, achieving synchronization of audio and video display. The video conferencing terminal device also includes a microphone, a camera, a display screen, and speakers. This system and video conferencing terminal device can test the relative delay time of audio and video signals and perform delay compensation, ensuring synchronization of the output audio and video images, improving video conferencing quality and user experience.

[0063] The aforementioned patent application with application number 202010171567.0, entitled "A Method and Apparatus for Joint Video and Audio Quality Evaluation," proposes a method and apparatus for joint video and audio quality evaluation. This method first describes the experience quality of each of the audio and video modalities separately, then processes the predicted quality of the two modalities and performs perceptual quality fusion to construct a joint video and audio quality evaluation model. It employs a reference-based method and does not involve time-dependent factors.

[0064] The aforementioned patent application with application number 202411160775.5, entitled "An Audio and Video Signal Processing System and a Video Conferencing Terminal Device Using the Same," provides an audio and video signal processing system and a video conferencing terminal device using the same. This system and terminal device can test the relative delay time of audio and video signals and perform delay compensation to ensure synchronization of the output audio and video, thereby improving video conferencing quality and user experience. The proposed signal delay measurement module measures the relative delay between the audio and video signals, and the signal synchronization processing module performs delay compensation based on the relative delay measured by the signal delay measurement module, achieving synchronization of audio and video display. The signal synchronization processing module generates a delay compensation time and a timestamp based on the relative delay measured by the signal delay measurement module. When the system time reaches the timestamp, the audio signal is synchronously played in the audio playback device, and the video signal is synchronously played on the display terminal such as the screen, achieving synchronization of audio playback and video display. The delay compensation time varies depending on the audio transmission method. After delay compensation, the audio signal is amplified by a power amplifier unit and then played on audio playback devices such as speakers.

[0065] In summary, the solutions in the relevant technologies have the following shortcomings:

[0066] First, the time factor was not considered when evaluating audio and video quality. The method of evaluating audio and video synchronization based on timestamps only compensates for delays and does not calculate user experience-related interaction latency quality or media synchronization quality.

[0067] Second, current methods use different models or calculation steps for delay quality calculation and synchronization quality calculation, which cannot adopt a unified approach and increases the complexity of quality assessment.

[0068] Based on this, in this embodiment of the disclosure, the audio and video quality is determined; a first factor is determined, and a second factor is determined; both the first factor and the second factor are related to the time parameters of the audio and video; based on the audio and video quality, as well as the first factor and the second factor, the audio and video temporal quality is determined.

[0069] Referring to Figure 3, which is a schematic flowchart of the quality assessment method according to an embodiment of this disclosure, as shown in Figure 3, it includes steps 301 to 303:

[0070] Step 301: Determine the audio and video quality.

[0071] It is understood that the audio and video quality referred to can refer to the audio and video quality in scenarios such as video calls, video conferencing, live video streaming, and video on demand.

[0072] It is understood that the audio and video quality is related to the audio and video encoding and decoding parameters.

[0073] It is understood that the audio and video quality refers to the combined quality of audio and video, which can be obtained through subjective user scoring or calculated using audio quality models and video quality models.

[0074] Specifically, the process of determining the audio and video quality may include:

[0075] First, an audio quality score can be determined based on the audio encoding / decoding parameters and corresponding weights, where different scores represent different audio qualities; the audio encoding / decoding parameters include at least one of the following: encoding format, bit rate, resolution, latency, and packet loss rate.

[0076] Alternatively, the audio encoding and decoding parameters can be input into a pre-trained audio quality model to obtain an audio quality score.

[0077] Then, the video quality score can be determined based on the video's encoding / decoding parameters, terminal parameters, and corresponding weights, where different score values ​​represent different video qualities; the video's encoding / decoding parameters include at least one of the following: encoding format, bitrate, resolution, frame rate, latency, stuttering level, and image clarity; the terminal parameters include at least one of the following: terminal type and terminal resolution.

[0078] Here, the video encoding / decoding parameters and terminal parameters can also be input into a pre-trained video quality model to obtain a video quality score.

[0079] Finally, the audio quality score and the video quality score are combined to obtain the audio and video quality score, where different audio and video quality scores represent different audio and video qualities.

[0080] Here, the audio and video quality scores can be determined based on the audio quality score, the video quality score, and their corresponding weights.

[0081] It is understood that the audio and video quality values ​​can range from 1 to 5 (inclusive). For example, a value of 3 indicates that the audio and video quality is normal; a value between 3 and 4 indicates that the audio and video quality is good; and a value between 4 and 5 indicates that the audio and video quality is optimal.

[0082] Step 302: Determine the first factor and the second factor; both the first factor and the second factor are related to the time parameters of the audio and video.

[0083] Here, the first factor is called the impact factor for the quality affected by the interaction delay.

[0084] Here, the second factor is called the impact factor for the quality affected by media synchronization.

[0085] Here, the factors affecting audio and video temporal quality include the factors affecting audio and video interaction latency quality and the factors affecting audio and video media synchronization quality, with values ​​ranging from 0 to 1 (inclusive).

[0086] It is understood that the audio time parameter may refer to the first time delay parameter of the audio, and the video time parameter may refer to the second time delay parameter of the video.

[0087] Here, the first delay parameter of the audio can specifically be a time in milliseconds. The first delay parameter represents the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0088] In other words, the first delay parameter represents the duration between the transmission and reception times of an audio frame from the receiving end's perspective. This delay can be the delay of a certain time period corresponding to any given time point. For example, assuming there are three audio frames, the duration between the transmission and reception times of the first audio frame is T1, the duration between the transmission and reception times of the second audio frame is T2, and the duration between the transmission and reception times of the third audio frame is T3. The first delay parameter can refer to the delay of a certain time period corresponding to the starting transmission time point of any audio frame, such as the first audio frame, for example, T1; or it can refer to the sum of the delays of a certain time period corresponding to the starting transmission times of any two audio frames, such as the first and second audio frames, for example, T1+T2; or it can refer to the sum of the delays of a certain time period corresponding to the starting transmission times of any three audio frames, such as the first, second, and third audio frames, for example, T1+T2+T3.

[0089] Here, the second delay parameter of the video can specifically be a time in milliseconds. The second delay parameter represents the video delay, that is, the end-to-end delay of the video stream in the transmission network.

[0090] In other words, the second delay parameter represents the duration between the transmission and reception times of a video frame from the receiver's perspective. This delay can be the delay of a time interval corresponding to any given point in time. For example, assuming there are three video frames, the duration between the transmission and reception times of the first video frame is T4, the duration between the transmission and reception times of the second video frame is T5, and the duration between the transmission and reception times of the third video frame is T6. The second delay parameter can refer to the delay of a time interval corresponding to the starting transmission time of any video frame, such as the first video frame, for example, T4; or it can refer to the sum of the delays of a time interval corresponding to the starting transmission times of any two video frames, such as the first and second video frames, for example, T4+T5; or it can refer to the sum of the delays of a time interval corresponding to the starting transmission times of any three video frames, such as the first, second, and third video frames, for example, T4+T5+T6.

[0091] In practical applications such as video calls, video conferencing, live video streaming, and video-on-demand, optimizing the time required for data transmission and reception is crucial for improving user experience and enhancing interactivity. Accurately assessing the audio-video interaction latency quality can aid in this optimization. In this disclosure, after determining the audio-video quality using audio encoding / decoding parameters, video encoding / decoding parameters, and terminal parameters, a first factor related to the timing parameters of the audio and video is determined. Thus, the audio-video interaction latency quality obtained based on this first factor and the audio-video quality can simultaneously reflect both the clarity and smoothness of the audio and video.

[0092] Based on this, in some embodiments, determining the first factor includes:

[0093] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0094] The first factor is determined based on the first delay parameter of the audio and the second delay parameter of the video.

[0095] Here, the first delay parameter can specifically be a time in milliseconds. The first delay parameter represents the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0096] Here, the second delay parameter can specifically be a time in milliseconds. The second delay parameter represents the video delay, that is, the end-to-end delay of the video stream in the transmission network.

[0097] Here, the audio and video sending end can send the first delay parameter of the audio and the second delay parameter of the video to the audio and video receiving end. The audio and video receiving end determines the first factor based on the first delay parameter of the audio and the second delay parameter of the video. Subsequently, based on the first factor and the audio and video quality, the audio and video interaction delay quality is evaluated.

[0098] In some embodiments, determining the first factor based on a first delay parameter of the audio and a second delay parameter of the video includes:

[0099] A first operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain the third delay parameter;

[0100] The first factor is determined based on the third delay parameter and the first preset parameter.

[0101] Here, the first operation may refer to the square of the mean of the first delay parameter and the second delay parameter.

[0102] Here, determining the first factor based on the third time delay parameter and the first preset parameter may include: multiplying the third time delay parameter and the first preset parameter to obtain the value of the first factor.

[0103] In some embodiments, the first factor is determined according to the following formula:

[0104] Among them, f delayp1 represents the first factor, with a value ranging from 0 to 1 (inclusive). The first factor characterizes the impact factor for the quality affected by interaction delay; p1 represents the first preset parameter, with a value ranging from -1 to 1 (inclusive); T V The second delay parameter representing the video can specifically be a time unit in milliseconds. This second delay parameter characterizes the video delay, specifically the end-to-end delay of the video stream in the transmission network. a The first delay parameter representing the audio can be a time unit in milliseconds. The first delay parameter characterizes the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0105] Here, the first factor (or described as the audio-video interaction latency quality influencing factor) is calculated based on the audio and video time parameters. First, the combined audio-video latency is calculated, which is equal to the square of the mean of the audio latency and the video latency; then, the audio-video interaction latency quality influencing factor is calculated, which is equal to the product of the first preset parameter and the combined audio-video latency.

[0106] In practical applications such as video calls, video conferencing, live video streaming, and video-on-demand, optimizing the synchronization between audio and video is crucial for improving user experience and enhancing interactivity. Accurately assessing the synchronization quality of audio and video media can aid in this optimization. In this disclosure, after determining the audio and video quality using audio and video encoding / decoding parameters and the terminal, a second factor related to the timing parameters of the audio and video is determined. Thus, the audio and video media synchronization quality obtained based on this second factor and the audio and video quality can simultaneously reflect both the clarity and synchronization consistency of the audio and video.

[0107] Based on this, in some embodiments, determining the second factor includes:

[0108] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0109] The second factor is determined based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality.

[0110] Here, the first delay parameter can specifically be a time in milliseconds. The first delay parameter represents the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0111] Here, the second delay parameter can specifically be a time in milliseconds. The second delay parameter represents the video delay, that is, the end-to-end delay of the video stream in the transmission network.

[0112] Here, the audio and video sending end can send the first delay parameter of the audio and the second delay parameter of the video to the audio and video receiving end. The audio and video receiving end determines the second factor based on the first delay parameter of the audio, the second delay parameter of the video, and the audio and video quality. Subsequently, the audio and video media synchronization quality is evaluated based on the second factor and the audio and video quality.

[0113] Here, the process of determining the second factor includes the following two cases:

[0114] In the first case, the second delay parameter of the video is greater than or equal to the first delay parameter of the audio.

[0115] In the second case, the second delay parameter of the video is less than the first delay parameter of the audio.

[0116] Regarding the first scenario described above, in some embodiments, determining the second factor based on the first latency parameter of the audio, the second latency parameter of the video, and the audio / video quality includes:

[0117] If the second delay parameter of the video is greater than or equal to the first delay parameter of the audio, a second operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain a fourth delay parameter;

[0118] The second factor is determined based on the fourth delay parameter, the second preset parameter, the third preset parameter, the fourth preset parameter, and the audio / video quality.

[0119] Here, the second operation can refer to the square of the difference between the second delay parameter and the first delay parameter.

[0120] Here, determining the second factor based on the fourth delay parameter, the second preset parameter, the third preset parameter, the fourth preset parameter, and the audio / video quality may include:

[0121] The first value is obtained by summing the audio / video quality with the fourth preset parameter.

[0122] The first value is multiplied by the third preset parameter to obtain the second value;

[0123] The second value is used as the exponent of e, and an operation is performed with e as the base to obtain the third value;

[0124] Summing the third value with the value 1 yields the fourth value;

[0125] The first ratio is obtained by dividing the second preset parameter by the fourth value;

[0126] The value of the second factor is obtained by multiplying the first ratio with the fourth time delay parameter.

[0127] Regarding the second scenario described above, in some embodiments, determining the second factor based on the first time parameter of the audio, the second time parameter of the video, and the audio / video quality includes:

[0128] If the second delay parameter of the video is less than the first delay parameter of the audio, a third operation is performed on the first delay of the audio and the second delay of the video to obtain a fifth delay parameter;

[0129] The second factor is determined based on the fifth delay parameter, the fifth preset parameter, the sixth preset parameter, the seventh preset parameter, and the audio / video quality.

[0130] Here, the third operation may refer to the square of the difference between the first delay parameter and the second delay parameter.

[0131] Here, determining the second factor based on the fifth delay parameter, the fifth preset parameter, the sixth preset parameter, the seventh preset parameter, and the audio / video quality may include:

[0132] The fifth value is obtained by summing the audio and video quality with the seventh preset parameter.

[0133] The sixth value is obtained by multiplying the fifth value by the sixth preset parameter.

[0134] Using the sixth value as the exponent of e, perform an operation with e as the base to obtain the seventh value;

[0135] Summing the seventh value with the value 1 yields the eighth value;

[0136] The second ratio is obtained by dividing the fifth preset parameter by the eighth value;

[0137] The value of the second factor is obtained by multiplying the second ratio with the fifth time delay parameter.

[0138] In some embodiments, the second factor is determined according to the following formula:

[0139] Among them, f sync p1 represents the second factor, with a value ranging from 0 to 1 (inclusive). The second factor characterizes the impact factor for the quality affected by media synchronization. p2 represents the second preset parameter, p3 represents the third preset parameter, and p4 represents the fourth preset parameter. The values ​​of the second, third, and fourth preset parameters range from -5 to 5 (inclusive). av The audio-visual quality is represented by a value between 1 and 5 (inclusive), and the audio-visual quality characterizes the estimated audiovisual quality; T V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0140] Here, the second factor (or described as an influencing factor on audio and video media synchronization quality) is calculated based on the timing parameters of the audio and video, as well as the audio and video quality. When the video delay is greater than or equal to the audio delay, the square of the difference between the video delay and the audio delay is first calculated. Then, the exponent of e is calculated based on the third preset parameter, the fourth preset parameter, and the audio and video quality. An operation with e as the base is performed to obtain the calculation result. The calculation result is summed with the value 1 to obtain the summation result. The second preset parameter is then divided by the summation result to obtain the ratio. Finally, the ratio is multiplied by the square of the difference to obtain the value of the second factor.

[0141] In some embodiments, the second factor is determined according to the following formula:

[0142] Among them, f syncThe second factor represents the second factor, with a value range between 0 and 1 (inclusive). The second factor characterizes the impact factor for the quality affected by media synchronization. p5 represents the fifth preset parameter, p6 represents the sixth preset parameter, and p7 represents the seventh preset parameter. The values ​​of the fifth, sixth, and seventh preset parameters range from -5 to 5 (inclusive). Q av The audio and video quality is represented by a value between 1 and 5 (inclusive), and the audio and video quality characterizes the evaluated audiovisual quality; T V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0143] Here, the second factor (or described as an influencing factor on audio and video media synchronization quality) is calculated based on the timing parameters of the audio and video, as well as the audio and video quality. When the video latency is less than the audio latency, the square of the difference between the audio latency and the video latency is first calculated. Then, the exponent of e is calculated based on the sixth preset parameter, the seventh preset parameter, and the audio and video quality. An operation with e as the base is performed to obtain the calculation result. The calculation result is summed with the value 1 to obtain the summation result. The fifth preset parameter is then divided by the summation result to obtain the ratio. Finally, the ratio is multiplied by the square of the difference to obtain the value of the second factor.

[0144] Step 303: Determine the audio and video temporal quality based on the audio and video quality, as well as the first factor and the second factor.

[0145] Here, the audio and video temporal quality is obtained through a unified linear calculation method. This audio and video temporal quality includes audio and video interaction latency quality and audio and video media synchronization quality.

[0146] Here, the linear calculation of audio and video temporal quality refers to the linear calculation of audio and video quality and the influencing factors of audio and video temporal quality.

[0147] In practical applications, after determining the audio and video quality using the encoding and decoding parameters of the audio and video and the terminal parameters, the first and second factors related to the timing parameters of the audio and video are then determined. Thus, based on the first factor and the audio and video quality, the audio and video interaction latency quality is obtained, which can simultaneously reflect the clarity and smoothness of the audio and video. Based on the second factor and the audio and video quality, the audio and video media synchronization quality is obtained, which can simultaneously reflect the clarity and synchronization consistency of the audio and video.

[0148] Based on this, in some embodiments, determining the temporal quality of audio and video based on the audio and video quality, as well as the first factor and the second factor, includes:

[0149] Based on the audio and video quality and the first factor, the audio and video temporal quality includes the audio and video interaction latency quality.

[0150] Based on the audio and video quality and the second factor, the audio and video temporal quality includes the audio and video media synchronization quality.

[0151] In some embodiments, the audio and video temporal quality, including the audio and video interaction latency quality, is determined according to the following formula: Q delay =(p8+p9Q) av )-p 10 f delay

[0152] Among them, Q delay The audio-visual interaction latency quality is represented by a value between 1 and 5 (inclusive). The audio-visual interaction latency quality can also be described as the evaluated audiovisual interaction latency quality. p8 represents the eighth preset parameter, p9 represents the ninth preset parameter, and p... 10 The tenth preset parameter represents the value range of the eighth, ninth, and tenth preset parameters, which is between -1 and 5 (inclusive); Q av Indicates the audio / video quality, f delay This refers to the first factor.

[0153] Here, the product of the ninth preset parameter and the audio / video quality is first calculated to obtain a first product result. The first product result is then summed with the eighth preset parameter to obtain a summation result. The product of the tenth preset parameter and the first factor is then calculated to obtain a second product result. Finally, the difference between the summation result and the second product result is obtained to obtain a difference value, which is used as the audio / video interaction latency quality.

[0154] In some embodiments, the audio and video temporal quality, including the audio and video media synchronization quality, is determined according to the following formula:

[0155] Q sync =(p 11 +p 12 Q av )-p 13 f sync

[0156] Among them, Qsync This indicates the audio-visual media synchronization quality, with a value ranging from 1 to 5 (inclusive). The audio-visual media synchronization quality can also be described as the estimated audiovisual synchronization quality. 11 p represents the eleventh preset parameter. 12 p represents the twelfth preset parameter. 13 This refers to the thirteenth preset parameter, where the values ​​of the eleventh, twelfth, and thirteenth preset parameters range from -1 to 5 (inclusive); Q av Indicates the audio / video quality, f sync This refers to the second factor.

[0157] Here, the product of the twelfth preset parameter and the audio / video quality is first calculated to obtain the third product result. The third product result is then summed with the eleventh preset parameter to obtain the summation result. The thirteenth preset parameter is then multiplied with the second factor to obtain the fourth product result. Finally, the difference between the summation result and the fourth product result is obtained to obtain the difference value, which is used as the audio / video media synchronization quality.

[0158] The embodiments disclosed herein have the following advantages:

[0159] (1) A method for evaluating audio and video temporal quality is provided, wherein audio and video temporal quality includes audio and video interaction latency quality and audio and video media synchronization quality.

[0160] In this disclosure, audio and video quality related to the encoding and decoding parameters of audio and video is determined, and a first factor and a second factor related to the timing parameters of audio and video are also determined. Thus, based on the audio and video quality, as well as the first factor and the second factor, the audio and video temporal quality is determined, which allows the evaluated audio and video temporal quality, including the audio and video interaction latency quality, to simultaneously reflect the clarity and smoothness of the audio and video, and also allows the evaluated audio and video temporal quality, including the audio and video media synchronization quality, to simultaneously reflect the clarity and synchronization consistency of the audio and video.

[0161] It can be seen that when evaluating the temporal quality of audio and video, considering both time factors and audio and video quality is consistent with the actual user experience. Therefore, the evaluated temporal quality of audio and video can be used to maximize the optimization of user experience, and no model is required, making the evaluation process relatively simple.

[0162] (2) A unified linear function is used to calculate the audio and video interaction delay quality and audio and video media synchronization quality, which has a small computational load and good robustness.

[0163] In other words, the calculation of audio-visual interaction latency quality and audio-visual media synchronization quality uses the same linear model, only the parameters are different, which is conducive to implementation.

[0164] (3) Audio-visual interaction latency quality assessment and audio-visual media synchronization quality assessment have significant value in the business field. First, these assessments can significantly improve user experience. For example, they can ensure the smoothness and real-time performance of audio-visual interactions, thereby increasing user satisfaction and loyalty. Second, these assessments help companies optimize their products and services. By monitoring and evaluating audio-visual quality in real time, companies can promptly identify and resolve potential problems, such as latency, stuttering, and audio-visual synchronization issues, thereby improving the stability and reliability of services. This is particularly important for companies providing enterprise-level video calling, video conferencing, collaboration, and training services, as the quality of these services directly affects the company's brand image and market competitiveness. Finally, audio-visual interaction latency and synchronization quality assessment also helps promote technological innovation and industry standardization. By evaluating and comparing the performance of different technologies and solutions, companies can better understand market demands, thereby driving technological innovation and product upgrades. At the same time, these assessment methods also provide a common quality standard for the industry, helping to promote the healthy development of the industry.

[0165] Referring to Figure 4, which is a schematic diagram of the specific implementation flow of the quality assessment method according to an embodiment of this disclosure, as shown in Figure 4, it includes steps 401 to 405:

[0166] Step 401: Determine the audio and video quality.

[0167] It is understood that the audio and video quality referred to can refer to the audio and video quality in scenarios such as video calls, video conferencing, live video streaming, and video on demand.

[0168] It is understood that the audio and video quality is related to the audio and video encoding and decoding parameters.

[0169] It is understood that the audio and video quality refers to the combined quality of audio and video, which can be obtained through subjective user scoring or calculated using audio quality models and video quality models.

[0170] Specifically, the process of determining the audio and video quality may include:

[0171] First, an audio quality score can be determined based on the audio encoding / decoding parameters and corresponding weights, where different scores represent different audio qualities; the audio encoding / decoding parameters include at least one of the following: encoding format, bit rate, resolution, latency, and packet loss rate.

[0172] Alternatively, the audio encoding and decoding parameters can be input into a pre-trained audio quality model to obtain an audio quality score.

[0173] Then, the video quality score can be determined based on the video's encoding / decoding parameters, terminal parameters, and corresponding weights, where different score values ​​represent different video qualities; the video's encoding / decoding parameters include at least one of the following: encoding format, bitrate, resolution, frame rate, latency, stuttering level, and image clarity; the terminal parameters include at least one of the following: terminal type and terminal resolution.

[0174] Here, the video encoding / decoding parameters and terminal parameters can also be input into a pre-trained video quality model to obtain a video quality score.

[0175] Finally, the audio quality score and the video quality score are combined to obtain the audio and video quality score, where different audio and video quality scores represent different audio and video qualities.

[0176] Here, the audio and video quality scores can be determined based on the audio quality score, the video quality score, and their corresponding weights.

[0177] It is understood that the audio and video quality values ​​can range from 1 to 5 (inclusive). For example, a value of 3 indicates that the audio and video quality is normal; a value between 3 and 4 indicates that the audio and video quality is good; and a value between 4 and 5 indicates that the audio and video quality is optimal.

[0178] Step 402: Obtain the first delay parameter of the audio and the second delay parameter of the video; determine the first factor based on the first delay parameter of the audio and the second delay parameter of the video.

[0179] Here, the first delay parameter can specifically be a time in milliseconds. The first delay parameter represents the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0180] Here, the second delay parameter can specifically be a time in milliseconds. The second delay parameter represents the video delay, that is, the end-to-end delay of the video stream in the transmission network.

[0181] Here, the audio / video sending end can send the first latency parameter of the audio and the second latency parameter of the video to the audio / video receiving end. The audio / video receiving end determines the first factor based on the first latency parameter of the audio and the second latency parameter of the video. Subsequently, based on the first factor and the audio / video quality, the audio / video interaction latency quality is evaluated. And based on the first latency parameter of the audio, the second latency parameter of the video and the audio / video quality, the second factor is determined. Subsequently, based on the second factor and the audio / video quality, the audio / video media synchronization quality is evaluated.

[0182] Here, the first factor is determined according to the following formula:

[0183] Among them, f delay The first factor represents the value between 0 and 1 (inclusive), and it characterizes the impact factor for the quality affected by interaction delay; p1 represents the first preset parameter; T V The second delay parameter representing the video can specifically be a time unit in milliseconds. This second delay parameter characterizes the video delay, specifically the end-to-end delay of the video stream in the transmission network. a The first delay parameter representing the audio can be a time unit in milliseconds. The first delay parameter characterizes the audio delay, that is, the end-to-end delay of the audio stream in the transmission network.

[0184] Here, the first factor (or described as the audio-video interaction latency quality influencing factor) is calculated based on the audio and video time parameters. First, the combined audio-video latency is calculated, which is equal to the square of the mean of the audio latency and the video latency; then, the audio-video interaction latency quality influencing factor is calculated, which is equal to the product of the first preset parameter and the combined audio-video latency.

[0185] Step 403: Determine the second factor based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality.

[0186] Here, the second factor (or described as an influencing factor on the synchronization quality of audio and video media) is calculated based on the timing parameters of the audio and video and the audio and video quality.

[0187] Here, the second factor is determined according to the following formula:

[0188] Among them, f sync p1 represents the second factor, with a value ranging from 0 to 1 (inclusive). The second factor characterizes the impact factor for the quality affected by media synchronization. p2 represents the second preset parameter, p3 represents the third preset parameter, and p4 represents the fourth preset parameter. The values ​​of the second, third, and fourth preset parameters range from -5 to 5 (inclusive). av The audio-visual quality is represented by a value between 1 and 5 (inclusive), and the audio-visual quality characterizes the estimated audiovisual quality; T V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0189] Here, the second factor (or described as an influencing factor on audio and video media synchronization quality) is calculated based on the timing parameters of the audio and video, as well as the audio and video quality. When the video delay is greater than or equal to the audio delay, the square of the difference between the video delay and the audio delay is first calculated. Then, the exponent of e is calculated based on the third preset parameter, the fourth preset parameter, and the audio and video quality. An operation with e as the base is performed to obtain the calculation result. The calculation result is summed with the value 1 to obtain the summation result. The second preset parameter is then divided by the summation result to obtain the ratio. Finally, the ratio is multiplied by the square of the difference to obtain the value of the second factor.

[0190] Here, the second factor is determined according to the following formula:

[0191] Among them, f syncThe second factor represents the second factor, with a value range between 0 and 1 (inclusive). The second factor characterizes the impact factor for the quality affected by media synchronization. p5 represents the fifth preset parameter, p6 represents the sixth preset parameter, and p7 represents the seventh preset parameter. The values ​​of the fifth, sixth, and seventh preset parameters range from -5 to 5 (inclusive). Q av The audio and video quality is represented by a value between 1 and 5 (inclusive), and the audio and video quality characterizes the evaluated audiovisual quality; T V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0192] Here, the second factor (or described as an influencing factor on audio and video media synchronization quality) is calculated based on the timing parameters of the audio and video, as well as the audio and video quality. When the video latency is less than the audio latency, the square of the difference between the audio latency and the video latency is first calculated. Then, the exponent of e is calculated based on the sixth preset parameter, the seventh preset parameter, and the audio and video quality. An operation with e as the base is performed to obtain the calculation result. The calculation result is summed with the value 1 to obtain the summation result. The fifth preset parameter is then divided by the summation result to obtain the ratio. Finally, the ratio is multiplied by the square of the difference to obtain the value of the second factor.

[0193] Step 404: Based on the audio and video quality and the first factor, determine the audio and video temporal quality, including the audio and video interaction latency quality.

[0194] Here, the audio / video interaction latency quality is determined according to the following formula: Q delay =(p8+p9Q) av )-p 10 f delay

[0195] Among them, Q delay The audio-visual interaction latency quality is represented by a value between 1 and 5 (inclusive). The audio-visual interaction latency quality can also be described as the evaluated audiovisual interaction latency quality. p8 represents the eighth preset parameter, p9 represents the ninth preset parameter, and p... 10 The tenth preset parameter represents the value range of the eighth, ninth, and tenth preset parameters, which is between -1 and 5 (inclusive); Q av Indicates the audio / video quality, fdelay This refers to the first factor.

[0196] Here, the product of the ninth preset parameter and the audio / video quality is first calculated to obtain a first product result. The first product result is then summed with the eighth preset parameter to obtain a summation result. The product of the tenth preset parameter and the first factor is then calculated to obtain a second product result. Finally, the difference between the summation result and the second product result is obtained to obtain a difference value, which is used as the audio / video interaction latency quality.

[0197] Step 405: Based on the audio and video quality and the second factor, determine the audio and video temporal quality, including the audio and video media synchronization quality.

[0198] Here, the audio and video media synchronization quality is determined according to the following formula: Q sync =(p 11 +p 12 Q av )-p 13 f sync

[0199] Among them, Q sync This indicates the audio-visual media synchronization quality, with a value ranging from 1 to 5 (inclusive). The audio-visual media synchronization quality can also be described as the estimated audiovisual synchronization quality. 11 p represents the eleventh preset parameter. 12 p represents the twelfth preset parameter. 13 This refers to the thirteenth preset parameter, where the values ​​of the eleventh, twelfth, and thirteenth preset parameters range from -1 to 5 (inclusive); Q av Indicates the audio / video quality, f sync This refers to the second factor.

[0200] Here, the product of the twelfth preset parameter and the audio / video quality is first calculated to obtain the third product result. The third product result is then summed with the eleventh preset parameter to obtain the summation result. The thirteenth preset parameter is then multiplied with the second factor to obtain the fourth product result. Finally, the difference between the summation result and the fourth product result is obtained to obtain the difference value, which is used as the audio / video media synchronization quality.

[0201] To implement the quality assessment method of this disclosure embodiment, this disclosure embodiment also provides a quality assessment device, which is installed in an electronic device. Figure 5 is a schematic diagram of the composition structure of the quality assessment device of this disclosure embodiment. As shown in Figure 5, the device includes:

[0202] The first processing module 51 is used to determine the audio and video quality;

[0203] The second processing module 52 is used to determine a first factor and a second factor; both the first factor and the second factor are related to the time parameters of the audio and video; based on the audio and video quality, as well as the first factor and the second factor, the audio and video temporal quality is determined.

[0204] In some embodiments, the second processing module 52 has the function of:

[0205] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0206] The first factor is determined based on the first delay parameter of the audio and the second delay parameter of the video.

[0207] In some embodiments, the second processing module 52 has the function of:

[0208] A first operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain the third delay parameter;

[0209] The first factor is determined based on the third delay parameter and the first preset parameter.

[0210] In some embodiments, the second processing module 52 has the following functions:

[0211] The first factor is determined according to the following formula:

[0212] Among them, f delay p1 represents the first factor, p1 represents the first preset parameter, and T represents the first preset parameter. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0213] In some embodiments, the second processing module 52 has the function of:

[0214] Obtain the first delay parameter of the audio and the second delay parameter of the video;

[0215] The second factor is determined based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality.

[0216] In some embodiments, the second processing module 52 has the function of:

[0217] If the second delay parameter of the video is greater than or equal to the first delay parameter of the audio, a second operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain a fourth delay parameter;

[0218] The second factor is determined based on the fourth delay parameter, the second preset parameter, the third preset parameter, the fourth preset parameter, and the audio / video quality.

[0219] In some embodiments, the second processing module 52 has the function of:

[0220] If the second delay parameter of the video is less than the first delay parameter of the audio, a third operation is performed on the first delay of the audio and the second delay of the video to obtain a fifth delay parameter;

[0221] The second factor is determined based on the fifth delay parameter, the fifth preset parameter, the sixth preset parameter, the seventh preset parameter, and the audio / video quality.

[0222] In some embodiments, the second processing module 52 has the function of:

[0223] The second factor is determined using the following formula:

[0224] Among them, f sync p2 represents the second factor, p3 represents the second preset parameter, p4 represents the third preset parameter, and p4 represents the fourth preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0225] In some embodiments, the second processing module 52 has the function of:

[0226] The second factor is determined using the following formula:

[0227] Among them, f sync p5 represents the second factor, p6 represents the fifth preset parameter, p7 represents the sixth preset parameter, and p7 represents the seventh preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

[0228] In some embodiments, the second processing module 52 has the function of:

[0229] Based on the audio and video quality and the first factor, the audio and video temporal quality includes the audio and video interaction latency quality.

[0230] Based on the audio and video quality and the second factor, the audio and video temporal quality includes the audio and video media synchronization quality.

[0231] In some embodiments, the second processing module 52 has the function of:

[0232] The audio and video temporal quality, including the audio and video interaction latency quality, is determined according to the following formula: Q delay =(p8+p9Q) av )-p 10 f delay

[0233] Among them, Q delay p8 represents the audio / video interaction latency quality, p9 represents the eighth preset parameter, and p represents the ninth preset parameter. 10 Q represents the tenth preset parameter. av Indicates the audio / video quality, f delay This refers to the first factor.

[0234] In some embodiments, the second processing module 52 has the function of:

[0235] The audio and video temporal quality, including the audio and video media synchronization quality, is determined according to the following formula: Q sync =(p 11 +p 12 Q av )-p 13 f sync

[0236] Among them, Q sync p represents the synchronization quality of the audio and video media. 11 p represents the eleventh preset parameter. 12 p represents the twelfth preset parameter. 13 This represents the thirteenth preset parameter, Q. av Indicates the audio / video quality, f sync This refers to the second factor.

[0237] In practical applications, the first processing module 51 and the second processing module 52 can be implemented by the processor in the quality assessment device.

[0238] It should be noted that the quality assessment device provided in the above embodiments is only illustrated by the division of the above-described program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the quality assessment device and the quality assessment method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0239] This disclosure also provides an electronic device, as shown in FIG6, comprising:

[0240] Communication interface 61 enables information exchange with other devices;

[0241] The processor 62, connected to the communication interface 61, is used to execute the methods provided by one or more technical solutions on the electronic device side when running a computer program. The computer program is stored in the memory 63.

[0242] It should be noted that the specific processing procedures of the processor 62 and the communication interface 61 are detailed in the method embodiment and will not be repeated here.

[0243] Of course, in practical applications, the various components in electronic device 60 are coupled together through bus system 64. It can be understood that bus system 64 is used to achieve communication between these components. In addition to the data bus, bus system 64 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 64 in Figure 6.

[0244] The memory 63 in this embodiment is used to store various types of data to support the operation of the electronic device 60. Examples of such data include any computer program used to operate on the electronic device 60.

[0245] The methods disclosed in the embodiments of this disclosure can be applied to, or implemented by, the processor 62. The processor 62 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the form of software within the processor 62. The processor 62 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 62 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically memory 63. The processor 62 reads information from memory 63 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0246] In an exemplary embodiment, the electronic device 60 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0247] It is understood that the memory (memory 63) in this embodiment of the present disclosure can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this disclosure are intended to include, but are not limited to, these and any other suitable types of memories.

[0248] In an exemplary embodiment, this disclosure also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory that stores a computer program, which can be executed by the processor 62 of the electronic device 60 to complete the steps described in the aforementioned electronic device-side method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0249] By way of example, this disclosure also provides a computer program product, including a computer program that can be executed by a processor 62 of an electronic device 60 to perform the steps described in any of the foregoing methods.

[0250] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0251] Furthermore, the technical solutions described in the embodiments of this disclosure can be combined arbitrarily without conflict.

[0252] The above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure.

Claims

1. A quality assessment method, the method comprising: Determine audio and video quality; Identify the first factor, and then identify the second factor; Both the first factor and the second factor are related to the time parameters of audio and video. Based on the audio and video quality, as well as the first and second factors, the temporal quality of the audio and video is determined.

2. The method according to claim 1, wherein, The determination of the first factor includes: Obtain the first delay parameter of the audio and the second delay parameter of the video; The first factor is determined based on the first delay parameter of the audio and the second delay parameter of the video.

3. The method according to claim 2, wherein, Determining the first factor based on the first delay parameter of the audio and the second delay parameter of the video includes: A first operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain the third delay parameter; The first factor is determined based on the third delay parameter and the first preset parameter.

4. The method according to claim 1, wherein, The first factor is determined according to the following formula: Among them, f delay p1 represents the first factor, p1 represents the first preset parameter, and T represents the first preset parameter. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

5. The method according to claim 1, wherein, The determination of the second factor includes: Obtain the first delay parameter of the audio and the second delay parameter of the video; The second factor is determined based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality.

6. The method according to claim 5, wherein, The determination of the second factor based on the first delay parameter of the audio, the second delay parameter of the video, and the audio / video quality includes: If the second delay parameter of the video is greater than or equal to the first delay parameter of the audio, a second operation is performed on the first delay parameter of the audio and the second delay parameter of the video to obtain a fourth delay parameter; The second factor is determined based on the fourth delay parameter, the second preset parameter, the third preset parameter, the fourth preset parameter, and the audio / video quality.

7. The method according to claim 5, wherein, The step of determining the second factor based on the first time parameter of the audio, the second time parameter of the video, and the audio / video quality includes: If the second delay parameter of the video is less than the first delay parameter of the audio, a third operation is performed on the first delay of the audio and the second delay of the video to obtain a fifth delay parameter; The second factor is determined based on the fifth delay parameter, the fifth preset parameter, the sixth preset parameter, the seventh preset parameter, and the audio / video quality.

8. The method according to claim 1, wherein, The second factor is determined using the following formula: Among them, f sync p2 represents the second factor, p3 represents the third preset parameter, p4 represents the fourth preset parameter, and Q represents the fourth preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

9. The method according to claim 1, wherein, The second factor is determined using the following formula: Among them, f sync p5 represents the second factor, p6 represents the fifth preset parameter, p7 represents the sixth preset parameter, and Q represents the seventh preset parameter. av T represents the audio / video quality. V T represents the second time delay parameter of the video. a The first time delay parameter represents the audio.

10. The method according to claim 1, wherein, The determination of audio and video temporal quality based on the audio and video quality, as well as the first factor and the second factor, includes: Based on the audio and video quality and the first factor, the audio and video temporal quality includes the audio and video interaction latency quality. Based on the audio and video quality and the second factor, the audio and video temporal quality includes the audio and video media synchronization quality.

11. The method according to claim 1, wherein, The audio and video temporal quality, including the audio and video interaction latency quality, is determined according to the following formula: Q delay =(p8+p9Q) av )-p 10 f delay Among them, Q delay This indicates the audio / video interaction latency quality, p8 represents the eighth preset parameter, p9 represents the ninth preset parameter, and p... 10 Q represents the tenth preset parameter. av Indicates the audio / video quality, f delay This refers to the first factor.

12. The method according to claim 1, wherein, The audio and video temporal quality, including the audio and video media synchronization quality, is determined according to the following formula: Q sync =(p 11 +p 12 Q av )-p 13 f sync Among them, Q sync p represents the synchronization quality of the audio and video media. 11 p represents the eleventh preset parameter. 12 p represents the twelfth preset parameter. 13 This represents the thirteenth preset parameter, Q. av Indicates the audio / video quality, f sync This refers to the second factor.

13. A quality assessment device, comprising: The first processing module is used to determine the audio and video quality; The second processing module is used to determine the first factor and the second factor; Both the first factor and the second factor are related to the time parameters of the audio and video; based on the audio and video quality, as well as the first factor and the second factor, the audio and video temporal quality is determined.

14. An electronic device comprising a processor and a memory for storing a computer program capable of running on the processor, wherein, When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images