Evaluation method and system for high-definition video call perception

Abstract

The invention discloses an evaluation method and system for high-definition video call perception. The method comprises the following steps: S1, preparing equipment with a high-definition video call function; s2, selecting participants of video call evaluation; s3, setting evaluation indexes and scoring standards; s4, carrying out video calls of the participants and carrying out data collection; s5, collecting data recorded by the participants, and generating a questionnaire; s6, feedback suggestions are collected, and prominent problems and improvement suggestions are extracted; and S7, performing targeted adjustment on the evaluation index and the call flow. According to the method, firstly, the participants are trained, the participants are informed of the content needing to be evaluated, so that the control ability of the participants in the data recording direction in the video call process is improved, meanwhile, in the video call process of the participants, through real-time recording, the evaluation accuracy of the participants in the whole call process can be more accurately guaranteed, and the user experience is improved. Therefore, the finally analyzed result can be more accurate.

Description

Technical Field

[0001] The present invention relates to the field of video call evaluation, and in particular to an evaluation method and system for high-definition video call perception. Background Art

[0002] HD video call perception evaluation involves systematically analyzing and evaluating the user experience and call quality during HD video calls. Its primary goal is to understand users' subjective feelings and objective performance indicators when using HD video calling technology, thereby identifying call quality issues, optimizing the call experience, and improving user satisfaction.

[0003] The existing evaluation method for high-definition video call perception is to first organize participants to conduct high-definition video calls, maintain the participants' high-definition calls by debugging hardware equipment, and after the set call time ends, collect network record data during the video call and participant feedback questions after the call, and finally draw an evaluation conclusion through comprehensive analysis of the two parts of data; although the above evaluation method combines network record data and participant feedback data, the participants provide feedback after the video call ends. Feedback during this time period will cause some experience content during the video call to be ignored or forgotten, resulting in deviations in the feedback content of the participants' experience, which in turn affects the accuracy of the final evaluation results; in view of this, this solution proposes an evaluation method and system for high-definition video call perception to solve the above problems. Summary of the Invention

[0004] The object of the present invention is to provide a method and system for evaluating high-definition video call perception to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for evaluating high-definition video call perception, comprising the following steps:

[0006] S1. First, prepare a device with high-definition video calling capabilities, install the video calling software, and complete the video calling operation configuration.

[0007] S2, select participants for the video call assessment and introduce the purpose and process of the assessment;

[0008] S3, setting evaluation indicators and scoring criteria for participants to learn and master, the evaluation indicators include subjective indicators and objective indicators;

[0009] S4, conduct a video call with the participants and collect call data, while the participants mark the status of the call in real time;

[0010] S5, collect the data recorded by the participants, insert customized questions based on the recorded data, and generate the questionnaire;

[0011] S6: Distribute questionnaires to participants and collect feedback, extracting prominent issues and improvement suggestions from participants;

[0012] In S7, based on the participants’ suggestions for improvement and the data recorded by the equipment, targeted adjustments are made to the evaluation indicators and call processes, and an improvement plan is developed and implemented in the next evaluation.

[0013] Preferably, the specific steps of the S1 operation include:

[0014] For the S101, first select a device that supports high-definition video calls, including a 4K camera, an omnidirectional microphone array, noise-canceling headphones, and a stable light source;

[0015] S102: Configure a test network with a bandwidth of ≥50 Mbps and QoS guarantee.

[0016] S103, selecting video call software and analysis tools, and configuring and setting the software and analysis tools;

[0017] S104, prepare the video call environment, use light sources to adjust the video lighting and keep the environment quiet.

[0018] Preferably, the content of the software configuration settings includes video and audio input and output selection, network connection test and screen ratio setting;

[0019] The content of the analysis tool configuration setting is to enable developer mode to obtain the original data stream.

[0020] Preferably, the selection of participants first requires determining the number of people participating in the evaluation, selecting user groups of different ages and different technical levels, and after the participants are selected, explaining the purpose, importance and expected results of the evaluation through conference training, as well as introducing a detailed scoring mechanism and recording method.

[0021] Preferably, the subjective indicators include participant satisfaction, call fluency, and audio and video quality perception, and the objective indicators include call delay, image frame rate, and audio quality;

[0022] The scoring criteria are to develop quantitative scoring criteria and provide specific scoring instructions for each indicator. After the evaluation indicators and scoring criteria are developed, participants will be trained.

[0023] Preferably, the situation marking is that the participants use marking tools to record the situation and experience of the call in real time during the call.

[0024] Preferably, the data collection is to collect data recorded by participants during the call, including detailed information on key issues and experiences;

[0025] The customized question insertion is to design customized questions based on the collected data, and combine the customized questions with evaluation indicators to generate a structured questionnaire.

[0026] Preferably, the questionnaire is distributed to participants via email and an online questionnaire platform, and then a feedback deadline is set to collect questionnaire data. After the questionnaires are collected, common problems and improvement suggestions are extracted to form a preliminary feedback analysis report.

[0027] Preferably, when formulating the improvement plan, firstly, the collected questionnaire data is quantitatively analyzed using statistical tools, then the open feedback of the participants is qualitatively analyzed, and finally, based on the analysis results, the indicators and call processes that need to be improved are determined, and specific improvement measures and implementation schedules are formulated.

[0028] An evaluation system for high-definition video call perception, applied to the above-mentioned evaluation method for high-definition video call perception, the evaluation system comprising video hardware equipment, a hardware configuration management module, a participant management module, a data acquisition module, a questionnaire design and feedback collection module, and a data analysis and improvement plan module;

[0029] The video hardware device is a hardware device used to provide video calls;

[0030] The hardware configuration management module is used for configuration management of video hardware equipment and environment;

[0031] The participant management module is used to select assessment participants for training and introduce the assessment purpose and process;

[0032] The data acquisition module is used to conduct video calls and collect call data, while recording the call experience and status of participants in real time;

[0033] The questionnaire design and feedback collection module is used to design customized questionnaires based on the collected data and distribute them to participants to collect feedback;

[0034] The data analysis and improvement plan module is used to perform quantitative and qualitative analysis on the collected questionnaire and call data and formulate improvement plans.

[0035] Technical effects and advantages of the present invention:

[0036] When conducting a high-definition video call perception assessment, the present invention first trains the participants and informs them of the content that needs to be assessed, thereby improving the participants' ability to control the direction of data recording during the video call. At the same time, during the participants' video call, real-time recording can more accurately ensure the accuracy of the participants' assessment of the entire call process, so that the final analysis results can be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the operation flow of the evaluation method of the present invention.

[0038] Figure 2 This is a framework diagram of the evaluation system of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] In embodiment 1, the present invention provides Figure 1 The evaluation method for high-definition video call perception shown includes the following steps:

[0041] S1. First, prepare a device with high-definition video calling capabilities, install the video calling software, and complete the video calling operation configuration.

[0042] It should be noted that the device must work properly, have high-definition video call capabilities, and choose a suitable video call environment (sufficient lighting and clean background), ensure a stable network connection, and have a bandwidth of at least 50Mbps uplink and downlink.

[0043] Specifically, the specific steps of S1 operation include:

[0044] For the S101, first select a device that supports high-definition video calls, including a 4K camera, an omnidirectional microphone array, noise-canceling headphones, and a stable light source;

[0045] S102: Configure a test network with a bandwidth of ≥50 Mbps and QoS guarantee.

[0046] It should be noted that configuring QoS is a key technical step in evaluating HD video calls, ensuring that network resources are prioritized for video call traffic, thereby reducing lag, latency, and image quality loss. The core role of QoS in HD video calls includes:

[0047] Traffic prioritization: video (H.264 / HEVC encoded stream), audio (Opus / AAC encoded stream), and signaling (SIP / WebRTC control packets) are marked as the highest priority; other traffic (file downloads, web browsing) is assigned a low priority.

[0048] Key performance indicators include bandwidth reservation of ≥10 Mbps for video calls (higher bandwidth required for 4K calls); latency control, end-to-end latency ≤150 ms (International Telecommunication Union ITU-G.114 standard); jitter suppression, network jitter ≤30 ms; and packet loss rate ≤1% (requires forward error correction (FEC) / retransmission mechanism compensation).

[0049] S103, selecting video call software and analysis tools, and configuring and setting the software and analysis tools;

[0050] Furthermore, the software configuration settings include video and audio input and output selection, network connection test and screen ratio setting, and the analysis tool configuration settings include turning on developer mode to obtain the original data stream.

[0051] It should be noted that the analysis tool uses one of Wireshark and Obs Studio. The configuration settings include forcibly enabling 1080p and above resolutions, turning off AI enhancements such as beauty / noise reduction, and turning on developer mode to obtain the original data stream.

[0052] S104, prepare the video call environment, use light sources to adjust the video lighting and keep the environment quiet.

[0053] S2, select participants for the video call assessment and introduce the purpose and process of the assessment;

[0054] Specifically, the selection of participants first requires determining the number of people participating in the evaluation and selecting user groups of different ages and technical levels. After the participants are selected, the purpose, importance and expected results of the evaluation will be explained through conference training, as well as the detailed scoring mechanism and recording method.

[0055] It should be noted that the selection of evaluators and evaluated persons, and the clarification of assessment objectives and processes require strengthened training and training on the use of marking tools, including demonstration of the functions of preset marking buttons (video freeze, audio interruption and echo); guidance on how to quickly record problem phenomena ("10:05 The picture is blurry for 5 seconds"); emphasis on the need for marking to be immediate, concise and objective to avoid interfering with calls; and reiteration of traditional precautions (video angle, microphone distance).

[0056] S3, after setting evaluation indicators and scoring criteria for participants to learn and master, the evaluation indicators include subjective indicators and objective indicators;

[0057] Specifically, subjective indicators include participant satisfaction, call fluency, and audio and video quality perception, while objective indicators include call latency, image frame rate, and audio quality.

[0058] The scoring criteria are to develop quantitative scoring criteria and provide specific scoring instructions for each indicator. After the evaluation indicators and scoring criteria are developed, participants will be trained.

[0059] It's important to note that objective metrics (latency, packet loss rate) are fed back in real time to the QoS policy engine (connected to the SDN controller via an API). When latency exceeds 200ms, bandwidth expansion or path switching is automatically triggered. When users mark a "stutter" in real time, the system captures network data (TCP retransmission rate, Bufferbloat value) at that time to identify the root cause of the QoS failure.

[0060] S4: Conduct a video call with participants and collect call data. Organize participants to conduct the actual video call, ensuring that it is conducted under the same network conditions. At the same time, participants mark the status of the call in real time, focusing on key events and issues in the call.

[0061] Specifically, status tagging is when participants use tagging tools (instant messaging or tagging applications) to record the status and experience of the call in real time during the call, ensuring that the record can reflect the actual situation in a timely manner.

[0062] S5, collect the data recorded by the participants, insert customized questions based on the recorded data, and generate the questionnaire;

[0063] Specifically, data collection involves collecting data recorded by participants during calls, which contains detailed information on key issues and experiences;

[0064] Customized question insertion is to design customized questions based on the collected data, and combine the customized questions with evaluation indicators to generate a structured questionnaire to ensure that the questions are concise and easy to fill out.

[0065] S6: Distribute questionnaires to participants and collect feedback, extracting prominent issues and improvement suggestions from participants;

[0066] Specifically, questionnaires were distributed to participants via email and online questionnaire platforms. A feedback deadline was set to collect questionnaire data. Participants were encouraged to fill out the questionnaires, and support was provided to answer their questions. After the questionnaires were collected, common problems and improvement suggestions were extracted to form a preliminary feedback analysis report.

[0067] In S7, based on the participants’ suggestions for improvement and the data recorded by the equipment, targeted adjustments are made to the evaluation indicators and call processes, and an improvement plan is developed and implemented in the next evaluation.

[0068] Specifically, when formulating an improvement plan, we first use statistical tools to conduct quantitative analysis on the collected questionnaire data, then qualitatively analyze the participants' open feedback, and finally, based on the analysis results, determine the indicators and call processes that need to be improved, and formulate specific improvement measures and implementation timetables.

[0069] In embodiment 2, the present invention provides Figure 2 The evaluation system for high-definition video call perception shown is applied to the evaluation method for high-definition video call perception in Example 1. The evaluation system consists of video hardware equipment, a hardware configuration management module, a participant management module, a data acquisition module, a questionnaire design and feedback collection module, and a data analysis and improvement plan module.

[0070] Video hardware devices are hardware devices used to provide video calls;

[0071] The hardware configuration management module is used for configuration management of video hardware devices and environments. Its functions include device library management (4K camera / microphone array), network QoS configuration (bandwidth ≥ 50Mbps guarantee), environment simulation system (light source / background noise control), and device status monitoring (real-time detection of hardware anomalies).

[0072] The Participant Management module is used to select assessment participants for training and to introduce the assessment purpose and process. The Participant Management module ensures that participants have the necessary background knowledge and skills to provide effective feedback in the assessment.

[0073] The data collection module is used to conduct video calls and collect call data, while also recording participants' call experience and status in real time. As the core of the evaluation, the data collection module affects the quality of the data and the effectiveness of subsequent analysis.

[0074] The questionnaire design and feedback collection module is used to design customized questionnaires based on the collected data and distribute them to participants to collect feedback. The questionnaire design and feedback collection module converts the actual experience of the call into structured data for subsequent analysis and the formulation of improvement plans;

[0075] The data analysis and improvement plan module is used to conduct quantitative and qualitative analysis on the collected questionnaire and call data and formulate improvement plans. Based on the feedback and data analysis results, the data analysis and improvement plan module identifies problems and proposes improvement measures, thereby continuously optimizing the evaluation process.

[0076] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for evaluating high-definition video call perception, characterized in that: The evaluation method includes the following steps: S1. First, prepare a device with high-definition video calling capabilities, install the video calling software, and complete the video calling operation configuration. S2, select participants for the video call assessment and introduce the purpose and process of the assessment; S3, setting evaluation indicators and scoring criteria for participants to learn and master, the evaluation indicators include subjective indicators and objective indicators; S4, conduct a video call with the participants and collect call data, while the participants mark the status of the call in real time; S5, collect the data recorded by the participants, insert customized questions based on the recorded data, and generate the questionnaire; S6: Distribute questionnaires to participants and collect feedback, extracting prominent issues and improvement suggestions from participants; In S7, based on the participants’ suggestions for improvement and the data recorded by the equipment, targeted adjustments are made to the evaluation indicators and call processes, and an improvement plan is developed and implemented in the next evaluation.

2. The method for evaluating high-definition video call perception according to claim 1, wherein: The specific steps of the S1 operation include: For the S101, first select a device that supports high-definition video calls, including a 4K camera, an omnidirectional microphone array, noise-canceling headphones, and a stable light source; S102: Configure a test network with a bandwidth of ≥50 Mbps and QoS guarantee. S103, selecting video call software and analysis tools, and configuring and setting the software and analysis tools; S104, prepare the video call environment, use light sources to adjust the video lighting and keep the environment quiet.

3. The method for evaluating high-definition video call perception according to claim 2, wherein: The software configuration settings include video and audio input and output selection, network connection test and aspect ratio setting; The content of the analysis tool configuration setting is to enable developer mode to obtain the original data stream.

4. The method for evaluating high-definition video call perception according to claim 1, wherein: The selection of participants first requires determining the number of people participating in the evaluation and selecting user groups of different ages and technical levels. After the participants are selected, the purpose, importance and expected results of the evaluation will be explained through conference training, as well as a detailed scoring mechanism and recording method.

5. The method for evaluating high-definition video call perception according to claim 1, wherein: The subjective indicators include participant satisfaction, call fluency, and audio and video quality perception; the objective indicators include call delay, image frame rate, and audio quality; The scoring criteria are to develop quantitative scoring criteria and provide specific scoring instructions for each indicator. After the evaluation indicators and scoring criteria are developed, participants will be trained.

6. The method for evaluating high-definition video call perception according to claim 1, wherein: The status marking is that participants use marking tools to record the status and experience of the call in real time during the call.

7. The method for evaluating high-definition video call perception according to claim 1, wherein: The data collection mentioned above is to collect data recorded by participants during the calls, which contains details of key issues and experiences; The customized question insertion is to design customized questions based on the collected data, and combine the customized questions with evaluation indicators to generate a structured questionnaire.

8. The method for evaluating high-definition video call perception according to claim 1, wherein: The questionnaire distribution is carried out by sending questionnaires to participants via email and online questionnaire platform, and then setting a feedback deadline to collect questionnaire data. After the questionnaires are collected, common problems and improvement suggestions are extracted to form a preliminary feedback analysis report.

9. The method for evaluating high-definition video call perception according to claim 1, wherein: When formulating the improvement plan, the collected questionnaire data is first quantitatively analyzed using statistical tools, and then the open feedback from the participants is qualitatively analyzed. Finally, based on the analysis results, the indicators and call processes that need to be improved are determined, and specific improvement measures and implementation schedules are formulated.

10. A system for evaluating high-definition video call perception, applied to the method for evaluating high-definition video call perception according to any one of claims 1 to 9, characterized in that: The evaluation system consists of video hardware equipment, hardware configuration management module, participant management module, data acquisition module, questionnaire design and feedback collection module and data analysis and improvement plan module; The video hardware device is a hardware device used to provide video calls; The hardware configuration management module is used for configuration management of video hardware equipment and environment; The participant management module is used to select assessment participants for training and introduce the assessment purpose and process; The data acquisition module is used to conduct video calls and collect call data, while recording the call experience and status of participants in real time; The questionnaire design and feedback collection module is used to design customized questionnaires based on the collected data and distribute them to participants to collect feedback; The data analysis and improvement plan module is used to perform quantitative and qualitative analysis on the collected questionnaire and call data and formulate improvement plans.

Images