A service experience quality testing method, device and equipment and storage medium

Abstract

This application provides a service experience quality testing method, apparatus, device, and storage medium, relating to the field of information technology. The method is applied to a test node on the terminal side or network side, comprising: sending a test command to a service node on the terminal side, the test command instructing the service node to initiate a test process for a target service on the terminal side, the target service being a network application service on the terminal side; during the process of the service node responding to the test command and executing the test process for the target service, acquiring target test data for evaluating the service experience quality, the target test data including network performance indicators and service performance indicators; and inputting the target test data into a service evaluation model to determine the test values ​​of the target service experience quality indicators.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, apparatus, device and storage medium for testing business experience quality. Background Technology

[0002] In the field of wireless communication, quantifying service experience is crucial for evaluating users' subjective perception of quality, and it is typically measured using methods such as mean opinion score (MOS) or quality of experience (QoE). These methods define a series of performance indicators for typical services such as voice and video, thereby reflecting users' subjective feelings about the service experience.

[0003] Currently, common evaluation methods mainly rely on network equipment to statistically analyze service performance indicators, or on obtaining experience quality data through multiple manual service testing sessions conducted on terminal devices.

[0004] However, statistics based on the network side cannot obtain key performance indicators on the terminal side, leading to distorted evaluation results. Manual terminal testing, on the other hand, is cumbersome and inefficient. It is difficult to balance accuracy and efficiency in the evaluation. Summary of the Invention

[0005] This application provides a business experience quality testing method, apparatus, device, and storage medium to address the problem that business experience quality testing struggles to balance accuracy and efficiency in evaluation.

[0006] Firstly, this application provides a service experience quality testing method. This method is applied to a test node. The executing entity of this method can be an electronic device, a component or device applied to the electronic device (e.g., a processor, chip, or chip system), or a logic module or software capable of implementing all or part of the functions of the electronic device, including: Send test instructions to the service nodes on the terminal side. The test instructions are used to instruct the service nodes to initiate the test process of the target service on the terminal side. The target service is the network application service on the terminal side. During the testing process of business nodes responding to test commands and executing target services, target test data is obtained to evaluate the quality of the target service experience. The target test data includes network performance indicators and service performance indicators. Input the target test data into the business evaluation model to determine the test value of the target business experience quality index.

[0007] In the first aspect, test commands are issued from test nodes to terminal-side business nodes, automatically triggering the initiation of test processes for terminal-side network application services. This achieves automated startup of business testing without manual intervention in terminal-side test operations. During the execution of the target business test process at the business nodes, target test data covering network performance indicators and business performance indicators are collected simultaneously. This captures core performance information during business operation, solving the problems of scattered and single-dimensional indicator statistics in traditional evaluations. It provides comprehensive and objective quantitative data support for business experience quality evaluation. The multi-dimensional target test data is input into the business evaluation model, which automatically analyzes, processes, and outputs quantitative target business experience quality indicator test values. This replaces the traditional evaluation method that relies on subjective human judgment, forming a standardized automated evaluation process based on objective data. This improves the overall execution efficiency of business experience quality testing, avoids subjective biases in manual evaluation, and ensures the repeatability and consistency of evaluation results under different test scenarios.

[0008] In conjunction with the first aspect, in one possible implementation, during the test process where the business node responds to test commands and executes the target business, target test data for evaluating the quality of the target business experience is acquired, including: During the process of a business node responding to test commands and executing the test process of the target business, the first data generated by the business node and the network performance indicators are obtained. The network performance indicators are measured by the business node or network device. The business performance indicators are determined based on the first data, which includes at least one of the following: the business data packets of the target business, the log files of the business node, or the display record data of the business node.

[0009] In this implementation, during the execution of the test process at the business node, raw information reflecting the actual operational status of the business can be directly collected from the terminal side by acquiring the first data such as the business data packets, log files, or display record data generated by the node. Based on the parsing or analysis of this first data, business performance indicators can be obtained, accurately capturing key performance characteristics of the business layer, such as video stuttering and response latency. Combined with network performance indicators measured by the terminal or network devices, a complete test dataset covering both business and network perspectives can be constructed, thereby supporting a more comprehensive and accurate evaluation of the business experience.

[0010] In conjunction with the first aspect, in one possible implementation, the business evaluation model is either a model trained based on sample data or a pre-defined function mapping relationship; The sample data includes historical service performance indicators, historical network performance indicators, and historical target service experience quality indicators.

[0011] In this implementation, sample data is used for training, enabling the model to learn the complex and potential correlation between objective performance metrics and users' subjective feelings. The pre-defined function mapping reflects validated evaluation experience within the domain. Both approaches aim to establish a reliable and stable conversion relationship from measurable objective data to subjective experience.

[0012] In conjunction with the first aspect, in one possible implementation, the method further includes: If the test value of the target service experience quality indicator is less than the preset threshold, the first message is sent to the service node, and the first message instructs the service node to send an adjustment request to the network device. An adjustment request is used to request a network device to perform at least one of the following operations: Adjust the performance parameters corresponding to the target service; Alternatively, perform a preset network optimization operation, which is a network-side control and optimization operation for the target service being transmitted.

[0013] In this implementation, after obtaining the test value of the target service experience quality index, a preset threshold is introduced as a judgment benchmark. When the test value is lower than the threshold, a first message is automatically sent to the service node to instruct it to improve the experience quality. The service node is instructed to send an adjustment request to the network device. Adjusting performance parameters can directly change the service's demand for network transmission capabilities. Upon receiving the request, the network device can allocate appropriate transmission resources to the target service based on the adjusted parameters. Through this mechanism of service-side parameter adjustment and network-side resource allocation, the service data transmission process can be optimized, thereby specifically improving service transmission performance. On the other hand, executing preset network optimization operations can directly and proactively regulate and optimize the network side carrying the target service, enhancing the network's real-time guarantee capability for service experience and improving overall network resource utilization efficiency and service transmission stability.

[0014] In conjunction with the first aspect, in one possible implementation, a preset network optimization operation is performed, including at least one of the following: Adjust the transmission resources for the target service; Alternatively, adjust the priority of the target service transmission resources; Alternatively, troubleshoot the transmission links or device nodes corresponding to the target service. Alternatively, instruct the service node to switch cells.

[0015] In this implementation, the pre-defined network optimization operations performed by network devices include allocating more transmission resources, increasing scheduling priority, troubleshooting, or switching to a new cell. These operations directly address the network side, covering multiple key dimensions such as resource assurance, scheduling strategies, fault recovery, and access optimization. When service experience quality fails to meet standards, these network-side measures can be flexibly invoked to quickly address different network bottlenecks or problems. This provides diverse and direct network optimization paths, enhancing the system's adaptability and processing efficiency in ensuring and improving service experience in complex network environments.

[0016] In conjunction with the first aspect, in one possible implementation, the test instruction may include at least one of the following: target service type, service node identifier, test start time, test end time, number of times the service is initiated, or time interval between initiating the service.

[0017] This implementation allows for flexible configuration of information such as target business type, business node identifier, test start time, test end time, number of business initiations, or time interval between business initiations within the test instructions. This enables the testing process to be customized according to specific needs, such as conducting specialized tests for different business types or repeatedly initiating business at set intervals to obtain continuous evaluation data. Through flexible parameter configuration, it can adapt to diverse testing scenarios and evaluation requirements, improving the applicability and operability of test instructions. It also supports automated batch testing, reducing manual intervention and further enhancing the efficiency and flexibility of business experience evaluation.

[0018] Secondly, this application provides a business experience quality testing device, comprising: The instruction sending module is used to send test instructions to the service nodes on the terminal side. The test instructions are used to instruct the service nodes to initiate a test process for a target service on the terminal side. The target service is a network application service on the terminal side. The data acquisition module is used to acquire target test data for evaluating the experience quality of the target service during the test process of the service node responding to the test command and executing the target service. The target test data includes network performance indicators and service performance indicators. The quality assessment module is used to input the target test data into the business assessment model to determine the test values ​​of the target business experience quality indicators.

[0019] In conjunction with the second aspect, in one possible implementation, the data acquisition module is also used to acquire first data generated by the business node and network performance indicators during the process of the business node responding to test instructions and executing the test process of the target business. The network performance indicators are measured by the business node or network device. The business performance indicators are determined based on the first data, which includes at least one of the following: the business data packets of the target business, the log files of the business node, or the display record data of the business node.

[0020] In conjunction with the second aspect, in one possible implementation, the business evaluation model is either a model trained based on sample data or a pre-defined function mapping relationship; The sample data includes historical service performance indicators, historical network performance indicators, and historical target service experience quality indicators.

[0021] In conjunction with the second aspect, in one possible implementation, the device further includes a quality control module, which is used to send first information to the service node if the test value of the target service experience quality index is less than a preset threshold. The first information instructs the service node to send an adjustment request to the network device. An adjustment request is used to request a network device to perform at least one of the following operations: Adjust the performance parameters corresponding to the target service; Alternatively, perform a preset network optimization operation, which is a network-side control and optimization operation for the target service being transmitted.

[0022] In conjunction with the second aspect, in one possible implementation, the quality control module is also used to perform preset network optimization operations, including at least one of the following: Adjust the transmission resources for the target service; Alternatively, adjust the priority of the target service transmission resources; Alternatively, troubleshoot the transmission links or device nodes corresponding to the target service. Alternatively, instruct the service node to switch cells.

[0023] In conjunction with the second aspect, in one possible implementation, the test instruction may include at least one of the following: target service type, service node identifier, test start time, test end time, number of times the service is initiated, or time interval between initiating the service.

[0024] Thirdly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0025] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0026] Fifthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, it causes the electronic device to implement the method described in the first aspect.

[0027] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0028] Figure 1 A schematic diagram of the application environment for a service experience quality testing method provided in this application embodiment; Figure 2 A flowchart illustrating a service experience quality testing method provided in an embodiment of this application; Figure 3 A schematic diagram of the composition of a service experience quality testing device provided in this application embodiment; Figure 4 This is a schematic diagram of the composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The following is a detailed description, with reference to the accompanying drawings, of a service experience quality testing method, apparatus, device, and storage medium provided in this application.

[0030] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0031] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0032] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0033] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0034] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0035] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0036] In wireless communication networks, service experience quality is a key indicator for measuring network service levels and user satisfaction.

[0037] Currently, MOS and QoE methods are commonly used to quantify subjective user experience. MOS is primarily used for voice services, averaging subjective ratings from multiple users under different network conditions. QoE, as a core indicator reflecting the overall user experience, has had its performance metrics defined by standards organizations such as the 3rd Generation Partnership Project (3GPP) for typical service types. For example, for streaming media services, parameters such as average throughput, initial playback latency, buffering time, and video bitrate are specified, and the quality of user experience is evaluated based on these parameters.

[0038] Currently, common evaluation methods mainly fall into two categories: one is to indirectly infer the quality of service experience by statistically analyzing user service performance indicators through network equipment; the other is to directly obtain subjective user feedback by conducting manual service testing through terminal equipment.

[0039] However, because some key service performance indicators can only be collected from the terminal side, network devices cannot obtain complete data, leading to discrepancies between the service experience quality they infer and the actual user experience, resulting in poor evaluation effectiveness. Furthermore, the terminal testing method, which relies on manual intervention, is time-consuming and labor-intensive, making it difficult to achieve large-scale, routine service experience testing and resulting in low evaluation efficiency.

[0040] To address the aforementioned technical issues, this application provides a service experience quality testing method, apparatus, device, and storage medium. The approach involves: issuing test commands from a test node to a terminal-side service node, automatically triggering the testing process for the terminal-side network application service, thus automating the service testing process without manual intervention on the terminal side; simultaneously collecting target test data covering network performance indicators and service performance indicators during the execution of the target service test process by the service node, capturing core performance information during service operation, and solving the problems of scattered and single-dimensional indicator statistics in traditional evaluations, providing comprehensive and objective quantitative data support for service experience quality evaluation; inputting multi-dimensional target test data into a service evaluation model, which automatically analyzes, processes, and outputs quantitative target service experience quality indicator test values, replacing the traditional evaluation method that relies on subjective human judgment, forming a standardized automated evaluation process based on objective data, improving the overall execution efficiency of service experience quality testing, avoiding subjective biases in manual evaluation, and ensuring the repeatability and consistency of evaluation results under different testing scenarios.

[0041] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0042] This application provides a business experience quality testing method that can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes: terminal device 100 and network device 101, wherein terminal device 100 and network device 101 are interconnected.

[0043] Terminal device 100 supports access to network device 101, and can be a terminal device in a 5G or 6G communication system, specifically a user equipment (UE). Its types include, but are not limited to, smartphones, tablets, wearable smart devices, Internet of Things (IoT) terminals, vehicle-to-everything (V2X) terminals, as well as androids, drones, and industrial-grade sensing devices. The terminal accesses the access network via a wireless air interface, possessing the capability to carry voice services, data transmission services, and multimedia services. It can also achieve direct communication between different terminals based on device-to-device (D2D) direct connection technology or V2X technology. Terminal device 100 includes an intelligent agent service module, used to initiate intelligent agent services and execute corresponding tasks.

[0044] The application environment also includes test nodes, used to implement service experience quality testing functions. These can be applications (APPs), mini-programs, or other forms of software functional modules on terminal devices, integrated into terminal devices, network devices, or standalone physical devices. Test nodes send test commands to service nodes to initiate target service testing processes and are responsible for collecting and integrating network performance indicators and service performance indicators during the testing process to form target test data for evaluating experience quality. Test nodes also integrate or can invoke service evaluation models, which are used to determine the target service experience quality indicator test values ​​based on the target test data. Furthermore, when the experience quality indicator test values ​​do not meet the requirements, the test node can instruct the service node to initiate a network optimization request to the network device.

[0045] The application environment also includes service nodes, which initiate target services. These can be at least one of the following wireless communication services: voice, web browsing, file download, QR code scanning, video, extended reality (XR), cloud gaming, mobile gaming, live video streaming, and smart agents. They support message interaction with network devices and can be apps, mini-programs, or other software modules integrated into the terminal device. Service nodes can receive and respond to test instructions from test nodes and execute the corresponding test process for the target service. During testing, service nodes generate initial data, including service data packets, log files, or displayed record data. This data is used to determine service performance metrics. Service nodes can also send adjustment requests to network devices, as instructed by the test nodes, to request adjustments to the target service performance or to perform network optimization operations.

[0046] Network device 101 can be any wireless communication device, supporting any of the network standards such as 5G or 6G. Network device 101 interacts with service nodes and test nodes. During service testing, the network device is used to provide or assist in measuring network performance indicators. The network device can also receive adjustment requests from service nodes and perform corresponding operations based on these requests. These operations include adjusting performance parameters corresponding to the target service, adjusting transmission resources or resource priorities, troubleshooting transmission links or device nodes, or instructing service nodes to switch cells.

[0047] It should be noted that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0048] See Figure 2This is a flowchart illustrating a service experience quality testing method provided in an embodiment of this application. Figure 2 As shown, the business experience quality testing method provided in this application can be implemented through a test node, specifically including the following steps S200~S202.

[0049] S200, the test node sends test commands to the terminal-side service node.

[0050] The test node is an entity used to implement service experience quality testing functions, and can be deployed on the terminal side, the network side, or as a standalone device. The terminal-side service node is a terminal-side functional module used to initiate the target service. Please refer to the previous description of the application environment.

[0051] The test command instructs the service node to initiate a test process for the target service on the terminal side. The target service is a network application service on the terminal side. The target service being tested is a network application service that requires bidirectional data exchange between the terminal and network devices to achieve its functionality. For example, target services include voice, web browsing, file download, QR code scanning, video, XR, or smart agent services. The service node identifier is used to uniquely identify the corresponding service node and can be a service node name or number, such as a web browser, short video app, long video app, or smart agent service app.

[0052] The time interval for initiating a business test indicates the interval between two consecutive target business test initiation times. For example, a new short video browsing service may be initiated every minute. After receiving the test instruction, the business node initiates the target service at the test start time according to the information items in the test instruction, and repeats the initiation of the target service a specified number of times according to the time interval for initiating the service, until the initiation of the target service stops at the test end time.

[0053] S201. During the process of the service node responding to test commands and executing the test process of the target service, the test node acquires target test data used to evaluate the user experience quality of the target service. Target test data refers to the set of measurement data collected during the execution of the target service, used to quantitatively evaluate the quality of the user's perceived experience. Target test data includes network performance indicators and service performance indicators.

[0054] Network performance metrics indicate the transmission performance of the wireless network used by the target service, including the following metrics: signaling plane connection establishment delay, user plane connection establishment delay, uplink / downlink transmission delay, uplink / downlink retransmission rate, uplink / downlink delay jitter, uplink / downlink packet loss rate, uplink / downlink throughput, serving cell wireless signal strength / signal-to-noise ratio, etc.

[0055] Service performance metrics indicate the data transmission performance of a target service and are related to the type of target service; for example, for instant messaging services, the corresponding service performance metrics include data transmission rate, round-trip time (RTT), and other metrics.

[0056] For video services, the corresponding performance metrics include video encoding bitrate, video encoding frame rate, video encoding resolution, initial loading latency, video stuttering frequency, duration of a single stutter, data loss rate, end-to-end latency, downlink throughput, transmission control protocol (TCP) connection latency, RTT latency, TCP retransmission rate, and TCP packet loss rate.

[0057] For intelligent agent business, the corresponding business performance indicators include data performance indicators, interaction performance indicators, perception performance indicators, decision performance indicators, execution performance indicators, etc.

[0058] For voice services, the corresponding performance metrics include voice call success rate, call drop rate, connection establishment latency, and one-way communication ratio.

[0059] For web browsing services, the corresponding performance metrics include webpage response latency, download speed, TCP connection latency, RTT latency, TCP retransmission rate, and TCP packet loss rate.

[0060] S202. The test node inputs the target test data into the business evaluation model to determine the test value of the target business experience quality index.

[0061] The business evaluation model is a set of calculation rules or algorithms used to establish a quantitative correspondence between network performance indicators, business performance indicators, and user subjective experience. The business experience quality indicator test value is a quantitative score that characterizes the degree of user subjective perception of the overall experience of the target business in this test.

[0062] The input parameters of the business evaluation model are network performance metrics and service performance metrics from the target test data. The business evaluation model processes and maps the input metrics, outputting a numerical result, namely the test value of the target service experience quality metric. This test value is usually defined within a continuous numerical range, such as 1 to 5 points, with higher scores indicating a better subjective user experience.

[0063] By processing data through a business evaluation model, multiple objective indicators reflecting network and service performance are mapped to a score that represents the user's subjective experience. This business evaluation model determines the test values ​​for service experience quality indicators, transforming multidimensional and heterogeneous objective performance data into an intuitive and unified evaluation of experience quality.

[0064] In this embodiment, test commands are sent from the test node to the terminal-side service node, automatically triggering the test process of the terminal-side network application service. This achieves automated start-up of service testing without manual intervention in the terminal-side test operations. During the execution of the target service test process by the service node, target test data covering network performance indicators and service performance indicators are collected simultaneously. This captures core performance information during service operation, solving the problems of scattered and single-dimensional indicator statistics in traditional evaluations. It provides comprehensive and objective quantitative data support for service experience quality evaluation. The multi-dimensional target test data is input into the service evaluation model, which automatically analyzes, processes, and outputs quantitative target service experience quality indicator test values. This replaces the traditional evaluation method that relies on subjective human judgment, forming a standardized automated evaluation process based on objective data. This improves the overall execution efficiency of service experience quality testing, avoids subjective bias in manual evaluation, and ensures the repeatability and consistency of evaluation results under different test scenarios.

[0065] In one embodiment, step S201 includes: S2011. During the test process of the business node responding to test instructions and executing the target business, obtain the first data generated by the business node and obtain network performance indicators.

[0066] Network performance metrics are measured by service nodes or network devices.

[0067] When the test node executes the test process on the business node, it first obtains the first data generated by the business node.

[0068] Simultaneously, the test nodes acquire network performance metrics. These metrics are measured by service nodes or network devices. Specifically: Test nodes can receive network performance metrics measured by service nodes. That is, service nodes (terminal side) measure and obtain network performance metrics for the target service during the test period and send them to test nodes via message exchange.

[0069] Test nodes can also receive network performance metrics measured by network devices. Specifically, network devices measure and acquire network performance metrics for the target service test period and send them to the test nodes via message exchange.

[0070] S2012. Determine business performance indicators based on the first data.

[0071] The first data includes at least one of the following: the target service's business data packet, the business node's log file, or the business node's display record data.

[0072] Service performance metrics are obtained by parsing the service data packets of the target service. Specifically, the test node acquires and parses the service data packets of the target service from the terminal device, extracting service performance metrics such as video bitrate, resolution, frame rate, initial loading latency, and stuttering from the data files or data packet bitstreams.

[0073] Business performance metrics are obtained by analyzing the log files of business nodes. Specifically, the test node obtains the log files of the terminal device and extracts business performance metrics based on information such as timestamps, event types, specific behaviors, and log keywords contained in the log files. For example, relevant metrics such as video stuttering duration, number of stutters, initial playback latency, and bitrate can be extracted from the video business log files.

[0074] Business performance metrics are obtained by analyzing the display data (e.g., screen recording files) of business nodes. Specifically, the test node acquires screen recording files captured by the terminal device's screen recording tool, analyzes frame features using video processing algorithms or computer vision techniques, identifies abnormal events such as stuttering, and calculates business performance metrics.

[0075] In this embodiment, during the execution of the test process at the service node, by acquiring the first data such as the generated service data packets, log files, or display record data, raw information reflecting the actual operating status of the service can be directly collected from the terminal side. Based on the parsing or analysis of this first data, service performance indicators can be obtained, accurately capturing key performance characteristics of the service layer, such as video stuttering and response latency. Combined with network performance indicators measured by the terminal or network device, a complete test dataset covering both service and network perspectives can be constructed, thereby supporting a more comprehensive and accurate evaluation of the service experience.

[0076] In one embodiment, the business evaluation model is a model trained on sample data, or a pre-defined function mapping relationship. Models trained on sample data include regression models, decision tree models, random forest models, and deep learning models. Model parameters can be obtained through training with sample data.

[0077] The sample data includes historical service performance indicators, historical network performance indicators, and historical service experience quality labels.

[0078] Sample data can be obtained through business testing methods. For example, within a preset period, the test node controls the business node to initiate multiple target services, obtains the values ​​of each service performance indicator and network performance indicator corresponding to each service, and allows users to subjectively rate each service to obtain the corresponding service experience quality label value. Each service performance indicator, network performance indicator, and label value corresponding to a single service are used as sample data.

[0079] The business evaluation model can also be a pre-defined function mapping relationship, which can be determined based on historical data or expert experience. The function mapping relationship can be a linear or non-linear function relationship. The input to the function includes business performance indicators or network performance indicators. The output is a business experience quality indicator. For example, the business experience quality indicator = Wi is the weight of the i-th performance metric, and Ki is the normalized value of the i-th performance metric.

[0080] The purpose of defining a service evaluation model is to establish a mapping relationship between service performance indicators, network performance indicators, and service experience quality indicators. During the testing phase, the acquired target test data is input into this model to obtain the test values ​​of the target service experience quality indicators, thereby completing the evaluation of the target service experience quality. Because this model is built or calibrated based on service testing data, its mapping relationship can more accurately reflect the correlation between user subjective experience and objective performance indicators.

[0081] In one embodiment, the service experience quality testing method further includes: S203. If the target service experience quality indicator test value is less than the preset threshold, send the first message to the service node. After obtaining the target service experience quality indicator test value, the test node can determine whether the target service experience quality indicator test value is less than the preset threshold. The preset threshold is a pre-set standard numerical limit used to define whether the experience quality meets the standard.

[0082] If the test value is less than a preset threshold, the test node sends a first message to the service node. This first message instructs the service node to send an adjustment request to the network device. The first message is a control command generated by the test node and sent to the service node; its content instructs the service node to send an adjustment request to the network device to improve the target service experience quality.

[0083] The comparison between the target service experience quality indicator test value and the preset threshold constitutes the decision condition for whether to trigger the sending of the first message. The first message, as the output action of this judgment condition, carries specific improvement instructions.

[0084] In one possible implementation, the adjustment request is used to request the network device to perform at least one of the following operations: S2031. Adjust the performance parameters corresponding to the target service.

[0085] After receiving the initial information, the business node first adjusts the expected values ​​of the performance parameters.

[0086] Performance parameters indicate the minimum service transmission performance parameters for the target service. These parameters can be Quality of Service (QoS) or a 5G QoS identifier (5QI), such as Guaranteed Bit Rate (GBR), Packet Delay Budget (PDB), and Packet Error Rate (PER). A higher GBR results in a higher data transmission rate; a lower PDB indicates better real-time performance; and a lower PER indicates higher reliability.

[0087] Adjusting the expected performance parameters of a target service needs to be done according to preset rules. These preset rules include adjusting at least one of the expected performance parameters by preset step sizes. For example, increasing the guaranteed bit rate, decreasing the packet delay budget, or decreasing the packet error rate.

[0088] For example, a method for adjusting the desired performance parameters according to preset rules may include ensuring that the bit rate increases by 1 Mbps, the packet delay budget decreases by 1 ms, and the packet error rate decreases by 1 × 10⁻⁵.

[0089] Subsequently, the service node sends an adjustment request to the network device to adjust the performance parameters according to the service node's expected values.

[0090] An adjustment request is a message initiated by a service node to request the allocation of transmission resources based on the adjusted expected values ​​of performance parameters. It carries the target service identifier and the adjusted expected values ​​of performance parameters.

[0091] Adjustment requests can reuse existing air interface signaling, such as radio resource control (RRC) messages or non-access stratum (NAS) messages. Upon receiving an adjustment request, the network device will parse the target service identifier and the expected adjusted performance parameters. Based on this, it will allocate or reallocate corresponding radio resources, scheduling priorities, or transmission link configurations for the data transmission of the target service, ensuring that the actual transmission performance of the service meets the adjusted performance parameter requirements.

[0092] By sending an adjustment request, the business node formally submits its need for higher performance to the network side, triggering resource configuration adjustments on the network side. This is a key network interaction step for achieving a closed-loop improvement in business experience quality.

[0093] In one possible implementation, after completing step S2031, steps S200-S203 and S2031 can be repeated until the test value is greater than or equal to a preset threshold.

[0094] S2032. Perform preset network optimization operations. The adjustment request can also instruct network devices to perform preset network optimization operations.

[0095] The preset network optimization operation is a network-side control and optimization operation for transmitting the target service.

[0096] In one possible implementation, a preset network optimization operation is performed, including at least one of the following: Adjust the transmission resources for the target service.

[0097] Alternatively, adjust the priority of the target service transmission resources.

[0098] Alternatively, troubleshoot the transmission links or device nodes corresponding to the target service.

[0099] Alternatively, instruct the service node to switch cells.

[0100] Adjust the transmission resources for the target service. For example, increase the amount of physical resource block (PRB) resources and time slot resources called, configure multi-carrier aggregation and activate secondary carriers for service data transmission, and configure multi-antenna transmission.

[0101] Adjust the priority of transmission resources for the target service. For example, increase the resource scheduling priority and allocate resources more frequently for the data transmission of the target service.

[0102] Troubleshooting is performed on the transmission links or device nodes corresponding to the target service. For example, by obtaining alarm information or sending packet internet probe (PING) packets, it can be determined whether there are device nodes or transmission links that are faulty, thus ensuring the performance of service transmission through troubleshooting.

[0103] Instructing service nodes to switch cells to obtain better service transmission performance. In a communication network, a cell is a basic wireless signal coverage area, typically provided by a sector or antenna of a base station. The cell handover process refers to replacing the current serving cell with a target cell that offers better transmission performance.

[0104] The method for network devices to determine the target cell includes: the network device acquiring a measurement report reported by the service node, which contains wireless signal strength information of the serving cell and surrounding neighboring cells. From among neighboring cells whose signal strength difference with the serving cell is within a preset range and whose historical service performance statistics are better than those of the serving cell, the network device selects the neighboring cell with the best service performance statistics or the strongest signal strength, and identifies it as the target cell. The selected target cell is typically a cell with the same coverage as the serving cell and can provide better service transmission performance.

[0105] In one possible implementation, after completing step S2032, steps S200-S203 and S2032 can be repeated until the test value is greater than or equal to a preset threshold.

[0106] In this embodiment, after obtaining the test value of the target service experience quality index, a preset threshold is introduced as a judgment benchmark. When the test value is lower than the threshold, first information is automatically sent to the service node to instruct it to improve the experience quality. This instructs the service node to send an adjustment request to the network device. Adjusting performance parameters can directly change the service's demand for network transmission capabilities. Upon receiving the request, the network device can allocate appropriate transmission resources to the target service based on the adjusted parameters. Through this mechanism of service-side parameter adjustment and network-side resource allocation, the service data transmission process can be optimized, thereby specifically improving service transmission performance. Furthermore, executing preset network optimization operations can directly and proactively regulate and optimize the network side carrying the target service, enhancing the network's real-time guarantee capability for service experience and improving overall network resource utilization efficiency and service transmission stability.

[0107] The pre-defined network optimization operations performed by network devices include allocating more transmission resources, increasing scheduling priority, troubleshooting, or switching to a new cell. These operations directly address the network side, covering multiple key dimensions such as resource assurance, scheduling strategies, fault recovery, and access optimization. When service experience quality fails to meet standards, these network-side measures can be flexibly invoked to quickly address different network bottlenecks or problems. This provides diverse and direct network optimization paths, enhancing the system's adaptability and processing efficiency in ensuring and improving service experience in complex network environments.

[0108] In one embodiment, the test instruction may include at least one of the following: target service type, service node identifier, test start time, test end time, number of times to initiate a service, or time interval between initiating a service.

[0109] The test command carries several parameters controlling the service test behavior. The target service type specifies the type of network application to be tested, enabling the terminal to accurately launch the corresponding application. The service node identifier uniquely identifies the functional entity within the terminal responsible for executing the service, ensuring the test command is delivered to the correct processing unit. The test start time and test end time together define the start and end times of the entire test cycle, ensuring the terminal only performs service initiation operations within the specified time window. The number of service initiations defines the total number of times the service is repeatedly executed throughout the test cycle, while the service initiation time interval controls the waiting time between two adjacent service initiation actions. By jointly setting the number of initiations and the interval, service nodes can simulate user behavior at a preset frequency, thereby collecting sufficient and statistically significant test samples.

[0110] This implementation allows for flexible configuration of information such as target business type, business node identifier, test start time, test end time, number of business initiations, or time interval between business initiations within the test instructions. This enables the testing process to be customized according to specific needs, such as conducting specialized tests for different business types or repeatedly initiating business at set intervals to obtain continuous evaluation data. Through flexible parameter configuration, it can adapt to diverse testing scenarios and evaluation requirements, improving the applicability and operability of test instructions. It also supports automated batch testing, reducing manual intervention and further enhancing the efficiency and flexibility of business experience evaluation.

[0111] In this embodiment, sample data is used for training, enabling the model to learn the complex and potential correlation between objective performance indicators and users' subjective feelings. The pre-defined function mapping reflects validated evaluation experience within the domain. Both construction methods aim to establish a reliable and stable conversion relationship from measurable objective data to subjective experience.

[0112] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0113] This application embodiment can divide the service experience quality testing device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0114] In some embodiments, this application also provides a service experience quality testing apparatus. This service experience quality testing apparatus may include one or more functional modules for implementing the service experience quality testing method of the above method embodiments.

[0115] For example, Figure 3 This is a schematic diagram illustrating the composition of a service experience quality testing device provided in an embodiment of this application. Figure 3As shown, the service experience quality testing device 300 includes: an instruction sending module 301, a data acquisition module 302, and a quality evaluation module 303.

[0116] The instruction sending module 301 is used to send a test instruction to the service node on the terminal side. The test instruction is used to instruct the service node to initiate a test process for a target service on the terminal side. The target service is a network application service on the terminal side. The data acquisition module 302 is used to acquire target test data for evaluating the experience quality of the target service during the process of the service node responding to the test command and executing the test process of the target service. The target test data includes network performance indicators and service performance indicators. The quality assessment module 303 is used to input the target test data into the business assessment model to determine the test value of the target business experience quality index.

[0117] In one embodiment, the data acquisition module 302 is further configured to acquire first data generated by the service node and network performance indicators during the process of the service node responding to test instructions and executing the test process of the target service. The network performance indicators are measured by the service node or network device. The service performance indicators are determined based on the first data, which includes at least one of the following: service data packets of the target service, log files of the service node, or display record data of the service node.

[0118] In one embodiment, the business evaluation model is a model trained based on sample data, or a preset function mapping relationship.

[0119] The sample data includes historical service performance indicators, historical network performance indicators, and historical target service experience quality indicators.

[0120] In one embodiment, the device further includes a quality control module 304, which is used to send first information to the service node if the test value of the target service experience quality index is less than a preset threshold. The first information instructs the service node to send an adjustment request to the network device.

[0121] An adjustment request is used to request a network device to perform at least one of the following operations: Adjust the performance parameters corresponding to the target service.

[0122] Alternatively, perform a preset network optimization operation, which is a network-side control and optimization operation for the target service being transmitted.

[0123] In one embodiment, the quality control module 304 is further configured to perform a preset network optimization operation, including at least one of the following: Adjust the transmission resources for the target service.

[0124] Alternatively, adjust the priority of the target service transmission resources.

[0125] Alternatively, troubleshoot the transmission links or device nodes corresponding to the target service.

[0126] Alternatively, instruct the service node to switch cells.

[0127] In one embodiment, the test instruction may include at least one of the following: target service type, service node identifier, test start time, test end time, number of times to initiate a service, or time interval between initiating a service.

[0128] When implementing the functions of the integrated modules described above in hardware, this embodiment of the invention provides a possible schematic diagram of the electronic device involved in the above embodiments. For example... Figure 4 As shown, the electronic device 400 includes: a processor 402, a communication interface 403, and a bus 404. Optionally, the electronic device 400 may also include a memory 401.

[0129] Processor 402 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 402 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0130] Communication interface 403 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0131] The memory 401 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0132] As one possible implementation, the memory 401 can exist independently of the processor 402. The memory 401 can be connected to the processor 402 via a bus 404 and is used to store instructions or program code. When the processor 402 calls and executes the instructions or program code stored in the memory 401, it can implement the service experience quality testing method provided in this embodiment of the invention.

[0133] In another possible implementation, the memory 401 can also be integrated with the processor 402.

[0134] Bus 404 can be an extended industry standard architecture (EISA) bus, etc. Bus 404 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0135] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0136] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned service invocation device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned service invocation device. Further, the aforementioned computer-readable storage medium can include both internal storage units of the aforementioned service invocation device and external storage devices. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned service invocation device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0137] This application also provides computer instructions. All or part of the processes in the above method embodiments can be executed by computer instructions to instruct related hardware (such as computers, processors, network devices, and terminals). The program can be stored in the aforementioned computer-readable storage medium.

[0138] This application also provides a computer program product that, when run on a computer, causes the above-described method embodiments to be executed.

[0139] This application also provides a chip system. The chip system may be composed of chips or may include chips and other discrete devices, without limitation. The chip system includes a processor and a transceiver. All or part of the processes in the above method embodiments can be completed by this chip system, such as the chip system being used to implement the functions performed by the network devices or terminals in the above method embodiments.

[0140] In one possible design, the chip system further includes a memory for storing program instructions and / or data. When the chip system is running, the processor executes the program instructions stored in the memory to enable the chip system to perform the functions performed by the network device or terminal in the above method embodiments.

[0141] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for testing the quality of business experience, characterized in that, The method is applied to test nodes on the terminal side or network side, including: Send a test instruction to the service node on the terminal side. The test instruction is used to instruct the service node to initiate a test process for the target service on the terminal side. The target service is a network application service on the terminal side. During the process of the service node responding to the test command and executing the test process of the target service, target test data for evaluating the experience quality of the target service is obtained, and the target test data includes network performance indicators and service performance indicators; The target test data is input into the business evaluation model to determine the test value of the target business experience quality index.

2. The method according to claim 1, characterized in that, During the process of the business node responding to the test command and executing the test process of the target business, target test data for evaluating the experience quality of the target business is acquired, including: During the process of the service node responding to the test command and executing the test process of the target service, the first data generated by the service node and the network performance index are obtained, wherein the network performance index is measured by the service node or the network device; the service performance index is determined based on the first data, wherein the first data includes at least one of the following: the service data packet of the target service, the log file of the service node, or the display record data of the service node.

3. The method according to claim 1 or 2, characterized in that, The business evaluation model is a model trained based on sample data, or a preset function mapping relationship; The sample data includes historical service performance indicators, historical network performance indicators, and historical target service experience quality indicator test values.

4. The method according to claim 1, characterized in that, The method further includes: If the test value of the target service experience quality index is less than a preset threshold, a first message is sent to the service node, and the first message instructs the service node to send an adjustment request to the network device; The adjustment request is used to request the network device to perform at least one of the following operations: Adjust the performance parameters corresponding to the target service; Alternatively, a preset network optimization operation may be performed, which is a network-side control and optimization operation for transmitting the target service.

5. The method according to claim 4, characterized in that, The execution of the preset network optimization operation includes at least one of the following: Adjust the transmission resources of the target service; Alternatively, adjust the priority of the target service transmission resources; Alternatively, troubleshoot the transmission link or device node corresponding to the target service; Alternatively, instruct the service node to switch cells.

6. The method according to claim 1, characterized in that, Test instructions may include at least one of the following: target service type, service node identifier, test start time, test end time, number of times to initiate a service or time interval between initiating a service.

7. A business experience quality testing device, characterized in that, include: The instruction sending module is used to send test instructions to the service nodes on the terminal side. The test instructions are used to instruct the service nodes to initiate a test process for a target service on the terminal side. The target service is a network application service on the terminal side. The data acquisition module is used to acquire target test data for evaluating the experience quality of the target service during the test process of the service node responding to the test command and executing the target service. The target test data includes network performance indicators and service performance indicators. The quality assessment module is used to input the target test data into the business assessment model to determine the test values ​​of the target business experience quality indicators.

8. An electronic device, characterized in that, The device includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the business experience quality testing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the business experience quality testing method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the service experience quality testing method as described in any one of claims 1 to 6.

Images