Application service testing methods, systems, electronic devices, and storage media

Abstract

This application provides a testing method, system, electronic device, and storage medium for application services. In this application, the testing terminal can simulate various weak network environments, control various types of playback terminals, and analyze various performance indicators of application services. This enables automated testing of application services to cover various weak network environments, various terminal types, and various performance indicators, thereby achieving automated testing of application services, improving testing efficiency, increasing the accuracy of test results, and reducing testing costs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of testing, in particular to a testing method and system of application service, an electronic device and a storage medium. BACKGROUND

[0002] For various PaaS (Platform as a Service) services related to streaming media such as on-demand, live broadcast and real-time audio and video, it is necessary to optimize various performance indicators of PaaS services under different networks, such as latency, stalling rate, second opening rate and start playing success rate.

[0003] Generally, the performance of PaaS services is tested in an online gray scale manner. The online gray scale manner groups real online users, collects performance indicators corresponding to each group and user experience data of users of each group using PaaS services, analyzes the collected data, and obtains performance test results of PaaS services. However, this performance test method has low test efficiency and inaccurate test results. SUMMARY

[0004] Aspects of the present application provide a testing method and system of application service, an electronic device and a storage medium to improve the test efficiency and accuracy of the test results of the application service.

[0005] The present application provides a testing method of application service, which is applied to a test end. The method comprises: selecting a weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and building a current weak network environment according to the current weak network configuration strategy; triggering a plurality of playing ends running the application service to access the current weak network environment respectively, the plurality of playing ends corresponding to a plurality of terminal types; triggering a pushing end to push each audio and video frame in media stream data to the plurality of playing ends, adding a time identifier to the currently pushed audio and video frame, and triggering the plurality of playing ends to play the currently received audio and video frame respectively; determining a performance indicator of a first dimension of the application service according to log data of the plurality of playing ends respectively, and determining a performance indicator of a second dimension of the application service according to the time identifier of each audio and video frame; and determining a performance test result of the application service under the current weak network environment according to the performance indicator of the first dimension and the performance indicator of the second dimension.

[0006] The embodiment of the application further provides a test system of an application service, the system comprising: a test terminal, a plurality of playing terminals and a pushing terminal, the playing terminals and the pushing terminal performing data transmission through a transmission network; the test terminal is configured to: select a weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and build a current weak network environment according to the current weak network configuration strategy; trigger a plurality of playing terminals running the application service to access the current weak network environment respectively, the plurality of playing terminals corresponding to a plurality of terminal types; trigger the pushing terminal to push each audio and video frame in media stream data to the plurality of playing terminals, and add a time identifier to the current pushed audio and video frame, and trigger the plurality of playing terminals to play the current received audio and video frame respectively; determine a performance index of a first dimension of the application service according to log data of the plurality of playing terminals respectively, and determine a performance index of a second dimension of the application service according to the time identifier of each audio and video frame; and determine a performance test result of the application service in the current weak network environment according to the performance index of the first dimension and the performance index of the second dimension. The working principle of the system can be referred to the related description in the foregoing method embodiment, and will not be described here.

[0007] The embodiment of the application further provides an electronic device, comprising: a memory and a processor; the memory is configured to store a computer program; the processor is coupled to the memory and is configured to execute the computer program to perform the steps in the test method of the application service.

[0008] The embodiment of the application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement the steps in the test method of the application service.

[0009] In the embodiment of the application, the test terminal can simulate a variety of weak network environments, control a variety of playing terminals of terminal types and analyze a variety of performance indexes of the application service, so that the automatic test of the application service can cover a variety of weak network environments, cover a variety of terminal types and cover a variety of performance indexes, the automatic test of the application service is realized, the test efficiency of the application service is improved, the accuracy of the test result is improved, and the test cost is low. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings used to provide further understanding of the application and form a part of the application, the illustrative embodiments of the application and the description thereof serve to explain the application, and do not constitute improper limitation on the application. In the drawings:

[0011] Figure 1 A schematic diagram of an iteration cycle of an application service;

[0012] Figure 2a An exemplary application scenario provided by the embodiment of the application;

[0013] Figure 2b Another exemplary application scenario provided for the embodiments of the present application is shown in FIG. 3;

[0014] Figure 3 A flowchart of a method for testing an application service provided for the embodiments of the present application is shown in FIG. 4;

[0015] Figure 4 An exemplary time delay division diagram is shown in FIG. 5;

[0016] Figure 5 A structural schematic diagram of a testing device for an application service provided for the embodiments of the present application is shown in FIG. 6;

[0017] Figure 6 A structural schematic diagram of an electronic device provided for the embodiments of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0019] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the access relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, and A and B can be singular or plural. In the textual description of the present application, the character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, in the embodiments of the present application, "first", "second", "third", etc. are only to distinguish the contents of different objects, and have no other special meanings.

[0020] Figure 1An iteration cycle of a PaaS service. The iteration cycle of the PaaS service includes, in sequence, a PaaS development, a PaaS test, a customer integration, a customer test, and a customer gray-scale verification. In the PaaS development, a developer of the PaaS service provider performs a development task of the PaaS service. In the PaaS test, an operator of the PaaS service provider performs a performance test task of the PaaS service. In the customer integration, a user of the PaaS service integrates the PaaS service in a service product of the user. In the customer test, an operator of the user of the PaaS service performs a performance test task of the service product after the integration of the PaaS service. In the customer gray-scale verification, the operator of the user of the PaaS service performs a gray-scale verification of the service product after the integration of the PaaS service.

[0021] Currently, the PaaS service is tested in performance by using an online gray-scale method. This method has the following advantages: ①, it can verify the influence of different optimization strategies on the performance indicators of the PaaS service, and the improvement of the performance indicators is the ultimate goal of technical optimization. ②, it can better avoid the problem of "overfitting" of the test sample by the laboratory method. The real users online often cover various weak network environments due to different operators, different geographical regions, and different times, while the number of laboratory samples is limited, which leads to difficulty in complete coverage. However, the PaaS service uses the online gray-scale method for testing, which faces many challenges: ①, the iteration cycle is long, especially when the customer needs to update the client SDK (Software Development Kit), which may take up to a month. In the extreme case, if the test results of the PaaS test node of the PaaS service provider and the test results of the customer test node of the PaaS service user are quite different, it will greatly affect the customer experience. ②, the fragmentation of the long-tail customers of the PaaS service further limits the use of the online gray-scale method. First, different customers have different scenarios, and the optimization strategy of one customer cannot be simply transferred to another customer; second, enough samples are needed for gray-scale verification to demonstrate which optimization strategy is better, and small and medium-sized customers may not have enough gray-scale user volume; finally, sometimes the customer does not want their users to participate in the test of optimizing the PaaS service.

[0022] Currently, the performance of PaaS services is usually tested in an online grayscale manner. However, the test efficiency is low and the test result is not accurate enough. To this end, the embodiments of the present application provide a test method for PaaS services, an electronic device and a storage medium. In the embodiments of the present application, the test end can simulate a variety of weak network environments, control a variety of terminal types of the playback end, and analyze a variety of performance indicators of the PaaS service, so that the automatic test for the PaaS service can cover a variety of weak network environments, cover a variety of terminal types, and cover a variety of performance indicators, the automatic test for the PaaS service is realized, the test efficiency of the PaaS service is improved, the accuracy of the test result is improved, and the test cost is low. In addition, the test method is a new laboratory analysis method, which can effectively solve the overfitting problem existing in the traditional laboratory analysis method. In the traditional laboratory analysis method, there may be network dimension overfitting problem, that is, the performance indicators of the PaaS service are optimized under one network, but the performance indicators are deteriorated under another network; there may be performance indicator dimension overfitting problem, that is, one performance indicator of the PaaS service is optimized, but another performance indicator is deteriorated; there may be terminal type dimension overfitting problem, that is, the performance indicators of the PaaS service are optimized under one terminal type, but the performance indicators are deteriorated under another terminal type.

[0023] Figure 2a An example application scenario is provided for the embodiments of the present application. In the application scenario, the test end can build a variety of weak network environments; trigger the playback end running the application service to access any weak network environment covering a variety of terminal types; the test end also triggers the push stream end to push media stream data, and the push stream end adds a time identifier to the audio and video frames in the media stream data when pushing the audio and video frames. The audio and video frames with the added time identifier are distributed through a transmission network such as a content delivery network (CDN), transmitted to the playback end through a downlink, and the test end also triggers the playback end to play the audio and video frames with the added time identifier. The test end can also collect log data of a plurality of playback ends respectively, determine the performance indicators of the first dimension of the application service through analysis of the log data, determine the performance indicators of the second dimension of the application service through analysis of the time identifier of the audio and video frames, and finally determine the performance test result of the application service under the current weak network environment based on the performance indicators of the first dimension and the performance indicators of the second dimension.

[0024] The playing terminal, the pushing terminal or the testing terminal can be a terminal device or a server. The terminal device can be hardware or software. When the terminal device is hardware, the terminal device is, for example, a mobile phone, a tablet computer, a desktop computer, a wearable smart device, a smart home device, etc. When the terminal device is software, the terminal device can be installed in the hardware devices listed above, and the terminal device is, for example, a plurality of software modules or a single software module, etc. The server can be hardware or software. When the server is hardware, the server is a single server or a distributed server cluster composed of multiple servers. When the server is software, the server can be a plurality of software modules or a single software module, etc. The embodiments of the present application are not limited.

[0025] Figure 2b Another exemplary application scenario provided by the embodiments of the present application is shown in the figure. A traffic controller such as Linux Traffic Control is installed in a Linux host, and the traffic controller is used to build an automatic control weak network environment. The traffic controller can also use various automation tools to automatically control the UI (User Interface) of the pushing terminal and the playing terminal. The automation tools are, for example, Sikuli, which is an automatic tool software based on OpenCV image recognition technology. The user behavior is simulated by UI automation to complete the test of the user interface. With the help of UI automation, the host can control the pushing terminal to start, push and close, etc. The host can also control the playing terminal to start, play and close, etc. to realize the complete automation of the pulling behavior. In addition, the playing terminal establishes a remote long connection with the host, and interacts through a private protocol. The playing terminal can program the network parameters of the weak network environment at any time before, during and after playing, by calling the Linux Traffic Control command, to realize the programmability of the network parameters of the weak network environment.

[0026] Specifically, the playing terminal programs the network parameters of the weak network environment by calling the Linux Traffic Control command, the host builds the corresponding current weak network environment based on the network parameters of the weak network environment, and the host triggers the playing terminal to access the built current weak network environment. The host triggers the pushing terminal to push the media stream, and the media stream is transmitted to the playing terminal through a transmission network such as CDN. The pushing terminal stamps a two-dimensional code including a time identifier in the pushed audio and video frames when pushing the audio and video frames in the media stream. The playing terminal is triggered to play the received media stream.

[0027] In the performance index analysis stage, the center control and data analysis module can perform online or offline analysis. Specifically, the center control and data analysis module collects log data of the pusher, the player, the flow controller and the transmission network, analyzes the performance indexes of the PaaS service such as the latency information, the stall rate, the second opening rate and the playing success rate based on the log data. In addition, the audio and video frames pushed by the pusher at the same time and the audio and video frames played by the player are sent into a mixer for mixing processing to obtain mixed audio and video frames; the delay analysis module identifies the two-dimensional code in the mixed audio and video frames to obtain the time identifier corresponding to the audio and video frames pushed by the pusher and the time identifier corresponding to the audio and video frames played by the player, and analyzes the latency information of the PaaS service based on the time identifier corresponding to the audio and video frames pushed by the pusher and the time identifier corresponding to the audio and video frames played by the player. The center control and data analysis module can also analyze the stall rate of the PaaS service based on the time identifiers corresponding to the adjacent two audio and video frames played by the player. Further, the center control and data analysis module cross- verifies the performance indexes based on the log analysis and the performance indexes obtained based on the time identifiers corresponding to the audio and video frames to determine whether the performance indexes based on the log analysis are reliable. In the case that the performance indexes based on the log analysis are reliable, the performance indexes based on the log analysis determine the performance test result of the PaaS service in the current weak network environment.

[0028] It is worth noting that the Linux Traffic Control command can be called to program the network parameters of different weak network environments to highly simulate the real and changing weak network environment and ensure the confidence of the network dimension. Various types of players of different terminal types can be prepared to ensure the confidence of different terminal implementations; various performance indexes can be analyzed to ensure the confidence of the customer required scene; and then the weak network environment, the performance index, the terminal type and other multiple dimensions are densely covered to increase the number of laboratory samples as much as possible, solve the problem of easy "overfitting" of the traditional laboratory method, and have high confidence.

[0029] The Linux Traffic Control command can realize the completeness and programmability of the network parameters of the weak network environment; the Linux Traffic Control is used to build an automatically controlled network, so that the players and the pushers of different terminal types are connected to the network to ensure the consistency of the network behavior of different terminal types; only a small amount of hardware devices such as Linux hosts and mixers are needed to complete, which has the advantage of low cost.

[0030] In summary, Figure 2b The PaaS service performance test system has the advantages of low cost, high efficiency and high confidence.

[0031] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0032] Figure 3 A flowchart of a method for testing an application service is provided in an embodiment of the present application. Referring to Figure 3 , the method can include the following steps:

[0033] 301. Selecting one weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and building a current weak network environment according to the current weak network configuration strategy.

[0034] 302. Triggering a plurality of players running an application service to access the current weak network environment respectively, the plurality of players covering a plurality of terminal types.

[0035] 303. Triggering a pusher to push each audio and video frame in media stream data to the plurality of players, adding a time identifier to the currently pushed audio and video frame, and triggering the plurality of players to play the currently received audio and video frame respectively.

[0036] 304. Determining a performance indicator of a first dimension of the application service according to log data of the plurality of players respectively, and determining a performance indicator of a second dimension of the application service according to the time identifier of each audio and video frame.

[0037] 305. Determining a performance test result of the application service in the current weak network environment according to the performance indicator of the first dimension and the performance indicator of the second dimension.

[0038] The method provided in the embodiment can be executed by a test terminal. The test terminal can test any application service, which includes but is not limited to PaaS service, SaaS (Infrastructure as a service, Infrastructure as a Service) service and IaaS (Software as a Service, Software as a Service) service.

[0039] In a real scenario, weak network environments change rapidly. For example, weak network environments appear when a user enters or exits an elevator, a carrier switches or is preempted by other users' burst traffic. In the embodiment, the test terminal is configured with a plurality of weak network configuration strategies, one weak network configuration strategy can simulate one weak network environment, and a plurality of weak network configuration strategies can simulate a plurality of weak network environments, thereby highly simulating the rapidly changing weak network environments in the real scenario.

[0040] In a specific application, a weak network configuration strategy can be defined from multiple network indicators such as bandwidth, packet loss, packet loss type, latency, latency type, jitter, and router cache. Bandwidth (also referred to as throughput) refers to the amount of data transmitted per unit of time, and the unit is usually bits per second (bps); bandwidth reflects the transmission capacity of the network, and the larger the better. Packet loss can be represented by the number of data packet losses, that is, the difference between the number of sent data packets and the number of received data packets; packet loss reflects the network reliability, and the smaller the better. Packet loss types include, but are not limited to, random packet loss, periodic packet loss (Loss-Cycle), fixed probability packet loss (Loss-Random), and burst packet loss (Loss-Burst). Latency refers to the time taken from the start of data packet transmission to reception, and the unit is usually milliseconds (ms); latency reflects the speed of the network, and the smaller the better. Latency types include, but are not limited to, constant latency, latency conforming to uniform distribution, and latency conforming to normal distribution. Among them, constant latency can be understood as fixed-size latency. Jitter refers to the variation of latency, which can be obtained from the variance of multiple latencies; jitter reflects the stability of the network, and the smaller the better. The size of the router cache can have multiple specifications, thereby covering different sizes of router caches.

[0041] In this embodiment, one weak network configuration strategy is selected from the plurality of pre-configured weak network configuration strategies as a current weak network configuration strategy, and a current weak network environment is built based on the current weak network configuration strategy. After the current weak network environment is built, the test end can trigger a plurality of playing ends running application services to access the current weak network environment respectively. Specifically, the test end can send a network access instruction to each playing end, and the network access instruction includes network access parameters of the current weak network environment, such as network identification of the current weak network environment and a password required for accessing the current weak network environment. Each playing end responds to the network access instruction and accesses the current weak network environment based on the network access parameters.

[0042] In this embodiment, the plurality of playing ends cover multiple terminal types, thereby highly simulating the diverse terminal types in real scenarios. Terminal types can be defined from multiple aspects such as device models, operating system types, and player types. It can be understood that terminal types are different due to different device models, operating system types, or player types. For example, playing end 1 corresponds to a certain brand of mobile phone with an Android system and using player 1; playing end 2 corresponds to a certain brand of mobile phone with an iOS system and using player 1; playing end 1 and playing end 2 belong to different terminal types respectively.

[0043] In this embodiment, after confirming that each playing end accesses the current weak network environment, the test end can trigger the stream pushing end to push media stream data externally, and trigger each playing end to play the received media stream data.

[0044] In the embodiment, when the push stream end pushes the audio and video frames in the media stream data, a time identifier is added to the currently pushed audio and video frame, and the time identifier can indicate the pushing time of the currently pushed audio and video frame. In actual application, the time identifier can be used as the associated data of the currently pushed audio and video frame and is transmitted synchronously with the audio and video frame. Further, in order to facilitate the management and maintenance of the time identifier of the audio and video frame, a two-dimensional code carrying the time identifier can be superimposed in the currently pushed audio and video frame, and the time identifier corresponding to the audio and video frame can be obtained by identifying the two-dimensional code subsequently.

[0045] In the embodiment, the performance indicators include, for example, but are not limited to, latency information, freezing rate, second opening rate and playing success rate. The test end can collect the log data of each of the plurality of playing ends, and determine the performance indicators of the application service in the first dimension by analyzing the log data. The performance indicators in the first dimension are the performance indicators obtained by log analysis.

[0046] In the embodiment, the performance indicators can include latency information. Accordingly, according to the log data of each of the plurality of playing ends, one optional implementation manner of determining the performance indicators of the application service in the first dimension is as follows: according to the log data of the push stream end and the third playing end, the latency information of a third audio and video frame in each stage of the life cycle from being pushed by the push stream end to being played by the third playing end is determined, the third audio and video frame is one audio and video frame in the media stream data, and the third playing end is one of the plurality of playing ends; according to the latency information of the third audio and video frame in each stage of the life cycle, the final latency information of the third audio and video frame is determined; and according to the final latency information of at least one third audio and video frame, the latency information of the application service in the first dimension under the terminal type of the third playing end is determined.

[0047] Specifically, the log analysis manner supports accumulating the latency information of each stage in the life cycle in a segment-by-segment accumulation manner. For example, the push stream latency, the uplink latency, the transmission latency, the uplink latency and the playing latency are accumulated to obtain the final latency information between the push stream end and the playing end. Among them, the push stream latency can be obtained by accumulating the latency of the collection node, the latency of the preprocessing node, the latency of the encoding node and the latency of the sending buffer area; and the playing latency can be obtained by accumulating the latency of the receiving buffer area, the latency of the decoding node and the latency of the rendering node. Figure 4 For example, the push stream latency, the uplink latency, the transmission latency, the uplink latency and the playing latency are accumulated to obtain the final latency information between the push stream end and the playing end. Among them, the push stream latency can be obtained by accumulating the latency of the collection node, the latency of the preprocessing node, the latency of the encoding node and the latency of the sending buffer area; and the playing latency can be obtained by accumulating the latency of the receiving buffer area, the latency of the decoding node and the latency of the rendering node.

[0048] In the embodiment, the latency information of the third audio and video frame in each stage of the life cycle is accumulated to obtain the final latency information of the third audio and video frame. Then, the final latency information of a plurality of third audio and video frames is weighted and summed or averaged to obtain the latency information of the application service in the first dimension under the terminal type of the third playing end.

[0049] In the embodiment, the performance indicator can include a freezing rate. Accordingly, according to the log data of each of the plurality of playback terminals, an optional implementation manner of determining the performance indicator of the first dimension of the application service is as follows: determining the freezing rate of the first dimension of the application service under the terminal type corresponding to the fourth playback terminal according to the playback time of the adjacent two audio and video frames recorded in the log information of the fourth playback terminal, the fourth playback terminal being one of the plurality of playback terminals.

[0050] Specifically, the playback time difference corresponding to the adjacent two audio and video frames is determined based on the playback time of the adjacent two audio and video frames. If the playback time difference is greater than a preset freezing duration, the playback time difference is taken as a freezing time. The total freezing time is obtained by accumulating the freezing times corresponding to the plurality of adjacent two audio and video frames. The ratio between the total freezing time and the total playback duration of the media stream data played by the fourth playback terminal is calculated, and the ratio is taken as the freezing rate.

[0051] In the embodiment, the performance indicator can include a second opening rate. Accordingly, according to the log data of each of the plurality of playback terminals, an optional implementation manner of determining the performance indicator of the first dimension of the application service is as follows: obtaining the playback time consumption of at least one fourth audio and video frame from the log information of the fifth playback terminal, the fifth playback terminal being one of the plurality of playback terminals, and the fourth audio and video frame being the first played audio and video frame of the fifth playback terminal. The proportion of the fourth audio and video frame with a playback time consumption less than 1 second in the at least one fourth audio and video frame is determined as the second opening rate of the first dimension of the application service under the terminal type corresponding to the fifth playback terminal.

[0052] In the embodiment, the performance indicator can include a playback success rate. Accordingly, according to the log data of each of the plurality of playback terminals, an optional implementation manner of determining the performance indicator of the first dimension of the application service is as follows: obtaining the total number of plays and the number of successful plays of the sixth playback terminal from the log information of the sixth playback terminal, the sixth playback terminal being one of the plurality of playback terminals; and determining the playback success rate of the first dimension of the application service under the terminal type corresponding to the sixth playback terminal according to the number of successful plays and the total number of plays.

[0053] Specifically, the total number of plays refers to the total number of times of starting to play of the sixth playback terminal, and the number of successful plays refers to the number of times of successfully playing the media stream data of the sixth playback terminal. The ratio between the number of successful plays and the total number of plays is taken as the playback success rate.

[0054] In the embodiment, the test terminal can also determine the performance indicator of the second dimension of the application service by analyzing the time identifiers of the audio and video frames. The performance indicator of the second dimension is the performance indicator obtained by analyzing the time identifiers of the audio and video frames.

[0055] In the embodiment, the performance index can include time delay information, and accordingly, according to the time identifiers of the respective audio and video frames, an optional implementation manner of determining the performance index of the second dimension of the application service is: simultaneously acquiring a first audio and video frame currently pushed by the push streaming end and a second audio and video frame currently played by a first playing end, the first playing end being one of the multiple playing ends; acquiring a time identifier of the first audio and video frame and a time identifier of the second audio and video frame; and determining the time delay information of the second dimension of the application service under the terminal type of the first playing end according to the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

[0056] Specifically, the test end acquires the first audio and video frame currently pushed by the push streaming end and the second audio and video frame currently played by the first playing end at the same time point; calculates a time difference value between a time corresponding to the time identifier of the first audio and video frame and a time corresponding to the time identifier of the second audio and video frame, and takes the time difference value as the time delay information of the second dimension of the application service under the terminal type of the first playing end.

[0057] In some optional embodiments, when the push streaming end adds the time identifier to the currently pushed audio and video frame, a two-dimensional code carrying the time identifier is superimposed on the currently pushed audio and video frame. Accordingly, when the time identifier of the first audio and video frame and the time identifier of the second audio and video frame are acquired, the first audio and video frame and the second audio and video frame can be mixed to obtain a mixed audio and video frame; and the mixed audio and video frame is subjected to two-dimensional code recognition to acquire the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

[0058] Specifically, the first audio and video frame and the second audio and video frame are both superimposed with the two-dimensional code carrying the time identifier; the first audio and video frame and the second audio and video frame are input to a mixer for mixed processing to obtain a mixed audio and video frame; and the mixed audio and video frame is subjected to two-dimensional code recognition by using two-dimensional code recognition technology to acquire the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

[0059] In the embodiment, the performance index can include a stall rate, and accordingly, according to the time identifiers of the respective audio and video frames, an optional implementation manner of determining the performance index of the second dimension of the application service is: for a second playing end of the multiple playing ends, determining the stall rate of the second dimension of the application service under the terminal type corresponding to the second playing end according to the respective time identifiers of two adjacent audio and video frames played by the second playing end.

[0060] Specifically, based on the time identifiers in the two adjacent audio and video frames respectively corresponding to the time, the playing time difference corresponding to the two adjacent audio and video frames is determined; if the playing time difference is greater than the preset stall duration, the playing time difference is taken as a stall time; the stall times corresponding to the plurality of adjacent two audio and video frames are accumulated to obtain a total stall time; the ratio between the total stall time and the total playing duration of the media stream data played by the second playing end is calculated, and the ratio is taken as a stall rate.

[0061] In the embodiment, after determining the performance indicators of the application service in the first dimension and the performance indicators of the application service in the second dimension, the test end can determine the performance test result of the application service in the current weak network environment according to the performance indicators in the first dimension and the performance indicators in the second dimension.

[0062] In actual application, for the same performance indicator, the larger or smaller value can be selected from the performance indicators in the first dimension and the performance indicators in the second dimension as the actual performance indicator obtained by the test, or the performance indicators in the first dimension and the performance indicators in the second dimension are weighted and summed or averaged to obtain the actual performance indicator; then, according to the comparison result between the actual performance indicator and the expected performance indicator of the application service, the performance test result of the application service in the current weak network environment is generated. Specifically, if the actual performance indicator is close to the expected performance indicator, it is determined that the performance test result of the application service in the current weak network environment is passed; if the actual performance indicator is greatly different from the expected performance indicator, it is determined that the performance test result of the application service in the current weak network environment is failed. If the test is failed, the developer needs to optimize the application service and continuously improve the application service until the test is passed. The actual performance indicator being close to the expected performance indicator can be considered that the difference between the actual performance indicator and the expected performance indicator falls within the set difference range; the actual performance indicator being greatly different from the expected performance indicator can be considered that the difference between the actual performance indicator and the expected performance indicator does not fall within the set difference range.

[0063] Further optionally, in order to improve the accuracy of the test result, the optional implementation manner of determining the performance test result of the application service in the current weak network environment according to the performance indicators in the first dimension and the performance indicators in the second dimension is: according to the comparison result of the performance indicators in the first dimension and the performance indicators in the second dimension, the performance indicators in the first dimension are verified for credibility; in the case that the performance indicators in the first dimension pass the credibility verification, the performance test result of the application service in the current weak network environment is determined according to the performance indicators in the first dimension.

[0064] Specifically, the performance indicator of the first dimension is obtained through log analysis and can be regarded as an actual value. The performance indicator of the second dimension is obtained through time identifier analysis in the audio and video frames and can be regarded as a theoretical value. The reliability of the performance indicator of the first dimension can be verified by the performance indicator of the second dimension. For the same performance indicator, if the performance indicator of the first dimension is close to the performance indicator of the second dimension, it indicates that the reliability of the performance indicator of the first dimension is high. If the performance indicator of the first dimension is greatly different from the performance indicator of the second dimension, it indicates that the reliability of the performance indicator of the first dimension is not high.

[0065] In the case that the performance indicator of the first dimension is verified by the trusted party, for the same performance indicator, if the performance indicator of the first dimension is close to the corresponding expected performance indicator, it is determined that the performance test result of the application service in the current weak network environment is test passed. If the performance indicator of the first dimension is greatly different from the corresponding expected performance indicator, it is determined that the performance test result of the application service in the current weak network environment is test failed. If the test is failed, the developer needs to optimize the application service, and continuously improve the application service until the test is passed.

[0066] The technical scheme provided by the embodiment of the application can simulate a variety of weak network environments, control a variety of terminal types of the playback end, and analyze a variety of performance indicators of the application service, so that the automatic test of the application service can cover a variety of weak network environments, cover a variety of terminal types, and cover a variety of performance indicators, the automatic test of the application service is realized, the test efficiency of the application service is improved, the accuracy of the test result is improved, and the test cost is low.

[0067] Figure 5 A structural schematic diagram of an application service test device provided by the embodiment of the application is shown in FIG. 1. The device can include:

[0068] The building module 51 is configured to select one weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and build a current weak network environment according to the current weak network configuration strategy;

[0069] The triggering module 52 is configured to trigger a plurality of playback ends running the application service to access the current weak network environment respectively, and the plurality of playback ends correspond to a plurality of terminal types;

[0070] The triggering module 52 is further configured to trigger the streaming end to push each audio and video frame in the media stream data to the plurality of playback ends, add a time identifier to the currently pushed audio and video frame, and trigger the plurality of playback ends to play the currently received audio and video frame respectively;

[0071] The index determination module 53 is configured to determine a performance index of the application service in a first dimension according to log data of the plurality of players, and determine a performance index of the application service in a second dimension according to the time identifiers of the audio and video frames.

[0072] The result determination module 54 is configured to determine a performance test result of the application service in the current weak network environment according to the performance index in the first dimension and the performance index in the second dimension.

[0073] Further, when the result determination module 54 determines the performance test result of the application service in the current weak network environment according to the performance index in the first dimension and the performance index in the second dimension, the result determination module 54 is specifically configured to perform a credibility verification on the performance index in the first dimension according to a comparison result of the performance index in the first dimension and the performance index in the second dimension, and determine the performance test result of the application service in the current weak network environment according to the performance index in the first dimension when the performance index in the first dimension passes the credibility verification.

[0074] Further, the performance index includes time delay information, and correspondingly, when the index determination module 53 determines the performance index of the application service in the second dimension according to the time identifiers of the audio and video frames, the index determination module 53 is specifically configured to simultaneously obtain a first audio and video frame currently pushed by the pusher and a second audio and video frame currently played by a second player, the second player being one of the plurality of players, obtain a time identifier of the first audio and video frame and a time identifier of the second audio and video frame, and determine time delay information of the application service in the second dimension under a terminal type of the first player according to the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

[0075] Further, when the trigger module 52 adds the time identifier to the currently pushed audio and video frame, the trigger module 52 is specifically configured to superimpose a two-dimensional code carrying the time identifier on the currently pushed audio and video frame.

[0076] Correspondingly, when the index determination module 53 obtains the time identifier of the first audio and video frame and the time identifier of the second audio and video frame, the index determination module 53 is specifically configured to perform a mixing process on the first audio and video frame and the second audio and video frame to obtain a mixed audio and video frame, and perform two-dimensional code recognition on the mixed audio and video frame to obtain the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

[0077] Further, the performance index includes a freezing rate, and correspondingly, when the index determination module 53 determines the performance index of the application service in the second dimension according to the time identifiers of the audio and video frames, the index determination module 53 is specifically configured to determine, for a second player in the plurality of players, a freezing rate of the application service in the second dimension under a terminal type corresponding to the second player according to time identifiers of two adjacent audio and video frames played by the second player.

[0078] Further optionally, the performance indicator includes time delay information, and accordingly, the indicator determination module 53, when determining the performance indicator of the first dimension of the application service according to the log data of each of the plurality of players, is specifically configured to: determine time delay information of each stage in the life cycle of the third audio video frame from being pushed by the push stream end to being played by the third player according to the log data of the push stream end and the third player, the third audio video frame being one audio video frame in the media stream data, and the third player being one of the plurality of players; determine final time delay information of the third audio video frame according to the time delay information of each stage in the life cycle of the third audio video frame; and determine the time delay information of the first dimension of the application service under the terminal type of the third player according to the final time delay information of at least one third audio video frame.

[0079] Further optionally, the performance indicator includes a stall rate, and accordingly, the indicator determination module 53, when determining the performance indicator of the first dimension of the application service according to the log data of each of the plurality of players, is specifically configured to: determine the stall rate of the first dimension of the application service under the terminal type corresponding to the fourth player according to the playing time of adjacent two audio video frames recorded in the log information of the fourth player, the fourth player being one of the plurality of players.

[0080] Further optionally, the performance indicator includes a second opening rate, and accordingly, the indicator determination module 53, when determining the performance indicator of the first dimension of the application service according to the log data of each of the plurality of players, is specifically configured to: obtain playing time consumption of at least one fourth audio video frame from the log information of the fifth player, the fifth player being one of the plurality of players, and the fourth audio video frame being the first played audio video frame of the fifth player; and determine the proportion of the fourth audio video frame with a playing time consumption less than 1 second in the at least one fourth audio video frame as the second opening rate of the first dimension of the application service under the terminal type corresponding to the fifth player.

[0081] Further optionally, the performance indicator includes a playing success rate, and accordingly, the indicator determination module 53, when determining the performance indicator of the first dimension of the application service according to the log data of each of the plurality of players, is specifically configured to: obtain the total number of playing and the number of successful playing from the log information of the sixth player, the sixth player being one of the plurality of players; and determine the playing success rate of the first dimension of the application service under the terminal type corresponding to the sixth player according to the number of successful playing and the total number of playing.

[0082] Figure 5 The apparatus shown can perform the method shown Figure 3 The method shown, the implementation principle and technical effect will not be repeated. For the Figure 5 The apparatus shown, the implementation principle and technical effect will not be repeated. For the

[0083] The detailed implementation process of the processor performing each action can be referred to the related description in the foregoing method embodiments or device embodiments, which will not be repeated here.

[0084] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 301 to 305 can be device A; for another example, the execution subject of steps 301 and 302 can be device A, and the execution subject of steps 303 to 305 can be device B; and so on.

[0085] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations appearing in a certain order are included, but it should be clear that these operations can be executed or executed in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 301, 302, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the "first", "second", and the like in this text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0086] The embodiment of the present application also provides a test system of an application service, which comprises a test end, a plurality of playing ends and a pushing end, the playing ends and the pushing end perform data transmission through a transmission network; the test end is used for: selecting a weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and building a current weak network environment according to the current weak network configuration strategy; triggering a plurality of playing ends running the application service to access the current weak network environment respectively, the plurality of playing ends correspond to a plurality of terminal types; triggering the pushing end to push each audio / video frame in media stream data to the plurality of playing ends, and adding a time identifier to the current pushed audio / video frame, and triggering the plurality of playing ends to play the current received audio / video frame respectively; determining a performance index of the first dimension of the application service according to the log data of the plurality of playing ends respectively, and determining a performance index of the second dimension of the application service according to the time identifier of each audio / video frame; and determining a performance test result of the application service in the current weak network environment according to the performance index of the first dimension and the performance index of the second dimension. The working principle of the system can be referred to the related description in the foregoing embodiments, which will not be repeated here.

[0087] Figure 6 A structural schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 1. As shown in the figure, the electronic device comprises a memory 61 and a processor 62. Figure 6

[0088] ​The memory 61 is used to store computer programs and can be configured to store other various data to support operations on the computing platform. Examples of these data include instructions for any application or method operating on the computing platform, contact data, phonebook data, messages, pictures, videos, etc.

[0089] The memory 61 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable read only memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable read-only memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0090] The processor 62 is coupled to the memory 61 and is used to execute the computer programs in the memory 61 for: selecting one weak network configuration policy from a plurality of weak network configuration policies as a current weak network configuration policy, and building a current weak network environment according to the current weak network configuration policy; triggering a plurality of players running application services to access the current weak network environment respectively, the plurality of players corresponding to a plurality of terminal types; triggering a push stream end to push each audio and video frame in a media stream data to the plurality of players, and adding a time identifier to the current pushed audio and video frame, and triggering the plurality of players to play the current received audio and video frame respectively; determining a performance indicator of the first dimension of the application service according to the log data of the plurality of players respectively, and determining a performance indicator of the second dimension of the application service according to the time identifier of each audio and video frame; determining a performance test result of the application service in the current weak network environment according to the performance indicator of the first dimension and the performance indicator of the second dimension.

[0091] Further, as shown in Figure 6 , the electronic device further includes a communication component 63, a display 64, a power supply component 65, an audio component 66 and other components. Figure 6 Only part of the components are shown in the figure, which does not mean that the electronic device only includes Figure 6 the components shown. In addition, Figure 6The components in the dashed box are optional components, not mandatory components, and can be determined according to the product form of the electronic device. The electronic device in the embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT (Internet of things) device, or a server device such as a general server, a cloud server, or a server array. Figure 6 The components in the dashed box; if the electronic device in the embodiment is implemented as a server device such as a general server, a cloud server, or a server array, the electronic device can not include Figure 6 The components in the dashed box.

[0092] The detailed implementation process of the processor performing each action can be referred to the related description in the foregoing method embodiment or device embodiment, which will not be described here.

[0093] Correspondingly, the embodiment of the application further provides a computer readable storage medium storing a computer program, and the computer program is executed to implement each step that can be executed by the electronic device in the foregoing method embodiment.

[0094] Correspondingly, the embodiment of the application further provides a computer program product, including a computer program / instruction, when the computer program / instruction is executed by the processor, the processor can implement each step that can be executed by the electronic device in the foregoing method embodiment.

[0095] The communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on a radio frequency identification (RFID) technology, an infrared data association (IrDA) technology, an ultra wide band (UWB) technology, a Bluetooth (BT) technology, and other technologies.

[0096] The display includes a screen, which can include a Liquid Crystal Display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding movement, but also detect a duration and intensity of the touch or slide operation.

[0097] The power component provides power to various components of the device in which the power component is located. The power component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

[0098] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device in which the audio component is located is in a particular mode of operation, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0099] Those skilled in the art will understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied therein.

[0100] The present application is described with reference to the flowchart illustrations and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 The function(s) specified in the flowchart illustrations and / or block diagrams Figure 1 The function(s) specified in the flowchart illustrations and / or block diagrams

[0101] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 The flow or flows and / or blocks Figure 1 The flow or flows and / or blocks

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 The flow or flows and / or blocks Figure 1 Figure 1 The flow or flows and / or blocks

[0103] In one typical configuration, a computing device includes one or more processors (central processing units, CPUs), input / output interfaces, network interfaces, and memory.

[0104] The memory can include non-persistent memory and / or persistent memory, both of which can be volatile and / or non-volatile. Non-persistent memory can include, for example, a random access memory (RAM), which can be a volatile memory device, and non-volatile memory, such as read only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer readable media.

[0105] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0106] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0107] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of testing an application service, characterized by, The method applied to a test terminal comprises: selecting a weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and building a current weak network environment according to the current weak network configuration strategy; wherein the plurality of weak network configuration strategies configured by the test terminal are used to simulate a plurality of weak network environments; triggering a plurality of playing terminals running an application service to access the current weak network environment respectively, and the plurality of playing terminals corresponding to a plurality of terminal types; triggering a pushing terminal to push each audio and video frame in media stream data to the plurality of playing terminals, adding a time identifier to the currently pushed audio and video frame, and triggering the plurality of playing terminals to play the currently received audio and video frame respectively; determining a performance index of the first dimension of the application service according to the log data of the plurality of playing terminals respectively, and determining a performance index of the second dimension of the application service according to the time identifier of each audio and video frame; performing credible verification on the performance index of the first dimension according to the comparison result of the performance index of the first dimension and the performance index of the second dimension; in the case that the performance index of the first dimension passes the credible verification, determining a performance test result of the application service in the current weak network environment according to the performance index of the first dimension.

2. The method of claim 1, wherein, The performance index comprises time delay information, and correspondingly, determining the performance index of the second dimension of the application service according to the time identifier corresponding to each audio and video frame comprises: simultaneously acquiring a first audio and video frame currently pushed by the pushing terminal and a second audio and video frame currently played by a first playing terminal, the first playing terminal being one of the plurality of playing terminals; acquiring the time identifier of the first audio and video frame and the time identifier of the second audio and video frame; determining the time delay information of the second dimension of the application service under the terminal type of the first playing terminal according to the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

3. The method of claim 2, wherein, adding the time identifier to the currently pushed audio and video frame comprises: superimposing a two-dimensional code carrying the time identifier on the currently pushed audio and video frame; correspondingly, acquiring the time identifier of the first audio and video frame and the time identifier of the second audio and video frame comprises: mixing the first audio and video frame and the second audio and video frame to obtain a mixed audio and video frame; performing two-dimensional code recognition on the mixed audio and video frame to obtain the time identifier of the first audio and video frame and the time identifier of the second audio and video frame.

4. The method of claim 1, wherein, The performance index comprises a stall rate, and correspondingly, determining the performance index of the second dimension of the application service according to the time identifier corresponding to each audio and video frame comprises: for a second playing terminal in the plurality of playing terminals, determining the stall rate of the second dimension of the application service under the terminal type corresponding to the second playing terminal according to the time identifier of each of two adjacent audio and video frames played by the second playing terminal.

5. The method of claim 1, wherein, The performance index comprises time delay information, and correspondingly, determining the performance index of the first dimension of the application service according to the log data of the plurality of playing terminals respectively comprises: According to the log data of the push stream end and the third playback end, respectively, determine the time delay information of the third audio and video frame at each stage of the life cycle from being pushed by the push stream end to being played by the third playback end, the third audio and video frame being one audio and video frame in the media stream data, and the third playback end being one of the plurality of playback ends; According to the time delay information of the third audio and video frame at each stage of the life cycle, determine the final time delay information of the third audio and video frame; According to the final time delay information of at least one third audio and video frame, determine the first dimension time delay information of the application service under the terminal type of the third playback end.

6. The method of claim 1, wherein, The performance indicators include the stall rate, and accordingly, according to the log data of the plurality of playback ends, respectively, determine the performance indicators of the first dimension of the application service, including: According to the playback time of adjacent two audio and video frames recorded in the log information of the fourth playback end, determine the stall rate of the first dimension of the application service under the corresponding terminal type of the fourth playback end, the fourth playback end being one of the plurality of playback ends.

7. The method of claim 1, wherein, The performance indicators include the second opening rate, and accordingly, according to the log data of the plurality of playback ends, respectively, determine the performance indicators of the first dimension of the application service, including: Obtain the playback time consumption of at least one fourth audio and video frame from the log information of the fifth playback end, the fifth playback end being one of the plurality of playback ends, and the fourth audio and video frame being the first played audio and video frame of the fifth playback end; Determine the proportion of the fourth audio and video frame with a playback time consumption less than 1 second in at least one fourth audio and video frame as the second opening rate of the first dimension of the application service under the corresponding terminal type of the fifth playback end.

8. The method of claim 1, wherein, The performance indicators include the playback success rate, and accordingly, according to the log data of the plurality of playback ends, respectively, determine the performance indicators of the first dimension of the application service, including: Obtain the total number of playback and the number of successful playback from the log information of the sixth playback end, the sixth playback end being one of the plurality of playback ends; According to the number of successful playback and the total number of playback, determine the playback success rate of the first dimension of the application service under the corresponding terminal type of the sixth playback end.

9. A test system for application services, characterized by The system includes a test end, a plurality of playback ends and a push stream end, the playback ends and the push stream end performing data transmission through a transmission network; The test terminal is configured to: select one weak network configuration strategy from a plurality of weak network configuration strategies as a current weak network configuration strategy, and build a current weak network environment according to the current weak network configuration strategy, wherein the plurality of weak network configuration strategies configured by the test terminal are used to simulate a plurality of weak network environments; trigger a plurality of playing terminals running an application service to access the current weak network environment respectively, the plurality of playing terminals corresponding to a plurality of terminal types; trigger a pushing terminal to push each audio and video frame in media stream data to the plurality of playing terminals, and add a time identifier to a currently pushed audio and video frame, and trigger the plurality of playing terminals to play the currently received audio and video frame respectively; determine a performance indicator of a first dimension of the application service according to log data of the plurality of playing terminals respectively, and determine a performance indicator of a second dimension of the application service according to the time identifier of each audio and video frame; perform trusted verification on the performance indicator of the first dimension according to a comparison result of the performance indicator of the first dimension and the performance indicator of the second dimension; and in a case where the performance indicator of the first dimension passes the trusted verification, determine a performance test result of the application service in the current weak network environment according to the performance indicator of the first dimension.

10. An electronic device, comprising: Comprise: a memory and a processor; the memory is configured to store a computer program; the processor is coupled to the memory and is configured to execute the computer program to perform the steps in the method of any one of claims 1-8.

11. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor is enabled to implement the steps in the method of any one of claims 1-8.

Images