A method and device for automatic testing of P2SP live broadcast

Abstract

The application discloses a kind of P2SP live automatic testing method and device, link is established between at least one client by socket;Test task and evaluation index are obtained, and the test task and evaluation index are distributed to at least one target client by the link;The performance data of all the target client under the test task is obtained, and the performance data is evaluated according to the evaluation index, and evaluation data is obtained;And unit consumption performance data is obtained according to all the performance data;The log data of all the target client is obtained, and the subscription state of each target client is obtained according to the log data;According to the number of the target client in current test and unit consumption performance data, total performance consumption data is obtained;The evaluation data, subscription state and total performance consumption data of all the target client are displayed, so as to solve the existing problem for P2SP test.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of P2SP testing, in particular to a P2SP live broadcast automatic testing method and device. BACKGROUND

[0002] In recent years, the rapid development of the Internet live broadcast industry, whether it is live broadcast of goods, online education, cannot do without the support of live streaming capabilities, and the traffic cost of live broadcast is a major challenge faced by major application manufacturers. P2SP (Peer to Server & Peer) live broadcast has become the core direction of streamlining, especially in some internal network environment, P2SP live broadcast undoubtedly greatly reduces the bandwidth cost.

[0003] But it is a certain degree of difficulty to guarantee the viewing effect of each live broadcast client in the P2SP environment while achieving a certain sharing rate. The whole development process is often a multi-version iteration, even a long-term iteration process. The test method of P2SP is different from that of traditional P2S business. The traditional P2S is convenient in business function test, and the service pressure test can be realized through the conventional pressure test tool. The live broadcast client of P2SP needs to pay attention to the stability of the process Peer and Seed switching process, as well as the service bandwidth usage change under the dynamic change process of the number of online clients. Therefore, it is more developed through the "human sea tactics", but the data collection for this process is very difficult to carry out, as follows:

[0004] Shortcoming one: the business test of P2SP will dynamically adjust the pressure on the service with the increase and decrease of the number of online clients, so it is difficult to output a conversion relationship trend for the performance overhead data of the P2SP client and the service.

[0005] Shortcoming two: P2SP client testing often needs to occupy a large number of device terminals, and one or two test personnel's devices cannot support the real scale application test.

[0006] Shortcoming three: even if there are enough test devices, the test personnel cannot operate multiple devices at the same time and observe the process data dynamically, and record the data in the process one by one.

[0007] Shortcoming four: even if there are enough test devices and test personnel, the test personnel cannot easily quantify the live broadcast viewing effect on each computer; the subjective consciousness of each test personnel is different, and it is difficult to provide objective quantitative data.

[0008] Disadvantage five: the current subscription stream source of the client in the service of P2SP cannot be known through functional performance observation, that is, the user is unaware of whether the current user subscribes to the source stream or the P2P stream. SUMMARY

[0009] The technical problem solved by the present application is to provide a P2SP live automatic testing method and device, and to solve the existing problems in the P2SP test.

[0010] In order to solve the above technical problems, the technical scheme adopted by the present application is:

[0011] A P2SP live automatic testing method, comprising:

[0012] Establish a link between the socket and at least one client;

[0013] Obtain a test task and an evaluation index, and distribute the test task and the evaluation index to at least one target client through the link;

[0014] Obtain performance data of all target clients under the test task, evaluate the performance data according to the evaluation index to obtain evaluation data, and obtain unit consumption performance data according to all performance data;

[0015] Obtain log data of all target clients, and obtain the subscription state of each target client according to the log data;

[0016] Obtain total performance consumption data according to the number of target clients in the current test and the unit consumption performance data;

[0017] Display the evaluation data, subscription state and total performance consumption data of all target clients.

[0018] In order to solve the above technical problems, another technical scheme adopted by the present application is:

[0019] A P2SP live automatic testing device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to realize each step of the above-mentioned P2SP live automatic testing method.

[0020] The application has the beneficial effects that: the link between the client and the server is established through the socket, the problem of testing the equipment in different network segments during the test process is solved, the client equipment is tested through the test task during the test process, the performance data of the client equipment is evaluated based on the evaluation index, the problem of not providing objective quantitative data is avoided, the unit consumption performance data is obtained through the performance data, the total performance consumption data is reflected through the unit consumption performance data and the number of clients in the current test, and the problem of analyzing the state of the client is solved through the subscription state of the client during the test process. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A step flow chart of a P2SP live automatic testing method in an embodiment of the application is shown in the figure.

[0022] Figure 2 A module interaction schematic diagram of a system service platform in an embodiment of the application is shown in the figure.

[0023] Figure 3 An interaction schematic diagram of a device management module in an embodiment of the application is shown in the figure.

[0024] Figure 4 An interaction schematic diagram of a terminal scheduling module in an embodiment of the application is shown in the figure.

[0025] Figure 5 A step flow chart of a task management module in an embodiment of the application is shown in the figure.

[0026] Figure 6 An interaction schematic diagram of a steering processing module in an embodiment of the application is shown in the figure.

[0027] Figure 7 An interaction schematic diagram of a data statistics module in an embodiment of the application is shown in the figure.

[0028] Figure 8 A structure schematic diagram of a P2SP live automatic testing device in an embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0029] To make the technical content, the achieved purposes and the effects of the application clear, the following will be described in detail in combination with the embodiments and the accompanying drawings.

[0030] Please refer to Figure 1 A P2SP live automatic testing method, comprising:

[0031] establishing a link between the client and the server through the socket;

[0032] obtaining a test task and an evaluation index, distributing the test task and the evaluation index to at least one target client through the link;

[0033] obtaining performance data of all the target clients under the test task, evaluating the performance data according to the evaluation index to obtain evaluation data, and obtaining unit consumption performance data according to all the performance data;

[0034] obtaining log data of all the target clients, and obtaining a subscription state of each target client according to the log data;

[0035] obtaining total performance consumption data according to the number of the target clients in the current test and the unit consumption performance data;

[0036] displaying the evaluation data, the subscription state and the total performance consumption data of all the target clients.

[0037] As can be seen from the above description, the application has the beneficial effects that the link between the client and the server is established through the socket, and the problem of accessing different network segments of the test equipment in the test process is solved; in the test process, the client equipment is tested through the test task issued, the performance data is evaluated based on the evaluation index while solving the problem of test performance statistics, and the problem of being unable to provide objective quantitative data is avoided; and the unit consumption performance data is obtained from the performance data, so that the total performance consumption data can be reflected through the unit consumption performance data and the number of the clients in the current test; meanwhile, the problem of analyzing the state of the client is solved by obtaining the subscription state of the client in the test process.

[0038] Further, the link between the client and the server is established through the socket, and the link between the client and the server is established through the socket.

[0039] receiving the link information uploaded by the client;

[0040] judging whether the client is a new device according to the link information, if yes, generating login information and returning the login information, if no, directly returning the login information; the login information is used to instruct the client to initiate the link to establish the socket long link; meanwhile, through the establishment of the socket long link, even if the location of the client and the server are not in the same local area network, the server can obtain real-time data update of the client terminal.

[0041] As can be seen from the above description, the corresponding login information is generated after the link information of the client is obtained, and the client can establish the socket long link with the server through the login information, so as to ensure the two-way real-time communication between the client and the server.

[0042] Further, the link between the socket and the at least one client is established, and the method further comprises:

[0043] obtaining an extension field of the client, the extension field comprising a disable state and a priority;

[0044] setting a link state of the client according to the disable state;

[0045] setting a data transmission priority of the client according to the priority.

[0046] According to the above description, the extension field is used to note the information of the client device, so that the link between the client and the server can be controlled by the disable state to avoid security problems, and the priority can be set to prioritize data processing for specific client devices.

[0047] Further, the test task and the evaluation index are distributed to at least one target client, and the method further comprises:

[0048] obtaining a test condition of each target client;

[0049] distributing the test task, the evaluation index, and the test condition to the corresponding target client.

[0050] According to the above description, by setting the test condition for the client, such as setting the test loop number and the delay time, different clients can perform the test task of joining the live room, sending information, and exiting the live room at different time periods, and the test task is executed for a corresponding number of times.

[0051] Further, the test task and the evaluation index are distributed to at least one target client, and the method further comprises:

[0052] determining whether the performance of the system environment where the target client is located is insufficient, and if so, stopping the test on the target client.

[0053] According to the above description, by detecting the performance of the system environment where the client device is located, such as when the CPU or memory capacity is small, it is suggested that the current device is not recommended to be used, which facilitates to avoid the unstable performance of the test terminal caused by the shortage of device environment resources.

[0054] Further, the performance data is evaluated according to the evaluation index to obtain evaluation data, and the method further comprises:

[0055] the evaluation index comprises an evaluation algorithm and an evaluation threshold;

[0056] calculating a performance value of the performance data according to the evaluation algorithm;

[0057] comparing the evaluation threshold with the performance value of the performance data to obtain the evaluation data.

[0058] From the above description, the performance data is calculated by the algorithm, and the evaluation threshold is used for scoring. Compared with the existing subjective judgment based on the test personnel, a relatively fair score result can be obtained.

[0059] Further, the subscription state of each target client obtained according to the log data comprises:

[0060] The subscription state comprises a server state and a network node state.

[0061] The log data is parsed to obtain the subscription state, and the target client is marked as a server state or a network node state according to the subscription state.

[0062] From the above description, the client is marked according to the subscription state after parsing the log data, which can effectively identify the current client subscription state.

[0063] Further, the link between the socket and at least one client is established, and further comprises:

[0064] Creating a client update task, configuring the download address and installation directory address of the client to be tested;

[0065] Determining whether the client update file is received, if yes, instructing the client to download the client update file according to the download address and installation directory address and update.

[0066] From the above description, based on the client update task, the client can quickly update the installation package, avoid manual installation replacement, and improve the test efficiency.

[0067] Further, the test task and evaluation index are distributed to at least one target client, and further comprises:

[0068] Instructing the target client to perform local recording;

[0069] Determining whether the test task is completed, if yes, instructing the target client to complete the local recording to obtain a test video, and uploading the test video.

[0070] From the above description, when the client device is tested, the local recording function of the client device is started to record the test picture, and a test video is generated and uploaded to the server, so that the corresponding client device can be played back and checked in the future.

[0071] Another embodiment of the present application provides an automatic testing device for P2SP live broadcast, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements each step of the automatic testing method for P2SP live broadcast when executing the computer program.

[0072] The automatic testing method and device for P2SP live broadcast can be applied to the service testing of P2SP, can effectively quantify the testing result, and improve the efficiency of the P2SP live broadcast testing. The following will be described through specific embodiments.

[0073] Embodiment one

[0074] Please refer to Figure 1 An automatic testing method for P2SP live broadcast, wherein the method implements the automatic testing of P2SP live broadcast based on a system service platform; please refer to Figure 2 The system service platform mainly comprises a background management and a terminal management; wherein the background management mainly comprises a device management module, a task management module, and a data statistics module; the terminal management mainly comprises a terminal scheduling module, a special processing module, and a performance statistics module; the background management is the core of the whole system, is used to store the registration information of each online terminal, and keeps communication with the client; the task scheduling scheme can be formulated and issued on the background management, and the statistical data from the client and the service performance can be collected and analyzed. The terminal management is the task implementation party of the whole system, is used to receive the task of the background management, execute and analyze a series of analysis results, and report the analysis results to the background management, so that the technical personnel can clearly know the performance of the testing client on each terminal in the whole testing process.

[0075] For example, the system service platform distributes and combines the tasks through the task management module of the background management, simulates the user behavior in the live broadcast watching process by issuing to the online device, observes the audio and video performance of each live broadcast watching terminal in the process, records the client subscription state and the bandwidth change of the server, can directly complete the audio and video quality analysis of the P2SP live broadcast, and can complete the evaluation of the bandwidth saving rate of the P2SP live broadcast server. The method specifically comprises the following steps:

[0076] S1, establishing a link between the socket and at least one client; wherein the link is mainly established through the device management module, and specifically:

[0077] Please refer to Figure 3When the client terminal program starts, a login request is initiated to the server (i.e., the device management module) with link information such as a Mac address and an IP; when the server determines that the Mac address is a new device, a new device number is registered as login information, and the corresponding login number and address are returned; if the server determines that the Mac address is an existing device, the corresponding login number and address are directly returned;

[0078] After the client device obtains the login address, it initiates a Socket long link with the login number to ensure two-way instant communication between the client and the server; when the Socket link between the client and the server is online, the server can be identified, for example, as online; when the link is disconnected, it is identified as offline; compared with the existing http / https request mode, the embodiment can solve the problem of inaccessible terminals in different intranet situations. The server background provides an extension field for remarking the information of the client device; for example, the disabled state and the priority; the disabled state means that a device can be manually disabled, at which time the client cannot initiate a link through the device to avoid security problems; the priority means that the priority of a device can be set, and the higher the value, the higher the priority; for example, in some special device environment, a bug has occurred on the corresponding device, so it needs to be set to high priority for priority regression.

[0079] Among them, please refer to Figure 4 The terminal scheduling module is mainly responsible for receiving task instructions issued by the server, i.e., receiving tasks issued by the task management module, and parsing them into executable instructions, and passing them to the special processing module for data collection, process synchronization, and reporting the performance occupation of its client, as follows:

[0080] First, when the terminal scheduling module is deployed, it will initiate a registration to the server to report the device information, mainly including IP, Mac information, and CPU core number information, among which the Mac will be used as the unique identifier of the device, and the CPU core number will be used to evaluate the call priority (the higher the CPU core number, the higher the priority coefficient); then the terminal scheduling will establish a Socket long link with the server, and after the server and the terminal establish the link, the current Mac device will activate a state in the background;

[0081] When the background publishes a new client update version, the client will automatically download the latest client; that is, each time the task instruction is parsed, the address of the currently configured test client package is requested, if it is the first time to test the local does not save the address of the related test package, the test package is directly downloaded to the corresponding file directory and the address configuration is saved; if it is not the first time to test, it is compared with the local file address, if it is consistent, it is not updated, if it is not consistent, the latest package address is downloaded and the original file execution path is replaced, and the latest downloaded file address is saved and enabled; when the latest client is found to be enabled, the program is closed, so that the terminal program always maintains a latest state;

[0082] When receiving the instruction request of a task issued by the background, the terminal scheduling module parses the service issued parameters, parses the behavior set to be executed, and the index information to be executed, such as the implementation of audio, video quality evaluation and other capabilities; at the same time, when the task is started, the local desktop recording is started, and when the task is completed, the corresponding video recording is completed, which is used for test playback troubleshooting; and when the task executed by the background is completed or the task is forcibly stopped by the background, the corresponding test application is closed to stop data collection, and a task completion flag is sent to the background.

[0083] S2, obtain a test task and an evaluation index, and distribute the test task and the evaluation index to at least one target client through a link; wherein the task is issued through a task management module, and specifically:

[0084] Please refer to Figure 5 S21, create a client update task, configure the download address of the client to be tested and the installation directory address; that is, by creating a task, the download address of the client to be tested can be configured, and the directory address to be installed can be specified, which is used for the client to quickly update the installation package, thereby saving the time of manual installation replacement; so that when the background publishes a new client update version, the client will automatically download the latest client;

[0085] S22, specify a behavior set to be executed by the client; the behavior set refers to the UI automation capability managed by the terminal, for example: opening a live client, joining a live room, sending chat information in the live room at a specified frequency, executing a specified time after exiting the live room, and a series of test tasks;

[0086] S23, distribute the device to be executed, that is, determine the target client; one-key full selection or manual specification of the device to be executed is supported; at the same time, the test conditions such as the number of test cycles and the delay time can be configured, that is, the test conditions of each target client are obtained;

[0087] Meanwhile, when the device is distributed, the background can obtain the performance of the system environment where the target client is located by calling the performance query interface. If it is judged that the performance of the system environment where the target client is located is insufficient, for example, the CPU or memory capacity is small, it is suggested that the current device is not recommended to be used, the test on the target client can be stopped, and the unstable performance of the test terminal caused by the shortage of device environment resources can be avoided.

[0088] S24, further select a test task, for example, whether to enable audio and video capability evaluation, wherein the audio capability evaluation includes voice quality evaluation and integrity evaluation, and the video capability evaluation includes definition evaluation, stall rate evaluation and delay rate evaluation. Each of the above-mentioned capabilities is an independent switch, which can be customized to be turned on according to requirements.

[0089] S25, set evaluation indexes and configure abnormal threshold values of each index, for example, if the delay time exceeds N seconds, the definition is lower than M value, and the integrity is lower than K, the corresponding early warning is triggered; meanwhile, the way and content of the early warning need to be pushed can be configured, including but not limited to internal instant message, common third-party instant message notification, such as Dingding and Wechat.

[0090] S26, distribute the test task, evaluation index and test condition to the corresponding target client; that is, when the above-mentioned configuration items are completed, the above-mentioned task is issued to the terminal scheduling module for further implementation.

[0091] S3, obtain performance data of all target clients under the test task, evaluate the performance data according to the evaluation indexes to obtain evaluation data, and obtain unit consumption performance data according to all performance data.

[0092] Please refer to Figure 6 The process is mainly realized by a special processing module, which is a module for evaluating audio quality, video quality and counting P2SP business state. If the video quality evaluation capability is started in the task configuration, and the evaluation of delay time, stall rate and picture quality is checked, the system will perform real-time picture sampling and analysis on the picture of the live page, for example, calculate the performance value of the performance data according to the evaluation algorithm, compare the evaluation threshold value with the performance value to obtain the evaluation data, and the specific process is as follows:

[0093] For the delay index: after obtaining the picture, the time delay is obtained by analyzing the timestamp on the picture and the local timestamp; this part of data is displayed in the form of a real-time trend chart in the background, which is convenient for the test / development to dynamically observe the change, and can also be used for stall rate observation.

[0094] For the frame freezing rate index: take the number of judgments as the denominator, and judge whether it is frame freezing according to the time interval of less than 500ms per second, and adopt the normalized statistical method, if the result of this time is frame freezing, the value is 1, otherwise 0, if the frame freezing times in the whole live broadcast are 100 times, and last 1000 seconds, the final frame freezing rate of the whole live broadcast is converted to 100 / 1000; wherein, the module receives the latest 10 frame freezing values in a first-in first-out manner through an array of length 10 (which can be configured according to actual conditions), 1 second is 1 detection, 10 frame freezing samples, that is, the frame freezing weight of the previous 10 seconds at a certain time point can be circumscribed, then the real-time frame freezing rate is calculated by dividing the sum of the 10 frame freezing values by 10, and the data is returned to the background, so as to facilitate the background to locate the frame freezing time point, as follows:

[0095] In one scenario: the video is 10 seconds long, the first 5 seconds are not frame freezing, so the time interval is 1 (1>500ms), that is, no frame freezing; the following 5 seconds have frame freezing, so the time interval is 0 (0<500ms), that is, frame freezing; the normalization operation is to judge that the frame freezing times are 5 times, and the frame freezing rate is (5*1+0*5) / 10=0.5.

[0096] In another scenario: the video is 10 seconds long, frame freezing occurs from 2 seconds to 4 seconds and from 6 seconds to 8 seconds, that is, the time interval is 1s (1>500ms) in the time period of 0->1, 1->2, 4->5, 5->6, 8->9 and 9->10, and the value is 1, that is, no frame freezing; while the time interval is 0ms in the time period of 2->3, 3->4, 6->7 and 7->8, and the value is 0 (0<500ms), that is, frame freezing; the frame freezing rate is (4*1+0*6) / 10=0.4.

[0097] For the clarity index: a picture is transmitted to the corresponding evaluation algorithm, for example, including: Brenner, Laplacian, SMD, Variance, Energy, Vollath, Entropy and other algorithms, the results of all algorithm evaluation are calculated according to the corresponding weight ratio, and the score result is obtained and notified to the background.

[0098] Bottom-up verification: if the video has abnormal conditions such as black screen and screen flower, the above-mentioned analysis of the picture timestamp process will trigger an exception capture, at this time, the exception will be directly reported to the background, and the user will be notified of the abnormal condition on the corresponding device according to the notification channel set by the background.

[0099] If the task configuration starts the audio quality evaluation capability, and the voice quality evaluation and integrity evaluation are checked, the system will record the sound of the sound card, and perform a collection analysis on the audio data, specifically:

[0100] Data collection: the module will reuse the picture recognition results in video collection data, when the data found that the prompt can initiate audio collection of the identification, such as: audio start*, then start recording the corresponding audio data, when receiving audioend*, then end recording, and audio file storage record name audio*; At the same time, the page corresponding to audio end will have the corresponding audio content, that is, the text content corresponding to the voice expression.

[0101] For voice quality index: the module will score the corresponding audio file by nonparametric NISQA algorithm and record it.

[0102] For completeness index: the module will perform asr recognition on the audio file, output the corresponding text content, and calculate the similarity rate with the data results in audio content to output the completeness; and finally report to the background management.

[0103] S4, obtain the log data of all target clients, and obtain the subscription state of each target client according to the log data; specifically: the P2SP business state is a default enabled function, which is used to judge whether the execution state of the current live terminal is peer or seed; the fundamental difference between peer and seed is whether to subscribe to the source stream or to subscribe to the PSP stream; in this embodiment, the subscription role of the current client is refreshed in real time by analyzing the log of the client, wherein if it is peer, it is marked as 1, and if it is seed, it is marked as 0; the collection process will transmit data to the server in real time.

[0104] S5, obtain total performance consumption data according to the number of target clients in the current test and unit consumption performance data; the process is mainly realized by a performance statistics module, which feeds back the performance of the current device, and can also listen to the performance occupation of all programs on the current device, and report to the background service, that is, feed to the data statistics module, as follows:

[0105] For example, when the user tests the program on one of the PCs and selects the IP of the current PC in the background corresponding to the task, the corresponding process name is configured and sent to the corresponding terminal; the data request will be obtained by the terminal scheduling module and transmitted to the performance statistics module; then the performance statistics module calls the performance query API of the task manager to query the process name of the program to be tested, and records the CPU, memory, disk, bandwidth, handle and other occupation indicators in real time, and reports to the background; mainly used to detect the performance occupation of the live viewing terminal, and provide basis for the background when comparing the performance changes of the client; at the same time, the data statistics module displays the current number of online clients and the performance data of the program server; the normal P2S data is linearly related, for example N*Base; N refers to the number of clients watching; Base refers to the performance data consumed per unit, which is usually constant; while the unit performance consumption of P2SP will decrease with the increase of N.

[0106] S6, display all target client evaluation data, subscription state and total performance consumption data; wherein, the background can select a certain device terminal under the report of the corresponding task execution, observe the change of one index of the process, and display the change of the process in the form of trend chart;

[0107] Please refer to Figure 7 The process is mainly realized by the data statistics module, which is responsible for the process data collection and statistics of the client and server, as well as the collection and data presentation of special indicators. The process data collection and statistics of the client mainly refers to: the performance process collection of the tested client and the performance collection of the running environment; the process data collection and statistics of the server refers to: the server of the tested program; for example, receiving the related client performance data reported by the terminal management; and collecting the performance data of the server of the tested program, specifically:

[0108] The module receives the index items uploaded by each terminal in real time, such as delay time, stuttering rate, picture quality, intelligibility, completeness and the current business state machine of the terminal, i.e. subscription state; the module has the ability to collect service performance, as long as any service provides an interface to query the load, the service can record the performance data of each service according to the configuration requirements; at the same time, the module provides an interface for the client to report the load of the corresponding client program, such as: program name, memory occupation, CPU, GPU, uplink and downlink bandwidth, disk, frame rate, etc.;

[0109] Module integrates the data: when a test is completed, the client performance reporting data and the server performance data are combined to generate a visual report; the data export function is supported, and the performance data in the entire process can be exported for specific performance analysis; at the same time, the data statistics module can also re-initiate the task, and the data statistics module re-sends the corresponding task ID to the task management module to re-initiate the task through the task management module, and then the same process is repeated with the same device terminal, and then the statistical differences between the two tests are compared.

[0110] Embodiment two

[0111] In this embodiment, the method in embodiment one is applied to a specific application scenario, which is as follows:

[0112] Scenario one

[0113] As the scale of the business increases after going online, some performance problems are exposed on the server; the tester needs to perform corresponding performance testing on the live streaming service of the project, and the requirement is a relatively full-link test, which involves testing of different services such as APIs and streaming media; if single-point testing is used, the test results will deviate from the actual stress of the full-link and process; if real clients are used for stress testing, it is difficult to implement concurrent group control testing.

[0114] Through the scheme in embodiment one, the tester only needs to configure the live streaming viewing terminal to be tested, and configure the corresponding installation package address in the background; a certain number of terminals are simulated to join a live streaming room at the same time through background configuration, and corresponding chat message application behaviors are performed in the live streaming room; at this time, the data changes of the server are observed to know the impact on the server and the performance of the client in the whole process.

[0115] Scenario two

[0116] In the process of testing the project, there may be low-probability problems such as lag, black screen, and delay, and the tester needs to invest a lot of time and effort to reproduce and locate the problems.

[0117] Through the scheme in embodiment one, the tester can select a batch of terminal devices of different models through the background and set to join a live streaming room, and select to start audio and video detection and verification; when a device has video lag, black screen, or other problems, an abnormal prompt will be received immediately, and the video playback at the corresponding abnormal time point is watched to quickly locate the problem.

[0118] Scenario three

[0119] Currently, the test method of P2SP needs multiple terminals to observe; a networking process needs at least 3 or more devices to verify networking, and the tester cannot know the subscription state of the corresponding live viewing terminal from the functional level, whether it is a subscription source stream or a subscription P2P stream.

[0120] Through the scheme in embodiment one, the tester only needs to configure through the background to quickly schedule batch scheduling of live viewing, and can clearly observe in the background that each calling terminal currently subscribes to a stream source for live viewing; at the same time, the bandwidth trend of the server can also be observed with the change of the number of live viewing terminals, and the throttling value of P2SP is quantified from two dimensions of service bandwidth application and client sharing rate.

[0121] Scenario four:

[0122] In order to improve the live experience, it is usually necessary to optimize from the angles of clarity, delay, etc., and at the same time, it is necessary to consider improving various indicators from the program without increasing the performance occupation of the client; therefore, it is necessary to perform compatibility regression testing on different devices; in this process, the tester needs to log in to different computers to install the iteration package and record the process data.

[0123] Through the scheme in embodiment one, the tester can select different models of devices through the background management list, issue the address of the latest test package and issue task execution, record various special indicators of different devices such as delay and clarity, and upload the process client performance occupation; and the tester can clearly know the difference between the two versions of the client by comparing the results of the two tests in the background.

[0124] Scenario five:

[0125] When performing compatibility testing and stress testing on live streaming, if the test devices are not in a local area network, the tester cannot directly access the devices through remote devices.

[0126] In this embodiment one, a communication between the socket long connection establishment platform and the test terminal is established, even if the test devices are in different network environments, the tester can issue tasks through the background, and the corresponding test devices can perform corresponding test execution and data collection after receiving the tasks, to realize compatibility testing and stress testing; at the same time, it can also be more close to the various network conditions of actual users, to perform real user device compatibility testing before release, and to perform test simulation in different network environments.

[0127] Embodiment three

[0128] Please refer to Figure 8The application provides a P2SP live broadcast automatic testing device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and each step in the P2SP live broadcast automatic testing method of the embodiment one is realized when the processor executes the computer program.

[0129] In conclusion, the P2SP live broadcast automatic testing method and device provided by the application can establish a link with a client through a Socket, can ignore the internal and external network environment of the test client, can perform corresponding scheduling access even if the client to be scheduled is not in a local area network, and can solve the problem of accessing different network segments of the test equipment in the test process. In the test process, a plurality of live broadcast watching ends are controlled by a background group to perform concurrent live broadcast watching, and the live broadcast watching effect on all terminal machines can be monitored in the background, an abnormal live broadcast client can be acquired in real time, and problems can be quickly analyzed and positioned. Through the provided report statistics, the change of a P2SP sharing rate under different user amounts can be quickly analyzed, objective data support for developing an optimization scheme can be provided, a service end running condition and performance under different user amounts and user behaviors can be quickly analyzed, and the influence of each version optimization on the client can be quickly judged, thereby providing a basis for whether the product needs to be further optimized in performance. Through the provided special processing capability, video playing abnormality and audio playing abnormality of a certain device can be quickly found, and specific indexes of the abnormality are provided, including: a freezing rate, definition, delay condition, audio quality, similarity and the like. Meanwhile, a client and a time point of the fault abnormality can be quickly positioned from a plurality of test devices.

[0130] The above-mentioned embodiments are merely examples of the application, and do not limit the patent scope of the application, and any equivalent transformation or direct or indirect application in the related technical field by using the content of the specification and drawings of the application is also included in the patent protection scope of the application.

Claims

1. A method for automated testing of P2SP live streaming, the method comprising: The method comprises: establishing a link with at least one client through a socket; obtaining a test task and an evaluation index, and distributing the test task and the evaluation index to at least one target client through the link; obtaining performance data of all the target clients under the test task, and evaluating the performance data according to the evaluation index to obtain evaluation data; obtaining unit consumption performance data according to all the performance data; obtaining log data of all the target clients, and obtaining a subscription state of each target client according to the log data; judging whether an execution state of a current live terminal is a subscription source stream or a subscription PSP stream according to the subscription state of each target client obtained according to the log data; obtaining total performance consumption data according to the number of the target clients in the current test and the unit consumption performance data; displaying the evaluation data, the subscription state and the total performance consumption data of all the target clients.

2. The method of claim 1, wherein the method further comprises: The method of establishing a link with at least one client through a socket comprises: receiving link information uploaded by the client; judging whether the client is a new device according to the link information, generating login information and returning the login information if the client is a new device, and directly returning the login information if the client is not a new device; the login information is used to instruct the client to initiate a link to establish a socket long link.

3. The method of claim 1 or 2, wherein the method further comprises: The method of establishing a link with at least one client through a socket further comprises: obtaining an extension field of the client, the extension field comprising a disabled state and a priority; setting a link state of the client according to the disabled state; setting a data transmission priority of the client according to the priority.

4. The method of claim 1, wherein the method further comprises: The method of distributing the test task and the evaluation index to at least one target client comprises: obtaining a test condition of each target client; distributing the test task, the evaluation index and the test condition to the corresponding target client.

5. The method of claim 1 or 4, wherein the method further comprises: The method of distributing the test task and the evaluation index to at least one target client further comprises: judging whether the performance of a system environment in which the target client is located is insufficient, and stopping the test on the target client if the performance of the system environment is insufficient.

6. The method of claim 1, wherein the method further comprises: The method of evaluating the performance data according to the evaluation index to obtain evaluation data comprises: the evaluation index comprises an evaluation algorithm and an evaluation threshold; calculating a performance value of the performance data according to the evaluation algorithm; comparing the evaluation threshold with the performance value to obtain the evaluation data.

7. The method of claim 1, wherein the method further comprises: The method of obtaining a subscription state of each target client according to the log data comprises: the subscription state comprises a server state and a network node state; parsing the log data to obtain the subscription state, and marking the target client as the server state or the network node state according to the subscription state.

8. The method of claim 1, wherein the method further comprises: The method of establishing a link with at least one client through a socket further comprises: creating a client update task, and configuring a download address and an installation directory address of the client to be tested; determining whether the client update file is received, and if so, instructing the client to download the client update file according to the download address and the installation directory address and update.

9. The method of claim 1, wherein the method further comprises: The distributing the test task and the evaluation index to at least one target client further comprises: instructing the target client to perform local recording; determining whether the test task is completed, and if so, instructing the target client to complete the local recording to obtain a test video and upload the test video.

10. An apparatus for automated testing of P2SP live streaming, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program comprises the following steps of: The processor implements each step of the automatic test method for P2SP live broadcast according to any one of claims 1-9 when executing the computer program.

Images