Application program testing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an application program test method and device, equipment and a storage medium, and the method comprises the steps: determining first equipment configuration information of a plurality of test playing equipment, configuring the plurality of test playing equipment according to the first equipment configuration information, and determining first video configuration information of a plurality of test videos, configuring a plurality of test videos according to the first video configuration information; playing a plurality of test videos on the application program of the first version before updating through a plurality of test playing devices, and obtaining first test information of preset indexes corresponding to the plurality of test playing devices when the plurality of test playing devices play the plurality of test videos; playing a plurality of test videos on the updated application program of the second version through a plurality of test playing devices, and obtaining second test information of preset indexes corresponding to the plurality of test playing devices when the plurality of test playing devices play the plurality of test videos; and generating test result information corresponding to the application program according to the first test information and the second test information. The method improves the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of application development technology, and in particular to a method, apparatus, device, and storage medium for testing applications. Background Technology

[0002] Before a new version of the application is launched, it needs to be tested to verify the effectiveness of the improvements made.

[0003] In existing technologies, A / B testing is used to test new versions of applications. Specifically, a group of users (Group A) uses the new version of the application, while another group continues to use the old version (Group B). Preset metrics (e.g., the player's home screen) are then tested for both Group A and Group B, and the test results for Group A and Group B are obtained. These results are then used to verify the improvement effect of the new version of the application.

[0004] However, the inventors have found that the existing technology has at least the following technical problems: if the modifications to the new version of the application do not meet expectations or contain program errors, it will directly affect the user's experience, thereby reducing the user experience of the application. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, and storage medium for testing applications, which can obtain test result information for applications offline, thereby improving user experience.

[0006] In a first aspect, embodiments of this disclosure provide a method for testing an application, comprising:

[0007] The system determines first device configuration information for multiple test playback devices and configures the multiple test playback devices according to the first device configuration information; it also determines first video configuration information for multiple test videos and configures the multiple test videos according to the first video configuration information; wherein the test playback devices test preset metrics of the application by playing the test videos.

[0008] By playing the multiple test videos on the first version of the application before the update using the multiple test playback devices, the first test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained;

[0009] By playing the multiple test videos on the updated second version of the application using the multiple test playback devices, second test information corresponding to the preset indicators of each of the multiple test playback devices when playing the multiple test videos is obtained;

[0010] Based on the first test information and the second test information, test result information corresponding to the application is generated, wherein the test result information is used to represent the comparison result information between the preset indicators of the first version of the application and the preset indicators of the second version of the application.

[0011] Secondly, embodiments of this disclosure provide an application testing apparatus, comprising:

[0012] A determination module is used to determine first device configuration information of multiple test playback devices and configure multiple test playback devices according to the first device configuration information; and to determine first video configuration information of multiple test videos and configure multiple test videos according to the first video configuration information; wherein, the test playback devices test the preset indicators of the application by playing the test videos;

[0013] The first test module is used to play the multiple test videos on the first version of the application before the update through the multiple test playback devices, and to obtain the first test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos.

[0014] The second testing module is used to play the multiple test videos on the updated second version of the application through the multiple test playback devices, and to obtain second test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos.

[0015] The result generation module is used to generate test result information corresponding to the application based on the first test information and the second test information, wherein the test result information is used to represent the comparison result information between the preset indicators of the first version of the application and the preset indicators of the second version of the application.

[0016] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0017] A processor, and a memory communicatively connected to the processor;

[0018] The memory stores computer-executed instructions;

[0019] The processor executes computer execution instructions stored in the memory to implement the application testing method described in the first aspect above.

[0020] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the application testing method described in the first aspect above.

[0021] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the testing method for the application described in the first aspect above.

[0022] This embodiment provides a testing method, apparatus, device, and storage medium for an application. The method includes: determining first device configuration information for multiple test playback devices and configuring the multiple test playback devices according to the first device configuration information; and determining first video configuration information for multiple test videos and configuring the multiple test videos according to the first video configuration information. The test playback devices test preset metrics of the application by playing the test videos. The method involves playing the multiple test videos on a first version of the application before an update using the multiple test playback devices, and obtaining first test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos. The method also involves playing the multiple test videos on a second version of the application using the multiple test playback devices, and obtaining second test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos. Finally, the method generates test result information corresponding to the application based on the first and second test information, wherein the test result information represents a comparison result between the preset metrics of the first version of the application and the preset metrics of the second version of the application. In this embodiment of the disclosure, by using multiple test playback devices and multiple test videos, the preset indicators of the application before the update and the preset indicators of the application after the update are tested, thereby obtaining the test result information corresponding to the application. Since the test playback devices are not the playback devices used by the user, even if the modifications to the new version of the application do not meet expectations or have program errors, it will not affect the user's experience, thus improving the user experience. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of a testing method for an application in the prior art.

[0025] Figure 2 A flowchart illustrating a testing method for an application provided in this embodiment of the disclosure;

[0026] Figure 3 A flowchart of another application testing method provided in this disclosure embodiment;

[0027] Figure 4 A flowchart of another application testing method provided in this disclosure embodiment;

[0028] Figure 5 A flowchart of another application testing method provided in this disclosure embodiment;

[0029] Figure 6 A structural block diagram of an application testing device provided in an embodiment of this disclosure;

[0030] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0033] Currently, after the application is developed and updated, in order to determine whether the new version of the application meets expectations, it is necessary to test the new version of the application before it is launched, and to verify the improvement effect of the new version of the application through the test results.

[0034] In existing technologies, A / B testing is used to test new versions of applications. Specifically, a group of users (Group A) uses the new version of the application, while another group continues to use the old version (Group B). Preset metrics (e.g., the player's home screen) are then tested for both Group A and Group B, and the test results for Group A and Group B are obtained. These results are then used to verify the improvement effect of the new version of the application.

[0035] However, if the changes in the new version of the application do not meet expectations or contain program errors, it will directly affect the user experience, thereby reducing the user experience of the application.

[0036] To address the aforementioned issues, this embodiment proposes the following technical concept: by simulating an experimental environment offline that matches the online "device model distribution," "network speed distribution," "video type distribution," and "viewing duration distribution," the goal of verifying experiments offline can be achieved, avoiding impact on user experience and shortening the experimental cycle.

[0037] Accordingly, the specific method may include: First, determining first device configuration information for multiple test playback devices and configuring the multiple test playback devices according to the first device configuration information; and second, determining first video configuration information for multiple test videos and configuring the multiple test videos according to the first video configuration information; wherein, the test playback devices test the application's preset metrics by playing the test videos. Then, playing the multiple test videos on the first version of the application before the update using the multiple test playback devices, obtaining first test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos; playing the multiple test videos on the second version of the application using the multiple test playback devices, obtaining second test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos. Finally, generating test result information corresponding to the application based on the first and second test information, wherein the test result information is used to represent the comparison result information between the preset metrics of the first version of the application and the preset metrics of the second version of the application.

[0038] In this scenario, by using multiple test playback devices and multiple test videos, the preset metrics of the application before and after the update are tested, thereby obtaining the corresponding test results information for the application. Since the test playback devices are not the playback devices used by the user, even if the modifications to the new version of the application do not meet expectations or contain program errors, it will not affect the user's experience, thus improving the user experience.

[0039] The application scenarios of the embodiments of this disclosure are explained below:

[0040] First, the technical terms involved in the embodiments of this disclosure will be explained:

[0041] Offline simulation testing refers to using a test playback device to play test videos from the application to test the application's preset metrics. The test playback device can simulate a user's playback device.

[0042] A / B testing: In an A / B test, a preset number (e.g., 100) of playback devices can be selected as group A, and another 100 playback devices as group B. The playback devices in group A have the unupdated application installed, while the playback devices in group B have the updated application installed. Then, the metric 1 corresponding to the unupdated application in group A and the updated application in group B are tested respectively, yielding the test results for groups A and B. Finally, by comparing the test results of groups A and B, the experimental results of the A / B test are obtained. The playback devices in the A / B test can be the playback devices used by the user.

[0043] The application testing method provided in this disclosure can be applied to application scenarios where various improvement schemes for applications are verified. Figure 1 This is a schematic diagram illustrating an application scenario for a testing method for an application provided in this embodiment of the disclosure. For example... Figure 1 As shown, after developing an improvement plan for an application, a demo (example) program is created based on the improvement plan. This demo (example) program is then subjected to offline simulation testing using the application testing methods provided in this application. If the test passes, the demo (example) program is integrated into the official version of the application. If the test fails, the improvement plan can be adjusted based on data analysis.

[0044] Furthermore, after merging the demo program into the official application version, offline simulation testing can be performed on the updated application using the testing methods provided in this application. If the test passes, A / B testing is conducted for verification; if the test fails, the improvement plan can be adjusted based on data analysis. At this point, because two offline simulation tests have been performed before the A / B testing, the probability of unexpected changes or program errors in the application is greatly reduced, thus improving the user experience. Afterwards, if the A / B testing passes, the new version of the application is fully launched.

[0045] The following detailed embodiments illustrate the testing methods for the applications provided in this disclosure.

[0046] Figure 2 This is a flowchart illustrating a testing method for an application provided in this embodiment. In this embodiment, the testing method is described using an electronic device as an example. (Reference) Figure 2 The method includes:

[0047] S201. Determine first device configuration information for multiple test playback devices, and configure multiple test playback devices according to the first device configuration information; and determine first video configuration information for multiple test videos, and configure multiple test videos according to the first video configuration information; wherein, the test playback devices test the preset indicators of the application by playing the test videos.

[0048] In this embodiment, test simulation data corresponding to the application can be obtained by acquiring real online data within a preset duration. Optionally, the test simulation data includes, but is not limited to, first device configuration information of multiple test playback devices and first video configuration information of multiple test videos. In this embodiment, the example of test simulation data including first device configuration information of multiple test playback devices and first video configuration information of multiple test videos will be used for illustration.

[0049] S202. Play multiple test videos on the first version of the application before the update using multiple test playback devices, and obtain the first test information of the preset indicators corresponding to each of the multiple test playback devices when playing multiple test videos.

[0050] In this embodiment of the disclosure, a first version of the application can be pre-installed on each test playback device before testing. This embodiment of the disclosure does not specifically limit the preset metric. In some embodiments, the preset metric may be: the player's first screen. Here, the player's first screen means: the time elapsed from when the player is invoked to when the first video frame is displayed. In other embodiments, the preset metric may also be: the player's stuttering duration.

[0051] In some embodiments, when multiple test playback devices play multiple test videos, the first test information for each corresponding preset indicator may be the average value of each preset indicator.

[0052] For example, the preset metric could be: the player's first screen. The first test information corresponding to the preset metric for the first version of the application is: 200ms (milliseconds).

[0053] In this step, multiple test playback devices need to play multiple test videos on the first version of the application before the update a preset number of times to ensure the accuracy of the obtained first test information. In this embodiment, the value of the preset number of playbacks is not specifically limited. Optionally, the preset number of playbacks is 1000 times.

[0054] S203. Play multiple test videos on the updated second version of the application using multiple test playback devices, and obtain the second test information of the preset indicators corresponding to each of the multiple test playback devices when playing multiple test videos.

[0055] In this embodiment of the disclosure, after testing the first test information, the updated second version of the application can be directly installed on each test playback device; or, after testing the first test information, the first version of the application can be updated to obtain the updated second version of the application.

[0056] In some embodiments, the second test information for the preset metrics may be the average value of various preset metrics. For example, the preset metric may be: the player's first screen. The second test information corresponding to the preset metric of the second version of the application is: 150ms (milliseconds).

[0057] It should be noted that in this step, the number of times multiple test videos are played on the updated second version of the application through multiple test playback devices needs to reach a preset number of plays to ensure the accuracy of the obtained second test information. In this embodiment, the value of the preset number of plays is not specifically limited. Optionally, the preset number of plays is 1000 times.

[0058] S204. Based on the first test information and the second test information, generate test result information corresponding to the application, wherein the test result information is used to represent the comparison result information between the preset indicators of the first version of the application and the preset indicators of the second version of the application.

[0059] In this embodiment of the disclosure, the improvement effect of the updated second version of the application can be demonstrated by comparing the results information.

[0060] Optionally, the comparison result information may include first test information, second test information, difference information between the first test information and the second test information, and relative value information between the first test information and the second test information. Accordingly, this step may include: determining the difference information between the first test information and the second test information, and determining the relative value information between the first test information and the second test information; and generating test result information corresponding to the application based on the first test information, the second test information, the difference information, and the relative value information.

[0061] For example, the preset metric is the first screen of the player. The first test information for the first version of the application is 200ms, and the second test information for the second version of the application is 150ms. The difference between the first and second test information is determined to be -50ms; where the negative sign indicates that the second test information is 50ms lower than the first test information. The relative value between the first and second test information is determined to be -0.25. Wherein the negative sign indicates that the second test information is 0.25 lower than the first test information.

[0062] Based on the first test information, the second test information, the difference information, and the relative value information, the test result information corresponding to the application is generated as follows: "Indicator: Player first screen; Control group: 200ms; Experimental group: 150ms; Absolute value: -50ms; Relative value: -0.25. Among them, the control group is the first test information corresponding to the first version (old version) of the application, and the experimental group is the second test information corresponding to the second version (new version) of the application."

[0063] This disclosure provides a method for testing an application: First, first device configuration information for multiple test playback devices is determined, and the multiple test playback devices are configured according to the first device configuration information. Second, first video configuration information for multiple test videos is determined, and the multiple test videos are configured according to the first video configuration information. The test playback devices test preset metrics of the application by playing the test videos. Then, the multiple test videos are played on a first version of the application before the update using the multiple test playback devices, and first test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained. The multiple test videos are then played on a second version of the application using the multiple test playback devices, and second test information for the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained. Finally, test result information corresponding to the application is generated based on the first and second test information, wherein the test result information represents a comparison result between the preset metrics of the first version of the application and the preset metrics of the second version of the application. In this embodiment of the disclosure, by using multiple test playback devices and multiple test videos, the preset indicators of the application before the update and the preset indicators of the application after the update are tested, thereby obtaining the test result information corresponding to the application. Since the test playback devices are not the playback devices used by the user, even if the modifications to the new version of the application do not meet expectations or have program errors, it will not affect the user's experience, thus improving the user experience.

[0064] In some embodiments, such as Figure 3 As shown, the method for determining the first device configuration information of multiple test playback devices in step S201 may include the following steps S301 and S302:

[0065] S301. Obtain the second device configuration information of multiple first playback devices, wherein the first playback device is the playback device used by the user to play any video on the application within a preset duration.

[0066] In some embodiments, the second device configuration information includes multiple second device model information, each second device model information corresponding to multiple second hardware configuration information, and a second quantity ratio of the playback device corresponding to each second hardware configuration information among all playback devices.

[0067] In this step, the second model information is used to distinguish different playback devices. Optionally, the second model information includes a model identifier. The model identifier is used to distinguish playback devices with different identifiers. For example, the model identifiers include: identifier A, identifier B, identifier C, identifier D, and identifier E.

[0068] In this step, each second model information corresponds to multiple second hardware configuration information, which are used to distinguish playback devices with different hardware configurations for the same model information.

[0069] Optionally, the second hardware configuration information includes: a first model, a second model, and a third model. The first model, the second model, and the third model can be determined based on parameters such as the playback device's RAM and storage space. In this embodiment, the value of the preset duration is not specifically limited.

[0070] In this embodiment, the preset duration is not specifically limited. For example, the historical preset duration is 2 weeks. The second model identifiers of the multiple first playback devices include: identifier A, identifier B, identifier C, identifier D, and identifier E. Each second model information corresponds to multiple second hardware configuration information, including: first model, second model, and third model. The second proportion of the playback device corresponding to each second hardware configuration information among all playback devices is: 10% (first model), 50% (second model), and 40% (third model). Here, the playback device can be a first playback device.

[0071] Optionally, the second device configuration information may further include the proportion of playback devices corresponding to each second device model among all playback devices. For example, the preset duration is 2 weeks. Within these 2 weeks, the proportion of playback devices corresponding to each second device model among all playback devices is: 30% (identifier A), 20% (identifier B), 30% (identifier C), 10% (identifier D), and 10% (identifier E). Here, the playback device can be the first playback device.

[0072] S302. Determine the first device configuration information of multiple test playback devices based on the second device configuration information of multiple first playback devices.

[0073] In some embodiments, the first device configuration information includes multiple first device model information, each first device model information corresponding to multiple first hardware configuration information, and a first quantity ratio of playback devices corresponding to each first hardware configuration information in all playback devices; the second device configuration information includes multiple second device model information, each second device model information corresponding to multiple second hardware configuration information, and a second quantity ratio of test playback devices corresponding to each second hardware configuration information in all test playback devices. Accordingly, this step may include: determining that multiple first device model information is the same as multiple second device model information; for each set of the same first device model information and second device model information, determining that the multiple first hardware configuration information corresponding to the first device model information is the same as the multiple second hardware configuration information corresponding to the second device model information; for each set of the same first hardware configuration information of the first device model information and the second hardware configuration information of the second device model information, determining that the first quantity ratio of playback devices corresponding to the first hardware configuration information of the first device model information in all playback devices is the same as the second quantity ratio of test playback devices corresponding to the second hardware configuration information of the second device model information in all test playback devices.

[0074] In other embodiments, the first device configuration information further includes: the proportion of test playback devices corresponding to each first model information in all test playback devices. The second device configuration information further includes: the proportion of playback devices corresponding to each second model information in all playback devices. Accordingly, this step may also include: for each identical group of first model information and second model information, determining that the proportion of test playback devices corresponding to the first model information in that group in all test playback devices is the same as the proportion of playback devices corresponding to the second model information in that group in all playback devices.

[0075] The following describes the steps for configuring multiple test playback devices based on the first device configuration information, using three different scenarios.

[0076] In the first scenario, multiple test playback devices are configured based on the proportion of each hardware configuration information. Accordingly, the steps of configuring multiple test playback devices based on the first device configuration information may include: determining the number of multiple test playback devices as a first preset number; determining the multiple test playback devices based on the first preset number and the proportion of the playback device corresponding to each second hardware configuration information in the total number of playback devices; wherein the proportion of the playback device corresponding to the first hardware configuration information in the total number of playback devices is the same as the proportion of the test playback device corresponding to the second hardware configuration information in the total number of test playback devices.

[0077] For example, the number of multiple test playback devices is 10. The first hardware configuration information includes a first model, a second model, and a third model; the second hardware configuration information also includes the first model, the second model, and the third model. The respective proportions of the first, second, and third models among the multiple playback devices are: "10% (first model), 50% (second model), 40% (third model)". Therefore, it is determined that the first model includes 1 device (representing 10% of all test playback devices), the second model includes 5 devices (representing 50% of all test playback devices), and the third model includes 4 devices (representing 40% of all test playback devices).

[0078] In the second scenario, multiple test playback devices are configured based on the proportion of each hardware configuration information for each device model. Accordingly, the steps for configuring multiple test playback devices based on the first device configuration information may include: determining the number of test playback devices for the first device model as a second preset number; determining multiple test playback devices based on the second preset number and the second proportion of the playback devices corresponding to the second hardware configuration information of each second device model in all playback devices; wherein the first proportion of the playback devices corresponding to the first hardware configuration information of the first device model in all playback devices is the same as the second proportion of the test playback devices corresponding to the second hardware configuration information of the second device model in all test playback devices.

[0079] For example, both the first and second model information are labeled A; the second preset number of test playback devices labeled A is 5. Among the multiple playback devices labeled A, the respective proportions of the first, second, and third models are as follows: the first model accounts for 20% of the total number of playback devices labeled A, the second model accounts for 60%, and the third model accounts for 20%. In this case, the 5 test playback devices labeled A are determined as follows: 1 device of the first model (accounting for 20% of all test playback devices), 3 devices of the second model (accounting for 60% of all test playback devices), and 1 device of the third model (accounting for 20% of all test playback devices).

[0080] In the third scenario, multiple test playback devices are configured based on the proportion of each device model information. Accordingly, the steps for configuring multiple test playback devices based on the first device configuration information may include: determining the number of multiple test playback devices as a first preset number; determining multiple first device model information and the number of each first device model information based on the first preset number, multiple second device model information, and the proportion of playback devices corresponding to each second device model information in the total number of playback devices; wherein the multiple first device model information is the same as the multiple second device model information, and the proportion of test playback devices corresponding to each first device model information in the total number of test playback devices is the same as the proportion of playback devices corresponding to each second device model information in the total number of playback devices.

[0081] For example, the number of multiple test playback devices is 10. The second device type information includes identifiers A, B, C, D, and E. The proportion of each second device type information's playback device among all playback devices is: 30% (identifier A), 20% (identifier B), 30% (identifier C), 10% (identifier D), and 10% (identifier E). In this case, multiple first device type information is determined to include identifiers A, B, C, D, and E. The number of each first device type information is: 3 for identifier A (30% of all test playback devices), 2 for identifier B (20% of all test playback devices), 3 for identifier C (30% of all test playback devices), 1 for identifier D (10% of all test playback devices), and 1 for identifier E (10% of all test playback devices).

[0082] In some embodiments, such as Figure 4 As shown, the method for determining the first video configuration information of multiple test videos in step S201 may include the following steps S401 to S402:

[0083] S401. Obtain the second video configuration information of multiple first videos, wherein the first video is any video played on the application by the first playback device within a preset duration.

[0084] In some embodiments, the second video configuration information includes second playback duration information corresponding to each of the plurality of first videos, and second quantity percentage information of each video format of the first video in all the first videos. Optionally, the video format includes video container format and / or video encoding format.

[0085] In some embodiments, the video format includes a video encoding format. The video encoding format indicates the algorithm used for video compression. Optionally, the video encoding format includes H.264, H.265, and H.266 encoding formats. The percentage of each of the multiple video formats in the first video is: 10% (H.264 encoding format), 70% (H.265 encoding format), and 20% (H.266 encoding format).

[0086] In other embodiments, the video format includes a video container format. The video container format indicates the format in which the video is saved. Optionally, the video container format includes mp4 and dash formats. The percentage of each of the first videos in multiple video formats is 50% (mp4 format) and 50% (dash format).

[0087] For example, the first playback duration (i.e., the second playback duration information corresponding to each of the multiple first videos) can be used as the basis for simulating user playback behavior.

[0088] For example, the first playback duration information corresponding to the first video in multiple video formats includes: 10 seconds (264 encoding format), 15 seconds (265 encoding format), and 20 seconds (266 encoding format).

[0089] S402. Determine the first video configuration information of multiple test videos based on the second video configuration information of multiple first videos.

[0090] In some embodiments, the first video configuration information includes first playback duration information corresponding to each of the multiple test videos and first quantity percentage information of each video format test video in all test videos; the second video configuration information includes second playback duration information corresponding to each of the multiple first videos and second quantity percentage information of each video format first video in all first videos.

[0091] Accordingly, this step may include: determining that the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos; for each video format, determining that the first quantity percentage information of the test videos of the video format in all test videos is the same as the second quantity percentage information of the first videos of the video format in all first videos; wherein, the video format includes video container format and / or video encoding format.

[0092] In some embodiments, the number of multiple test videos is the same as the number of multiple first videos; that is, there is a one-to-one correspondence between the test videos and the multiple first videos. The first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos, which may include: the first playback duration information of each test video is the same as the second playback duration information of its corresponding first video.

[0093] In other embodiments, the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos. This can include test videos and first videos of the same video format, with the same playback duration information. For example, the playback duration information corresponding to the MP4 format test videos is the same as that corresponding to the MP4 format first videos.

[0094] In this embodiment of the disclosure, an automated software tool can be developed based on the playback duration information (i.e., the "playback duration distribution") corresponding to each format of test video. During the automated playback process, the playback duration of test videos of different formats can be simulated proportionally, thereby simulating the usage of online users.

[0095] In other embodiments, the first video configuration information further includes first network speed configuration information corresponding to multiple test videos, and the second video configuration information further includes second network speed configuration information corresponding to multiple first videos; accordingly, as Figure 5 As shown, based on the second video configuration information of multiple first videos, the first video configuration information of multiple test videos is determined, including:

[0096] S501. Determine that the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos.

[0097] S502. For each video format, determine that the first quantity percentage information of the test video of the video format in all test videos is the same as the second quantity percentage information of the first video of the video format in all first videos.

[0098] For example, the first quantity percentage information corresponding to the test video in mp4 format is 30%, and the second quantity percentage information corresponding to the first video in mp4 format is also 30%.

[0099] S503. Determine that the third quantity percentage information of the test video corresponding to each network speed level in the first network speed configuration information is the same as the fourth quantity percentage information of the first video corresponding to each network speed level in the second network speed configuration information.

[0100] This step does not specifically limit the classification of network speed levels. Optionally, network speed levels can be divided into a first speed level, a second speed level, and a third speed level based on network speed. For example, the network speed range corresponding to the first speed level is less than or equal to 1 Mbps. The network speed range corresponding to the second speed level is greater than 1 Mbps and less than or equal to 2 Mbps. The network speed range corresponding to the third speed level is greater than 2 Mbps.

[0101] For example, the third quantity percentage information corresponding to the test video of the first network speed level is 20%, and the fourth quantity percentage information corresponding to the first video of the first network speed level is also 20%.

[0102] In this embodiment of the disclosure, the network speed in the laboratory can be continuously adjusted during the automated testing process using professional network equipment based on the network speed configuration information (i.e., the "network speed distribution") of the test video, thereby simulating a similar online network environment.

[0103] In other embodiments, the first video configuration information further includes the average playback duration of the test video corresponding to each network speed level. The second video configuration information further includes the average playback duration of the first video corresponding to each network speed level. Accordingly, determining the first video configuration information of the multiple test videos based on the second video configuration information of the multiple first videos includes: for each network speed level, determining that the average playback duration of the test video corresponding to that network speed level is the same as the average playback duration of the first video corresponding to that network speed level.

[0104] For example, the average playback duration of the first video at each of the multiple network speed levels is: 10 seconds (first video at the first network speed level), 15 seconds (first video at the second network speed level), and 20 seconds (first video at the third network speed level). In this case, the average playback duration of the test videos at each of the multiple network speed levels is determined to be: 10 seconds (test video at the first network speed level), 15 seconds (test video at the second network speed level), and 20 seconds (test video at the third network speed level).

[0105] In this embodiment of the disclosure, an automated software tool can be developed based on the playback duration information corresponding to each network speed level. During the automated playback process, the playback duration of test videos at different network speed levels can be simulated proportionally, thereby simulating the usage of online users.

[0106] Figure 6 This is a structural block diagram of an application testing apparatus provided in an embodiment of this disclosure. See also... Figure 6The device includes: a determination module 601, a first test module 602, a second test module 603, and a result generation module 604. The system includes a determining module 601, configured to determine first device configuration information for multiple test playback devices and configure the multiple test playback devices according to the first device configuration information; and a determining module 602, configured to determine first video configuration information for multiple test videos and configure the multiple test videos according to the first video configuration information; wherein the test playback devices test the preset metrics of the application by playing the test videos; a first testing module 602, configured to play the multiple test videos on the application before the update using the multiple test playback devices and obtain first test information of the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos; a second testing module 603, configured to play the multiple test videos on the application after the update using the multiple test playback devices and obtain second test information of the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos; and a result generation module 604, configured to generate test result information corresponding to the application based on the first test information and the second test information, wherein the test result information represents the comparison result information between the preset metrics of the application in the first version and the preset metrics of the application in the second version.

[0107] According to one or more embodiments of this disclosure, determining multiple test playback devices corresponding to an application includes: obtaining device configuration information of playback devices that play multiple videos on the application within a historical preset duration, wherein the device configuration information includes quantity ratio information corresponding to multiple device models; and determining multiple test playback devices corresponding to the application based on the quantity ratio information corresponding to the multiple device models.

[0108] According to one or more embodiments of this disclosure, the determining module 601 determines first device configuration information of a plurality of test playback devices, specifically including: obtaining second device configuration information of a plurality of first playback devices, wherein the first playback device is a playback device used by a user playing any video on the application within a preset duration; and determining first device configuration information of a plurality of test playback devices based on the second device configuration information of the plurality of first playback devices.

[0109] According to one or more embodiments of this disclosure, the determining module 601 determines first video configuration information of a plurality of test videos, specifically including: obtaining second video configuration information of a plurality of first videos, wherein the first video is any video played by the first playback device on the application within a preset duration; and determining first video configuration information of a plurality of test videos based on the second video configuration information of the plurality of first videos.

[0110] According to one or more embodiments of this disclosure, the first device configuration information includes multiple first model information, each first model information corresponding to multiple first hardware configuration information, and a first quantity ratio of test playback devices corresponding to each first hardware configuration information in all test playback devices; the second device configuration information includes multiple second model information, each second model information corresponding to multiple second hardware configuration information, and a second quantity ratio of playback devices corresponding to each second hardware configuration information in all playback devices; correspondingly, the determining module 601 determines the first device configuration information of multiple test playback devices based on the second device configuration information of the multiple first playback devices, specifically including: determining that the multiple first model information is the same as the multiple second model information; for each group of the same first model information and second model information, determining that the multiple first hardware configuration information corresponding to the first model information is the same as the multiple second hardware configuration information corresponding to the second model information; for each group of the same first hardware configuration information and second hardware configuration information, determining that the first quantity ratio of playback devices corresponding to the first hardware configuration information of the first model information in all playback devices is the same as the second quantity ratio of test playback devices corresponding to the second hardware configuration information of the second model information in all test playback devices.

[0111] According to one or more embodiments of this disclosure, the first video configuration information includes first playback duration information corresponding to each of the plurality of test videos and first quantity percentage information of each video format test video in all test videos; the second video configuration information includes second playback duration information corresponding to each of the plurality of first videos and second quantity percentage information of each video format first video in all first videos; correspondingly, the determining module 601 determines the first video configuration information of the plurality of test videos based on the second video configuration information of the plurality of first videos, specifically including: determining that the first playback duration information corresponding to each of the plurality of test videos is the same as the second playback duration information corresponding to each of the plurality of first videos; for each video format, determining that the first quantity percentage information of the test video of that video format in all test videos is the same as the second quantity percentage information of the first video of that video format in all first videos; wherein, the video format includes video container format and / or video encoding format.

[0112] According to one or more embodiments of this disclosure, the first video configuration information further includes first network speed configuration information corresponding to multiple test videos, and the second video configuration information further includes second network speed configuration information corresponding to multiple first videos; correspondingly, the determining module 601 determines the first video configuration information of multiple test videos based on the second video configuration information of the multiple first videos, specifically including: determining that the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos; for each video format, determining that the first quantity percentage information of the test videos of the video format in all test videos is the same as the second quantity percentage information of the first videos of the video format in all first videos; determining that the third quantity percentage information of the test videos corresponding to each network speed level in the first network speed configuration information is the same as the fourth quantity percentage information of the first videos corresponding to each network speed level in the second network speed configuration information in all first videos.

[0113] According to one or more embodiments of this disclosure, the result generation module 604 generates test result information corresponding to the application based on the first test information and the second test information, specifically including: determining the difference information between the first test information and the second test information, and determining the relative value information between the first test information and the second test information; generating test result information corresponding to the application based on the first test information, the second test information, the difference information and the relative value information.

[0114] refer to Figure 7 The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of the present disclosure. The electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0115] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0116] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0117] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0118] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0119] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0120] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0121] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0123] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0124] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0125] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0126] In a first aspect, according to one or more embodiments of this disclosure, a method for testing an application is provided, comprising:

[0127] The system determines first device configuration information for multiple test playback devices and configures the multiple test playback devices according to the first device configuration information; it also determines first video configuration information for multiple test videos and configures the multiple test videos according to the first video configuration information; wherein the test playback devices test preset metrics of the application by playing the test videos; by playing the multiple test videos on a first version of the application before the update using the multiple test playback devices, first test information of the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained; by playing the multiple test videos on a second version of the application using the multiple test playback devices, second test information of the preset metrics corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained; and based on the first test information and the second test information, test result information corresponding to the application is generated, wherein the test result information is used to represent the comparison result information of the preset metrics of the first version of the application and the preset metrics of the second version of the application.

[0128] According to one or more embodiments of this disclosure, determining the first device configuration information of a plurality of test playback devices includes: obtaining second device configuration information of a plurality of first playback devices, wherein the first playback device is a playback device used by a user playing any video on the application within a preset duration; and determining the first device configuration information of a plurality of test playback devices based on the second device configuration information of the plurality of first playback devices.

[0129] According to one or more embodiments of this disclosure, determining the first video configuration information of a plurality of test videos includes: obtaining second video configuration information of a plurality of first videos, wherein the first video is any video played by the first playback device on the application within a preset duration; and determining the first video configuration information of a plurality of test videos based on the second video configuration information of the plurality of first videos.

[0130] According to one or more embodiments of this disclosure, the first device configuration information includes multiple first model information, each first model information corresponding to multiple first hardware configuration information, and a first quantity ratio of test playback devices corresponding to each first hardware configuration information in all test playback devices; the second device configuration information includes multiple second model information, each second model information corresponding to multiple second hardware configuration information, and a second quantity ratio of playback devices corresponding to each second hardware configuration information in all playback devices; correspondingly, determining the first device configuration information of multiple test playback devices based on the second device configuration information of the multiple first playback devices includes: determining that the multiple first model information is the same as the multiple second model information; for each set of the same first model information and second model information, determining that the multiple first hardware configuration information corresponding to the first model information is the same as the multiple second hardware configuration information corresponding to the second model information; for each set of the same first hardware configuration information and second hardware configuration information, determining that the first quantity ratio of playback devices corresponding to the first hardware configuration information of the first model information in all playback devices is the same as the second quantity ratio of test playback devices corresponding to the second hardware configuration information of the second model information in all test playback devices.

[0131] According to one or more embodiments of this disclosure, the first video configuration information includes first playback duration information corresponding to each of the plurality of test videos and first quantity percentage information of each video format test video in all test videos; the second video configuration information includes second playback duration information corresponding to each of the plurality of first videos and second quantity percentage information of each video format first video in all first videos; correspondingly, determining the first video configuration information of the plurality of test videos based on the second video configuration information of the plurality of first videos includes: determining that the first playback duration information corresponding to each of the plurality of test videos is the same as the second playback duration information corresponding to each of the plurality of first videos; for each video format, determining that the first quantity percentage information of the test video of that video format in all test videos is the same as the second quantity percentage information of the first video of that video format in all first videos; wherein, the video format includes video container format and / or video encoding format.

[0132] According to one or more embodiments of this disclosure, the first video configuration information further includes first network speed configuration information corresponding to multiple test videos, and the second video configuration information further includes second network speed configuration information corresponding to multiple first videos; correspondingly, determining the first video configuration information of multiple test videos based on the second video configuration information of the multiple first videos includes: determining that the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos; for each video format, determining that the first quantity percentage information of the test videos of that video format in all test videos is the same as the second quantity percentage information of the first videos of that video format in all first videos; determining that the third quantity percentage information of the test videos corresponding to each network speed level in the first network speed configuration information is the same as the fourth quantity percentage information of the first videos corresponding to each network speed level in the second network speed configuration information in all first videos.

[0133] According to one or more embodiments of this disclosure, generating test result information corresponding to the application based on the first test information and the second test information includes: determining difference information between the first test information and the second test information, and determining relative value information between the first test information and the second test information; generating test result information corresponding to the application based on the first test information, the second test information, the difference information, and the relative value information.

[0134] Secondly, according to one or more embodiments of this disclosure, an application testing apparatus is provided, comprising: a determining module, configured to determine first device configuration information of a plurality of test playback devices, and configure the plurality of test playback devices according to the first device configuration information; and to determine first video configuration information of a plurality of test videos, and configure the plurality of test videos according to the first video configuration information; wherein the test playback devices test preset metrics of the application by playing the test videos; a first testing module, configured to play the plurality of test videos on a first version of the application before an update using the plurality of test playback devices, and obtain first test information of preset metrics corresponding to each of the plurality of test playback devices when playing the plurality of test videos; a second testing module, configured to play the plurality of test videos on a second version of the application after an update using the plurality of test playback devices, and obtain second test information of preset metrics corresponding to each of the plurality of test playback devices when playing the plurality of test videos; and a result generating module, configured to generate test result information corresponding to the application based on the first test information and the second test information, wherein the test result information is used to represent a comparison result information between the preset metrics of the first version of the application and the preset metrics of the second version of the application.

[0135] According to one or more embodiments of this disclosure, the determining module determines first device configuration information of a plurality of test playback devices, specifically including: obtaining second device configuration information of a plurality of first playback devices, wherein the first playback device is a playback device used by a user playing any video on the application within a preset duration; and determining first device configuration information of a plurality of test playback devices based on the second device configuration information of the plurality of first playback devices.

[0136] According to one or more embodiments of this disclosure, the determining module determines first video configuration information of multiple test videos, specifically including: obtaining second video configuration information of multiple first videos, wherein the first video is any video played by the first playback device on the application within a preset duration; and determining first video configuration information of multiple test videos based on the second video configuration information of the multiple first videos.

[0137] According to one or more embodiments of this disclosure, the first device configuration information includes multiple first model information, each first model information corresponding to multiple first hardware configuration information, and a first quantity ratio of test playback devices corresponding to each first hardware configuration information in all test playback devices; the second device configuration information includes multiple second model information, each second model information corresponding to multiple second hardware configuration information, and a second quantity ratio of playback devices corresponding to each second hardware configuration information in all playback devices; correspondingly, the determining module determines the first device configuration information of multiple test playback devices based on the second device configuration information of the multiple first playback devices, specifically including: determining that the multiple first model information is the same as the multiple second model information; for each set of the same first model information and second model information, determining that the multiple first hardware configuration information corresponding to the first model information is the same as the multiple second hardware configuration information corresponding to the second model information; for each set of the same first hardware configuration information and second hardware configuration information, determining that the first quantity ratio of playback devices corresponding to the first hardware configuration information of the first model information in all playback devices is the same as the second quantity ratio of test playback devices corresponding to the second hardware configuration information of the second model information in all test playback devices.

[0138] According to one or more embodiments of this disclosure, the first video configuration information includes first playback duration information corresponding to each of the plurality of test videos and first quantity percentage information of each video format test video in all test videos; the second video configuration information includes second playback duration information corresponding to each of the plurality of first videos and second quantity percentage information of each video format first video in all first videos; correspondingly, the determining module determines the first video configuration information of the plurality of test videos based on the second video configuration information of the plurality of first videos, specifically including: determining that the first playback duration information corresponding to each of the plurality of test videos is the same as the second playback duration information corresponding to each of the plurality of first videos; for each video format, determining that the first quantity percentage information of the test video of that video format in all test videos is the same as the second quantity percentage information of the first video of that video format in all first videos; wherein, the video format includes video container format and / or video encoding format.

[0139] According to one or more embodiments of this disclosure, the first video configuration information further includes first network speed configuration information corresponding to multiple test videos, and the second video configuration information further includes second network speed configuration information corresponding to multiple first videos; correspondingly, the determining module determines the first video configuration information of multiple test videos based on the second video configuration information of the multiple first videos, specifically including: determining that the first playback duration information corresponding to each of the multiple test videos is the same as the second playback duration information corresponding to each of the multiple first videos; for each video format, determining that the first quantity percentage information of the test videos of that video format in all test videos is the same as the second quantity percentage information of the first videos of that video format in all first videos; determining that the third quantity percentage information of the test videos corresponding to each network speed level in the first network speed configuration information is the same as the fourth quantity percentage information of the first videos corresponding to each network speed level in the second network speed configuration information in all first videos.

[0140] According to one or more embodiments of this disclosure, the result generation module generates test result information corresponding to the application based on the first test information and the second test information, specifically including: determining the difference information between the first test information and the second test information, and determining the relative value information between the first test information and the second test information; generating test result information corresponding to the application based on the first test information, the second test information, the difference information and the relative value information.

[0141] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

[0142] The memory stores computer-executed instructions;

[0143] The processor executes computer execution instructions stored in the memory to implement the test method for the application described in the first aspect and various possible designs of the first aspect.

[0144] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, a test method for implementing the application described in the first aspect and various possible designs of the first aspect is provided.

[0145] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the testing method for the application described in the first aspect and various possible designs of the first aspect.

[0146] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0147] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0148] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for testing an application, characterized in that, include: The system determines first device configuration information for multiple test playback devices and configures the multiple test playback devices according to the first device configuration information; it also determines first video configuration information for multiple test videos and configures the multiple test videos according to the first video configuration information; wherein the test playback devices test preset metrics of the application by playing the test videos. By playing the multiple test videos on the first version of the application before the update using the multiple test playback devices, first test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos is obtained; By playing the multiple test videos on the updated second version of the application using the multiple test playback devices, second test information corresponding to the preset indicators of each of the multiple test playback devices when playing the multiple test videos is obtained; Based on the first test information and the second test information, test result information corresponding to the application is generated, wherein the test result information is used to represent the comparison result information between the preset indicators of the first version of the application and the preset indicators of the second version of the application.

2. The method according to claim 1, characterized in that, The determination of the first device configuration information for multiple test playback devices includes: Obtain second device configuration information for multiple first playback devices, wherein the first playback device is a playback device used by a user to play any video on the application within a preset time period; Based on the second device configuration information of the plurality of first playback devices, the first device configuration information of the plurality of test playback devices is determined.

3. The method according to claim 2, characterized in that, The determination of the first video configuration information for multiple test videos includes: Obtain second video configuration information for multiple first videos, wherein the first video is any video played by the first playback device on the application within a preset duration; Based on the second video configuration information of the plurality of first videos, the first video configuration information of the plurality of test videos is determined.

4. The method according to claim 2, characterized in that, The first device configuration information includes multiple first device model information, each first device model information corresponds to multiple first hardware configuration information, and the first quantity ratio of the test playback device corresponding to each first hardware configuration information in all test playback devices; the second device configuration information includes multiple second device model information, each second device model information corresponds to multiple second hardware configuration information, and the second quantity ratio of the playback device corresponding to each second hardware configuration information in all playback devices. Accordingly, determining the first device configuration information of the plurality of test playback devices based on the second device configuration information of the plurality of first playback devices includes: It is determined that the plurality of first model information and the plurality of second model information are the same; For each set of identical first model information and second model information, it is determined that the first model information corresponds to multiple first hardware configuration information and the second model information corresponds to multiple second hardware configuration information. For each group of first model information and second model information with the same first hardware configuration information, determine that the first number ratio of the playback device corresponding to the first hardware configuration information of the first model information in all playback devices is the same as the second number ratio of the test playback device corresponding to the second hardware configuration information of the second model information in all test playback devices.

5. The method according to claim 3, characterized in that, The first video configuration information includes first playback duration information corresponding to each of the multiple test videos, and first quantity percentage information of each video format test video in all test videos; the second video configuration information includes second playback duration information corresponding to each of the multiple first videos, and second quantity percentage information of each video format first video in all first videos; Accordingly, determining the first video configuration information of the multiple test videos based on the second video configuration information of the multiple first videos includes: It is determined that the first playback duration information corresponding to each of the plurality of test videos is the same as the second playback duration information corresponding to each of the plurality of first videos; For each video format, the first quantity percentage information of the test video of the video format in all test videos is determined to be the same as the second quantity percentage information of the first video of the video format in all first videos; The video format includes video container format and / or video encoding format.

6. The method according to claim 5, characterized in that, The first video configuration information also includes first network speed configuration information corresponding to multiple test videos, and the second video configuration information also includes second network speed configuration information corresponding to multiple first videos; Accordingly, determining the first video configuration information of the multiple test videos based on the second video configuration information of the multiple first videos includes: It is determined that the first playback duration information corresponding to each of the plurality of test videos is the same as the second playback duration information corresponding to each of the plurality of first videos; For each video format, the first quantity percentage information of the test video of the video format in all test videos is determined to be the same as the second quantity percentage information of the first video of the video format in all first videos; The third quantity percentage information of the test video corresponding to each network speed level in the first network speed configuration information is determined to be the same as the fourth quantity percentage information of the first video corresponding to each network speed level in the second network speed configuration information.

7. The method according to any one of claims 1-6, characterized in that, The step of generating test result information corresponding to the application based on the first test information and the second test information includes: Determine the difference information between the first test information and the second test information, and determine the relative value information between the first test information and the second test information; Based on the first test information, the second test information, the difference information, and the relative value information, test result information corresponding to the application is generated.

8. A testing apparatus for an application, characterized in that, include: A determination module is used to determine first device configuration information of multiple test playback devices and configure multiple test playback devices according to the first device configuration information; and to determine first video configuration information of multiple test videos and configure multiple test videos according to the first video configuration information; wherein, the test playback devices test the preset indicators of the application by playing the test videos; The first test module is used to play the multiple test videos on the first version of the application before the update through the multiple test playback devices, and to obtain the first test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos. The second testing module is used to play the multiple test videos on the updated second version of the application through the multiple test playback devices, and to obtain second test information of the preset indicators corresponding to each of the multiple test playback devices when playing the multiple test videos. The result generation module is used to generate test result information corresponding to the application based on the first test information and the second test information, wherein the test result information is used to represent the comparison result information between the preset indicators of the first version of the application and the preset indicators of the second version of the application.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the testing method for the application as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the test method for the application as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the testing method for the application as described in any one of claims 1 to 7.

Images