Software product test comparison method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a software product test comparison method and device, equipment and a storage medium, and the method comprises the steps: obtaining a test plan, and determining a target test template from a plurality of preset common test templates according to the business line information and product module information in the test plan; querying a test case identifier corresponding to the first target test scene identifier, and sending the test case identifier to a target test platform, so that the target test platform operates a target test case corresponding to the test case identifier, and test result data corresponding to the set software products are obtained and fed back; target result data corresponding to the first target test index are screened out from the fed-back test result data, statistics and processing are conducted based on the target result data, and test comparison results corresponding to the multiple software products are obtained. According to the scheme, the test scene and the test index required by the test can be matched, the test comparison of the software product is carried out, and a comprehensive and reliable test comparison result is provided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for testing and comparing software products. Background Art

[0002] With the rapid development of computer technology, various software products have become increasingly integrated into people's lives, work, entertainment, and other areas, meeting diverse digital needs. For example, live streaming applications, instant messaging applications, and short video applications can provide people with a convenient and rich online experience. As users' demands for software feature richness, smooth experience, and performance stability continue to rise, optimizing and iterating software products has become increasingly important. Software product testing and comparison is a key step in optimizing and iterating software products. It can evaluate the performance of different software products of the same type and is a key way to improve the functional completeness of one's own products and optimize the user experience.

[0003] Related technologies use basic testing tools to test and compare software products, which can only provide test comparison results for a relatively single test dimension. However, due to the increasing diversity and complexity of operating environments and functional requirements, these methods lack multi-dimensional test evaluation and have low execution efficiency, making them unable to provide comprehensive and reliable test comparison results. Summary of the Invention

[0004] The embodiments of the present application provide a test and comparison method, apparatus, device, and storage medium for a software product, which solve the problems in the related art of lacking multi-dimensional test evaluation, having low execution efficiency, and being unable to provide comprehensive and reliable test comparison results. The method can quickly match the test scenarios and test indicators required for the current test, can take into account multi-dimensional test evaluation, efficiently perform test comparison of software products, and provide comprehensive and reliable test comparison results.

[0005] In a first aspect, an embodiment of the present application provides a method for testing and comparing a software product, the method comprising:

[0006] Obtaining a test plan, and determining a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator;

[0007] Querying a test case identifier corresponding to the first target test scenario identifier, sending the test case identifier to a target test platform so that the target test platform runs a target test case corresponding to the test case identifier, obtaining test result data corresponding to a plurality of set software products and providing feedback, wherein the target test case is pre-set with a plurality of test indicators;

[0008] Target result data corresponding to the first target test indicator is screened out from the fed-back test result data, and statistics and processing are performed based on the target result data to obtain test comparison results corresponding to the multiple software products.

[0009] In a second aspect, an embodiment of the present application further provides a software product testing and comparison device, the device comprising:

[0010] a test template determination module configured to obtain a test plan and determine a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator;

[0011] a test case determination module configured to query a test case identifier corresponding to the first target test scenario identifier, send the test case identifier to a target test platform, so that the target test platform runs a target test case corresponding to the test case identifier, obtain test result data corresponding to a plurality of set software products, and provide feedback, wherein the target test case is pre-set with a plurality of test indicators;

[0012] The comparison result generation module is configured to filter out target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to the multiple software products.

[0013] In a third aspect, an embodiment of the present application further provides a software product testing and comparison device, the device comprising:

[0014] one or more processors;

[0015] a storage device configured to store one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the software product testing and comparison method described in the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are configured to execute the software product testing and comparison method described in the embodiment of the present application.

[0018] In a fifth aspect, an embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device executes the testing and comparison method of the software product described in the embodiment of the present application.

[0019] In an embodiment of the present application, by obtaining a test plan, a target test template is determined from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; the test case identifier corresponding to the first target test scenario identifier is queried, and the test case identifier is sent to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators; the target result data corresponding to the first target test indicator is filtered out from the fed-back test result data, and statistics and processing are performed based on the target result data to obtain test comparison results corresponding to the multiple software products. In the above scheme, by determining the target test template from multiple pre-set public test templates based on the business line information and product module information in the test plan, the pre-built public test template can be reasonably utilized to quickly match the test scenarios and test indicators required for the current test, thereby improving the test configuration efficiency; by querying the test case identifier corresponding to the first target test scenario identifier and sending the test case identifier to the target test platform, the target test platform can quickly locate the required test case and generate test result data for multiple software products corresponding to multiple different test indicators; by screening out the target result data corresponding to the first target test indicator from the feedback test result data, and performing statistics and processing based on the target result data, the test comparison results corresponding to multiple software products are obtained, which can take into account multi-dimensional test evaluation, efficiently perform test comparison of software products, and provide comprehensive and reliable test comparison results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a software product testing and comparison method provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a software product testing and comparison method including a process of constructing a target test template provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of a software product testing and comparison method including a process of updating a target test template provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of a software product testing and comparison method including a process for generating test and comparison results provided in an embodiment of the present application;

[0024] Figure 5 A flowchart of another software product testing and comparison method including a process for generating test and comparison results provided in an embodiment of the present application;

[0025] Figure 6 A flowchart of a software product testing and comparison method including a process for verifying target result data provided in an embodiment of the present application;

[0026] Figure 7 A structural block diagram of a software product testing and comparison device provided in an embodiment of the present application;

[0027] Figure 8 A schematic diagram of the structure of a testing and comparison device for a software product provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the embodiments of the present application, rather than all structures.

[0029] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0030] The software product testing and comparison method provided in the embodiment of the present application can perform test comparisons of multiple software products corresponding to different test scenarios and different test indicators, and provide comprehensive and reliable test comparison results. Taking audio and video applications as an example, related software products may include live broadcast applications, instant messaging applications, short video applications, etc. The several software products listed above are only exemplary and explanatory, and can also cover different software products in other fields for test comparison, which is not limited by the embodiment of the present application. The software product testing and comparison method, device, equipment and storage medium provided in the embodiment of the present application are intended to solve the problem of lack of multi-dimensional test evaluation in related technologies, low execution efficiency, and inability to provide comprehensive and reliable test comparison results.

[0031] In the software product testing and comparison method provided in the embodiment of the present application, the execution entity of each step can be a computer device, which refers to any electronic device with data calculation, processing and storage capabilities, such as a server, etc., and the embodiment of the present application does not limit this.

[0032] Figure 1 This is a flowchart of a software product testing and comparison method provided in an embodiment of the present application. Figure 1 As shown, the following steps are included:

[0033] Step S101: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0034] Among them, the test plan can record a variety of test configuration information pre-registered by developers based on specific test requirements. The various test configuration information can include business line information, product module information, test time information, description information, etc. The business line information can include one or more business line identifiers corresponding to the business line, each of which is used to represent the specific business field to which the software product to be tested belongs, such as instant messaging business, live video business, short video business, etc., and each business field can pre-deploy multiple similar software products to be tested. The product module information can include one or more product module identifiers corresponding to the product modules, each of which is used to represent the specific functional module corresponding to the software product to be tested, such as audio transmission, video transmission, client performance, client time consumption, etc., and each business line can be associated with one or more product modules. For example, for instant messaging business, the corresponding associated product modules can be video transmission, audio transmission, client performance, client time consumption, etc. For another example, for live broadcast business, the corresponding associated product modules can be live broadcast image quality, beauty and makeup, video transmission, client time consumption, etc. The public test template may record standardized test parameters pre-configured for a specific product module associated with a specific business line. The standardized test parameters specifically include a test scenario identifier corresponding to at least one test scenario and at least one test indicator. For example, a public test template may be pre-configured for instant messaging services, and its associated product module may be audio transmission. The corresponding test scenarios may include two-person instant call, three-person instant call, different weak network types, etc., and the corresponding test indicators may include end-to-end latency, uplink bit rate, downlink bit rate, sound quality smoothness, and subjective sound quality ratings. For another example, a public test template may be pre-configured for instant messaging services, and its associated product module may be client performance. The corresponding test scenarios may include two-person instant call, three-person instant call, etc., and the corresponding test indicators may include processor occupancy, memory usage, power consumption, etc. Therefore, multiple different public test templates can be configured for different business lines and different product modules to meet the testing requirements of different test scenarios and different test indicators, achieving the purpose of reusing test scenarios and test indicators and simplifying the configuration process. Since the business line identifier and product module identifier can be recorded in the public test template, the business line information and product module information in the test plan can be compared and matched with the business line identifier and product module identifier recorded in each public test template, thereby determining a target test template from multiple public test templates. The target test template can include a first target test scenario identifier and a first target test indicator. The number of the first target test scenario identifiers can be one or more, and the number of the first target test indicators can be one or more.

[0035] Step S102: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0036] Among them, since different test scenarios and different test indicators need to rely on different operating environments to perform tests, therefore, in this embodiment, the developer pre-deploys test cases corresponding to different test scenarios on multiple different types of external test platforms, and each test case is pre-set with multiple test indicators. For example, the test platform can include a data-driven platform, a performance monitoring platform, etc. Among them, taking audio and video applications as an example, the data-driven platform can be used to test audio transmission, video transmission and other product modules corresponding to similar software products such as live broadcast services and instant messaging services. The test scenarios associated with the test cases set can be multi-person instant calls, different weak network types, etc., and the test indicators set in each test case can be frame rate, freeze rate, end-to-end delay, uplink bit rate, downlink bit rate, sound quality smoothness, fluency, clarity, etc. The performance monitoring platform can be used to test client performance and other product modules corresponding to similar software products such as live broadcast services and instant messaging services. The test scenarios associated with the test cases set can be multi-person video calls with different numbers of people, multi-person audio calls, etc., and the test indicators set in each test case can be processor occupancy, memory usage, power consumption, etc. Of course, the specific test platform and test case can be deployed by the developer according to the test requirements of the actual application scenario, and this application is not limited to this. By querying the preset corresponding relationship or database, the test case identifier corresponding to each first target test scenario identifier can be obtained, which is used to determine the target test case that matches each first target test scenario. By sending the test case identifier to the target test platform, the target test case that the target test platform needs to run can be indicated. The target test case can test and obtain the result values corresponding to multiple test indicators of the set multiple software products under the first target test scenario, and the multiple test indicators can include the first target test indicator. In one embodiment, the target test platform can feedback the test result data to the server after running the target test case to obtain the test result data corresponding to multiple software products. In one embodiment, the target test platform can feedback the result URL (Uniform Resource Locator) to the server after running the target test case to obtain the test result data corresponding to multiple software products. The server can download the test result data based on the result URL, which is more flexible and convenient.

[0037] Step S103: Filter target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to multiple software products.

[0038] Among them, since the test result data contains the result values corresponding to multiple pre-set test indicators, it is necessary to further filter out the target result data corresponding to one or more first target test indicators. The target result data may include the result values of each first target test indicator associated with the product modules corresponding to different software products under each first target test scenario. For example, the first target test scenario may be a lossless transmission scenario, an uplink speed limit scenario, a downlink speed limit scenario, etc., and the target result data may be the uplink bit rate, downlink bit rate, frame rate, freeze rate and end-to-end delay obtained by measuring the video transmission of different software products. For another example, the first target test scenario may be a group video call of 3 people, a group video call of 4 people, a group video call of 6 people, etc., and the target result data may be the processor occupancy, memory usage, power consumption, etc. obtained by measuring the client corresponding to different software products running on a preset terminal device. In one embodiment, the target result data is counted and processed, which can be a one-by-one comparison or deviation calculation of each test indicator corresponding to different software products in each first target test scenario, thereby obtaining a test comparison result. The specific form of the test comparison result can be a statistical table, a radar chart, etc., which can be used to compare and assemble different indicators, which is beneficial for relevant personnel to compare the performance or functional differences between different software products. In one embodiment, the target result data is counted and processed, which can be a one-by-one comparison or deviation calculation of each test indicator corresponding to different software products in each first target test scenario, thereby obtaining a test comparison result. The specific form of the test comparison result can be a line chart, a bar chart, etc., which is beneficial for relevant personnel to observe the optimization status of software products based on the time dimension and decide on the optimization direction.

[0039] In the above, by obtaining the test plan, a target test template is determined from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; the test case identifier corresponding to the first target test scenario identifier is queried, and the test case identifier is sent to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators; the target result data corresponding to the first target test indicator is filtered out from the fed-back test result data, and statistics and processing are performed based on the target result data to obtain the test comparison results corresponding to the multiple software products. In the above scheme, by determining the target test template from multiple pre-set public test templates based on the business line information and product module information in the test plan, the pre-built public test template can be reasonably utilized to quickly match the test scenarios and test indicators required for the current test, thereby improving the test configuration efficiency; by querying the test case identifier corresponding to the first target test scenario identifier and sending the test case identifier to the target test platform, the target test platform can quickly locate the required test case and generate test result data for multiple software products corresponding to multiple different test indicators; by screening out the target result data corresponding to the first target test indicator from the feedback test result data, and performing statistics and processing based on the target result data, the test comparison results corresponding to multiple software products are obtained, which can take into account multi-dimensional test evaluation, efficiently perform test comparison of software products, and provide comprehensive and reliable test comparison results.

[0040] Figure 2 The flowchart of a software product test comparison method including the process of constructing a target test template is provided in an embodiment of the present application. Figure 2 As shown, the following steps are included:

[0041] Step S201: Acquire test description information, a test scenario set, and a test indicator set, wherein the test description information includes business line information, product module information, scenario description information, and indicator description information.

[0042] Among them, the test description information can be the key information required to construct the target test template, which can specifically include business line information, product module information, scenario description information and indicator description information. The scenario description information can be the text description content of the test scenario associated with the business line and product module pre-selected by the developer according to the specific test requirements. The indicator description information can be the text description content of the test indicator associated with the business line and product module pre-selected by the developer according to the specific test requirements. The test scenario set can be a collection of multiple test scenario identifiers and corresponding descriptive information corresponding to different businesses and different product modules. The test indicator set can be a collection of multiple test indicators and corresponding descriptive information corresponding to different businesses and different product modules pre-set.

[0043] Step S202: extract a first target test scenario identifier from the test scenario set according to the scenario description information, and extract a first target test indicator from the test indicator set according to the indicator description information.

[0044] The first target test scenario identifier associated with the test scenario set can be extracted by semantically matching the description information corresponding to each test scenario identifier in the test scenario set with the scenario description information. The first target test indicator associated with the test indicator set can be extracted by semantically matching the description information corresponding to each test indicator in the test indicator set with the indicator description information.

[0045] Step S203: construct a target test template based on the business line information, product module information, the first target test scenario identifier and the first target test indicator.

[0046] Among them, by combining business line information, product module information, the first target test scenario identifier and the first target test indicator, a target test template can be obtained. The target test template can be stored as a public test template for template matching in subsequent test plans.

[0047] Step S204: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0048] Step S205: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0049] Step S206: Filter target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to the multiple software products.

[0050] As described above, by extracting the first target test scenario identifier and the first target test indicator from the test scenario set and the test indicator set respectively according to the scenario description information and the indicator description information, the test scenarios and test indicators expected to be set for each test template can be effectively screened out. By constructing a target test template based on business line information, product module information, the first target test scenario identifier and the first target test indicator, a matchable public test template is provided for the test plan, the combination reuse of test scenarios and test indicators is improved, and the diversified testing needs of different business and product modules are met.

[0051] Figure 3 The flowchart of a software product test comparison method including a process of updating a target test template is provided in an embodiment of the present application. Figure 3 As shown, the following steps are included:

[0052] Step S301: Acquire test description information, a test scenario set, and a test indicator set, wherein the test description information includes business line information, product module information, scenario description information, and indicator description information.

[0053] Step S302: extract a first target test scenario identifier from the test scenario set according to the scenario description information, and extract a first target test indicator from the test indicator set according to the indicator description information.

[0054] Step S303: construct a target test template based on the business line information, product module information, the first target test scenario identifier and the first target test indicator.

[0055] Step S304: Acquire template update information, where the template update information includes a target template identifier, scenario update information, and indicator update information.

[0056] The template update information may be key information required to update the target test template, specifically including a target template identifier, scenario update information, and indicator update information. The target template identifier may be used to locate the target test template from multiple established public test templates. The scenario update information may be a textual description of the test scenario that the developer determines to be modified based on specific adjustments to actual test requirements. The indicator update information may be a textual description of the test indicator that the developer determines to be modified based on specific adjustments to actual test requirements.

[0057] Step S305: extract a second target test scenario identifier from the test scenario set according to the scenario update information, and extract a second target test indicator from the test indicator set according to the indicator update information.

[0058] Among them, by semantically matching the description information corresponding to each test scenario identifier in the test scenario set with the scenario update information, the second target test scenario identifier that needs to be added can be determined. By semantically matching the description information corresponding to each test indicator in the test indicator set with the indicator update information, the second target test indicator that needs to be added can be determined.

[0059] Step S306: Based on the scenario update information, the indicator update information, the second target test scenario identifier, and the second target test indicator, the target test template corresponding to the target template identifier is updated.

[0060] According to the scenario update information, the test scenario that needs to be replaced in the target test template can be deleted and a second target test scenario identifier can be added. Furthermore, according to the indicator update information, the test indicator that needs to be replaced in the target test template can be deleted and a second target test indicator can be added. Thus, the content of the target test template can be updated.

[0061] Step S307: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0062] Step S308: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0063] Step S309: Filter target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to multiple software products.

[0064] As described above, by extracting the second target test scenario identifier and the second target test indicator from the test scenario set and the test indicator set respectively according to the scenario update information and the indicator update information, the template change content that is adapted to the adjustment of the test requirements can be effectively screened out. By updating the target test template corresponding to the target template identifier based on the scenario update information, the indicator update information, the second target test scenario identifier and the second target test indicator, the public test template can be adjusted to adapt to the new test requirements and be compatible with the changes in the requirements of different business and product modules.

[0065] Figure 4 The present invention provides a flowchart of a method for testing and comparing software products including a process for generating test and comparison results. Figure 4 As shown, the following steps are included:

[0066] Step S401: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0067] Step S402: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0068] Step S403: Filter out target result data corresponding to the first target test indicator from the fed-back test result data.

[0069] Step S404: extract at least one first test result value corresponding to the first software product and at least one second test result value corresponding to the second software product from the target result data, perform numerical comparison between the first test result value and the corresponding second test result value to obtain a deviation ratio and a size relationship, and process the at least one first test result value, the at least one second test result value, the deviation ratio, and the size relationship to obtain a numerical comparison result.

[0070] In this embodiment, the multiple software products may include a first software product and a second software product. Specifically, the first test result values and second test result values corresponding to the first and second software products, respectively, may be extracted from the target result data, facilitating a comparison of the individual indicator results of different software products. The deviation ratio can be used to characterize the degree of data deviation between the first test result value and the corresponding second test result value. Specifically, the deviation ratio can be calculated by calculating the difference between the first test result value and the corresponding second test result value and dividing the difference by the first test result value or the second test result value to obtain the deviation ratio. The magnitude relationship can be obtained by comparing the numerical magnitudes of the first test result value and the corresponding second test result value. The numerical comparison results may include a statistical table and a radar chart. The radar chart may indicate multiple first test result values of the first software product under a specific test scenario, as well as multiple second test result values of the corresponding second software products, facilitating a combined comparison of multi-dimensional indicators. The statistical table may record, row by row, the first test result values of the first software product, the corresponding second test result values of the second software product under different test scenarios, the deviation ratio, and the magnitude relationship, facilitating an intuitive comparison result.

[0071] As mentioned above, by numerically comparing the first test result value with the corresponding second test result value to obtain the deviation ratio and size relationship, the advantages and disadvantages of indicators between different software products can be effectively measured, and by processing at least one first test result value, at least one second test result value, the deviation ratio and size relationship to obtain a numerical comparison result, the function or performance comparison results of different software products can be intuitively provided to relevant personnel, providing reliable reference information for the optimization of subsequent software products.

[0072] Figure 5 Flowchart of another software product test comparison method including the process of generating test comparison results provided by the embodiment of the present application. Figure 5 As shown, the following steps are included:

[0073] Step S501: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0074] Step S502: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0075] Step S503: Filter out target result data corresponding to the first target test indicator from the fed-back test result data.

[0076] Step S504: Acquire historical result data corresponding to each software product, extract current result data corresponding to each software product from the target result data, and process the historical result data and current result data corresponding to each software product in time series to obtain a numerical change result.

[0077] The historical result data may be the result data obtained from testing various software products over a period of time. Since software products are in a state of continuous optimization and iteration of their functions or performance, the historical result data and current result data corresponding to each software product can be processed in a time series. This is equivalent to aligning the indicator result data of each software product along the time axis from a temporal dimension. While observing the changes in the optimization of each test indicator of each software product over time, the optimization differences between different software products can be compared. The numerical change results may include line charts, bar charts, etc.

[0078] As mentioned above, by processing the historical result data and current result data corresponding to each software product in time series to obtain numerical change results, the indicator result change trends of different software products can be intuitively provided to relevant personnel, providing dynamic reference information in the time dimension for the optimization of subsequent software products.

[0079] Figure 6 The flowchart of a method for testing and comparing a software product including a process for verifying target result data is provided in an embodiment of the present application. Figure 6 As shown, the following steps are included:

[0080] Step S601: Obtain a test plan, and determine a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator.

[0081] Step S602: query the test case identifier corresponding to the first target test scenario identifier, and send the test case identifier to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators.

[0082] Step S603: Filter out target result data corresponding to the first target test indicator from the fed-back test result data.

[0083] Step S604: Perform format check on the target result data. If the format check passes, determine whether there is abnormal data in the target result data based on the set threshold range. If abnormal data exists, remove the abnormal data from the target result data, generate corresponding alarm information, and feed back the alarm information to the target test platform.

[0084] Among them, in order to ensure the reliability and correctness of the target result data, the target result data can be format checked to check whether the target result data conforms to the pre-set format specifications to avoid processing the erroneous data returned by the target test platform. If the format check passes, it can continue to be combined with the set threshold range to determine whether the target result data is within the normal value range, and then eliminate abnormal data to prevent abnormal data from interfering with subsequent data processing and failing to provide reliable test comparison results. In addition, an alarm message can be generated for abnormal data and fed back to the target test platform to prompt the target test platform to conduct relevant abnormality investigation.

[0085] Step S605: Perform statistics and processing based on the target result data to obtain test comparison results corresponding to multiple software products.

[0086] By performing format verification and removing abnormal data in this manner, the reliability and accuracy of the target result data can be guaranteed, preventing abnormal data from affecting subsequent data analysis or processing. In the event of abnormal data, corresponding alarm information is generated and fed back to the target test platform, prompting the target test platform to conduct anomaly investigation and optimize subsequent test feedback results.

[0087] Figure 7 This is a structural block diagram of a software product testing and comparison device provided in an embodiment of the present application. The device is configured to execute the software product testing and comparison method provided in the above embodiment, and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the device includes:

[0088] The test template determination module 101 is configured to obtain a test plan and determine a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator;

[0089] The test case determination module 102 is configured to query a test case identifier corresponding to the first target test scenario identifier, send the test case identifier to a target test platform, so that the target test platform runs the target test case corresponding to the test case identifier, obtain test result data corresponding to the set multiple software products, and provide feedback, wherein the target test case is pre-set with multiple test indicators;

[0090] The comparison result generating module 103 is configured to filter out target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to multiple software products.

[0091] In the above, by obtaining the test plan, a target test template is determined from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; the test case identifier corresponding to the first target test scenario identifier is queried, and the test case identifier is sent to the target test platform so that the target test platform runs the target test case corresponding to the test case identifier, obtains the test result data corresponding to the set multiple software products and provides feedback, wherein the target test case is pre-set with multiple test indicators; the target result data corresponding to the first target test indicator is filtered out from the fed-back test result data, and statistics and processing are performed based on the target result data to obtain the test comparison results corresponding to the multiple software products. In the above scheme, by determining the target test template from multiple pre-set public test templates based on the business line information and product module information in the test plan, the pre-built public test template can be reasonably utilized to quickly match the test scenarios and test indicators required for the current test, thereby improving the test configuration efficiency; by querying the test case identifier corresponding to the first target test scenario identifier and sending the test case identifier to the target test platform, the target test platform can quickly locate the required test case and generate test result data for multiple software products corresponding to multiple different test indicators; by screening out the target result data corresponding to the first target test indicator from the feedback test result data, and performing statistics and processing based on the target result data, the test comparison results corresponding to multiple software products are obtained, which can take into account multi-dimensional test evaluation, efficiently perform test comparison of software products, and provide comprehensive and reliable test comparison results.

[0092] In a possible embodiment, a template construction module is further included, configured to:

[0093] Obtain test description information, test scenario set, and test indicator set, where the test description information includes business line information, product module information, scenario description information, and indicator description information;

[0094] Extracting a first target test scenario identifier from the test scenario set according to the scenario description information, and extracting a first target test indicator from the test indicator set according to the indicator description information;

[0095] A target test template is constructed based on the business line information, product module information, the first target test scenario identifier and the first target test indicator.

[0096] In a possible embodiment, a template updating module is further included, configured to:

[0097] Acquire template update information, wherein the template update information includes a target template identifier, scenario update information, and indicator update information;

[0098] Extracting a second target test scenario identifier from the test scenario set according to the scenario update information, and extracting a second target test indicator from the test indicator set according to the indicator update information;

[0099] Based on the scenario update information, the indicator update information, the second target test scenario identifier and the second target test indicator, the target test template corresponding to the target template identifier is updated.

[0100] In a possible embodiment, the comparison result generation module 103 is further configured to:

[0101] extracting at least one first test result value corresponding to the first software product and at least one second test result value corresponding to the second software product from the target result data;

[0102] Comparing the first test result value with the corresponding second test result value to obtain the deviation ratio and magnitude relationship;

[0103] At least one first test result value, at least one second test result value, the deviation ratio and the size relationship are processed to obtain a numerical comparison result.

[0104] In a possible embodiment, the test comparison result includes a numerical change result, and the comparison result generating module 103 is further configured to:

[0105] Obtain historical result data corresponding to each software product;

[0106] Extracting current result data corresponding to each software product from the target result data;

[0107] The historical result data and current result data corresponding to each software product are processed in time series to obtain the numerical change results.

[0108] In a possible embodiment, a result data processing module is further included, configured to:

[0109] Perform format verification on the target result data. If the format verification passes, determine whether there is abnormal data in the target result data based on the set threshold range;

[0110] In the case of abnormal data, the abnormal data is removed from the target result data, the corresponding alarm information is generated, and the alarm information is fed back to the target test platform.

[0111] Figure 8 A schematic diagram of a software product testing and comparison device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the number of processors 201 in the device can be one or more. Figure 8 In the embodiment, a processor 201 is used as an example; the processor 201, the memory 202, the input device 203 and the output device 204 in the device can be connected by a bus or other means. Figure 8 The example of the connection via bus is taken. The memory 202, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the test and comparison method of the software product in the embodiment of the present application. The processor 201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 202, that is, implements the test and comparison method of the above-mentioned software product. The input device 203 can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device 204 may include a display device such as a display screen.

[0112] An embodiment of the present application also provides a non-volatile storage medium containing computer-executable instructions, which, when executed by a computer processor, is configured to execute a test comparison method for a software product described in the above embodiment, which includes: obtaining a test plan, determining a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; querying a test case identifier corresponding to the first target test scenario identifier, sending the test case identifier to a target test platform, so that the target test platform runs the target test case corresponding to the test case identifier, obtaining test result data corresponding to the set plurality of software products and providing feedback, wherein the target test case is pre-set with a plurality of test indicators; filtering out target result data corresponding to the first target test indicator from the fed-back test result data, performing statistics and processing based on the target result data, and obtaining test comparison results corresponding to the plurality of software products.

[0113] It is worth noting that in the embodiment of the testing and comparison device for the above-mentioned software product, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not configured to limit the scope of protection of the embodiments of this application.

[0114] In some possible implementations, various aspects of the methods provided herein may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is configured to cause the computer device to execute the steps of the methods according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the software product testing and comparison methods described in the embodiments of the present application. The program product may be implemented using any combination of one or more readable media.

Claims

1. A software product testing and comparison method, characterized in that: include: Obtaining a test plan, and determining a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; Querying a test case identifier corresponding to the first target test scenario identifier, sending the test case identifier to a target test platform so that the target test platform runs a target test case corresponding to the test case identifier, obtaining test result data corresponding to a plurality of set software products and providing feedback, wherein the target test case is pre-set with a plurality of test indicators; Target result data corresponding to the first target test indicator is screened out from the fed-back test result data, and statistics and processing are performed based on the target result data to obtain test comparison results corresponding to the multiple software products.

2. The software product testing and comparison method according to claim 1, characterized in that: Before determining the target test template from a plurality of pre-set public test templates according to the business line information and product module information in the test plan, the method further includes: Obtain test description information, a test scenario set, and a test indicator set, wherein the test description information includes business line information, product module information, scenario description information, and indicator description information; Extracting a first target test scenario identifier from the test scenario set according to the scenario description information, and extracting a first target test indicator from the test indicator set according to the indicator description information; A target test template is constructed based on the business line information, the product module information, the first target test scenario identifier and the first target test indicator.

3. The software product testing and comparison method according to claim 2, characterized in that: After constructing the target test template based on the business line information, the product module information, the first target test scenario identifier, and the first target test indicator, the method further includes: Acquire template update information, where the template update information includes a target template identifier, scenario update information, and indicator update information; Extracting a second target test scenario identifier from the test scenario set according to the scenario update information, and extracting a second target test indicator from the test indicator set according to the indicator update information; Based on the scenario update information, the indicator update information, the second target test scenario identifier and the second target test indicator, the target test template corresponding to the target template identifier is updated.

4. The software product testing and comparison method according to claim 1, characterized in that: The multiple software products include a first software product and a second software product, the test comparison result includes a numerical comparison result, and the statistical analysis and processing based on the target result data to obtain the test comparison results corresponding to the multiple software products includes: extracting at least one first test result value corresponding to the first software product and at least one second test result value corresponding to the second software product from the target result data; Comparing the first test result value with the corresponding second test result value to obtain a deviation ratio and a size relationship; The at least one first test result value, the at least one second test result value, the deviation ratio and the size relationship are processed to obtain the numerical comparison result.

5. The software product testing and comparison method according to claim 1, characterized in that: The test comparison results include numerical change results, and the statistics and processing based on the target result data to obtain the test comparison results corresponding to the multiple software products include: Obtaining historical result data corresponding to each of the software products; Extracting current result data corresponding to each of the software products from the target result data; The historical result data and the current result data corresponding to each of the software products are processed in time series to obtain the numerical value change result.

6. The software product testing and comparison method according to claim 1, characterized in that: Before performing statistics and processing based on the target result data to obtain the test comparison results corresponding to the multiple software products, the method further includes: Performing format verification on the target result data, and if the format verification passes, determining whether the target result data has abnormal data according to a set threshold range; In the case that the abnormal data exists, the abnormal data is removed from the target result data, corresponding alarm information is generated, and the alarm information is fed back to the target test platform.

7. A testing and comparison device for a software product, characterized in that: include: a test template determination module configured to obtain a test plan and determine a target test template from a plurality of pre-set public test templates based on the business line information and product module information in the test plan, wherein the target test template includes a first target test scenario identifier and a first target test indicator; a test case determination module configured to query a test case identifier corresponding to the first target test scenario identifier, send the test case identifier to a target test platform, so that the target test platform runs a target test case corresponding to the test case identifier, obtain test result data corresponding to a plurality of set software products, and provide feedback, wherein the target test case is pre-set with a plurality of test indicators; The comparison result generation module is configured to filter out target result data corresponding to the first target test indicator from the fed-back test result data, perform statistics and processing based on the target result data, and obtain test comparison results corresponding to the multiple software products.

8. A software product testing and comparison device, comprising: one or more processors; A storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the software product testing and comparison method according to any one of claims 1 to 6.

9. A non-volatile storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are configured to execute the software product testing and comparison method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that The computer program is stored in a computer-readable storage medium, and at least one processor of a device reads and executes the computer program from the computer-readable storage medium, so that the device executes the software product testing and comparison method according to any one of claims 1 to 6.

Images