Target test method, system, equipment and medium, program product

Abstract

The present disclosure provides an object testing method, system, apparatus, medium, and program product. The method includes obtaining a test object and a test scene, testing the test object in the test scene to obtain current index values ​​of a plurality of test indexes, determining at least one important index from the plurality of test indexes based on the current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes when the test object is executed in the test scene, and obtaining a test result for the test object in the test scene based on the at least one important index. The method tests the test object in different test scenes to obtain overall expressions of the test object in the different test scenes, improving the testing accuracy of the object and meeting different service needs.

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 202310476796.7, filed on April 27, 2023, the entire contents of which are hereby incorporated by reference into this application.

[0002] SUMMARY OF THE INVENTION The embodiments of the present disclosure relate to an object testing method, system, electronic device, computer-readable storage medium, and computer program product. [Background technology]

[0003] With the rapid development of computer technology, various applications (APPs) have emerged in response to the times. For example, to meet the online shopping needs of different users, various electronic commerce (e-commerce) applications, such as cross-border e-commerce and self-managed e-commerce, have emerged one after another. Furthermore, to meet the leisure and entertainment needs of different users, various long and short video applications have also emerged.

[0004] During the execution of these applications, there are more situations that may affect the user experience. For example, when an e-commerce application runs a large sales promotion, burst traffic may occur, which may cause system overload or freeze, making it difficult to ensure the user experience. Furthermore, when a video application performs live video streaming, if the network quality is relatively poor, delays or excessive delays may occur, which may affect the user experience.

[0005] Currently, the industry mainly tests applications in a layered manner (also known as target testing). For example, if the test target is an e-commerce application, the user clicks the order control to trigger an order operation, entering the order scene. In the order scene, an order event occurs in the client terminal, and the order event passes through a relatively long link, for example, passing through the cache layer, database layer, network layer, and application layer. Even when target testing is performed, target testing is performed hierarchically, and single-layer test results are obtained. However, the above target testing method generally solves the single-layer problem and is unable to test the overall performance of the test target, making it difficult to meet service needs. Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure provides an object testing method, which tests a test object in different test scenes to obtain a complete representation of the test object in the different test scenes, thereby improving the testing accuracy of the object and meeting different service needs. The present disclosure also provides a system, an electronic device, a computer-readable storage medium, and a computer program product corresponding to the above method. [Means for solving the problem]

[0007] In a first aspect, the present disclosure provides a method of subject testing, the method comprising: acquiring a test subject and a test scene; testing the test object in the test scene to obtain current index values ​​of a plurality of test indexes; determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene; and obtaining a test result for the test scene of the test subject based on the at least one important indicator.

[0008] In some possible implementations, the step of determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene comprises: determining a similarity between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators based on the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators; determining test indicators whose similarity satisfies a set condition as candidate indicators; determining at least one significant indicator from the candidate indicators based on tags of the candidate indicators, the tags of the candidate indicators being intended to describe historical test results of the candidate indicators.

[0009] In some possible implementations, the step of determining a similarity between the current index values ​​of the plurality of test indexes and the historical index values ​​of the plurality of test indexes based on the current index values ​​of the plurality of test indexes and the historical index values ​​of the plurality of test indexes comprises: mapping current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes into a vector space; determining a similarity based on a distance in the vector space between current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes.

[0010] In some possible implementations, the step of obtaining a test result for the test scene of the test object based on the at least one important indicator comprises: determining a set of reference values ​​for the at least one key indicator based on historical indicator values ​​of the at least one key indicator; and obtaining a test result for the test scene of the test object based on the current index value of the at least one important index and the reference value set.

[0011] In some possible implementations, the step of obtaining a test result in the test scene of the test object based on the current index value of the at least one important index and the reference value set includes: If the current indicator value of the at least one important indicator falls within the reference value set, determining that the test result in the test scene of the test object is normal; or The method includes determining a score for the at least one important indicator depending on whether the current indicator value of the at least one important indicator fits the reference value set, and determining that the test result in the test scene of the test subject is normal if the sum of the scores of the at least one important indicator is greater than a predetermined score.

[0012] In some possible implementations, the step of determining a reference value set for the at least one important indicator based on historical indicator values ​​of the at least one important indicator comprises: The method includes performing a fitting analysis of the relationship between the state of the test object and the important indicators based on historical indicator values ​​of the at least one important indicator and important events during the period in which the test object is executed in the test scene, to determine a reference value set for the at least one important indicator, wherein the important events are events that represent a change in the state of the test object.

[0013] In some possible implementations, the method further comprises: determining a change trend of test results in the test scene of the test object based on test results in a plurality of periods of the test object; If the change trend of the test result satisfies a set condition, the method further includes a step of transmitting presented information.

[0014] In a second aspect, the present disclosure provides a subject test system, the system comprising: an acquisition unit for acquiring a test subject and a test scene; a test unit for testing the test object in the test scene to obtain current index values ​​of a plurality of test indexes; a determining unit for determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene; Further, the apparatus comprises a determination unit for obtaining a test result in the test scene of the test object based on the at least one important indicator.

[0015] In some possible implementations, the decision unit specifically: determining a similarity between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators based on the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators; determining test indicators whose similarity satisfies a set condition as candidate indicators; determining at least one significant indicator from the candidate indicators based on the tags of the candidate indicators, the tags of the candidate indicators being used to describe historical test results of the candidate indicators;

[0016] In some possible implementations, the decision unit specifically: Mapping current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes into a vector space, respectively; The similarity is determined based on a distance in the vector space between current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes.

[0017] In some possible implementations, the decision unit specifically: determining a set of reference values ​​for the at least one key indicator based on historical indicator values ​​of the at least one key indicator; and obtaining a test result in the test scene of the test object based on the current index value of the at least one important index and the reference value set.

[0018] In some possible implementations, the decision unit specifically: If the current indicator value of the at least one important indicator falls within the reference value set, determining that the test result in the test scene of the test object is normal; or The method determines a score for the at least one important indicator depending on whether the current indicator value of the at least one important indicator fits the reference value set, and determines that the test result in the test scene of the test subject is normal if the sum of the scores of the at least one important indicator is greater than a predetermined score.

[0019] In some possible implementations, the decision unit specifically: The purpose is to determine a reference value set for the at least one important indicator by performing a fitting analysis of the relationship between the state of the test object and the important indicator based on historical indicator values ​​of the at least one important indicator and important events during the period in which the test object is executed in the test scene, wherein the important events are events that represent a change in the state of the test object.

[0020] In some possible implementations, the decision unit further comprises: determining a change trend of test results in the test scene of the test object based on test results in a plurality of periods of the test object; The system further comprises: The device further comprises a presentation unit for transmitting presentation information when the change trend of the test result satisfies a set condition.

[0021] In a third aspect, the present disclosure provides an electronic device, the electronic device comprising a processor and a memory, the processor and the memory being in communication with each other, the processor executing instructions stored in the memory to cause the electronic device to perform a subject testing method in the first aspect or any one of the implementations of the first aspect.

[0022] In a fourth aspect, the present disclosure provides a computer-readable storage medium having instructions stored thereon for instructing an electronic device to perform a target testing method according to the first aspect or any one of the implementations of the first aspect.

[0023] In a fifth aspect, the present disclosure provides a computer program product including instructions, which, when executed in an electronic device, cause the electronic device to perform the target testing method described in the first aspect or any one of the implementations of the first aspect.

[0024] The present disclosure may be further combined to provide more implementation methods based on the above aspects.

[0025] In order to more clearly describe the technical methods of the embodiments of the present disclosure, the following briefly describes the drawings required for the embodiments. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is an architecture diagram of a target test system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart of an object testing method according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a structural schematic diagram of a target test system according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0027] The terms "first" and "second" in the examples of the present disclosure are merely for descriptive purposes and should not be understood to indicate or imply the relative importance or number of technical features shown, whereby a feature qualified by "first" or "second" may explicitly or implicitly include one or more of the feature.

[0028] First, some technical terms related to the embodiments of the present disclosure will be explained.

[0029] Target testing refers to testing a test target. Specifically, it may be a process of testing multiple functional modules and execution processes of the test target to determine whether the test target is in a normal state. The test target may be a computer program that provides services to users. For example, the test target may include one or more of an application (APP), a cloud service, and a plug-in.

[0030] Take the example of an application being the test target. An application is a computer program created to meet a specific user need. Applications may be divided into different types depending on different user needs. For example, applications may be divided into multiple types, such as entertainment, e-commerce, and social, depending on the user's entertainment, e-commerce, and social needs. Each type may include different applications; for example, e-commerce type applications may include cross-border e-commerce, independent e-commerce, and idle transaction e-commerce applications. For example, entertainment type applications may include long videos, short videos, audiobooks, and other applications.

[0031] During application execution, more situations that affect the user experience may occur. Take e-commerce applications as an example. E-commerce applications involve a large number of enterprise and individual users, and they frequently hold large-scale sales promotions (abbreviated as "large-scale promotions"). During large-scale promotions, a large number of users may trigger corresponding operations (e.g., purchase operations) at specific times (e.g., midnight, 8:00, 14:00, or 20:00 when the large-scale promotions begin). Bursty traffic may cause the e-commerce applications to experience system overloads or freezes, making it difficult to ensure the user's product purchasing experience. Take entertainment applications as an example. Entertainment applications may provide users with live video streaming. When users stream live video, if the network quality is relatively poor, the entertainment applications may experience excessive delays or slowdowns, which may affect the viewers' viewing experience.

[0032] In consideration of the above situation, application providers or operators can test applications to immediately capture events that may occur during the application execution process and affect user experience, and can also upgrade or update the applications. Currently, application providers or operators generally test test objects including applications in a layered manner.

[0033] Specifically, a trigger operation on a user's client terminal can generate a corresponding event, for example, a purchase operation on a user's e-commerce application client can generate a purchase event, and the event passes through a relatively long link to process data and obtain a processing result. The link may include multi-layer technical assemblies, specifically, the multi-layer technical assemblies may include cache layer, database layer, network layer, or application layer technical assemblies, and the multi-layer technical assemblies can process data from different layers, thus completing data processing hierarchically and improving application maintainability.

[0034] When conducting a targeted test, an application provider or operator can test a technology assembly at a certain layer to obtain the test results of the technology assembly at that layer. For example, an application provider or carrier can conduct a targeted test on a network layer to observe indicators related to the network layer, such as bandwidth, delay, and bandwidth-delay product, to determine the performance status of the network layer and obtain the test results of the network layer.

[0035] However, the layered method of performing object testing generally solves the problem of a single layer, but is unable to test the overall performance of the test object, making it difficult to meet service needs.

[0036] In view of this, the present disclosure provides an object testing method, which specifically includes first obtaining a test object and a test scene, testing a test object in the test scene to obtain current index values ​​of a plurality of test indicators, then determining at least one important index from the plurality of test indicators based on the current index values ​​of the plurality of test indicators and historical index values ​​of the plurality of test indicators when the test object is executed in the test scene, and obtaining a test result of the test object in the test scene based on the at least one important index.

[0037] The method supports the test object to perform an overall test in a test scene, and determines important indicators in the test scene based on the current indicator value and historical indicator value of the test indicator. In this way, the overall test result of the test object in the test scene is obtained based on the important indicators in the test scene. Compared with the test result at a certain level of the test object, the method improves the completeness and accuracy of the object test and can meet different service needs.

[0038] To make the technical solution of the present disclosure clearer and easier to understand, the system architecture of the target testing method according to the embodiments of the present disclosure will be described below with reference to the drawings.

[0039] Referring to the architecture diagram of the target test system shown in FIG. 1 , the target test system 10 is for performing a target test on a test target. The target test system 10 may be a software system, which may be located on a server, and the server executes the program code of the software system to perform the target test method according to the embodiment of the present disclosure. In some possible embodiments, the target test system 10 may be a hardware system having a target test function, which executes the target test method according to the embodiment of the present disclosure when executed. For example, the target test system 10 may be a server having a target test function. The server may be a cloud server, such as a central server in a central cloud computing cluster or an edge server in an edge cloud computing cluster. Of course, the server may also be a server in a local data center. A local data center refers to a data center directly controlled by a user.

[0040] Specifically, the object testing system 10 may obtain a test object and a test scene, test the test object in the test scene, and obtain current index values ​​of multiple test indicators. The test scene is a service scene corresponding to a user's trigger operation, and the test scene may be an actual service scene in the execution process of the test object, or a service scene that the test object has not yet been put on the market, and this is not a limitation of the embodiments of the present disclosure.

[0041] In some possible embodiments, the target test system 10 may be connected to a terminal 20, thereby acquiring a test object and a test scene through the terminal 20. The terminal 20 may include, but is not limited to, a smartphone, a tablet computer, a laptop, a personal digital assistant (PDA), or a smart wearable device. Specifically, the terminal 20 may provide a test configuration interface to a user (e.g., a tester), allowing the tester to select a test object and a test scene in the test configuration interface. In response to the tester's selection, the terminal 20 transmits the selected test object and test scene to the target test system 10, allowing the target test system 10 to test the test object in the test scene and acquire current index values ​​of multiple test indexes.

[0042] Furthermore, the target test system 10 may be connected to a database 30, which includes historical execution data of the test object and significant events during the execution of the test object. The historical execution data may specifically include execution data when the test object is executed in at least one test scene within a historical time period, and the significant events may be events representing state changes of the test object during the execution of the test object. The target test system 10 may determine at least one significant indicator from the multiple test indicators based on current indicator values ​​of the multiple test indicators and historical indicator values ​​of the multiple test indicators when the test object is executed in the test scene.

[0043] In this way, the target test system 10 may obtain a test result for the test scene of the test target based on at least one important indicator, and may further transmit the test result to the terminal 20 to display the test result to the tester.

[0044] In some possible implementations, the object test system 10 may further perform tests on the test object within multiple periods, thereby obtaining test results for the test object in multiple periods, and determining the change trend of the test results in the test scene of the test object. In this way, if the change trend of the test results meets the set condition, presentation information will be sent to facilitate the tester to immediately process the test object.

[0045] In the embodiments of the present disclosure, the object testing system 10 may test the test object in different test scenes, and by determining the important indicators in the test scenes, the overall test results in the different test scenes of the test object can be obtained, thereby improving the completeness and accuracy of the object test.

[0046] The object test system 10 for performing the object test method has been described above. The object test method according to an embodiment of the present disclosure will now be described in detail.

[0047] Referring to the flowchart of the object test method shown in FIG. 2, the method includes the following steps S201 to S204.

[0048] S201: Obtain a test object and a test scene.

[0049] A test target refers to an object awaiting testing, which may be a computer program capable of providing services to users, including, but not limited to, one or more of an application, a cloud service, and a plug-in. In a specific implementation, an object after a version update may be determined as the test target. Taking an e-commerce application as an example, during a major promotion period, the e-commerce application may undergo a version update, which specifically includes updating the application's user interface (UI) and other interaction capabilities to support sales promotion activities. Testing the new version of the application as the test target before the new version is released can ensure the availability of the new version of the application.

[0050] The test object can provide more service scenarios. During the execution of the test object, the user can enter the service scenario corresponding to the trigger operation by a trigger operation, thereby completing the corresponding service in the service scenario. Different types of test objects can provide different service scenarios. For example, in an e-commerce application, the user can click a purchase control to trigger a product purchase operation and enter a purchase scenario to complete the product purchase. For example, in a video application, the user can touch a playback control to trigger a video playback operation and enter a playback scenario to complete the video playback.

[0051] The test scene may be one or more service scenes provided by the test object during the execution process. For example, the test scene may generally be a service scene provided by the test object in response to a high-frequency trigger operation during the execution process. The method determines a service scene corresponding to a high-frequency trigger operation, such as a purchase operation or a resource receiving operation (e.g., receiving a coupon or lucky money), as the test scene, thereby making the test scene of the target test relatively highly directional.

[0052] Furthermore, embodiments of the present disclosure support testing a test object in a single test scene or a combination test scene. A combination test scene refers to a test scene that includes multiple service scenes. The test frequency of a test object may vary depending on the test scene. Because a combination test scene includes multiple service scenes, testing a test object in a combination test scene requires consuming relatively large resources. Therefore, the test frequency of a test object in a combination test scene can be appropriately reduced. For example, the test frequency of a test object in a single test scene may be twice a week, while the test frequency of a test object in a combination test scene may be twice a month. In this way, the test object can be tested in different test scenes while saving resources, thereby enriching the test results of the test object.

[0053] Compared with the traditional hierarchical method of object testing, testing the test object in the test scene is an object testing method based on the user's perspective. In other words, the object testing is not limited to a certain technical assembly, but by simulating specific user operations during the execution process of the test object, the test object in the test scene can be realized, thereby obtaining the overall representation of the test object in different test scenes.

[0054] S202: Test the test object in the test scene to obtain current index values ​​of a plurality of test indexes.

[0055] Test indicators are performance indicators for representing the execution status of a test target in a test scene. Test indicators may vary depending on the test scene. In some embodiments, test indicators may be obtained from a scene model corresponding to the test scene. A scene model may be generated when creating a test scene, and multiple indicators may be defined in the scene model, and the multiple indicators may be used as test indicators for the test scene. For example, when testing a purchase scene for an e-commerce application, a user (e.g., a tester) may create the purchase scene as a test scene and perform corresponding configuration for the purchase scene. A scene model corresponding to the purchase scene may be generated in response to the tester's configuration operation. Multiple performance indicators, including response time, remote dictionary server (Redis) capacity, and network bandwidth, may be defined in the scene model. In this case, the test indicators may be the response time, Redis capacity, and network bandwidth. In this way, a scene model can be used to achieve overall management of test scenes and multiple test indicators in the test scene.

[0056] In specific implementation, the current index values ​​of the plurality of test indicators may be determined based on the test time of the test object within one cycle. For example, when the test object is tested within one cycle, the index values ​​of the plurality of test indicators may be continuously obtained, and the average value of the index values ​​of the plurality of test indicators may be determined based on the index values ​​of the plurality of test indicators and the test time, and the average value of the index values ​​of the plurality of test indicators may be used as the current index value of the plurality of test indicators. The test time of one cycle may be set according to the actual needs of the test object and the test scenario, and the embodiments of the present disclosure are not limited thereto.

[0057] In some embodiments, a logical isolation tool can be used to control the scope of the target test, i.e., the logical isolation tool controls the target test so that it only affects the test scene and does not affect other service scenes. In this way, the decomposition radius of the target test can be effectively controlled. Specifically, the logical isolation tool can be deployed within a multi-layer technology assembly related to the test scene. For example, the logical isolation tool can be deployed in the form of a plug-in, and the plug-in can be used to identify the test scene, so that the target test is concentrated in the test scene and does not affect other service scenes.

[0058] S203: Determine at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene.

[0059] The historical index value refers to historical execution data when the test object is executed within a historical time period, and the historical execution data may specifically include execution data when the test object is executed in at least one test scene within the historical time period. The historical time period may be the time period from when the test object has completed development to the present, or the time period from when the test object has been officially released on the market to the present, or may be a time period set by a user (e.g., a tester) according to actual testing needs, and the embodiments of the present disclosure are not limited thereto.

[0060] In a specific implementation, historical execution data may be obtained using a data collection tool, including but not limited to a data collector, a log collector, and a link data tracker. Furthermore, to facilitate storage of the historical execution data, the historical execution data may be pre-processed, including but not limited to data filtering, data consistency checking, and invalid and missing value processing of the historical execution data.

[0061] Since the historical execution data includes execution data when the test object is executed in different test scenes, the historical execution data may be filtered according to the current test scene, thereby obtaining historical index values ​​of multiple test indicators when the test object is executed in the test scene.

[0062] In embodiments of the present disclosure, the state (which may be referred to as the scene state) of a test object when it is tested or executed in a test scene may be characterized by index values ​​of multiple test indicators. For example, the current scene state of the test object may be characterized by current index values ​​of multiple test indicators, and the historical scene state of the test object may be characterized by historical index values ​​of multiple test indicators.

[0063] Furthermore, a relatively high similarity between the index values ​​of different test indicators indicates that different scene states are relatively similar. For example, when a test object is executed in a test scene, the history index values ​​of the multiple test indicators may include a first history index value and a second history index value, and a relatively high similarity between the first history index value and the second history index value indicates that the first history scene state corresponding to the first history index value and the second history scene state corresponding to the second history index value are relatively similar. This allows a history scene state that is relatively similar to the current scene state to be determined from the history scene state of the test object, thereby determining at least one important indicator based on the relatively similar history scene state.

[0064] Specifically, based on the current index values ​​of the multiple test indexes and the historical index values ​​of the multiple test indexes, the similarity between the current index values ​​of the multiple test indexes and the historical index values ​​of the multiple test indexes is determined, and test indexes whose similarity meets a set condition are determined as candidate indexes, and at least one important index is determined from the candidate indexes based on the tags of the candidate indexes.

[0065] In some embodiments, the similarity between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators may be characterized by a distance between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators in a vector space. Specifically, the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators may be mapped to a vector space, and the similarity may be determined based on the distance between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators in the vector space.

[0066] Distance in vector space includes, but is not limited to, cosine distance, Euclidean distance, Manhattan distance, and Minkowski distance. In the following, Euclidean distance will be taken as an example.

[0067] An e-commerce application was tested in a purchasing scenario, with the test parameters being response time, Redis capacity, and network bandwidth. The current response time parameter is 500 milliseconds, the current Redis capacity parameter is 100 gigabytes (GB), and the current network bandwidth parameter is 100 megabytes.

[0068] Because the multiple test indicators may have different dimensions, the current indicator values ​​of the multiple test indicators may be normalized to convert the current indicator values ​​of the multiple test indicators into dimensionless scalars, thereby facilitating calculations. In some embodiments, the current indicator values ​​of the multiple test indicators may be converted to numbers between 0 and 1, such that the current indicator value for response time is converted to 0.88, the current indicator value for Redis capacity is converted to 0.3, and the current indicator value for network bandwidth is converted to 0.46. Thus, the coordinates of the current indicator values ​​of the multiple test indicators mapped into the vector space may be (0.88, 0.3, 0.46).

[0069] Similarly, the coordinates of the history index values ​​of multiple test indicators in the vector space may be obtained by performing the above conversion process and mapping process on the history index values ​​of multiple test indicators. For example, the coordinates of the history index values ​​of multiple test indicators mapped in the vector space may be (0.79, 0.41, 0.4).

[0070] Based on the coordinates of the current index values ​​of the plurality of test indexes in the vector space and the coordinates of the history index values ​​of the plurality of test indexes in the vector space, the Euclidean distance between the current index values ​​of the plurality of test indexes and the history index values ​​of the plurality of test indexes in the vector space may be determined, the specific formula being as follows:

number

[0071] It should be noted that the embodiments of the present disclosure take three test indexes as an example, and in some other embodiments, the plurality of test indexes may include n test indexes, and in this case, the Euclidean distance may be determined by the following formula:

number

[0072] Based on the above steps, the Euclidean distance obtained by calculation may be used as the similarity between the current index values ​​of the multiple test indexes and the historical index values ​​of the multiple test indexes.

[0073] Then, test indicators whose similarity satisfies a set condition, for example, the similarity is greater than a set threshold, may be determined as candidate indicators. In this way, a spatial metric may be used to determine a historical scene state that is relatively similar to the current scene state based on the similarity between the current indicator values ​​of the multiple test indicators and the historical indicator values ​​of the multiple test indicators.

[0074] The tag of the candidate indicator may be used to determine at least one significant indicator from the candidate indicator. The tag of the candidate indicator is for describing the historical test result of the candidate indicator. In some embodiments, the tag of the candidate indicator may be represented by "0" or "1". When the tag of the candidate indicator is "0", it indicates that the historical test result of the candidate indicator is normal, for example, the historical index value of the candidate indicator may meet the reference range of the candidate indicator. When the tag of the candidate indicator is "1", it indicates that the historical test result of the candidate indicator is abnormal, for example, the historical index value of the candidate indicator may not meet the reference range of the candidate indicator.

[0075] Furthermore, the at least one significant indicator may be a candidate indicator whose tag is "1". In other words, the at least one significant indicator may be a test indicator whose historical test result is abnormal in a historical scene state that is relatively similar to the current scene state, so that the significant indicator represents a test indicator that should be focused on in the current scene state. In this way, efficient and directional target testing can be achieved by observing the significant indicators without needing to analyze a large number of test indicators.

[0076] S204: Obtain a test result in the test scene of the test object based on at least one key indicator.

[0077] When specifically implemented, by observing at least one important indicator, a current indicator value of the at least one important indicator can be obtained, and based on the current indicator value of the at least one important indicator and the reference value set of the at least one important indicator, a test result in the test scene to be tested can be obtained.

[0078] The set of reference values ​​of the at least one important indicator is for describing the indicator value of the at least one important indicator when the test object is in a normal state. In some possible implementations, the set of reference values ​​of the at least one important indicator may be determined based on the historical indicator value of the at least one important indicator.

[0079] Specifically, a fitting analysis may be performed on the relationship between the state of the test object and the key indicators based on the historical indicator value of at least one key indicator and the key events during the period when the test object is executed in the test scene, thereby determining a set of reference values ​​for the at least one key indicator. The key events are events that represent changes in the state of the test object, and the state of the test object may specifically include a normal state and an abnormal state.

[0080] In some embodiments, based on the historical index values ​​of at least one important index, a trend graph showing the changes in the historical index values ​​of at least one important index over time may be obtained, and then based on the time points corresponding to the important events (i.e., the time points when the test object state is changed), a set of times when the test object is in a normal state may be determined from the trend graph, and the historical index values ​​of the at least one important index corresponding to the set of times when the test object is in the normal state may be determined as a set of reference values ​​of the at least one important index.

[0081] In some other embodiments, the state of the test object may be determined based on a significant event of the test object. For example, for a significant event that changes the test object from a normal state to an abnormal state, the test object is in the normal state before the significant event and in the abnormal state after the significant event. In this way, by fitting the state of the test object to the historical indicator values ​​of at least one significant indicator, the historical indicator values ​​of the at least one significant indicator when the test object is in the normal state can be determined as the reference value set.

[0082] Furthermore, the test result may be obtained based on the current index value of at least one important index and the reference value set. In some embodiments, if the current index value of at least one important index fits the reference value set, it is determined that the test result in the test scene to be tested is normal. In other words, the test result in the test scene to be tested is normal only if the current index values ​​of all important indexes fit the reference value set, thereby realizing a target test with stricter judgment standards.

[0083] In some other embodiments, the score of at least one important indicator may be determined according to whether the current indicator value of at least one important indicator fits the reference value set, and if the sum of the scores of at least one important indicator is greater than a predetermined score, the test result of the test scene to be tested is determined to be normal. For example, if the current indicator value of an important indicator fits the reference value set, the score of the important indicator may be 1. If the current indicator value of an important indicator does not fit the reference value set, the score of the important indicator may be determined based on the current indicator value of the important indicator and the upper or lower limit of the reference value set.

[0084] Take the case where the important indicator is response time as an example. The current indicator value for response time is 70 milliseconds, and the reference value set for response time is between 10 milliseconds and 50 milliseconds. In this case, the current indicator value is greater than the upper limit of the reference value set. Therefore, the score for response time may be determined based on the current indicator value and the upper limit of the reference value set. Specifically, the ratio between the difference between the current indicator value and the upper limit of the reference value set and the upper limit of the reference value set is first determined. Then, the difference between this ratio and 1 (i.e., the score when the current indicator value for response time falls within the reference value set) may be determined as the score for response time: 1-(70-50) / 50=0.6. In this way, the score for at least one important indicator is obtained. If the sum of the scores for at least one important indicator is greater than a predetermined score, the test result for the test scene under test is determined to be normal.

[0085] In some possible implementations, a weight may be assigned to at least one important indicator, and the total score of the test scene of the test object may be determined based on a linear combination of the score and weight of the at least one important indicator. For example, if the score of a first important indicator is x1, the weight of the first important indicator is a1, the score of a second important indicator is x2, and the weight of the second important indicator is a2, then the total score may be a1x1+a2x2, and the test result is determined by comparing the total score with a predetermined score. In this way, the importance of at least one important indicator is taken into account when determining the test result, meeting the test needs of different objects.

[0086] In an embodiment of the present disclosure, a historical scene state relatively similar to a current scene state may be determined, and key indicators may be determined according to the similar historical scene state, thereby obtaining test results based on the key indicators. Therefore, when a test is performed on a test object in a test scene multiple times, for example, when a test is performed on a test object within multiple periods, the current scene state corresponding to each test may be different, and the historical scene state relatively similar to the current scene state may also be different, and the determined key indicators may also be different. In this way, the test results can be updated based on different key indicators in multiple periods, thereby accurately determining the state of the test object.

[0087] Furthermore, a change trend of the test results in the test scene of the test subject may be determined based on the test results in multiple periods of the test subject, and if the change trend of the test results satisfies the set condition, presentation information is sent.

[0088] The change trend of the test results may be determined in response to the fact that at least one important indicator matches the reference value set in multiple periods. In some embodiments, the change trend of the test results may be determined based on the change trend of the number of times at least one important indicator does not match the reference value set (which may be referred to as the number of times it does not match) in multiple periods. In other embodiments, the change trend of the test results may be determined based on the change trend of the sum of the scores of at least one important indicator in multiple periods by determining the sum of the scores of at least one important indicator. In this way, if the change trend of the test results satisfies a set condition, for example, the number of times it does not match is on an increasing trend or the sum of the scores of at least one important indicator is on a decreasing trend, presentation information may be sent, so that a user (e.g., a tester) can pay attention to the state change situation in the test scene of the test subject and immediately manually intervene.

[0089] According to the above description, an embodiment of the present disclosure provides an object testing method, which includes first obtaining a test object and a test scene, testing the test object in the test scene to obtain current index values ​​of a plurality of test indicators, then determining at least one important index from the plurality of test indicators based on the current index values ​​of the plurality of test indicators and historical index values ​​of the plurality of test indicators when the test object is executed in the test scene, and obtaining a test result of the test object in the test scene based on the at least one important index.

[0090] The method supports the test object to perform an overall test in a test scene, and determines important indicators in the test scene based on the current indicator value and historical indicator value of the test indicator. In this way, the overall test result of the test object in the test scene is obtained based on the important indicators in the test scene. Compared with the test result at a certain level of the test object, the method improves the completeness and accuracy of the object test and can meet different service needs.

[0091] The object testing method according to the embodiment of the present disclosure has been briefly described above with reference to FIGS. 1 and 2. Below, a system and electronic device according to the embodiment of the present disclosure will be described with reference to the drawings.

[0092] Referring to the structural diagram of the target test system shown in FIG. 3, the target test system may be a software system, which is located on a server and implements the target test method corresponding to the embodiment shown in FIG. 2; in some embodiments, the target test system may be a hardware system, which includes: an acquisition unit 101 for acquiring test objects and test scenes; a test unit 102 for testing the test object in the test scene to obtain current index values ​​of a plurality of test indexes; a determining unit 103 for determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene; Further, a determining unit 103 for obtaining a test result in the test scene of the test object based on the at least one important indicator.

[0093] In some possible implementations, the determining unit 103 specifically: determining a similarity between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators based on the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators; determining test indicators whose similarity satisfies a set condition as candidate indicators; determining at least one significant indicator from the candidate indicators based on the tags of the candidate indicators, the tags of the candidate indicators being used to describe historical test results of the candidate indicators;

[0094] In some possible implementations, the determining unit 103 specifically: Mapping current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes into a vector space, respectively; The similarity is determined based on a distance in the vector space between current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes.

[0095] In some possible implementations, the determining unit 103 specifically: determining a set of reference values ​​for the at least one key indicator based on historical indicator values ​​of the at least one key indicator; and obtaining a test result in the test scene of the test object based on the current index value of the at least one important index and the reference value set.

[0096] In some possible implementations, the determining unit 103 specifically: If the current indicator value of the at least one important indicator falls within the reference value set, determining that the test result in the test scene of the test object is normal; or The method determines a score for the at least one important indicator depending on whether the current indicator value of the at least one important indicator fits the reference value set, and determines that the test result in the test scene of the test subject is normal if the sum of the scores of the at least one important indicator is greater than a predetermined score.

[0097] In some possible implementations, the determining unit 103 specifically: The purpose is to determine a reference value set for the at least one important indicator by performing a fitting analysis of the relationship between the state of the test object and the important indicator based on historical indicator values ​​of the at least one important indicator and important events during the period in which the test object is executed in the test scene, wherein the important events are events that represent a change in the state of the test object.

[0098] In some possible implementations, the determining unit 103 further comprises: determining a change trend of test results in the test scene of the test object based on test results in a plurality of periods of the test object; The system further comprises: The presenting unit 104 is provided for transmitting presenting information when the change trend of the test result satisfies a set condition.

[0099] The target test system 10 according to the embodiments of the present disclosure may correspond to performing the methods described in the embodiments of the present disclosure, and the above and other operations and / or functions of each module / unit of the target test system 10 are for realizing the corresponding processes of each method in the embodiments shown in FIG. 2, and for the sake of brevity, detailed descriptions thereof will be omitted here.

[0100] An embodiment of the present disclosure further provides electronic equipment 40, specifically for implementing the functionality of target test system 10 in the embodiment shown in FIG.

[0101] 4 provides a structural schematic diagram of electronic device 40. As shown in FIG. 4, electronic device 40 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage device 408 into random access memory (RAM) 403. Various programs and data necessary for the operation of electronic device 40 are further stored in RAM 403. Processing unit 401, ROM 402, and RAM 403 are connected to each other via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0102] Typically, input devices 406, including, for example, a touch panel, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 407, including, for example, a liquid crystal display (LCD), loudspeaker, vibrator, etc.; storage devices 408, including, for example, a magnetic tape, hard disk, etc.; and communication devices 409 may be connected to the I / O interface 405. The communication devices 409 may allow the electronic device 40 to exchange data with other devices through wireless or wired communication. While FIG. 4 shows the electronic device 40 with various devices, it should be understood that not all of the illustrated devices are required to be implemented or included. More or fewer devices may alternatively be implemented or included.

[0103] An embodiment of the present disclosure further provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored in a computing device, or a data storage device, such as a data center, that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that direct a computing device to perform a target test method that is applied to the target test system 10 described above.

[0104] An embodiment of the present disclosure further provides a computer program product, which includes one or more computer instructions that, when loaded and executed on a computing device, cause all or part of the processes or functions described in the embodiment of the present disclosure to occur.

[0105] The computer instructions may be stored on a computer-readable storage medium or transmitted from one computer-readable storage medium to another; for example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital user line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) means.

[0106] When the computer program product is executed by a computer, the computer executes any one of the above page display methods. The computer program product may be a software installation package, and when it is necessary to use any one of the above page display methods, the computer program product can be downloaded and executed on the computer.

[0107] The description of the processes or structures corresponding to each of the above drawings is focused on, and the parts of a process or structure that are not detailed may refer to the relevant description of other processes or structures.

Claims

1. A subject test method comprising: acquiring a test subject and a test scene; testing the test object in the test scene to obtain current index values ​​of a plurality of test indexes; determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene; and obtaining a test result of the test object in the test scene based on the at least one important indicator.

2. determining at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene, determining a similarity between the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators based on the current index values ​​of the plurality of test indicators and the historical index values ​​of the plurality of test indicators; determining test indicators whose similarity satisfies a set condition as candidate indicators; 2. The method of claim 1, further comprising: determining the at least one significant indicator from the candidate indicators based on tags of the candidate indicators, the tags of the candidate indicators being intended to describe historical test results of the candidate indicators.

3. determining a similarity between the current index values ​​of the plurality of test indexes and the historical index values ​​of the plurality of test indexes based on the current index values ​​of the plurality of test indexes and the historical index values ​​of the plurality of test indexes, mapping current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes into a vector space; and determining the similarity based on a distance in the vector space between current index values ​​of the plurality of test indexes and historical index values ​​of the plurality of test indexes.

4. The step of obtaining a test result for the test scene of the test subject based on the at least one important indicator comprises: determining a set of reference values ​​for the at least one key indicator based on historical indicator values ​​for the at least one key indicator; and obtaining a test result for the test scene of the test subject based on the current index value of the at least one important index and the reference value set.

5. The step of obtaining a test result in the test scene of the test object based on the current index value of the at least one important index and the reference value set includes: If the current indicator value of the at least one important indicator falls within the reference value set, determining that the test result in the test scene of the test object is normal; or 5. The method of claim 4, further comprising: determining a score for the at least one important indicator depending on whether a current indicator value of the at least one important indicator fits the reference value set; and determining that a test result in the test scene of the test subject is normal if a sum of the scores of the at least one important indicator is greater than a predetermined score.

6. determining a set of reference values ​​for the at least one important indicator based on historical indicator values ​​of the at least one important indicator; 5. The method of claim 4, further comprising: determining a reference value set for the at least one important indicator by performing a fitting analysis of a relationship between the state of the test object and the important indicator based on historical indicator values ​​of the at least one important indicator and important events during a period in which the test object is executed in the test scene, wherein the important events are events representing a state change of the test object.

7. determining a change trend of test results in the test scene of the test object based on test results in a plurality of periods of the test object; The method according to any one of claims 1 to 6, further comprising the step of: transmitting presentation information when the change trend of the test results satisfies a set condition.

8. The step of testing the test object in the test scene and obtaining current index values ​​of a plurality of test indexes includes: Testing the test object within one period and continuously obtaining index values ​​of a plurality of test indexes; 8. The method according to claim 1, further comprising: determining an average value of the index values ​​of the plurality of test indexes based on the index values ​​of the plurality of test indexes and a test time; and setting the average value of the index values ​​of the plurality of test indexes as a current index value of the plurality of test indexes.

9. The step of obtaining a test result for the test scene of the test subject based on the at least one important indicator comprises: assigning weights to the at least one important indicator and determining a total score for the test scene of the test subject based on a linear combination of the scores and weights of the at least one important indicator; and determining the test result by comparing the total score with a predetermined score.

10. the step of determining a reference value set of the at least one important indicator by performing a fitting analysis of a relationship between the state of the test object and the important indicator based on a history indicator value of the at least one important indicator and important events during a period in which the test object is executed in the test scene, obtaining a trend graph of the historical index value of the at least one important index over time based on the historical index value of the at least one important index; determining a set of times from the trend graph during which the test object is in a normal state based on the time points corresponding to the significant events; and determining historical indicator values ​​of the at least one important indicator corresponding to a set of times when the test object is in a normal state as a reference value set of the at least one important indicator.

11. The step of determining a change trend of test results in the test scene of the test object based on test results in a plurality of periods of the test object includes: determining the number of deviations of the at least one key indicator from a reference set, and determining a change trend of the test results based on the change trend of the number of deviations over multiple periods; or 8. The method of claim 7, further comprising determining a sum of scores of the at least one important indicator, and determining a change trend of the test results based on a change trend of the sum of scores of the at least one important indicator over multiple periods.

12. 1. A target test system, comprising: an acquisition unit configured to acquire a test subject and a test scene; a test unit configured to test the test object in the test scene to obtain current index values ​​of a plurality of test indexes; a determining unit configured to determine at least one significant indicator from the plurality of test indicators based on current indicator values ​​of the plurality of test indicators and historical indicator values ​​of the plurality of test indicators when the test object is executed in the test scene; and a decision unit configured to obtain a test result of the test object in the test scene based on the at least one important indicator.

13. An electronic device, a processor and a memory, An electronic device, wherein the processor is configured to execute instructions stored in the memory to cause the electronic device to perform a method according to any one of claims 1 to 11.

14. 1. A computer-readable storage medium, comprising: A computer readable storage medium containing instructions for directing an electronic device to perform the method of any one of claims 1 to 11.

15. 1. A computer program product comprising: A computer program product that, when executed in an electronic device, causes the electronic device to perform the method according to any one of claims 1 to 11.

Images