Reliability test method and device, electronic equipment and storage medium

Abstract

The invention provides a reliability test method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target test environment and a target test object, matching at least one target test scene from a preset experience library, and obtaining scene configuration parameters corresponding to the target test scene, generating a test task and a corresponding expected execution result, performing initialization configuration on the target test environment, executing the test task in the target test environment after the initialization configuration, collecting execution data of the target test object to obtain a first execution result, and if the first execution result does not reach the expected execution result, executing the test task in the target test environment; the target test object is adjusted based on the first execution result, the test task is executed again, a second execution result is obtained, a reliability test conclusion is obtained according to the first execution result, the second execution result and the expected execution result, and the technical problem of high learning cost and use cost in related reliability test methods is solved through the method.

Description

Technical Field

[0001] This invention relates to the field of automated testing technology, and specifically to a reliability testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] As intelligent manufacturing practices deepen, the application of information technology in industrial enterprises has evolved from single-point and localized to global, covering a wide range of fields, from individual equipment to all elements, from single processes to the entire industrial chain, and from individual enterprises to the entire lifecycle. However, with the exponential growth in the complexity of industrial systems and the comprehensive coverage of informatization, the stability and reliability of these systems face unprecedented challenges. Therefore, conducting thorough reliability testing before system launch or iteration to verify its behavior under various stress and failure scenarios, and to identify and fix unknown potential problems in advance, has become crucial for ensuring the continuity, stability, and security of industrial production.

[0003] In related technologies, there is a general reliance on various open-source or commercial fault injection tools. While these tools are effective in specific scenarios, they are complex to operate in complex multi-environment testing scenarios. Users need to manually select and combine a large number of independent fault scenarios, resulting in high learning and usage costs. Summary of the Invention

[0004] This invention provides a reliability testing method, apparatus, electronic device, and storage medium to solve the technical problem of high learning and usage costs in related reliability testing methods.

[0005] This invention provides a reliability testing method, comprising: acquiring a target test environment and a target test object; matching at least one target test scenario from a preset experience base based on the target test environment and the target test object, and acquiring scenario configuration parameters corresponding to the target test scenario; generating a test task and a corresponding expected execution result according to the target test scenario, and initializing the target test environment based on the scenario configuration parameters and the target test object; executing the test task in the initialized target test environment and collecting execution data of the target test object to obtain a first execution result; if the first execution result does not reach the expected execution result, adjusting the target test object based on the first execution result, executing the test task again in the initialized target test environment, and collecting execution data of the target test object to obtain a second execution result; and obtaining a reliability test conclusion based on the first execution result, the second execution result, and the expected execution result.

[0006] In one embodiment of the present invention, matching at least one target test scenario from a preset experience library includes: calculating a matching degree for the test scenarios in the experience library based on the target test environment and the target test object; and selecting one or more test scenarios with the highest matching degree from the experience library as the target test scenario.

[0007] In one embodiment of the present invention, calculating the matching degree for test scenarios in the experience base includes: constructing a first feature vector based on the type of the target test environment and the type of the target test object; constructing a second feature vector based on the labels of the test scenarios in the experience base, wherein the labels are set based on the test environment and test object of the test scenario; and obtaining the matching degree by calculating the cosine similarity between the first feature vector and the second feature vector.

[0008] In one embodiment of the present invention, generating test tasks and corresponding expected execution results according to the target test scenario includes: the target test scenario includes multiple sub-scenarios, generating multiple test tasks and corresponding expected execution results according to the multiple sub-scenarios, and configuring unified shared parameters for the multiple sub-scenarios, wherein the shared parameters include at least an execution interval time or an execution waiting time.

[0009] In one embodiment of the present invention, adjusting the target test object based on the first execution result, executing the test task again in the target test environment after initialization and configuration, and collecting the execution data of the target test object to obtain the second execution result includes: generating a first-round test report containing problem location and adjustment suggestions based on the first execution result; adjusting the parameters of the target test object based on the first-round test report; and re-executing the test task in the target test environment after initialization and configuration using the target test object with adjusted parameters to obtain the second execution result.

[0010] In one embodiment of the present invention, obtaining a reliability test conclusion based on the first execution result, the second execution result, and the expected execution result includes: performing overlay analysis on the execution data of the same observation points in the first execution result, the second execution result, and the expected execution result using a time series comparison algorithm, wherein the first execution result, the second execution result, and the expected execution result include execution data of multiple observation points; obtaining a comparison curve of the first execution result, a comparison curve of the second execution result, and a comparison curve of the expected execution result based on the overlay analysis; obtaining a first reliability assessment result based on the comparison curve of the first execution result and the comparison curve of the second execution result; obtaining a second reliability assessment result based on the comparison curve of the second execution result and the comparison curve of the expected execution result; and obtaining the reliability test conclusion based on the first reliability assessment result and the second reliability assessment result.

[0011] In one embodiment of the present invention, the target test environment includes at least one of a cluster, a physical machine server, or a containerized environment, and the target test object includes at least one of a microservice, an operating system, or an application instance.

[0012] The present invention also provides a reliability testing apparatus, the apparatus comprising: an information input module, configured to acquire a target test environment and a target test object, and based on the target test environment and the target test object, match at least one target test scenario from a preset experience base, and acquire scenario configuration parameters corresponding to the target test scenario; a task generation module, configured to generate test tasks and corresponding expected execution results according to the target test scenario, and initialize the target test environment based on the scenario configuration parameters and the target test object; a first execution module, configured to execute the test task in the initialized target test environment, and collect execution data of the target test object to obtain a first execution result; a second execution module, configured to, if the first execution result does not achieve the expected execution result, adjust the target test object based on the first execution result, execute the test task again in the initialized target test environment, and collect execution data of the target test object to obtain a second execution result; and a conclusion generation module, configured to obtain a reliability test conclusion based on the first execution result, the second execution result, and the expected execution result.

[0013] The present invention also provides an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the reliability testing method as described in any of the above embodiments.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a computer's processor, causes the computer to perform any of the reliability testing methods described in the above embodiments.

[0015] The beneficial effects of this invention are as follows: This invention proposes a reliability testing method, apparatus, electronic device, and storage medium. By acquiring a target test environment and a target test object, and based on these, matching at least one target test scenario from a pre-set experience database, and acquiring scenario configuration parameters corresponding to the target test scenario, a test task and its corresponding expected execution result are generated according to the target test scenario. The target test environment is then initialized based on the scenario configuration parameters and the target test object. The test task is executed in the initialized target test environment, and execution data of the target test object is collected to obtain a first execution result. If the first execution result does not meet the expected execution result, the target test object is adjusted based on the first execution result, and the test task is executed again in the initialized target test environment, and execution data of the target test object is collected to obtain a second execution result. Based on the first execution result, the second execution result, and the expected execution result, a reliability test conclusion is obtained. This invention, with its multi-environment support, high usability, and low barrier to entry reliability testing method, solves the problems of complex reliability testing environments, difficult reliability testing parameter adaptation, cumbersome data collection, and difficult problem analysis and localization, thus improving the stability and robustness of reliability testing tools.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0018] In the attached diagram: Figure 1 A schematic diagram of the implementation environment for a reliability testing method provided in an embodiment of the present invention; Figure 2 This is a flowchart of a reliability testing method provided in one embodiment of the present invention; Figure 3 This is a block diagram of a reliability testing device provided in one embodiment of the present invention; Figure 4This is a schematic diagram of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] Please see Figure 1 , Figure 1 This is a schematic diagram of the implementation environment for a reliability testing method provided in an embodiment of the present invention.

[0023] like Figure 1 As shown, the implementation environment includes a reliability testing platform 110 and a target testing environment 120. The reliability testing platform 110 integrates one or more existing reliability testing tools, which can perform fault injection operations under different target testing scenarios. At the same time, the reliability testing platform 110 can also manage different target testing environments 120. The target testing environment 120 includes at least a cluster, physical machine servers, or a containerized environment. The physical machine servers consist of one or more independent physical servers. The clusters include, but are not limited to, Kubernetes (K8s) clusters and application clusters of various microservice architectures. The containerized environment includes environments based on container runtime technologies such as Docker. The target testing environment 120 can deploy target test objects, which include at least microservices, operating systems, or application instances.

[0024] For example, the implementation environment may also include a network, which can be a wide area network (WAN), a local area network (LAN), or other networks. Users connect to the reliability testing platform 110 via the network. Based on the different target test environments 120 input by the user, the reliability testing platform 110 recommends different target test scenarios and provides the corresponding scenario configuration parameters and observation points. The reliability testing platform 110 constructs the task scenario in the target test environment according to the scenario configuration parameters and the target test object to achieve the initial configuration of the target test environment. It then generates different test tasks based on the target test scenarios, converts the test tasks into task files such as YAML and CRD, and distributes them to the initialized target test environment. Finally, it collects data on the target test object based on the recommended observation points. The execution data of the target test object is used to obtain the first execution result. The first execution result is compared with the expected execution result. If the first execution result does not reach the expected execution result (for example, the execution data of a certain observation point is lower than the expected execution result), it means that the reliability test has failed. The target test object is adjusted and optimized according to the first execution result, and the test task is re-executed to obtain the second execution result. The first execution result, the second execution result and the expected execution result are compared to determine whether the second reliability test has passed and the effectiveness of the adjustment and optimization of the target test object. If the second execution result reaches the expected execution result, the second reliability test has passed. If the second execution result is better than the first execution result, the adjustment and optimization of the target test object is effective.

[0025] Please see Figure 2 , Figure 2 This is a flowchart illustrating a reliability testing method provided in one embodiment of the present invention. This method can be applied to... Figure 1 The implementation environment shown can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0026] like Figure 2 As shown, in an exemplary embodiment, the reliability testing method includes at least steps S210 to S250, which are described in detail below: Step S210: Obtain the target test environment and target test object. Based on the target test environment and target test object, match at least one target test scenario from the preset experience library and obtain the scenario configuration parameters corresponding to the target test scenario.

[0027] For example, the experience base contains multiple test scenarios. Based on the test environment and fault type corresponding to each test scenario, corresponding tags are constructed. Based on the degree of matching between the tags of the test scenarios in the experience base and the user's needs, at least one test scenario is recommended as the target test scenario. The scenario configuration parameters and observation points corresponding to the target test scenario are obtained. The observation points are the data types that the target test scenario focuses on.

[0028] In one embodiment of the present invention, the target test environment includes at least one of a cluster, a physical machine server, or a containerized environment, and the target test object includes at least one of a microservice, an operating system, or an application instance.

[0029] In one embodiment of the present invention, matching at least one target test scenario from a preset experience base includes: calculating the matching degree for test scenarios in the experience base based on the target test environment and the target test object; and selecting one or more test scenarios with the highest matching degree from the experience base as target test scenarios.

[0030] In one embodiment of the present invention, calculating the matching degree for test scenarios in the experience base includes: constructing a first feature vector based on the type of the target test environment and the type of the target test object; constructing a second feature vector based on the labels of the test scenarios in the experience base, wherein the labels are set based on the test environment and test object of the test scenario; and obtaining the matching degree by calculating the cosine similarity between the first feature vector and the second feature vector.

[0031] For example, let A be the second feature vector constructed based on the labels of the test scenarios in the experience base, and B be the first feature vector constructed based on the type of the target test environment and the type of the target test object required by the user. Then, the matching degree is calculated as: cos(θ) = A·B / |A||B|, where cos(θ) is the cosine similarity between the first and second feature vectors, and this value is used as the matching degree. A is the second feature vector, and B is the first feature vector. For example, if the user inputs a K8S environment as the target test environment, then the labels corresponding to the target test scenario obtained based on the highest matching degree include "Environment: K8S" and "Fault Type: Node Downtime".

[0032] Step S220: Generate test tasks and corresponding expected execution results based on the target test scenario, and initialize the target test environment based on scenario configuration parameters and target test objects.

[0033] In one embodiment of the present invention, generating test tasks and corresponding expected execution results according to a target test scenario includes: the target test scenario includes multiple sub-scenarios, generating multiple test tasks and corresponding expected execution results according to the multiple sub-scenarios, and configuring unified shared parameters for the multiple sub-scenarios, the shared parameters including at least execution interval time or execution waiting time.

[0034] For example, a target test scenario may include multiple sub-scenarios, and different sub-scenarios can generate different test tasks and different observation points. In this embodiment, multiple sub-scenarios can be configured with unified shared parameters, such as the execution interval or execution waiting time between test tasks, and the execution method of the tasks, such as manual execution, scheduled execution, or periodic execution. Observation points represent the criteria for judging the target test scenario. Different observation points collect different execution data, and the execution status of the test task can be judged based on the execution data of multiple observation points, including whether the reliability test scenario was executed successfully (whether the test function was completed), and whether the expected test results were achieved (whether the test performance reached the threshold of the expected test results).

[0035] For example, optional reliability testing scenarios include network-level fault injection (covering fault modes of network-related interfaces, links, physical connections, single network elements, time clocks, and other fault objects), system-level fault injection (covering fault modes of inter-frame interfaces, links, frames, boards, time clocks, and other fault objects within a single network element), resource-related fault injection (covering fault modes of system resource-related fault objects such as memory, CPU, and hard disk / Flash), data-related fault injection (covering fault modes of data-related fault objects such as databases and files), and hardware-related fault injection (covering fault modes of single boards, hard disks, memory, network card chips, CPUs, buses, controllers, and other fault objects in the hardware platform).

[0036] Step S230: Execute the test task in the target test environment after initialization configuration, and collect the execution data of the target test object to obtain the first execution result.

[0037] For example, after completing the first reliability test, the first execution result is obtained based on the execution data corresponding to the observation point of the target test scenario. The first execution result includes functional test results and performance test results. The functional test results are used to verify whether all the functions required by the test task have been completed, and the performance test results are used to verify whether the data threshold of the expected execution result has been reached. For example, in the reliability test of a microservice in a test scenario of node failure in a Kubernetes cluster, the restart time of the microservice after node failure is used as the observation point. After executing the test task, the collected execution data includes that the restart time of the microservice after node failure is 60 seconds. Then the functional test result of the first execution result is passed (the microservice has achieved restart after node failure), and the performance test result of the first execution result is failed (the restart time in the expected execution result is less than 10 seconds). Since the restart time is too long, the microservice needs to be optimized and adjusted before a second reliability test is performed.

[0038] Step S240: If the first execution result does not achieve the expected execution result, the target test object is adjusted based on the first execution result, and the test task is executed again in the target test environment after initialization configuration, and the execution data of the target test object is collected to obtain the second execution result.

[0039] For example, if the first execution result achieves the expected execution result, that is, the functional test result in the first execution result is passed (all functional requirements of the test task are completed) and the performance test result in the first execution result is passed (the performance threshold in the expected execution result is reached), then the reliability test in the target test scenario is passed, and a second reliability test is not required.

[0040] In one embodiment of the present invention, adjusting the target test object based on the first execution result, executing the test task again in the target test environment after initialization and configuration, and collecting the execution data of the target test object to obtain the second execution result includes: generating a first-round test report containing problem location and adjustment suggestions based on the first execution result; adjusting the parameters of the target test object based on the first-round test report; and re-executing the test task in the target test environment after initialization and configuration using the target test object with adjusted parameters to obtain the second execution result.

[0041] For example, taking a reliability test of a microservice in a Kubernetes cluster node failure test scenario, the execution data of the first execution result includes that the microservice restarted after the node failure in 60 seconds, which did not meet the expected restart time (10 seconds) specified in the execution result. Based on the slow restart time of the microservice, the fault was located and the problem was identified as the service health check parameters of the microservice. It was recommended to adjust the service health check parameters of the microservice, and a first round test report was generated based on the problem location and adjustment recommendations. Based on the first round test report, the service health check parameters of the microservice were adjusted, and the test task was executed again based on the adjusted microservice to obtain the second execution result. The execution data of the second execution result includes that the microservice restarted after the node failure in 5 seconds, and no new faults were caused.

[0042] Step S250: Based on the first execution result, the second execution result, and the expected execution result, obtain the reliability test conclusion.

[0043] In one embodiment of the present invention, obtaining a reliability test conclusion based on a first execution result, a second execution result, and an expected execution result includes: performing overlay analysis on the execution data of the same observation points in the first execution result, the second execution result, and the expected execution result using a time series comparison algorithm, wherein the first execution result, the second execution result, and the expected execution result include execution data of multiple observation points; obtaining a comparison curve of the first execution result, a comparison curve of the second execution result, and a comparison curve of the expected execution result based on the overlay analysis; obtaining a first reliability assessment result based on the comparison curve of the first execution result and the comparison curve of the second execution result; obtaining a second reliability assessment result based on the comparison curve of the second execution result and the comparison curve of the expected execution result; and obtaining a reliability test conclusion based on the first reliability assessment result and the second reliability assessment result.

[0044] For example, taking the reliability test of a microservice in a Kubernetes cluster node failure test scenario as an example, the execution data of one observation point in the first execution result includes that the restart time of the microservice after the node failure is 60 seconds. The execution data of the same observation point in the second execution result includes that the restart time of the microservice after the node failure is 5 seconds. The execution data of the same observation point in the expected execution result includes that the restart time of the microservice after the node failure is less than 10 seconds. The second reliability test is passed by comparing the second execution result and the expected execution result. The optimization and adjustment of the microservice is effective by comparing the first execution result and the second execution result.

[0045] For example, taking the reliability test of a microservice in a Kubernetes cluster node failure test scenario as an example, the execution data of another observation point in the first execution result includes the request latency of the microservice at multiple collection times before and after the node failure. The execution data of the same observation point in the second execution result includes the request latency of the microservice at multiple collection times before and after the node failure. The execution data of the same observation point in the expected execution result includes the maximum threshold of the request latency of the microservice before and after the node failure (if the maximum threshold is exceeded, the reliability test fails). The request latency in the three execution results is superimposed and analyzed by the time series comparison algorithm to obtain the comparison curve of the first execution result, the comparison curve of the second execution result, and the comparison curve of the expected execution result. If the comparison curve of the first execution result is above the comparison curve of the second execution result at each time point, the first reliability assessment result is that the optimization and adjustment of the microservice is effective. If the comparison curve of the second execution result is below the comparison curve of the expected execution result at each time point, the second reliability assessment result is that the reliability test passes. The reliability test conclusion is that after effective optimization and adjustment of the parameters of the microservice, the reliability test under the target test scenario (the reliability test of a microservice in a Kubernetes cluster node failure test scenario) has been passed.

[0046] For example, after completing a reliability test, users can update the experience base based on the optimized target test scenario, or add their own modified reliability test scenarios to the experience base at any time, thereby achieving dynamic updates to the experience base.

[0047] In embodiments of the present invention, by constructing a unified testing platform that integrates multiple target testing environments, users do not need to learn, deploy, and maintain multiple independent testing tools for different environments, which simplifies the operation process and significantly reduces the time and manpower costs of environment adaptation.

[0048] In the embodiments of the present invention, the intelligent matching recommendation of target test scenarios based on the experience base effectively solves the problem that users are at a loss and have complicated configurations in a large number of fault scenarios, enabling even non-expert users to quickly and accurately carry out high-level reliability testing.

[0049] In the embodiments of the present invention, by comparing and analyzing the first execution result and the second execution result, the actual effect of the optimization measures can be displayed intuitively and quantitatively, the performance improvement and potential regression problems can be accurately identified, a data-driven closed-loop verification process can be constructed, and the measurable and verifiable reliability optimization can be realized.

[0050] In embodiments of the present invention, the experience base is updated based on the test scenarios, parameter configurations, test results and analysis reports of successful tests, which promotes the accumulation and reuse of excellent test scenarios and builds a continuously evolving test knowledge system.

[0051] Please see Figure 3 , Figure 3 This is a block diagram of a reliability testing apparatus provided in one embodiment of the present invention. This apparatus can be applied to... Figure 1 The implementation environment shown can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0052] like Figure 3 As shown, the exemplary reliability testing apparatus includes: The information input module 310 is used to obtain the target test environment and the target test object, and based on the target test environment and the target test object, to match at least one target test scenario from the preset experience library and obtain the scenario configuration parameters corresponding to the target test scenario. The task generation module 320 is used to generate test tasks and corresponding expected execution results based on the target test scenario, and to initialize the target test environment based on the scenario configuration parameters and the target test object. The first execution module 330 is used to execute test tasks in the target test environment after initialization and configuration, and to collect the execution data of the target test object to obtain the first execution result; The second execution module 340 is used to adjust the target test object based on the first execution result if the first execution result does not achieve the expected execution result, and then execute the test task again in the target test environment after initialization and configuration, and collect the execution data of the target test object to obtain the second execution result; The conclusion generation module 350 is used to obtain the reliability test conclusion based on the first execution result, the second execution result, and the expected execution result.

[0053] The information input module 310 matches at least one target test scenario from a preset experience library, including: calculating the matching degree for the test scenarios in the experience library based on the target test environment and the target test object; and selecting one or more test scenarios with the highest matching degree from the experience library as target test scenarios.

[0054] The information input module 310 calculates the matching degree for the test scenarios in the experience base by: constructing a first feature vector based on the type of the target test environment and the type of the target test object; constructing a second feature vector based on the labels of the test scenarios in the experience base, wherein the labels are set based on the test environment and test object of the test scenario; and obtaining the matching degree by calculating the cosine similarity between the first feature vector and the second feature vector.

[0055] The task generation module 320 generates test tasks and corresponding expected execution results based on the target test scenario, including: the target test scenario includes multiple sub-scenarios, multiple test tasks and corresponding expected execution results are generated based on the multiple sub-scenarios, and unified shared parameters are configured for the multiple sub-scenarios. The shared parameters include at least the execution interval time or the execution waiting time.

[0056] The second execution module 340 adjusts the target test object based on the first execution result, executes the test task again in the target test environment after initialization, and collects the execution data of the target test object to obtain the second execution result, including: generating a first-round test report containing problem location and adjustment suggestions based on the first execution result; adjusting the parameters of the target test object based on the first-round test report; and re-executing the test task in the target test environment after initialization using the target test object with adjusted parameters to obtain the second execution result.

[0057] The conclusion generation module 350 obtains the following reliability test conclusions based on the first execution result, the second execution result, and the expected execution result: It performs overlay analysis on the execution data of the same observation points in the first execution result, the second execution result, and the expected execution result using a time series comparison algorithm; the first execution result, the second execution result, and the expected execution result include execution data from multiple observation points; based on the overlay analysis, it obtains the comparison curves of the first execution result, the second execution result, and the expected execution result; based on the comparison curves of the first execution result and the second execution result, it obtains the first reliability assessment result; based on the comparison curves of the second execution result and the expected execution result, it obtains the second reliability assessment result; and based on the first reliability assessment result and the second reliability assessment result, it obtains the reliability test conclusions.

[0058] The aforementioned device enables reliability testing in multiple environments with high ease of use and low barriers to entry. It solves the problems of complex reliability testing environments, difficulty in adapting reliability testing parameters, cumbersome data acquisition, and difficulty in analyzing and locating problems, thereby improving the stability and robustness of reliability testing tools.

[0059] It is understood that the reliability testing device provided in the above embodiments and the reliability testing method provided in the above embodiments belong to the same concept. The specific way in which the reliability testing method is executed has been described in detail in the above embodiments and will not be repeated here. In practical applications, the reliability testing device provided in the above embodiments can be assigned to different functional modules as needed. That is, the internal structure of the reliability testing device can be divided into different functional modules, and then all or part of the functions of the corresponding functional modules can be implemented by the reliability testing method described in the above embodiments. No specific limitations are imposed here. For example, the information input module 310 includes steps for executing step S210 and related steps, the task generation module 320 includes steps for executing step S220 and related steps, the first execution module 330 includes steps for executing step S230 and related steps, the second execution module 340 includes steps for executing step S240 and related steps, and the conclusion generation module 350 includes steps for executing step S250 and related steps.

[0060] Figure 4 This is a schematic diagram of an electronic device provided in one embodiment of the present invention. It should be noted that... Figure 4 The computer system 400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0061] like Figure 4As shown, the computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 402 or programs loaded from storage portion 408 into Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 403. The CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. An Input / Output (I / O) interface 405 is also connected to bus 404.

[0062] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0063] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of the present invention.

[0064] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0065] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0066] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0067] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer's processor, causes the computer to perform the reliability testing method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0068] Another aspect of the present invention provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the reliability testing methods provided in the various embodiments described above.

[0069] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A reliability testing method, characterized in that, The method includes: Obtain the target test environment and the target test object; based on the target test environment and the target test object, match at least one target test scenario from a preset experience base, and obtain the scenario configuration parameters corresponding to the target test scenario; Test tasks and corresponding expected execution results are generated based on the target test scenario, and the target test environment is initialized and configured based on the scenario configuration parameters and the target test object; The test task is executed in the target test environment after initialization configuration, and the execution data of the target test object is collected to obtain the first execution result; If the first execution result does not achieve the expected execution result, the target test object is adjusted based on the first execution result, and the test task is executed again in the target test environment after initialization configuration. The execution data of the target test object is collected to obtain the second execution result. Based on the first execution result, the second execution result, and the expected execution result, a reliability test conclusion is obtained.

2. The reliability testing method according to claim 1, characterized in that, At least one target test scenario is matched from the preset experience base, including: Based on the target testing environment and the target testing object, calculate the matching degree for the test scenarios in the experience base; One or more test scenarios with the highest matching degree are selected from the experience base as the target test scenarios.

3. The reliability testing method according to claim 2, characterized in that, Calculating the matching degree for the test scenarios in the experience base includes: A first feature vector is constructed based on the type of the target test environment and the type of the target test object; a second feature vector is constructed based on the labels of the test scenarios in the experience base, wherein the labels are set based on the test environment and the test object of the test scenario. The matching degree is obtained by calculating the cosine similarity between the first feature vector and the second feature vector.

4. The reliability testing method according to any one of claims 1-3, characterized in that, The test tasks and corresponding expected execution results generated based on the target test scenario include: The target test scenario includes multiple sub-scenarios. Multiple test tasks and corresponding expected execution results are generated based on the multiple sub-scenarios. Unified shared parameters are configured for the multiple sub-scenarios. The shared parameters include at least the execution interval time or the execution waiting time.

5. The reliability testing method according to any one of claims 1-3, characterized in that, Based on the first execution result, the target test object is adjusted, and the test task is executed again in the target test environment after initialization. The execution data of the target test object is collected to obtain the second execution result, which includes: Based on the first execution result, a first-round test report containing problem identification and adjustment suggestions is generated; Based on the first round of test reports, the parameters of the target test object are adjusted; Using the target test object with adjusted parameters, the test task is re-executed in the target test environment after initialization configuration to obtain the second execution result.

6. The reliability testing method according to any one of claims 1-3, characterized in that, Based on the first execution result, the second execution result, and the expected execution result, the reliability test conclusions include: The execution data of the same observation point in the first execution result, the second execution result, and the expected execution result are overlaid and analyzed based on the time series comparison algorithm. The first execution result, the second execution result, and the expected execution result include execution data of multiple observation points. Based on the superposition analysis, a comparison curve of the first execution result, a comparison curve of the second execution result, and a comparison curve of the expected execution result are obtained; The first reliability assessment result is obtained based on the comparison curve of the first execution result and the comparison curve of the second execution result; The second reliability assessment result is obtained based on the comparison curve of the second execution result and the comparison curve of the expected execution result; The reliability test conclusion is derived based on the first reliability assessment result and the second reliability assessment result.

7. The reliability testing method according to any one of claims 1-3, characterized in that, The target test environment includes at least one of a cluster, a physical machine server, or a containerized environment, and the target test object includes at least one of a microservice, an operating system, or an application instance.

8. A reliability testing device, characterized in that, The device includes: The information input module is used to obtain the target test environment and the target test object, and based on the target test environment and the target test object, to match at least one target test scenario from a preset experience base, and to obtain the scenario configuration parameters corresponding to the target test scenario; The task generation module is used to generate test tasks and corresponding expected execution results according to the target test scenario, and to initialize the target test environment based on the scenario configuration parameters and the target test object; The first execution module is used to execute the test task in the target test environment after initialization and configuration, and to collect the execution data of the target test object to obtain the first execution result; The second execution module is used to adjust the target test object based on the first execution result if the first execution result does not achieve the expected execution result, and then execute the test task again in the target test environment after initialization and configuration, and collect the execution data of the target test object to obtain the second execution result; The conclusion generation module is used to obtain a reliability test conclusion based on the first execution result, the second execution result, and the expected execution result.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the reliability testing method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the reliability testing method as described in any one of claims 1-7.

Images