Software application fault testing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a fault test method and device of a software application, an electronic device and a storage medium. After obtaining a feature portrait label of a running environment of a software application to be tested, a test configuration matched with the running environment is determined based on the feature portrait label, a fault is generated through the test configuration and injected into the running environment to perform fault test on the software application. The fault test on the software application is realized by generating differentiated faults for terminal devices providing different running environments. The fault test is more flexible and targeted, can reflect the performance of the software application on faults in the real running environment, and can configure test resources based on different running environments, thereby improving the accuracy of the software application fault test and the utilization rate of test resources. The potential fault risk of the software application when running on different terminal devices can be found in advance through the fault test, which provides effective and reliable improvement basis for software application development and operation, and reduces the fault rate of the software application.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a method, apparatus, electronic device, and storage medium for fault testing of software applications. Background Technology

[0002] In the fields of software development and operations, fault injection testing is often used to ensure the stability and reliability of software applications. A common practice in this field is to configure a uniform fault scenario (e.g., fixed network latency, fixed server error rate) in the test environment and apply this uniform fault to all terminal devices or operating environments under test to observe the overall performance of the software application.

[0003] However, due to significant differences in hardware performance, operating systems, and network conditions among terminal devices in actual deployment environments, the aforementioned testing method using a uniform fault scenario has obvious limitations. On one hand, it cannot differentiate test resource configurations based on the actual fault intensity that different types of terminal devices can tolerate. This can lead to testing being overly lenient on high-performance devices and overly stringent on low-performance devices, resulting in inefficient use of test resources. On the other hand, this "one-size-fits-all" testing strategy struggles to effectively simulate and cover the real-world impact of faults in edge scenarios such as low-performance devices, outdated systems, or weak networks. This means that potential stability risks in these scenarios cannot be detected and fixed in advance during the testing phase, potentially leading to online failures and unnecessarily consuming additional server computing resources, network bandwidth, and maintenance manpower for fault location and recovery. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for fault testing of software applications, which generates different faults based on differences in terminal devices, enabling differentiated fault testing of software applications on different terminal devices and improving the accuracy of software application evaluation.

[0005] Firstly, this disclosure provides a method for fault testing of a software application, including: Obtain the feature profile tags of the runtime environment of the software application to be tested; Based on the feature profile tags, a test configuration adapted to the operating environment is determined; A fault is generated based on the test configuration, and the fault is injected into the runtime environment to test the software application.

[0006] Secondly, this disclosure provides a fault testing apparatus for software applications, comprising: The feature profile tag acquisition module is used to acquire feature profile tags of the runtime environment of the software application under test. The test configuration determination module is used to determine a test configuration that is compatible with the operating environment based on the feature profile labels; The fault generation and injection module is used to generate faults according to the test configuration and inject the faults into the operating environment to test the software application.

[0007] Thirdly, this disclosure provides an electronic device, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the fault testing method for the software application described in the first aspect of this disclosure.

[0008] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the fault testing method for the software application described in the first aspect of this disclosure.

[0009] This embodiment of the disclosure obtains the feature profile tags of the operating environment of the software application to be tested, determines the test configuration adapted to the operating environment based on the feature profile tags, and further generates faults through the test configuration and injects them into the operating environment to test the software application. This realizes the generation of differentiated faults for terminal devices providing different operating environments to perform fault testing on the software application. It can provide faults in the real operating environment for software application fault testing. Compared with using the same faults to perform fault testing on software applications on different terminal devices, on the one hand, fault testing is more flexible and targeted. It can reflect the performance of the software application in the real operating environment, improve the accuracy of software application fault testing, and configure test resources according to different operating environments, thereby improving the utilization rate of test resources. On the other hand, fault testing can discover potential fault risks of software applications running on different terminal devices in advance, providing an effective and reliable basis for improvement in software development and operation and maintenance, and reducing the failure rate of software applications.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a fault testing method for a software application provided in an embodiment of this disclosure; Figure 2 This is a flowchart of a fault testing method for a software application provided in another embodiment of this disclosure; Figure 3 This is a flowchart illustrating the generation of the image library in this embodiment of the disclosure; Figure 4 This is a schematic diagram illustrating the coverage of statistical devices in an embodiment of this disclosure; Figure 5 This is a schematic diagram illustrating the determination of target feature profile tags for fault testing in an embodiment of this disclosure; Figure 6 This is a schematic diagram illustrating an example of the target feature profile tags selected in an embodiment of this disclosure; Figure 7 This is a schematic diagram illustrating the determination of fault levels in an embodiment of this disclosure; Figure 8 This is a schematic diagram illustrating fault generation and injection in an embodiment of this disclosure; Figure 9 This is a schematic diagram of the server gateway responding to a service request in an embodiment of this disclosure; Figure 10 This is a schematic diagram of fault level detection in an embodiment of this disclosure; Figure 11 This is a schematic diagram of the structure of a fault testing device for a software application provided in an embodiment of this disclosure; Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this disclosure. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0014] Mobile software applications are installed and run on terminal devices such as mobile phones and tablets. When evaluating the stability and compatibility of software applications, fault testing is required. Fault testing refers to injecting faults into the operating environment of the software application to test whether the operating state of the software application is abnormal. For example, testing whether the software application crashes, freezes, or times out after the fault is injected.

[0015] Traditional fault testing uses a uniform fault to test software applications on all terminal devices. Taking a high-concurrency scenario of a major product promotion as an example, to verify the interface response time of the software application during the promotion, the response time of all interfaces is forced to be controlled within 200ms. Based on this, faults with response times greater than 200ms are generated to test the software application. However, different terminal devices have huge differences in hardware performance, system version, and network environment. For example, a 500ms response delay on a low-end terminal device may make the software application completely unusable, while on a high-end terminal device, the software application may only show slight lag without affecting the user experience. Using the same fault to test software applications on different terminal devices makes it difficult to detect fault risks in advance when the software application is running on low-end devices, old systems, weak network environments, etc., leading to frequent online failures of the software application.

[0016] Based on this, embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for testing software applications, which can generate different faults according to the differences in terminal devices, realize differentiated fault testing of software applications on different terminal devices, improve the accuracy of software application evaluation, discover fault risks of software applications running on different terminal devices in advance, and reduce the failure rate of software applications. The following describes the software application fault testing method of this disclosure in conjunction with embodiments.

[0017] Figure 1 This is a flowchart illustrating a software application fault testing method according to an embodiment of this disclosure. This embodiment is applicable to situations involving fault testing of software applications. The method can be executed by a software application fault testing device, which can be implemented in hardware and / or software and configured in an electronic device. Figure 1 As shown, the fault testing methods for this software application include: S101. Obtain the feature profile tags of the operating environment of the software application to be tested.

[0018] The software application in this embodiment is an application to be tested for faults. This software application can be installed on various types and configurations of terminal devices, including mobile phones, tablets, desktop computers, laptops, and other devices. After installing the software application on the terminal device, the user can launch the software application, causing the terminal device to load the software application.

[0019] The runtime environment can refer to the hardware environment, software environment, and external resource environment in which the software application runs. The hardware environment can include the hardware resources of the terminal device on which the software application is installed and loaded. The software environment can include the terminal device's operating system, dependent libraries, etc. The external resource environment can be the network environment connected to the terminal device, etc. Feature profile tags can be information used to describe the comprehensive characteristics of the runtime environment. For example, feature profile tags can describe the comprehensive characteristics of the terminal device on which the software application is loaded and the network it is connected to. For instance, feature profile tags can describe the terminal device's hardware configuration, network environment, operating system, brand, software application information, etc. During fault testing, the feature profile tags of the software application's runtime environment can be obtained each time a business request from the software application is received.

[0020] In one embodiment, the operating environment is provided by a terminal device loaded with the software application to be tested. During the fault test, each time a service request from the software application is received, basic information such as the hardware configuration, network environment, operating system, brand, and software application version of the terminal device loaded with the software application can be obtained. Based on the above basic information, the characteristic profile label of the operating environment can be determined. In another embodiment, before the fault test begins, the characteristic profile label of each terminal device loaded with the software application can be determined in advance. When a service request from the software application is received, the characteristic profile label of the terminal device can be found based on the terminal device identifier in the service request.

[0021] S102. Test configuration based on feature profile labels to determine compatibility with the operating environment.

[0022] In this embodiment, the test configuration can be information describing the faults required for the test. This configuration may include fault types, the objects affected by the faults, strength parameters, triggering and matching rules, etc. Fault types can be broadly categorized as server-side faults and application-side faults. Server-side faults can be further categorized as interface return errors (500 / timeout / empty response), response time latency (fixed / random / progressive), service circuit breaking / rate limiting, etc. Application-side faults can be further categorized as weak network simulation (latency / packet loss / bandwidth limitation), CPU / memory resource limitation, application freeze / crash simulation, etc. Strength parameters for network-related faults may include latency, packet loss rate, bandwidth upper and lower limits, etc., while strength parameters for resource-related faults may include CPU utilization thresholds, available memory ratio upper and lower limits, etc. The objects affected by the faults can include both the server and the application. Server-side faults can affect gateways, interface paths, service modules, etc., while application-side faults can affect application processes, middleware, pages, components, etc. Triggering and matching rules can include matching terminal devices participating in the test based on feature profile tags, and can also trigger faults based on business requests, scenarios, and timed intervals. In one embodiment, the mapping relationship between profiles and test configurations can be pre-configured. After determining the feature profile labels of the runtime environment, the test configuration mapped to the feature profile labels can be determined based on the mapping relationship.

[0023] In another alternative embodiment, the tolerance score of the feature profile label can be calculated based on the pre-configured profile scoring rules, the fault level can be determined based on the tolerance score, and the test configuration pre-set for the fault level can be obtained.

[0024] S103. Generate faults based on the test configuration and inject them into the runtime environment to test the software application.

[0025] After determining the test configuration for the software application in different operating environments, corresponding faults can be generated using these test configurations. Faults can then be injected into the operating environment during the response to the software application's business requests to perform fault testing on the software application. In the process of fault testing, performance metrics of the software application can be collected, and the fault test results can be determined through these performance metrics to generate a fault test report.

[0026] In the fault testing process of this disclosure, the test configuration is determined based on the feature profile tags of the software application's operating environment. Faults are then generated through the test configuration and injected into the operating environment to perform fault testing on the software application. This achieves fault testing of the software application by generating different faults for terminal devices providing different operating environments. Compared to using the same fault to test software applications on different terminal devices, this method is more flexible and targeted, generating different faults based on the differences in terminal devices. It reflects the software application's performance in real operating environments, improving the accuracy of software application fault testing. Furthermore, test resources can be configured according to different operating environments, increasing the utilization rate of test resources. On the other hand, fault testing can identify potential fault risks of the software application running on different terminal devices in advance, providing effective and reliable improvement basis for software development and maintenance, and reducing the failure rate of the software application.

[0027] Figure 2 A flowchart illustrating a fault testing method for a software application provided in another embodiment of this disclosure is shown below. Figure 2 As shown, the software application fault testing method includes: S201. Generate a feature profile library, which includes multiple feature profile tags and the device coverage of each feature profile tag.

[0028] In this embodiment, the feature profile tag can be information used to describe the comprehensive characteristics of the operating environment, and the feature profile library can be an information library including multiple feature profile tags and the device coverage of each feature profile tag. In one embodiment, the feature profile library can be generated through the following steps: S2011. When a preset event is detected, acquire multi-dimensional device characteristics of multiple terminal devices.

[0029] In this embodiment, the device profile library is not fixed and can be updated by a preset event. The preset event can be a device profile library update command triggered by the tester, a preset time point set by the tester, or new device data reaching a threshold. This embodiment does not limit the preset event.

[0030] In this embodiment, the operating environment can be provided by the terminal device that loads the software application and the network environment it is connected to. When generating the feature profile library, the feature profile library can be established based on the terminal device. Specifically, when a preset event is detected, the device profile building engine can be triggered to collect device information of a preset number (e.g., 10,000) of terminal devices. Multi-dimensional device features of the terminal device are extracted through the device information. The device information can include hardware configuration information, operating system information, network environment information, and application information. The hardware configuration information can include the terminal device's CPU model, number of CPU cores, clock speed, memory size, storage capacity, screen resolution, and other hardware information. The operating system information can include the operating system type, version number, API level, system patch version, and other information. The network environment information can include network type (WiFi / 4G / 5G), signal strength, network latency, bandwidth, and other information. The application information can include the software application version, installation time, usage frequency, and other information.

[0031] After obtaining the device information of the terminal device, feature engineering can be performed on the device information to obtain multi-dimensional device features of the terminal device. For example, numerical information can be standardized, and non-numerical information can be quantified to characterize the device information. Multi-dimensional device features include features in at least two of the following dimensions: Hardware configuration features, including at least one of processor information, memory size, and storage capacity; System information characteristics, including at least one of the following: operating system type, operating system version number, and system patch version; Network environment characteristics, including at least one of network type, signal strength, and network latency; Application information characteristics, including at least one of application version and frequency of use.

[0032] S2012. Cluster multiple terminal devices based on multi-dimensional device features to obtain clustering results including multiple clustering categories.

[0033] After obtaining the multi-dimensional device features of terminal devices, clustering algorithms (such as K-means, DBSCAN, fuzzy clustering, etc.) can be used to cluster the multi-dimensional device features of multiple terminal devices to obtain clustering results. Each cluster category in the clustering results includes multiple multi-dimensional device features. By automatically grouping massive, multi-dimensional device features through clustering algorithms, the shortcomings of manual classification rules being singular and subjective can be overcome. It can automatically discover naturally formed feature combination patterns in the device group, thereby making the generated feature profile labels more realistic and objective in reflecting the actual distribution of the terminal device group, improving the accuracy and representativeness of the profile.

[0034] S2013. Generate feature profile labels for each cluster category based on the clustering results.

[0035] Each cluster category in the clustering results includes multiple multi-dimensional device features that are similar in distance, indicating that the category includes multiple terminal devices with similar device features. The multiple multi-dimensional device features of each category can be abstracted to obtain the feature profile label of each cluster category. For example, the terminal device production time, chip model, operating system version, terminal device model, terminal device brand, network environment, etc., which are represented by device features, can be abstracted to obtain the feature profile label.

[0036] Figure 3 This is a flowchart of the process for generating the image library in this embodiment, see reference. Figure 3 After standardizing the features of device information from multiple terminal devices, cluster analysis is performed. Based on the cluster analysis results, device profile 1 (low-end Android - weak network), device profile 2 (high-end iOS - 5G), device profile 3 (mid-range iOS - 4G), and device profile 4 (old system - WiFi) are generated. Among them, low-end, terminal, and high-end are relative terms. For example, terminal devices before 2020, chip models A1-A3, and ROM versions below 10 are defined as low-end; terminal devices from 2020 to 2024, chip models A4-A6, and ROM versions 11-12 are mid-range; and terminal devices from 2024 to the present, the most recent two years, chip models A7-A9, and ROM versions 12-16 are high-end.

[0037] The feature profile labels in the above-mentioned feature profile library are merely examples. Those skilled in the art can also set more fine-grained clustering to generate a feature profile library that includes more feature profile labels.

[0038] Furthermore, after generating the feature profile library, the device coverage of each feature profile tag can be calculated. This device coverage can be the proportion of terminal devices for each feature profile tag. Specifically, the number of terminal devices belonging to each type of feature profile tag can be counted, and the ratio of this number to the total number of terminal devices is the device coverage of the feature profile tag. Figure 4 The diagram shown illustrates the coverage of statistical equipment. (For reference) Figure 4 First, count the number of devices in each device profile, and then calculate the coverage percentage to obtain the device coverage of each device profile.

[0039] In one optional embodiment of this disclosure, after generating the feature profile library, operations such as adding, deleting, updating device coverage, and rolling back the version can be performed on the feature profile library. Specifically, when a preset maintenance event for the feature profile library is detected, the feature profile library is updated. The preset maintenance event may include at least one of the following: changes in business scenarios, abnormal data sampling, or the number of new terminal devices loading the software application to be tested reaching a preset number. This disclosed embodiment, through dynamic maintenance of the device profile library, addresses two main issues. First, it avoids the problem of a static feature profile library rapidly deviating from the real online environment due to continuous iteration of terminal devices, the constant release of new models / system versions, and dynamic fluctuations in the network environment. This prevents it from covering newly added devices and new environmental scenarios, ensuring that the feature profile library automatically incorporates new devices and discards old ones, thus guaranteeing long-term consistency between the profile library and the online terminal environment and providing a reliable basis for differentiated testing. Second, it avoids the problem of a static feature profile library failing to adapt to changes in user device structure and usage behavior due to business peaks, promotional events, and time-of-day differences, leading to a disconnect between the feature profile library and real needs. This embodiment achieves automatic updates to profiles as business and user structure change, ensuring that the feature profile library always matches the current real operating environment and preventing a disconnect between testing and online operations.

[0040] Changes in business scenarios can refer to changes in the business requirements of the software application. For example, during the annual report period, the frequency of user logins increases, with more inactive users logging in. The profiles collected during the annual report period differ from those collected on weekdays. Alternatively, the software application's business may differ at different times of day, such as morning, noon, evening, and early morning, resulting in different profiles. Data sampling anomalies can refer to the discovery of anomalies in the data sampled from terminal devices each time. For example, some terminal device models may be missed, or a preset proportion of data anomalies may occur (such as the proportion of terminal devices with incorrect CPU models reaching a preset proportion). In such cases, the feature profile library can be rolled back to the previous version. New terminal devices can refer to new models of terminal devices that have loaded the software application (such as a new mobile phone model that has been launched and has the software application to be tested installed). The number of new terminal devices that have loaded the software application can be counted in real time.

[0041] When a preset maintenance event is detected, the feature profile library can be updated by adding, deleting, updating device coverage, and rolling back versions. For example, the same terminal device may use different networks at different times (e.g., using 5G network outdoors and WiFi network indoors). Collecting device information from the terminal device at different times to generate the profile library will cause changes in the feature profile labels. New models of terminal devices may also generate new feature profile labels after they are put on the market. The feature profile labels in the feature profile library can be dynamically added or deleted. Alternatively, if some terminal devices of certain models are missed during the sampling of terminal devices, resulting in data anomalies, the feature profile library can be rolled back to the previous version and reused.

[0042] This embodiment constructs feature profile tags through multi-dimensional device characteristics of terminal devices, thereby improving the accuracy of feature profile tags by leveraging the operating environment provided by real devices. It can comprehensively cover edge scenarios that traditional verification schemes struggle to cover, such as low-end devices, outdated systems, and weak network environments. By analyzing the coverage of each tag in the feature profile library, the group of terminal devices participating in fault testing can be selected. This ensures that fault testing resources are concentrated on the device profile group with the highest user coverage. Compared to the existing technology of simply grouping terminal devices for fault testing, this approach ensures that feature profile tags accurately reflect the real operating environment of the software application and maximizes the representativeness of the risks discovered during fault testing to the overall user experience, even with limited resources. This achieves intelligent optimization and allocation of testing resources.

[0043] S202. Based on equipment coverage, determine at least one feature profile label that participates in the fault test from the feature profile library as the target feature profile label.

[0044] In one optional embodiment, the feature profile tags in the feature profile library can be sorted in descending order according to device coverage. The device coverage of each feature profile tag is then accumulated in descending order to obtain a cumulative coverage. When the cumulative coverage is greater than or equal to a preset coverage threshold, the feature profile tags with accumulated device coverage are determined as target feature profile tags for participating in fault testing. This coverage-priority-based selection strategy ensures that fault testing resources (such as computing resources and time costs) are concentrated on the device profile group with the highest user coverage, following the "Pareto Principle" (80 / 20 rule). This maximizes the representativeness of the risks discovered during fault testing on the overall user experience within limited resources, achieving intelligent optimization and allocation of testing resources.

[0045] Figure 5 This is a schematic diagram illustrating the identification of target feature profile labels for participating in fault testing. (Refer to...) Figure 5The feature profile tags are sorted in descending order according to device coverage, and then the device coverage is accumulated. When accumulating the device coverage for each device, it is determined whether the accumulated coverage is greater than or equal to the coverage threshold (e.g., 80%). If not, the device coverage of the next device profile is accumulated. If so, the feature profile tags with the highest device coverage of 80% are selected.

[0046] The coverage threshold can be flexibly determined in various ways according to business needs, test resources, and test objectives to adapt to different test scenarios and improve the rationality and relevance of fault testing. For example, a fixed percentage can be preset as the coverage threshold, such as 80%, 85%, or 90%, with 80% being the most commonly used (in accordance with the Pareto principle). Of course, the coverage threshold can also be determined based on the application scenario. For example, during major promotions, events, or the release of new versions of software applications, the coverage threshold can be increased to 90%~95%, while the coverage threshold for routine inspections and minor software application updates can be reduced to 70%~80%. This embodiment does not restrict the method of determining the coverage threshold.

[0047] Figure 6 This is an example of the final selected target feature profile labels. With an 80% threshold set, profile 1, profile 2, and profile 3 are the top 3 profiles in terms of device coverage, and the cumulative device coverage is equal to 80%. Profile 1, profile 2, and profile 3 are used as target feature profile labels for fault verification testing.

[0048] This embodiment uses a priority sorting algorithm for feature profile tags based on device coverage, and further filters out target feature profile tags with high coverage based on coverage thresholds. It can filter the top 80% of high-coverage feature profile tag groups based on the 80 / 20 rule, ensuring that fault testing resources are focused on the most widely used terminal devices, realizing intelligent allocation of fault testing resources, and maximizing the value of fault testing with limited resources.

[0049] S203. Obtain the feature profile tags of the operating environment of the software application to be tested.

[0050] In this embodiment, the operating environment can be the environment provided by the terminal device loaded with the software application to be tested and the network it is connected to. After the fault test begins, when a service request for the software application to be tested is received from each terminal device, the hardware configuration, network environment, operating system, brand, software application version and other configuration information of the terminal device can be determined from the service request. Based on the configuration information, the feature profile label of the operating environment of the software application can be determined. In another embodiment, before the fault test begins, the feature profile label of each terminal device loaded with the software application can also be determined in advance. When a service request is received, the feature profile label of the corresponding terminal device can be found based on the terminal device identifier in the service request.

[0051] S204. Determine whether the feature profile label is the target feature profile label.

[0052] In practical applications, not every software application in every operating environment participates in fault testing. The group of terminal devices participating in fault testing can be determined by setting target feature profile labels. It is determined whether the feature profile label of each operating environment is any one of the multiple target feature profile labels. If so, S205 is executed. If not, the business requests of the software application under test are responded to under normal circumstances, such as forwarding the business requests to the server under normal circumstances.

[0053] It should be noted that S204 is not a necessary step. When it is not necessary to determine whether the feature profile label is the target feature profile label, it means that all terminal devices participate in the fault test, which is suitable for scenarios of comprehensive fault testing.

[0054] S205. Determine the terminal equipment that provides the operating environment as the equipment participating in the fault test, and determine the fault level based on the feature profile label.

[0055] If it is determined that the feature profile label of the software application to be tested is the target feature profile label, then it can be determined that the terminal device providing the operating environment is the device participating in the fault test, and the fault level of the terminal device can be determined. In one embodiment, the tolerance score of the feature profile label can be calculated based on the feature profile label and the preset profile-scoring rules, and the target score range in which the tolerance score is located can be further determined. The fault level can be determined based on the target score range.

[0056] For example, the feature profile labels include three dimensions: device level, network, and available memory. The specific scoring rules are as follows: Equipment level: Low-end equipment is 5 points, mid-range equipment is 15 points, and high-end equipment is 20 points; Network: 3G is 5 points, 4G is 15 points, 5G & WIFI is 30 points; Available memory: 20% is worth 5 points, 50% is worth 15 points, and 60% is worth 30 points.

[0057] The above scoring rules can be used to calculate the resilience score of each feature profile label. This resilience score reflects the ability of the operating environment with each feature profile label to withstand failures. The higher the score, the greater the resilience.

[0058] like Figure 7 The diagram shows a fault level determination based on the tolerance score. A score less than 20 indicates a low-intensity fault, 20 to 80 indicates a medium-intensity fault, and 80 to 100 indicates a high-intensity fault. The fault level can be determined based on the range of the tolerance score. Of course, those skilled in the art can classify more fault levels than just the three levels of low, medium, and high intensity mentioned above.

[0059] S206. Obtain a test configuration that matches the fault level.

[0060] In this embodiment, testers can pre-determine the corresponding test configurations for different fault levels. For example, the test configuration for low-intensity faults is a delay of 200ms and a packet loss of 3%, the configuration for medium-intensity faults is a delay of 400ms and a packet loss of 6%, and the test configuration for high-intensity faults is a delay of 600ms and a packet loss of 10%. After determining the fault level, the corresponding test configuration can be generated.

[0061] S207. Generate the current fault based on the test configuration.

[0062] In this embodiment, the test configuration may include server-side fault configuration and application-side fault configuration. The server-side fault configuration is used to configure server-side faults, and the application-side fault configuration is used to configure application-side faults. Server-side faults and application-side faults can be generated based on the server-side fault configuration and application-side fault configuration in the test configuration, respectively.

[0063] S208. Inject the current fault into the runtime environment to perform fault testing on the software application and collect the performance indicators of the software application in the runtime environment.

[0064] In this embodiment, server-side faults can be injected into the server-side gateway connected to the terminal device, and application-side faults can be injected into the application-side middleware. Server-side faults can be return-related faults or response time-related faults, while application-side faults can be simulated by the application-side middleware, such as weak network conditions, CPU performance issues, or available memory problems. For example, the application-side middleware can be implemented using an SDK or AAR integration method. The application-side middleware includes a network proxy module, a resource limiting module, a configuration management module, and a monitoring and reporting module. The network proxy module uses an HTTP interceptor to implement latency injection, packet loss simulation, and bandwidth limiting. The resource limiting module simulates CPU / memory pressure through thread pool control and memory allocation. The configuration management module supports JSON configuration and dynamic updates. The monitoring and reporting module is used to collect performance indicators and crash data in real time. The application-side middleware is integrated into the software application in the form of an AAR / SDK, and application-side fault simulation is achieved through configuration-driven implementation, without modifying the business code of the software application.

[0065] Server-side failures can be addressed through the gateway layer. The server-side gateway intercepts business requests from software applications on the terminal devices participating in the failure test based on device profile matching rules and applies the following failure logic: Interface return type failure: directly return the specified error code (such as 500, timeout); RT-related faults: Inject a delay before the response; The server gateway supports dynamic configuration of fault rules, matching based on device profiles, API paths, and other conditions to achieve differentiated fault injection without modifying the backend service code.

[0066] Furthermore, the test configuration can also include only server-side fault configuration or only application-side fault configuration, thereby generating a single type of fault. As an alternative implementation, server-side faults can be injected only into the server-side gateway without enabling fault simulation in the application-side middleware to verify the impact of server-side anomalies on the software application; or, faults can be injected only into the application-side middleware (such as limiting CPU usage to 90%) without triggering server-side faults to verify the stability of the software application under local faults.

[0067] like Figure 8 The diagram shown illustrates fault generation and injection. (Refer to...) Figure 8The device profile for participating in the fault test is determined from the feature profile library: low-end Android. After analyzing the profile features, the fault configuration generator generates server-side fault configurations and application-side fault configurations. The generated server-side fault configuration is: interface returns a 500 error, RT latency is 300ms, and the generated application-side fault configuration is: weak network latency is 200ms, CPU is limited to 70%. The fault configuration generator sends the server-side fault configuration to the server-side gateway and the application-side fault configuration to the application-side middleware. The server-side gateway applies the server-side fault configuration and sends a configuration success message to the fault configuration generator after successful configuration. The application-side middleware configures the application-side fault (weak network latency 200ms, CPU limited to 70%) to simulate the fault. After successful configuration, it sends a configuration success message to the fault configuration generator. The fault configuration generator confirms that the collaborative fault injection has been successfully started.

[0068] like Figure 9 The diagram illustrates how a server-side gateway responds to a business request. After a software application sends a business request to the server-side gateway, the gateway matches the characteristic profile label of the operating environment provided by the terminal device with the target characteristic profile label. If they do not match, it is determined that the terminal device is not a device that needs to participate in the fault test, and the business request is forwarded to the backend service normally. If they match, it is determined that the terminal device is a device that needs to participate in the fault test, and the fault rules are applied. For interface return type faults, a 500 error, timeout error, or empty response can be returned. For RT time type faults, a fixed delay, random delay, or progressive delay can be executed, and the business request is forwarded to the backend service after the delay.

[0069] This embodiment injects faults collaboratively through the server-side gateway and application-side middleware, which can realistically reproduce the superposition effect of server-side faults and application-side faults, and can discover cross-terminal issues that cannot be found by single fault verification, providing a more comprehensive and accurate assessment of stability and compatibility risks.

[0070] S209. Determine whether to adjust the fault level of the current fault based on performance indicators.

[0071] In this embodiment, after a fault is injected, performance indicators of the software application on the terminal device are collected, and a first number of first terminal devices with the same characteristic profile tags participating in the fault test are counted. When the first number reaches a preset number, a second number of second terminal devices that failed the fault test are determined from the first terminal devices based on the performance indicators. The ratio of the second number to the first number is calculated to obtain the proportion of devices that failed the fault test. It is determined whether the proportion of devices is greater than or equal to a preset proportion threshold. If so, if the fault level of the current fault is not the lowest fault level, it is determined that the fault level of the current fault needs to be reduced, and S210 can be executed. If the fault level is already the lowest fault level, it is determined that the fault level does not need to be adjusted, and the fault test is determined to have failed, and S211 can be executed. If not, if the fault level of the current fault is not the highest fault level, it is determined that the fault level of the current fault needs to be increased, and S210 can be executed. If the fault level is already the highest level, it is determined that the fault level does not need to be adjusted, and the fault test is determined to have passed, and S211 can be executed.

[0072] The preset quantity can be the minimum number of terminal devices participating in the test under the same feature profile label, used to ensure the reliability of statistical results and avoid misjudgment due to small samples. In one embodiment, the preset quantity can be a fixed value, such as 500, 1000, or 2000 units. In another embodiment, the preset quantity can be calculated based on device coverage, for example, preset quantity = total number of devices × device coverage of feature profile label × preset ratio. In yet another embodiment, the preset quantity can be set according to the business scenario. For example, the preset quantity can be set higher (e.g., 2000 units) during major promotions or version releases, and lower (e.g., 500 units) during routine inspections. This embodiment does not restrict the setting method of the preset quantity.

[0073] The preset percentage threshold refers to the upper limit of the percentage of devices allowed to fail a fault test under the same feature profile label. When the percentage of devices failing a fault test is greater than or equal to the preset percentage threshold, the current fault level is determined to be too high and the level needs to be lowered; conversely, the level can be increased. Its function is to quantify the fault tolerance boundary of the device group and ensure that the fault intensity matches the device's capacity. The preset percentage threshold can be a fixed percentage, such as 50%, 60%, 70%, or 80%, with a general stability standard of 70%. In another example, the preset percentage threshold can also be determined according to the feature profile label. For example, a lower percentage threshold (e.g., 40%~50%) can be set for high-end devices / high-performance profiles, while a higher percentage threshold (e.g., 70%~85%) can be set for low-end devices / weak-performance profiles. This makes high-performance devices more stable and allows for a lower failure rate, while low-performance devices are allowed a higher failure rate. This makes the fault level determination more in line with the actual capacity of the device and avoids a one-size-fits-all approach. Of course, those skilled in the art can also set the preset percentage threshold according to the business scenario of the fault test. This embodiment does not limit the setting method of the preset percentage threshold.

[0074] Performance metrics can include whether the software application crashes, freezes, or the page response time. A failure test is determined if the software application crashes, freezes, or the page response time times out. For the same feature profile label participating in the failure test, taking feature profile label A as an example, a preset percentage threshold of 70% (configurable) is set. This means that software applications on 70% of terminal devices with feature profile label A are allowed to fail during the failure test. Once a preset number of terminal devices with feature profile label A participating in the failure test (e.g., 500, 1000, etc., configurable to avoid insufficient sample size) is reached, [further steps can be taken]. The ratio of the number of terminal devices that failed the fault test to a preset number is used to obtain the proportion of devices that failed the fault test. If the proportion of devices is greater than or equal to a preset proportion threshold, and if the current fault level is not the lowest level, then the current fault level is determined to be too high, and the fault level can be reduced to detect the highest fault level that the terminal device with feature profile label A can withstand. If the proportion of devices is less than a preset proportion threshold, and if the current fault level is not the highest level, then the current fault level is determined to be too low, and the fault level can be increased to detect the highest fault level that the terminal device with feature profile label A can withstand.

[0075] This disclosure introduces a comparison mechanism between the proportion of devices based on statistical samples and a preset proportion threshold, and dynamically adjusts the fault level accordingly, achieving adaptive and refined detection of the fault tolerance capacity of each device profile group. This method can automatically find the highest fault intensity (i.e., the "collapse threshold") that the device profile can withstand within an acceptable failure rate range, thus providing accurate quantitative data on the stability boundary of software applications on different devices. It overcomes the shortcomings of traditional fixed-level testing, which cannot obtain extreme performance data, and provides a direct and reliable decision-making basis for product compatibility optimization and resource allocation. In addition, by using the proportion of devices that failed fault tests under the same feature profile label as the basis for fault level adjustment, compared to using the abnormal performance of a single device as the basis for fault level adjustment, it achieves a scientific assessment of the fault tolerance capacity of the device group, fundamentally avoiding misjudgment of fault level caused by occasional factors such as hardware anomalies of individual devices, system lag, and instantaneous network fluctuations.

[0076] In another embodiment, the performance metrics include at least one of the following: application unresponsiveness metric, application crash metric, and average page response time metric. When any performance metric meets a corresponding preset anomaly condition, it is determined that the fault level needs to be lowered; when none of the performance metrics meet the corresponding preset anomaly conditions, it is determined that the fault level needs to be raised. Here, an anomaly condition refers to the condition corresponding to the performance metric that triggers an anomaly. For example, the anomaly condition corresponding to the average page response time metric is that the page response time is greater than a preset time (e.g., 3 seconds); the anomaly condition for the application unresponsiveness metric is that the application is unresponsive; and the anomaly condition for the application crash metric is that the application crashes. On the one hand, using at least one of the application unresponsiveness, application crash, and average page response time metrics simultaneously for judgment is more efficient than... A single indicator provides a more comprehensive view, covering different types of fault risks and improving the completeness of the assessment. On the other hand, if any indicator is abnormal, the fault level is reduced, which can quickly identify severe risk scenarios and avoid repeated application crashes, freezes, or prolonged unresponsiveness due to excessive fault intensity. This protects the testing process from disruption and ensures the stability and sustainable execution of the testing process. Only when all indicators are normal can the fault level be increased, ensuring that the application truly performs stably under the current intensity. This effectively avoids misjudgments caused by blindly increasing the fault intensity due to normal local indicators, allowing the fault level to gradually approach the true tolerance limit of the equipment group. It can automatically and accurately detect the maximum fault level that the characteristic profile group can stably withstand, providing the test report with a true, reliable, and reproducible fault limit value.

[0077] S210. Adjust the fault level of the feature profile label.

[0078] When it is determined that the fault level of the feature profile label needs to be adjusted, the fault level can be gradually reduced or increased, and return to S206 to generate a fault for the current feature profile label in order to continue fault testing on the terminal device of the current feature profile label.

[0079] Figure 10 The diagram shown is a schematic of fault level detection. (Reference) Figure 10 After obtaining feature profile labels and determining the initial fault level based on the feature profile labels, the fault is injected to perform the test. The collected indicators are monitored in real time, and the status is judged through multi-dimensional indicators, including crash judgment, crash judgment, and response time detection. If the status is abnormal, the fault level is reduced and the fault is re-injected for fault testing. If the indicators are normal, the fault level is increased and the fault is re-injected for fault testing. In this way, the limit of the highest fault level that the feature profile labels can withstand is continuously detected and determined.

[0080] In this embodiment of the invention, the initial fault level is determined after predicting the fault tolerance score through feature profile tags, and an initial fault injection is generated based on the initial fault level for testing. On the one hand, it enables the prediction of the fault tolerance of the terminal device for each feature profile tag before testing. Compared with starting testing from the lowest, highest, or random fault level, it can reduce the number of fault tests for each feature profile tag and improve the efficiency of fault testing. On the other hand, by determining the test results through performance indicators and comparing the proportion of test failures with a preset proportion threshold, it can determine whether to increase or decrease the fault level. This can detect the fault tolerance limit of the terminal device for each feature profile tag and provide a basis for fault analysis reports and design improvements.

[0081] S211. A fault test report for the software application based on performance metrics and current faults.

[0082] If it is determined that no adjustment to the fault level is needed, and the test is completed, a fault test report can be generated using performance indicators and current fault information. Specifically, the performance indicators and the percentage of devices that failed the fault test can be statistically analyzed for each fault level. The fault level at which the percentage of devices that failed the fault test reaches a preset threshold can be obtained as the fault limit value. A test report can be generated using the performance indicators for each fault level test, the percentage of devices that failed the fault test, and the fault limit value.

[0083] For example, a fault test report may include fault level, feature image label, number of devices participating in the fault test, percentage of devices that failed the fault test, various performance indicators, and maximum tolerable fault level, etc. It may also include anomaly detection analysis, such as identifying unreasonable improvement points. For example, the anomaly analysis results may include at least one of the following: low fault tolerance of high-performance devices, weak tolerance of low-performance devices, incompatibility between different operating systems, and incompatibility between different versions of software applications. After generating the fault test report, the fault test report can be visualized.

[0084] In one embodiment, the present disclosure can also pre-train a fault test analysis model. Specifically, real test reports and real improvement suggestions for various software applications can be obtained as training data. The fault test analysis model is trained using the training data to predict improvement suggestions through testing. For example, N test reports are randomly selected and input into the fault test analysis model to obtain N predicted improvement suggestions. A loss value (such as the mean squared error loss value) is calculated using the predicted and real improvement suggestions. If the loss value is less than or equal to a preset value, the training of the fault test analysis model is stopped. If the loss value is greater than the preset value, gradient descent is performed on the model parameters of the fault analysis model based on the loss value, and the fault analysis model is trained again. This enables the fault test analysis model to learn the ability to provide improvement suggestions based on fault test reports. After generating a fault test report, it is input into the fault test analysis model to obtain the improvement suggestions for the current fault test of the software application. The test report and improvement suggestions are displayed, providing developers with directions for improving the software application and improving the intelligence of software application fault testing.

[0085] In this embodiment, after generating a feature profile library, the target feature profile tags for fault testing are determined from the feature profile library based on the device coverage of the feature profile tags. When the feature profile tag of the software application's operating environment is the target feature profile tag, the fault level is determined based on the feature profile tag. A test configuration is generated based on the fault level, and the current fault is generated and injected for fault testing based on the test configuration. Based on the performance indicators collected during the test, it is determined whether the fault level of the current fault needs to be adjusted. If so, the fault level of the feature profile tag is adjusted and the fault test continues. If not, a fault test report is generated after the test. This realizes the generation of different faults for terminal devices in different operating environments to perform fault testing on the software application. Compared with using the same fault to perform fault testing on software applications on different terminal devices, different faults can be generated based on the differences of terminal devices. Fault testing is more flexible and targeted, and can reflect the performance of the software application in the real operating environment. This improves the accuracy of software application fault testing. Fault testing can detect the fault risks of the software application running on different terminal devices in advance, reducing the failure rate of the software application.

[0086] Furthermore, by leveraging the characteristics of real devices, highly accurate feature profile tags can be constructed. The feature profile library can be updated through preset maintenance events, enabling continuous updates of feature profile tags. It automatically adapts to dynamic changes in terminal devices and automatically identifies new device types to update the feature profile library. This comprehensively covers edge scenarios that traditional testing solutions struggle to cover, such as low-end devices, outdated systems, and weak network environments. It provides a precise software application operating environment for differentiated fault testing. Through fault testing, various fault risks in software applications can be identified in advance during the development phase, effectively reducing online failures and improving the stability of software applications.

[0087] Furthermore, the generation of feature profile library, fault generation and injection, and fault test report generation can all be completed without human intervention, reducing errors that are prone to occur due to human intervention, improving the consistency and reliability of fault testing, and allowing multiple feature profile tags to participate in fault testing, thereby improving fault testing efficiency and shortening the software application development cycle.

[0088] Figure 11 This is a schematic diagram of the structure of a software application fault testing device provided in an embodiment of this disclosure. Figure 11 As shown, the fault testing device for this software application includes: The feature profile tag acquisition module 1101 is used to acquire feature profile tags of the running environment of the software application to be tested. The test configuration determination module 1102 is used to determine a test configuration that is compatible with the operating environment based on the feature profile tags; The fault generation and injection module 1103 is used to generate faults according to the test configuration and inject the faults into the operating environment to test the software application.

[0089] Figure 12 A schematic diagram of the structure of an electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, mainframe computers, smartphones, tablet computers, game consoles, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0090] like Figure 12 As shown, the electronic device 1200 includes at least one processor 1201 and a memory, such as a read-only memory (ROM) 1202 or a random access memory (RAM) 1203, communicatively connected to the at least one processor 1201. The memory stores computer programs executable by the at least one processor. The processor 1201 can perform various appropriate actions and processes based on the computer program stored in the ROM 1202 or loaded into the RAM 1203 from storage unit 1208. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The processor 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0091] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, touch screen, game controller, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0092] Processor 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 1201 performs the various methods and processes described above, such as fault testing methods for software applications.

[0093] The fault testing methods for software applications include: Obtain the feature profile tags of the runtime environment of the software application to be tested; Based on the feature profile tags, a test configuration adapted to the operating environment is determined; A fault is generated based on the test configuration, and the fault is injected into the runtime environment to test the software application.

[0094] Optionally, before obtaining the feature profile tags of the runtime environment of the software application under test, the following steps are also included: Generate a feature profile library, which includes multiple feature profile tags and the device coverage of each feature profile tag; Based on the device coverage, at least one feature profile label participating in the fault test is determined from the feature profile library as the target feature profile label.

[0095] Optionally, after generating the feature profile library, the following may also be included: When a preset maintenance event is detected for the feature profile database, an update operation is performed on the feature profile database. The preset maintenance events include at least one of the following: changes in business scenarios, abnormal data sampling, and the number of new terminal devices loading the software application to be tested reaching a preset number. The update operation includes at least one of the following operations: adding, deleting, or modifying feature profile tags, and / or rolling back the feature profile library to a historical version.

[0096] Optionally, a feature profile library is generated, including: When a preset event is detected, multi-dimensional device characteristics of multiple terminal devices are acquired, wherein the terminal devices are devices that provide the runtime environment for the software application to be tested; Based on the multi-dimensional device features, clustering is performed on multiple terminal devices to obtain clustering results including multiple clustering categories; Based on the clustering results, feature profile labels are generated for each cluster category, and feature profile libraries are generated using feature profile labels from multiple cluster categories.

[0097] Optionally, the multi-dimensional device features include features in at least two of the following dimensions: Hardware configuration features, including at least one of processor information, memory size, and storage capacity; System information characteristics, including at least one of the following: operating system type, operating system version number, and system patch version; Network environment characteristics, including at least one of network type, signal strength, and network latency; Application information characteristics, including at least one of application version and frequency of use.

[0098] Optionally, based on the device coverage, at least one feature profile label participating in the fault test is determined from the feature profile library as the target feature profile label, including: Sort the feature profile tags in the feature profile library in descending order according to the device coverage; The cumulative coverage is obtained by accumulating the device coverage of the feature profile tags in descending order; When the cumulative coverage is greater than or equal to a preset coverage threshold, the feature profile label of the accumulated device coverage is determined as the target feature profile label for participating in the fault test.

[0099] Optionally, determining a test configuration that matches the operating environment based on the feature profile labels includes: The fault level is determined based on the aforementioned feature profile labels; Obtain a test configuration that matches the fault level.

[0100] Before determining the fault level based on the feature profile labels, the process also includes: Determine whether the feature profile label is the target feature profile label; If so, the terminal device providing the operating environment is identified as the device participating in the fault test.

[0101] Optionally, determining the fault level based on the feature profile labels includes: The tolerance score of the feature profile label is calculated based on the feature profile label and the preset profile-scoring rules; Determine the target score range in which the tolerance score falls, and determine the fault level based on the target score range.

[0102] Optionally, generating faults based on the test configuration and injecting the faults into the runtime environment to perform fault testing on the software application includes: The current fault is generated based on the test configuration; The current fault is injected into the operating environment to perform fault testing on the software application, and the performance indicators of the software application in the operating environment are collected. Based on the performance indicators, determine whether to adjust the fault level of the current fault; If so, adjust the fault level of the feature profile label and return to the step of generating a test configuration based on the fault level; If not, generate a test report for the software application based on the performance metrics and the current fault.

[0103] Optionally, the performance metrics include at least one of the following: application unresponsiveness metrics, application crash metrics, and average page response time metrics; Determining whether to adjust the fault level of the current fault based on the performance indicators includes: When any of the performance indicators meets the corresponding preset abnormal condition, it is determined that the fault level needs to be reduced. When none of the performance indicators meet the corresponding preset abnormal conditions, it is determined that the fault level needs to be increased.

[0104] Optionally, the current fault is generated based on the test configuration, including: Based on the server-side fault configuration and application-side fault configuration in the test configuration, server-side faults and application-side faults are generated respectively. Injecting the current fault into the runtime environment to perform fault testing on the software application includes: The server-side fault is injected into the server-side gateway connected to the terminal device participating in the fault test, and the application-side fault is injected into the application-side middleware.

[0105] Optionally, determining whether to adjust the fault level of the current fault based on the performance indicators includes: The first number of terminal devices in the operating environment that participated in the fault test and provided the same characteristic profile tags; When the first quantity reaches a preset quantity, a second quantity of second terminal devices that have failed the fault test is determined from the first terminal devices based on the performance indicators; The percentage of devices that failed the fault test is obtained by calculating the ratio of the second quantity to the first quantity. Determine whether the proportion of the device is greater than or equal to a preset proportion threshold; If so, when the fault level of the current fault is not the lowest fault level, it is determined that the fault level of the current fault needs to be reduced. If not, if the fault level of the current fault is not the highest fault level, it is determined that the fault level of the current fault needs to be increased.

[0106] A test report for the software application is generated based on the performance metrics and the current fault, including: Statistics were compiled on the performance indicators and the percentage of devices that failed the tests for each fault level. The fault level when the percentage of devices that fail the fault test reaches a preset threshold is used as the fault limit value. Test reports are generated using performance indicators for each fault level, the percentage of devices that failed the fault test, and the fault limit values.

[0107] Optionally, after generating a test report for the software application based on the performance metrics and the current fault, the report further includes: The test report is input into the fault test analysis model to obtain improvement suggestions for the software application. Display the test report and the improvement suggestions.

[0108] In some embodiments, the software application fault testing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by processor 1201, one or more steps of the software application fault testing method described above may be performed. Alternatively, in other embodiments, processor 1201 may be configured by any other suitable means (e.g., by means of firmware) to perform the software application fault testing method of the embodiments of this disclosure.

[0109] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] Computer programs used to implement the methods of this disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this disclosure, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0114] A computing system can include software applications and servers. Software applications and servers are generally geographically separated and typically interact via communication networks. The relationship between a software application and a server is created by computer programs running on the respective computers and having a software application-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0115] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for fault testing of a software application, characterized in that, include: Obtain the feature profile tags of the runtime environment of the software application to be tested; Based on the feature profile tags, a test configuration adapted to the operating environment is determined; A fault is generated based on the test configuration, and the fault is injected into the runtime environment to test the software application.

2. The software application fault testing method according to claim 1, characterized in that, Before obtaining the feature profile tags of the runtime environment of the software application under test, the following steps are also included: Generate a feature profile library, which includes multiple feature profile tags and the device coverage of each feature profile tag; Based on the device coverage, at least one feature profile label participating in the fault test is determined from the feature profile library as the target feature profile label.

3. The software application fault testing method according to claim 2, characterized in that, Determining a test configuration that matches the operating environment based on the feature profile tags includes: The fault level is determined based on the aforementioned feature profile labels; Obtain a test configuration that matches the fault level.

4. The software application fault testing method according to claim 3, characterized in that, Determining the fault level based on the aforementioned feature profile labels includes: The tolerance score of the feature profile label is calculated based on the feature profile label and the preset profile-scoring rules; Determine the target score range in which the tolerance score falls, and determine the fault level based on the target score range.

5. The software application fault testing method according to claim 3, characterized in that, Generating faults based on the test configuration and injecting these faults into the runtime environment to perform fault testing on the software application includes: The current fault is generated based on the test configuration; The current fault is injected into the operating environment to perform fault testing on the software application, and the performance indicators of the software application in the operating environment are collected. Based on the performance indicators, determine whether to adjust the fault level of the current fault; If so, adjust the fault level of the feature profile label and return to the step of generating a test configuration based on the fault level; If not, generate a test report for the software application based on the performance metrics and the current fault.

6. The software application fault testing method according to claim 5, characterized in that, The current fault is generated based on the test configuration, including: Based on the server-side fault configuration and application-side fault configuration in the test configuration, server-side faults and application-side faults are generated respectively. Injecting the current fault into the runtime environment to perform fault testing on the software application includes: The server-side fault is injected into the server-side gateway connected to the terminal device participating in the fault test, and the application-side fault is injected into the application-side middleware.

7. The software application fault testing method according to claim 5, characterized in that, Determining whether to adjust the fault level of the current fault based on the performance indicators includes: The first number of terminal devices in the operating environment that participated in the fault test and provided the same characteristic profile tags; When the first quantity reaches a preset quantity, a second quantity of second terminal devices that have failed the fault test is determined from the first terminal devices based on the performance indicators; The percentage of devices that failed the fault test is obtained by calculating the ratio of the second quantity to the first quantity. Determine whether the proportion of the device is greater than or equal to a preset proportion threshold; If so, when the fault level of the current fault is not the lowest fault level, it is determined that the fault level of the current fault needs to be reduced. If not, if the fault level of the current fault is not the highest fault level, it is determined that the fault level of the current fault needs to be increased.

8. A fault testing device for a software application, characterized in that, include: The feature profile tag acquisition module is used to acquire feature profile tags of the runtime environment of the software application under test. The test configuration determination module is used to determine a test configuration that is compatible with the operating environment based on the feature profile labels; The fault generation and injection module is used to generate faults according to the test configuration and inject the faults into the operating environment to test the software application.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the fault testing method of the software application according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the fault testing method of the software application according to any one of claims 1-7.

Images