Method and apparatus for failure early warning for distributed systems

Abstract

The present disclosure provides a fault early warning method and device for a distributed system. The fault early warning method can include the following steps: obtaining a test task to be executed on the distributed system, wherein the test task includes a target fault type, and the target fault type is used for the distributed system to determine a target test plan from a plurality of test plans, wherein the plurality of test plans are changed by adding or replacing another test plan; obtaining an execution result of the distributed system executing the test task including the target test plan; and outputting data related to the execution result, wherein the data related to the execution result is used for fault analysis of the distributed system.

Description

Technical Field

[0001] This disclosure relates to the field of system testing technology, and in particular to a fault early warning method and fault early warning device for distributed systems such as attribution systems. Background Technology

[0002] As system architecture evolves from monolithic applications to distributed systems, development efficiency and system scalability gradually improve. However, system complexity also increases, and traditional service testing methods can no longer fully cover all possible system behaviors. With the continuous development of microservices and the increasing scale of systems, the dependencies between services bring many uncertainties. In such a complex call network, an anomaly in any link can potentially affect other services. Furthermore, the increased number of service nodes also increases the probability and randomness of failures. Therefore, improving the stability and high availability of distributed systems has become an urgent problem to solve. Summary of the Invention

[0003] This disclosure provides a fault early warning method and a fault early warning device for distributed systems, to at least solve the problems mentioned above.

[0004] According to a first aspect of the present disclosure, a fault early warning method for a distributed system is provided, which may include: acquiring a test task to be executed on the distributed system, wherein the test task includes a target fault type, the target fault type being used by the distributed system to determine a target test plan from a plurality of test plans, wherein the plurality of test plans are modified by adding or replacing other test plans; acquiring an execution result of the distributed system executing the test task including the target test plan; and outputting the execution result, wherein the data related to the execution result is used for fault analysis of the distributed system.

[0005] Optionally, obtaining the execution result of the distributed system executing the test task including the target test plan may include: recording the failure execution result when the distributed system encounters an error while executing the test task including the target test plan; and recording the test data corresponding to the execution of the test task when the distributed system does not encounter an error while executing the test task including the target test plan and the test task has been completed.

[0006] Optionally, outputting data related to the execution result may include: comparing and verifying the test data with the runtime data generated by the distributed system executing the test task that does not include the target test plan; and outputting the data matching accuracy based on the comparison and verification result.

[0007] Optionally, the method may further include: when the amount of data in the execution result does not meet the predetermined amount of data, determining to execute the test task including the target test plan again.

[0008] Optionally, the method may further include: when an error occurs in the distributed system executing the test task including the target test plan, determining whether to send a fault alarm message based on the importance of the error.

[0009] Optionally, the distributed system can be an attribution system.

[0010] Optionally, the test plan may be code used to inject faults into the normal program of the distributed system.

[0011] According to a second aspect of the present disclosure, a fault early warning device for a distributed system is provided, which may include: an acquisition module configured to acquire a test task to be executed on the distributed system, wherein the test task includes a target fault type, the target fault type being used by the distributed system to determine a target test plan from a plurality of test plans, wherein the plurality of test plans are modified by adding or replacing other test plans; a processing module configured to acquire the execution result of the distributed system executing the test task including the target test plan; and an output module configured to output data related to the execution result, wherein the data related to the execution result is used for fault analysis of the distributed system.

[0012] Optionally, the processing module can be configured to: record the failure execution result when the distributed system encounters an error while executing the test task including the target test plan; and record the test data corresponding to the execution of the test task after the distributed system executes the test task including the target test plan without encountering an error and after the test task is completed.

[0013] Optionally, the processing module may be configured to compare and verify the test data with the runtime data generated by the distributed system executing the test task that does not include the target test plan, in order to obtain the data matching accuracy.

[0014] Optionally, the processing module can be configured to: when the amount of data in the execution result does not meet the predetermined amount of data, determine to execute the test task including the target test plan again.

[0015] Optionally, the processing module can be configured to: when the distributed system encounters an error while executing the test task including the target test plan, determine whether to send a fault alarm message based on the importance of the error.

[0016] Optionally, the distributed system can be an attribution system.

[0017] Optionally, the test plan may be code used to inject faults into the normal program of the distributed system.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device may include: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the fault warning method as described above.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores instructions which, when executed by at least one processor, cause the at least one processor to perform the fault warning method as described above.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, wherein instructions in the computer program product are executed by at least one processor in an electronic device to perform the fault warning method as described above.

[0021] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0022] This disclosure modifies the test tasks of a distributed system by automatically adding or replacing test plans, making the test tasks uncertain. It enables the automatic execution of different forms of fault detection on the distributed system, making the testing process more in line with the actual production environment and better able to discover more unknown problems in the distributed system, thereby improving the overall stability of the distributed system.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0025] Figure 1 This is a flowchart of a fault warning method according to an embodiment of the present disclosure;

[0026] Figure 2 This is a schematic flowchart of a fault warning method executed through a fault warning platform according to an embodiment of the present disclosure;

[0027] Figure 3 This is a block diagram of a fault warning device according to an embodiment of the present disclosure;

[0028] Figure 4 This is a schematic diagram of the structure of a fault early warning device according to an embodiment of the present disclosure;

[0029] Figure 5 This is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0030] Throughout the accompanying drawings, it should be noted that the same reference numerals are used to denote the same or similar elements, features, and structures. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0032] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0033] The terms and words used in the following description and claims are not limited to their literal meaning, but are intended solely by the inventors to achieve a clear and consistent understanding of this disclosure. Therefore, those skilled in the art will understand that the following description of various embodiments of this disclosure is provided for illustrative purposes only and is not intended to limit the purpose of this disclosure as defined by the claims and their equivalents.

[0034] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0035] Before describing the technical solution of this patent, let's first explain a few terms.

[0036] Chaos Engineering: In real-world production environments, distributed systems inevitably encounter various unpredictable events. Simultaneously, the development of cloud-native technologies continues to drive the further decoupling of microservices, and the massive scale of data and users has led to the large-scale distributed evolution of infrastructure. Distributed systems inherently have numerous interdependencies, making them prone to errors. Poor handling can result in business disruptions or other unpredictable anomalies. While it's impossible to prevent these failures in complex distributed systems, efforts can be made to identify as many risks as possible before these anomalies are triggered. Then, targeted hardening and prevention can be implemented to avoid the severe consequences of failures. Chaos engineering is precisely such a methodology—experimenting on production distributed systems to proactively identify vulnerabilities—aiming to build confidence in the system's ability to withstand uncontrolled conditions in production environments. This empirically validated approach can clearly help users build more resilient systems while gaining a more thorough understanding of the system's operational behavior. Users can build confidence in running highly available distributed systems while continuously developing more resilient systems (resilience: the system's ability to cope with and recover from failures). Chaos engineering practices can range from simple, such as running a forced exit command (e.g., kill -9) in a production environment to simulate a sudden service node crash, to complex, such as selecting a small (but sufficiently representative) portion of online traffic and automatically running a series of experiments according to certain rules or frequencies. Therefore, chaos engineering is of great significance in the field of system stability.

[0037] Attribution System: To enable more users to understand product information and serve more users, promotional information needs to be set up for the product. However, this promotional information may be presented on multiple platforms to facilitate users' final download, installation, and use of the product. Attribution aims to use technical means to determine which platform the client application is providing services through, thereby enabling faster identification of the fault problem. Therefore, real-time and offline attribution are of great significance.

[0038] Currently, offline task computation accounts for a large proportion of attribution scenarios, with major offline computing frameworks including Spark and Flink. Attribution computation logic can be executed within these frameworks. Because attribution involves complex data processing and numerous shuffle operations (the key reason for shuffle operations is that data with common characteristics needs to be aggregated onto a single computing node for computation), this places high demands on the stability of the framework's computing node infrastructure (such as memory, CPU, and bandwidth).

[0039] A common practice for improving the stability of distributed systems, such as attribution systems, is testing, including functional testing, stress testing, and fault testing. However, many of these tests are conducted in test environments or with specific test cases, while problems that may arise in real-world production environments are often undetectable by these test environments or test cases. Furthermore, these tests typically use specific methods to test a specific part of the system; if the test results do not meet expectations, the test is considered faulty. This has a very clear characteristic: it aims to disrupt the system in a predetermined way. However, in real-world production environments, many scenarios are difficult to predict, and these unpredictable scenarios can often have a significant impact on the system and business operations.

[0040] Based on the above, this patent designs a fault early warning platform based on chaos engineering to specifically address the aforementioned problems, thereby improving the overall stability of distributed systems such as attribution systems. For example, the fault early warning platform disclosed herein can possess the following capabilities: 1. Continuously run tests in a production environment and have the ability to control the test blast radius; 2. Construct a general automated test execution method; 3. Establish general indicators that can describe service stability; 4. Automatically detect changes in stability indicators; 5. The ability to automatically terminate tests. For example, by using such a fault early warning platform to test the attribution system, more problems can be discovered in advance, enabling the attribution system to have a better response when the aforementioned availability issues occur, avoiding the production of erroneous results and impacting business operations.

[0041] In the following, the methods, apparatus and systems of this disclosure will be described in detail with reference to the accompanying drawings, according to various embodiments of this disclosure.

[0042] Figure 1 This is a flowchart of a fault prediction method according to an embodiment of the present disclosure. The fault prediction method according to the present disclosure can be applied to any distributed system, such as an attribution system.

[0043] The fault warning method according to this disclosure can be executed by any electronic device with data processing capabilities. The electronic device can be a user's terminal, such as a terminal used by the user when testing the system. The electronic device can be at least one of a smartphone, tablet, laptop, and desktop computer. The electronic device can also be a server. The electronic device is equipped with a fault warning platform according to this disclosure for testing distributed systems.

[0044] Reference Figure 1In step S101, the test tasks to be executed on the distributed system are obtained. The distributed system can be an attribution system. For example, a user can initiate a test task on a distributed system through a fault warning platform installed on an electronic device. The electronic device can schedule the test task to cause the distributed system to begin executing the test task.

[0045] According to embodiments of this disclosure, a test task may include a target fault type, which can be used by the distributed system to determine a target test plan from multiple test plans. Here, the test plan may be code used to inject faults into the normal program of the distributed system. Users can randomly set multiple fault types in the test task, or they can set the order in which the test plans are executed in the test task, causing the distributed system to execute the test plans in that order.

[0046] As an example, a test task can include multiple fault types for detecting system failures, each corresponding to a fault code. Fault types can be represented by labels, numbers, or letters, for example, 1 indicates full memory, 2 indicates insufficient CPU, 3 indicates insufficient bandwidth, 4 indicates injecting a specific time delay into selected network connections, and 5 indicates randomly causing some internal functions to throw exceptions, etc. After the fault warning platform schedules the test task, the distributed system can begin executing the test task and, based on the fault types included in the test task, select a target test plan related to the fault types in multiple test plans. Then, during the execution of the test task, the corresponding test plan is executed as needed.

[0047] Assuming the test task includes three fault types (first, second, and third), the distributed system queries the fault types in the test task when it begins execution, and then finds the fault codes corresponding to the first, second, and third fault types respectively from the fault injection code set. During the execution of the test in the distributed system, the specified fault codes in the test task can be injected according to a pre-set method (e.g., randomly or sequentially) to complete the test task.

[0048] According to embodiments of this disclosure, multiple test plans can be arbitrarily modified. As an example, a previous test plan can be changed by adding or replacing other test plans. For instance, a fault warning platform can periodically update the fault codes in the fault injection code set. Here, updating / changing can refer to changing the fault code sequence number in the fault injection code set, so that when the distributed system executes test tasks again later, other fault codes can be matched from the fault injection code set based on the fault code type. Updating / changing can refer to adding new fault codes to the fault injection code set, or it can refer to replacing all or part of the original fault codes in the fault injection code set with new fault codes. The purpose of updating / changing is to make the test tasks more uncertain, to better reflect the actual production environment, which differs from existing methods of performing specific tests on a specific fault.

[0049] The fault early warning platform disclosed herein can automatically perform fault drills of various forms. For example, a predetermined execution order can be set in the scheduled test tasks, so that the distributed system executes the test plan according to the fault sequence, or the distributed system can randomly execute fault codes corresponding to the fault types. By adopting the ideas of chaos engineering in designing the fault early warning platform, more unknown weaknesses of the distributed system in the production environment can be discovered.

[0050] In step S102, the execution results of the distributed system executing the test tasks, including the target test plan, are obtained.

[0051] As an example, when a distributed system encounters an error while executing a test task that includes a target test plan, the distributed system can exit the test task. The fault warning platform can then retrieve and record the execution result from the distributed system. For instance, in the case of a distributed system exiting a test task due to an error, the fault warning platform can obtain the message indicating that the test task has exited, along with data such as the cause of the exit and the location of the fault code.

[0052] When the distributed system executes the test task, including the target test plan, without encountering any errors, the distributed system can exit the test task after its completion. The fault warning platform can then retrieve and record the test data corresponding to the execution of the test task from the distributed system. For example, if the distributed system successfully completes the test task, the fault warning platform can obtain the relevant test data from the distributed system.

[0053] In step S103, data related to the execution results is output for analyzing the stability of the distributed system. This data can be used for fault diagnosis of the distributed system to analyze its stability. Testers can determine the fault type, location, and cause of the fault based on the data, and then find corresponding solutions. This allows developers to make targeted modifications and adjustments to the distributed system program, ultimately improving the system's stability.

[0054] As an example, a user interface can be displayed on the fault warning platform, showing the execution results of the distributed system. For instance, if the distributed system encounters an error and exits a test task, the user interface can display a message indicating the exit and the reason for the failure. If the distributed system successfully completes the test task, the user interface can display not only the test data but also the comparison and verification results between the test data and normal operating data. Here, the test data can be Hive data generated by the distributed system executing a test task that includes the target test plan, while the normal operating data can be Hive data generated by the distributed system executing a test task that does not include the target test plan.

[0055] For example, a fault warning platform can acquire test data and normal operation data separately, compare and verify the test data and normal operation data, and then display the data matching accuracy on the user interface. Alternatively, the data comparison and verification operation can be performed by other electronic devices, and the fault warning platform can then obtain the data matching accuracy from these other electronic devices and display it on the user interface for users to perform stability analysis on the tested distributed system. The elements displayed in the above user interface are merely exemplary; the user interface can also display other information about the fault warning platform and the tested distributed system. Furthermore, the layout of the user interface can be configured differently according to user needs.

[0056] According to embodiments of this disclosure, the fault early warning platform can establish general indicators that can describe service stability. By strictly verifying the execution result data of test tasks that end normally and abnormally, it can automatically detect changes in stability indicators and evaluate the stability of the distributed system based on the accuracy of the actual output data.

[0057] Furthermore, when the amount of data from the execution results does not meet the predetermined data volume, the fault warning platform can determine to re-execute the test task, including the target test plan. The predetermined data volume can be pre-set by the testers in the fault warning platform for each test task according to the test requirements. After the distributed system exits the test task, if the amount of data from the test results does not reach the preset data volume corresponding to this test task, the fault warning platform can automatically schedule the test task, allowing the distributed system to execute the same test task again until the amount of data from the execution results of the test task reaches the preset data volume. In this way, the fault warning platform can not only automatically execute and terminate tests, but also present more data to users, enabling them to better analyze the system's stability.

[0058] By automatically identifying and recording fault data and automatically repeating test tasks through the fault early warning platform, not only is an effective amount of test data guaranteed, but the cost of manual testing is also reduced.

[0059] Furthermore, when an error occurs during the execution of a test task that includes the target test plan in a distributed system, the fault early warning platform can determine whether to send a fault alarm based on the severity of the error. For example, if an error occurs during the execution of a test task that includes fault codes, the distributed system can exit the test task and send the fault information to the fault early warning platform. The platform can then determine whether to send a text message or voice notification to the user to inform them of the relevant test task based on the severity level of the fault. The severity level of the fault can be preset in the fault early warning platform by the testers. This ensures that the fault early warning platform has the capability to control the experimental blast radius, thereby preventing damage to the system due to major faults or other problems.

[0060] Figure 2 This is a schematic flowchart of a fault warning method executed through a fault warning platform according to an embodiment of the present disclosure.

[0061] Reference Figure 2 The fault warning platform can initialize a set of codes for injecting faults and can freely change the fault codes in the set.

[0062] The fault early warning platform can schedule the execution of test tasks. During the execution of a distributed system, test tasks can inject corresponding fault code into the normal code of the distributed system according to the fault injection type. For example, the fault early warning platform can schedule the execution of offline tasks, which can inject specified fault code as needed during execution.

[0063] During the operation of a distributed system, if an error is encountered (such as memory exhaustion, insufficient CPU, or unknown internal system errors), the test task will terminate. Simultaneously, the distributed system can send a task termination message to a fault warning platform, which can record the fault execution history. For example, refer to... Figure 2 If the distributed system encounters an error while executing a test task using an offline computing framework (i.e., a distributed system), the distributed system will exit the test task. The fault warning platform can then call back the system's task exit notification and record fault execution results such as the system's fault point and the corresponding fault code.

[0064] If the distributed system successfully completes the test task, it can notify the data verification service to compare and verify the offline Hive data (such as offline Hive tables produced in the experiment) generated by the test task with the offline Hive data (such as offline Hive tables produced normally) generated normally. Then, it outputs the matching accuracy and exits the test task. At the same time, it sends a callback notification of the matching result to the fault warning platform, which can record the fault execution record.

[0065] As another example, if the distributed system successfully completes a test task, it can receive the offline Hive data produced by the test task and then compare and verify this offline Hive data with the normally produced offline Hive data. In other words, Figure 2 The data verification service shown can be implemented by the fault warning platform, or it can be completed by another device and the comparison verification data can be sent back to the fault warning platform.

[0066] The fault early warning platform can determine whether to conduct a new round of testing based on the execution results of the fault. For example, if the amount of data from the execution results does not meet the data volume required for this test, the fault early warning platform can reschedule the same test task for a new round of testing.

[0067] The fault warning platform can determine whether to issue an emergency fault warning based on the importance of the fault, such as notifying relevant developers by phone or SMS.

[0068] Figure 3 This is a block diagram of a fault warning device according to an embodiment of the present disclosure.

[0069] Reference Figure 3The fault warning device 300 may include an acquisition module 301, a processing module 302, and an output module 303. Each module in the fault warning device 300 may be implemented by one or more modules, and the names of the corresponding modules may vary depending on the type of module. In various embodiments, some modules in the fault warning device 300 may be omitted, or additional modules may be included. Furthermore, modules / elements according to various embodiments of this disclosure may be combined to form a single entity, and thus perform the functions of the respective modules / elements equivalently before combination.

[0070] The fault warning device 300 can be in the form of a single device, platform, or system.

[0071] The acquisition module 301 can acquire test tasks to be executed on the distributed system. For example, a user can schedule test tasks for the distributed system through the acquisition module 301. The distributed system is, for example, an attribution system.

[0072] Test tasks can include target fault types, which can be used by the distributed system to determine a target test plan from multiple test plans. Testers can randomly set fault types in test tasks, or they can set the order in which the test plans are executed, so that the distributed system executes the test plans in the test tasks in that order.

[0073] As an example, a test plan could be code used to inject faults into the normal procedures of a distributed system. The set of fault codes could be pre-stored in the storage device (not shown) of the fault warning device 300, or in a code database communicatively connected to the fault warning device 300.

[0074] The acquisition module 301 can modify multiple test plans arbitrarily. It can add new test plans or replace previous ones with others. For example, the acquisition module 301 can freely change the fault code set or periodically add new fault codes to the code database. This allows for uncertainty in the testing process, making the testing more production-ready.

[0075] The processing module 302 can obtain the execution results of the test tasks executed by the distributed system, including the target test plan.

[0076] The output module 303 can output data related to the execution results for analysis of the stability of the distributed system.

[0077] Optionally, when an error occurs during the execution of a test task including the target test plan by the distributed system, the processing module 302 may record the failure execution result. When no error occurs during the execution of a test task including the target test plan by the distributed system and the test task is completed, the processing module 302 may record the test data corresponding to the executed test task.

[0078] Optionally, the processing module 302 can compare and verify the test data with the runtime data generated by the distributed system executing test tasks that do not include the target test plan, in order to obtain the data matching accuracy. The output module 303 can output the data matching accuracy.

[0079] Optionally, when the amount of data in the execution result does not meet the predetermined amount of data, the processing module 302 may determine to execute the test task including the target test plan again.

[0080] Optionally, when an error occurs during the execution of a test task including the target test plan by the distributed system, the processing module 302 may determine whether to send a fault alarm message based on the importance of the error.

[0081] The above has been based on Figure 1 and Figure 2 The fault early warning and detection process of distributed systems has been described in detail, and will not be described again here.

[0082] Figure 4 This is a schematic diagram of the structure of a fault early warning device for the hardware operating environment according to an embodiment of this disclosure.

[0083] like Figure 4 As shown, the fault warning device 400 may include: a processing component 401, a communication bus 402, a network interface 403, an input / output interface 404, a memory 405, and a power supply component 406. The communication bus 402 is used to enable communication between these components. The input / output interface 404 may include a video display (such as a liquid crystal display), a microphone and speaker, and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). Optionally, the input / output interface 404 may also include standard wired interfaces and wireless interfaces. The network interface 403 may optionally include standard wired interfaces and wireless interfaces (such as a Wi-Fi interface). The memory 405 may be a high-speed random access memory or a stable non-volatile memory. The memory 405 may also optionally be a storage device independent of the aforementioned processing component 401.

[0084] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the fault warning device 400, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0085] like Figure 4 As shown, the memory 405, which serves as a storage medium, may include an operating system (such as a MAC operating system), a data storage module, a network communication module, a user interface module, a fault warning program corresponding to the fault warning method of this disclosure, and a database.

[0086] exist Figure 4 In the fault warning device 400 shown, the network interface 403 is mainly used for data communication with external electronic devices / terminals; the input / output interface 404 is mainly used for data interaction with users; the processing component 401 and the memory 405 in the fault warning device 400 can be set in the fault warning device 400. The fault warning device 400 calls the fault warning program stored in the memory 405 and various APIs provided by the operating system through the processing component 401 to execute the fault warning method provided in the embodiments of this disclosure.

[0087] Processing component 401 may include at least one processor, and memory 405 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by at least one processor, a fault warning method according to an embodiment of this disclosure is performed. However, the above examples are merely exemplary, and this disclosure is not limited thereto.

[0088] For example, processing component 401 can test the stability of a distributed system based on the fault warning method of this disclosure. Input / output interface 404 can display test results, such as fault points and comparison verification data, in the form of a user interface.

[0089] The processing component 401 can control the components included in the fault warning device 400 by executing a program.

[0090] The fault warning device 400 can receive or output video, audio, and documents via the input / output interface 404. For example, the fault warning device 400 can output a user interface via the input / output interface 404, in which test results can be displayed. Users can analyze the stability of the distributed system through the user interface.

[0091] As an example, the fault warning device 400 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, the fault warning device 400 is not necessarily a single electronic device, but may be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The fault warning device 400 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0092] In the fault warning device 400, the processing component 401 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processing component 401 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0093] Processing component 401 can execute instructions or code stored in memory, wherein memory 405 can also store data. Instructions and data can also be sent and received over a network via network interface 403, wherein network interface 403 can employ any known transport protocol.

[0094] The memory 405 can be integrated with the processing component 401, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 405 can include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory and processing component 401 can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processing component 401 to read data stored in the memory 405.

[0095] According to embodiments of this disclosure, an electronic device may be provided. Figure 5 This is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 500 may include at least one memory 502 and at least one processor 501. The at least one memory 502 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor 501, a fault warning method according to an embodiment of the present disclosure is executed.

[0096] Processor 501 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 501 may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0097] The memory 502, which serves as a storage medium, may include an operating system (e.g., a MAC operating system), a data storage module, a network communication module, a user interface module, a fault warning program, and a database.

[0098] The memory 502 may be integrated with the processor 501; for example, RAM or flash memory may be arranged within an integrated circuit microprocessor. Alternatively, the memory 502 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 502 and the processor 501 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 501 to read files stored in the memory 502.

[0099] In addition, electronic device 500 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic device 500 can be interconnected via a bus and / or network.

[0100] As will be understood by those skilled in the art, Figure 5 The structure shown does not constitute a limitation on the structure and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0101] According to embodiments of this disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, they cause at least one processor to perform a fault warning method according to this disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0102] According to embodiments of this disclosure, a computer program product may also be provided, wherein the instructions in the computer program product can be executed by the processor of a computer device to complete the above-described fault warning method.

[0103] It should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.

[0104] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0105] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A fault early warning method for distributed systems, characterized in that, include: Obtain test tasks to be executed on a distributed system, wherein the test tasks include target fault types, the target fault types being used by the distributed system to determine a target test plan from multiple test plans, wherein the multiple test plans are arbitrarily changed by automatically adding or replacing other test plans; When an error occurs during the execution of the test task including the target test plan by the distributed system, the failure execution result is recorded; When the distributed system executes the test task including the target test plan without errors and after the test task is completed, the test data corresponding to the execution of the test task is recorded; The test data is compared and verified with the runtime data generated by the distributed system executing the test task that does not include the target test plan; The data matching accuracy is output based on the comparison and verification results, wherein the data matching accuracy is used for fault analysis of the distributed system. If the amount of data from the execution result does not meet the predetermined amount of data, the test task including the target test plan will be executed again. The distributed system is an attribution system.

2. The fault early warning method according to claim 1, characterized in that, The method further includes: When the distributed system encounters an error while executing the test task that includes the target test plan, it determines whether to send a fault alarm message based on the importance of the error.

3. A fault early warning device for a distributed system, characterized in that, include: The acquisition module is configured to acquire test tasks to be executed on the distributed system, wherein the test tasks include target fault types, the target fault types being used by the distributed system to determine a target test plan from multiple test plans, wherein the multiple test plans are arbitrarily changed by automatically adding or replacing other test plans; The processing module is configured to: record the failure execution result when an error occurs during the execution of the test task including the target test plan by the distributed system; record the test data corresponding to the execution of the test task after the test task is completed without an error by the distributed system executing the test task including the target test plan; compare and verify the test data with the running data generated by the distributed system executing the test task excluding the target test plan to obtain the data matching accuracy; and determine to execute the test task including the target test plan again when the amount of data in the execution result does not meet the predetermined amount of data. An output module is configured to output the data matching accuracy, wherein the data matching accuracy is used for fault analysis of the distributed system. The distributed system is an attribution system.

4. The fault early warning device according to claim 3, characterized in that, The processing module is configured as follows: When the distributed system encounters an error while executing the test task that includes the target test plan, it determines whether to send a fault alarm message based on the importance of the error.

5. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. Wherein, when the computer-executable instructions are executed by the at least one processor, they cause the at least one processor to perform the fault warning method as described in any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that, A storage instruction, when executed by at least one processor, causes the at least one processor to perform the fault warning method as described in any one of claims 1 to 2.

7. A computer program product, wherein instructions in the computer program product are executed by at least one processor in an electronic device to perform the fault warning method as claimed in any one of claims 1 to 2.

Images