Chaotic engineering experiment data analysis method and device based on AI large model

Abstract

The invention relates to a chaos engineering experiment data analysis method and device based on an AI large model, and relates to the technical field of data processing, and the method comprises the steps: obtaining target fault data and target system operation data of a chaos engineering platform; the target system operation data is operation data of the application system after the chaotic engineering platform injects the target fault data into the application system; calling an API interface of the large language model according to the target fault data and the target system operation data to obtain a target data analysis result returned by the large language model; providing the target data analysis result to a chaos engineering platform; wherein the target data analysis result is determined in combination with the large language model, analysis and processing by analysts can be avoided, and the data analysis cost is reduced; and the data analysis efficiency can be improved based on the large parameter quantity and accuracy of the large language model.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method and apparatus for analyzing chaotic engineering experimental data based on a large AI model. Background Technology

[0002] Currently, in the chaos engineering platform, fault data is injected into the application system to be tested, and the running data of the application system after the fault data is injected is obtained; the running data is analyzed to determine the performance of the application system.

[0003] In the above scheme, the analysis of operational data requires high technical skills and experience from the analysts, resulting in high analysis costs and poor analysis efficiency. Summary of the Invention

[0004] This disclosure provides a method and apparatus for analyzing chaotic engineering experimental data based on a large AI model, aiming to at least partially address one of the technical problems in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a method for analyzing chaotic engineering experimental data based on an AI large model is provided, comprising: acquiring target fault data and target system operation data of a chaotic engineering platform; the target system operation data being the operation data of an application system after the chaotic engineering platform injects the target fault data into the application system; calling the API interface of a large language model based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model; and providing the target data analysis results to the chaotic engineering platform.

[0005] Optionally, the input data format of the API interface is obtained; the input data format includes the position of each fault parameter and the position of each system operating parameter; based on the position of each fault parameter and the position of each system operating parameter, the position adjustment processing of each fault parameter in the target fault data and each system operating parameter in the target system operating data is performed to obtain structured data; the API interface is called based on the structured data to obtain the target data analysis results.

[0006] Optionally, the target data analysis result includes: target analysis judgment result and target solution measure; the target analysis judgment result is used to indicate whether there is a potential system problem in the application system; the target solution measure is a solution strategy for the potential system problem; the method further includes: obtaining the target feedback result corresponding to the target data analysis result; the target feedback result indicates whether the target solution measure solves the potential system problem indicated in the target analysis judgment result; when the target feedback result indicates that the target solution measure solves the potential system problem, the target fault data, the target system operation data, and the target data analysis result are determined as fine-tuning data for fine-tuning training of the large language model.

[0007] Optionally, the method further includes: when the target feedback result indicates that the target solution has not solved the potential system problem, marking the target solution as a failed solution to obtain a marked target solution; calling the API interface based on the target fault data, the target system operation data, and the marked target solution to obtain the adjusted data analysis results returned by the large language model; the adjusted data analysis results include the adjusted solution; and providing the adjusted data analysis results to the chaos engineering platform.

[0008] Optionally, obtaining the target feedback result corresponding to the target data analysis result includes: obtaining the processed system; the processed system is obtained by adjusting the application system according to the target solution; injecting the target fault data into the processed system to obtain processed system operation data; calling the API interface according to the target fault data and the processed system operation data to obtain the processed data analysis result returned by the large language model; and determining the target feedback result corresponding to the target data analysis result according to the processed analysis judgment result in the processed analysis result.

[0009] Optionally, determining the target feedback result corresponding to the target data analysis result based on the post-processing analysis judgment result in the post-processing analysis result includes: determining the target feedback result as a first feedback result when the post-processing analysis judgment result indicates that the potential system problem does not exist in the post-processed system; the first feedback result indicates that the target solution solves the potential system problem; determining the target feedback result as a second feedback result when the post-processing analysis judgment result indicates that the potential system problem exists in the post-processed system; the second feedback result indicates that the target solution has not solved the potential system problem.

[0010] Optionally, the method further includes: acquiring log data from the chaos engineering platform during the experiment on the application system; extracting historical fault data, historical system operation data, and historical data analysis results from the log data; and providing the historical fault data, historical system operation data, and historical data analysis results to the large language model for training the large language model.

[0011] Optionally, the historical data analysis results include: historical analysis judgment results and historical solutions; the training method of the large language model includes: determining feedback results corresponding to the historical data analysis results based on the historical solutions; the feedback results include a first feedback result and a second feedback result; the first feedback result indicates that the historical solutions address the potential system problems indicated in the historical analysis judgment results; the second feedback result indicates that the historical solutions do not address the potential system problems indicated in the historical analysis judgment results; determining positive samples based on the first feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the first feedback result; determining negative samples based on the second feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the second feedback result; and training the large language model based on the positive samples and the negative samples.

[0012] According to a second aspect of the present disclosure, a chaotic engineering experimental data analysis device based on an AI large model is provided, comprising: a first acquisition module, configured to acquire target fault data and target system operation data of a chaotic engineering platform; the target system operation data being the operation data of an application system after the chaotic engineering platform injects the target fault data into the application system; a calling module, configured to call the API interface of a large language model based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model; and a first providing module, configured to provide the target data analysis results to the chaotic engineering platform.

[0013] Optionally, the calling module is specifically used to: obtain the input data format of the API interface; the input data format includes the position of each fault parameter and the position of each system operating parameter; adjust the position of each fault parameter in the target fault data and each system operating parameter in the target system operating data according to the position of each fault parameter and the position of each system operating parameter to obtain structured data; and call the API interface according to the structured data to obtain the target data analysis results.

[0014] Optionally, the target data analysis result includes: a target analysis judgment result and a target solution measure; the target analysis judgment result is used to indicate whether there is a potential system problem in the application system; the target solution measure is a solution strategy for the potential system problem; the device further includes: a second acquisition module and a determination module; the second acquisition module is used to acquire the target feedback result corresponding to the target data analysis result; the target feedback result indicates whether the target solution measure solves the potential system problem indicated in the target analysis judgment result; the determination module is used to determine the target fault data, the target system operation data, and the target data analysis result as fine-tuning data when the target feedback result indicates that the target solution measure solves the potential system problem, for fine-tuning training processing of the large language model.

[0015] Optionally, the apparatus further includes: a marking processing module and a second providing module; the marking processing module is configured to mark the target solution as a failed measure when the target feedback result indicates that the target solution has not solved the potential system problem, thereby obtaining a marked target solution; the calling module is further configured to call the API interface based on the target fault data, the target system operation data, and the marked target solution to obtain the adjusted data analysis results returned by the large language model; the adjusted data analysis results include the adjusted solution; the second providing module is configured to provide the adjusted data analysis results to the chaos engineering platform.

[0016] Optionally, the second acquisition module is specifically used to: acquire the processed system; the processed system is obtained by adjusting the application system according to the target solution; inject the target fault data into the processed system to obtain processed system operation data; call the API interface according to the target fault data and the processed system operation data to obtain the processed data analysis results returned by the large language model; and determine the target feedback result corresponding to the target data analysis result according to the processed analysis judgment result in the processed analysis result.

[0017] Optionally, the second acquisition module is further configured to, when the post-processing analysis and determination result indicates that the potential system problem does not exist in the post-processing system, determine the target feedback result as a first feedback result; the first feedback result indicates that the target solution addresses the potential system problem; when the post-processing analysis and determination result indicates that the potential system problem exists in the post-processing system, determine the target feedback result as a second feedback result; the second feedback result indicates that the target solution has not addressed the potential system problem.

[0018] Optionally, the device further includes: a third acquisition module, an extraction and processing module, and a third providing module; the third acquisition module is used to acquire log data from the chaos engineering platform during the experiment on the application system; the extraction and processing module is used to extract the historical fault data, the historical system operation data, and the historical data analysis results from the log data; the third providing module is used to provide the historical fault data, the historical system operation data, and the historical data analysis results to the large language model for training the large language model.

[0019] Optionally, the historical data analysis results include: historical analysis judgment results and historical solutions; the training method of the large language model includes: determining feedback results corresponding to the historical data analysis results based on the historical solutions; the feedback results include a first feedback result and a second feedback result; the first feedback result indicates that the historical solutions address the potential system problems indicated in the historical analysis judgment results; the second feedback result indicates that the historical solutions do not address the potential system problems indicated in the historical analysis judgment results; determining positive samples based on the first feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the first feedback result; determining negative samples based on the second feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the second feedback result; and training the large language model based on the positive samples and the negative samples.

[0020] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method for analyzing chaotic engineering experimental data based on a large AI model as described in the first aspect of the present disclosure.

[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the chaotic engineering experimental data analysis method based on a large AI model as described in the first aspect of the present disclosure.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the method for analyzing chaotic engineering experimental data based on a large AI model as described in the first aspect of the present disclosure.

[0023] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: In this technical solution, the target data analysis results are determined by combining a large language model, target fault data from a chaos engineering platform, and target system operation data. This avoids the need for analysis and processing by analysts, thus reducing data analysis costs. Simultaneously, the large number of parameters and accuracy of the large language model improve data analysis efficiency. Furthermore, based on the input data format of the API interface, the positions of various parameters in the target fault data and target system operation data are adjusted to obtain structured data. The API interface is then called based on this structured data, enabling the corresponding processing logic to accurately extract and process each parameter, further improving call efficiency. Additionally, the target data analysis results, target fault data, and target system operation data, based on the corresponding target feedback results indicating that the target solutions have addressed potential system problems, are used to fine-tune the large language model, further improving its accuracy and consequently enhancing subsequent data analysis efficiency.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0026] Figure 1 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the first embodiment of this disclosure. Figure 2 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the second embodiment of this disclosure. Figure 3 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the third embodiment of this disclosure. Figure 4 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the fourth embodiment of this disclosure. Figure 5 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the fifth embodiment of this disclosure. Figure 6 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the sixth embodiment of this disclosure. Figure 7 This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the seventh embodiment of this disclosure. Figure 8 This is a schematic diagram of the structure of the chaotic engineering experimental data analysis device based on an AI large model, as shown in the eighth embodiment of this disclosure; Figure 9 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present disclosure. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0029] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution disclosed herein are all carried out with the consent of the user, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0030] Currently, in the chaos engineering platform, fault data is injected into the application system to be tested, and the running data of the application system after the fault data is injected is obtained; the running data is analyzed to determine the performance of the application system.

[0031] The analysis of operational data requires human intervention. This process can take several hours, resulting in long analysis times and low efficiency. Furthermore, analyzing operational data demands a high level of technical skill and experience from personnel. Those with lower technical skills or less experience are more prone to making errors in their analysis and are unable to provide effective solutions.

[0032] To address at least one of the aforementioned problems, this disclosure provides a method and apparatus for analyzing chaotic engineering experimental data based on a large AI model. The method and apparatus for analyzing chaotic engineering experimental data based on a large AI model according to embodiments of this disclosure are described below with reference to the accompanying drawings.

[0033] Figure 1This is a flowchart illustrating the method for analyzing chaotic engineering experimental data based on a large AI model, as shown in the first embodiment of this disclosure.

[0034] It should be noted that, in this embodiment of the disclosure, the method for analyzing chaotic engineering experimental data based on a large AI model can be applied to electronic devices. These electronic devices can be, for example, electronic devices that are communicatively connected to a chaotic engineering platform; or, one of the electronic devices within the chaotic engineering platform, etc., without specific limitations here.

[0035] Furthermore, the chaotic engineering experimental data analysis method based on large AI models can be applied to software in electronic devices, such as chaotic engineering experimental data analysis software based on large AI models. The following embodiments use an electronic device as an example for illustration.

[0036] like Figure 1 As shown, this method for analyzing chaotic engineering experimental data based on a large AI model may include the following steps: Step 101: Obtain the target fault data and target system operation data from the chaos engineering platform; the target system operation data is the operation data of the application system after the chaos engineering platform injects the target fault data into the application system.

[0037] In this embodiment, the chaos engineering platform can be a platform for verifying application systems through controlled experiments. The chaos engineering platform is used to verify at least one of the following aspects of the application system: application system architectural weaknesses, application system fault recovery capabilities, and application system robustness. No specific limitations are imposed here; the processing can be configured according to actual needs.

[0038] In this embodiment of the disclosure, the target fault data may include at least one of the following: fault scenario type, fault parameters, and fault time data; the fault scenario type may include at least one of the following: infrastructure fault, network fault, application fault, and service fault.

[0039] These include infrastructure failures such as Central Processing Unit (CPU) overload, disk failures, and memory failures; network failures such as packet loss and Domain Name System (DNS) hijacking; application failures such as slow Structured Query Language (SQL) injection attacks; and service failures such as container (pod) termination, process crashes, and Java Virtual Machine (JVM) failures.

[0040] The fault parameters, such as CPU load percentage, memory usage threshold, disk space utilization, packet loss rate, bandwidth limit, and service degradation ratio, are not specifically described here. Fault time data, such as fault duration, latency, and fault recovery time, are also included.

[0041] There is a correspondence between fault scenario types and fault parameters. Different fault scenario types can correspond to different fault parameters.

[0042] The flexible setting of various fault data allows the large language model to consider more features, further improving the accuracy of the target data analysis results.

[0043] In this embodiment of the disclosure, the target system operation data may include at least one of the following: number of requests, request frequency, number of requests per second, response duration, response frequency, etc., without specific limitations.

[0044] Step 102: Call the API interface of the large language model based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model.

[0045] As an example, such as Figure 2 As shown, step 102 may include the following steps: Step 1021: Obtain the input data format of the API interface; the input data format includes the location of each fault parameter and the location of each system operating parameter.

[0046] Step 1022: Based on the location of each fault parameter and the location of each system operating parameter, perform position adjustment processing on each fault parameter in the target fault data and each system operating parameter in the target system operating data to obtain structured data.

[0047] In this embodiment of the disclosure, the format of the structured data can be, for example, key-value pairs. In the input data format, each fault parameter and each system operating parameter has a fixed position and arrangement order. The electronic device can organize and process each fault parameter in the target fault data and each system operating parameter in the target system operating data according to the position and / or arrangement order of the parameters in the input data format to obtain structured data.

[0048] In this process, the target fault data and target system operation data are structured, enabling the processing logic corresponding to the API interface to accurately extract various parameters. This facilitates the processing of large language models based on the extracted parameters, further improving the accuracy of the target data analysis results.

[0049] Step 1023: Call the API interface based on the structured data to obtain the target data analysis results.

[0050] In this embodiment of the disclosure, the target data analysis result may include: a target analysis determination result and target solutions. The target analysis determination result indicates whether the application system has potential system problems. The target solutions are strategies for addressing potential system problems. A potential system problem is a system problem that does not occur during normal operation of the application system but arises during operation after injecting target fault data.

[0051] The results of the target analysis and judgment can be represented by phenomena. These phenomena include, for example, steady-state and unsteady-state system phenomena. Steady-state phenomena indicate the absence of potential system problems, while unsteady-state phenomena indicate the presence of potential system problems.

[0052] Specifically, based on the input data format of the API interface, the positions of various parameters in the target fault data and the target system operation data are adjusted and processed to obtain structured data. The API interface is then called based on the structured data, so that the processing logic corresponding to the API interface can accurately extract and process each parameter, thereby further improving the calling efficiency.

[0053] In this embodiment of the disclosure, to further improve the accuracy of the obtained target data analysis results, the electronic device can also obtain application system deployment information. Correspondingly, the electronic device can perform step 102, for example, by calling the API interface of the large language model based on the target fault data, target system operation data, and application system deployment information, to obtain the target data analysis results returned by the large language model.

[0054] To further improve the efficiency of data analysis in chaotic engineering experiments based on AI large models, historical fault data, historical system operation data, and historical data analysis results can be provided to the large language model for training. For example... Figure 3 As shown, prior to step 102, the electronic device may also perform the following processes: Step 104: Obtain log data from the chaos engineering platform during the experimentation with the application system.

[0055] In this embodiment of the disclosure, the chaos engineering platform can deploy at least one probe in the application system to obtain log data during the experiment process of the application system through at least one probe.

[0056] Extracting historical fault data, historical system operation data, and historical data analysis results from log data during the experimentation process of the application system can improve data extraction efficiency.

[0057] Step 105: Extract historical fault data, historical system operation data, and historical data analysis results from the log data.

[0058] In this embodiment of the disclosure, after step 105, the electronic device can store historical fault data, historical system operation data, and historical data analysis results in a knowledge base, so that the electronic device can retrieve them from the knowledge base when needed.

[0059] In this embodiment of the disclosure, there is a correlation between historical fault data, historical system operation data, and historical data analysis results in the knowledge base. Specifically, historical system operation data refers to the application system's operation data after the chaos engineering platform injects historical fault data into the application system. Historical data analysis results are the data analysis results corresponding to the historical system operation data.

[0060] Step 106: Provide historical fault data, historical system operation data, and historical data analysis results to the large language model for training.

[0061] In this embodiment of the disclosure, prior to step 106, the electronic device may also perform the following processes: determining the target environment; and deploying a large language model in the target environment.

[0062] The target environment can be, for example, an electronic device, or other devices that communicate with the electronic device.

[0063] The deployment of large language models facilitates subsequent training and processing of these models.

[0064] In this embodiment of the disclosure, the historical data analysis results may include: historical analysis judgment results and historical solution measures. The historical analysis judgment results are used to indicate whether there are potential system problems in the application system; the historical solution measures are strategies for addressing potential system problems. In one example, the training method for a large language model may include the following process: structuring historical fault data and historical system operation data to obtain historical structured data; inputting the historical structured data into the large language model to obtain the predicted data analysis results output by the large language model; determining the value of the loss function based on the historical data analysis results corresponding to the historical system operation data, the predicted data analysis results, and the loss function of the large language model; and adjusting the parameters of the large language model based on the loss function value to obtain the trained large language model.

[0065] In another example, the training method for a large language model may include the following process: Based on historical solutions, determine the feedback results corresponding to the historical data analysis results; the feedback results include a first feedback result and a second feedback result; the first feedback result indicates that the historical solutions addressed the potential system problems indicated in the historical analysis results; the second feedback result indicates that the historical solutions did not address the potential system problems indicated in the historical analysis results; based on the first feedback result, and the corresponding historical data analysis results, historical fault data, and historical system operation data, determine positive samples; based on the second feedback result, and the corresponding historical data analysis results, historical fault data, and historical system operation data, determine negative samples; and based on the positive and negative samples, train the large language model.

[0066] The feedback settings facilitate the large language model in distinguishing between positive and negative samples, thereby further improving the accuracy of the trained large language model.

[0067] In this embodiment of the disclosure, in order to further improve the accuracy of the trained large language model, the electronic device can also obtain application system deployment information. Correspondingly, the process of the electronic device executing step 106 can be, for example, providing historical fault data, historical system operation data, historical data analysis results, and application system deployment information to the large language model for training the large language model.

[0068] Step 103: Provide the target data analysis results to the chaos engineering platform.

[0069] In this embodiment of the disclosure, the target data analysis result may include: target analysis judgment result and target solution measures; the target analysis judgment result is used to indicate whether there are potential system problems in the application system; the target solution measures are the solution strategies for potential system problems.

[0070] To further improve the analytical accuracy of the large language model, electronic devices can acquire target feedback results corresponding to the target data analysis results. When the target feedback results indicate that the target solutions have resolved potential system problems, the target fault data, target system operation data, and target data analysis results are identified as fine-tuning data for fine-tuning the large language model. For example... Figure 4 As shown, after step 103, the electronic device may also perform the following process: Step 107: Obtain the target feedback result corresponding to the target data analysis result; the target feedback result indicates whether the target solution measures have solved the potential system problems indicated in the target analysis and judgment results.

[0071] As an example, such as Figure 5As shown, step 107 may include the following steps: Step 1071, obtain the processed system; the processed system is obtained by adjusting the application system according to the target solution.

[0072] Step 1072: Inject the target fault data into the processed system to obtain the processed system's operating data.

[0073] Step 1073: Call the API interface based on the target fault data and the processed system operation data to obtain the processed data analysis results returned by the large language model.

[0074] Step 1074: Based on the post-processing analysis judgment result in the post-processing analysis results, determine the target feedback result corresponding to the target data analysis result.

[0075] As an example, such as Figure 6 As shown, step 1074 may include the following steps: Step 10741: If the post-processing analysis and judgment result indicates that there are no potential system problems in the post-processing system, the target feedback result is determined as the first feedback result; the first feedback result indicates the target solution to solve the potential system problems.

[0076] Step 10742: If the post-processing analysis and judgment result is used to indicate that there is a potential system problem in the post-processing system, the target feedback result is determined as the second feedback result; the second feedback result indicates that the target solution has not solved the potential system problem.

[0077] Step 108: When the target feedback results indicate that the target solution measures can solve potential system problems, the target fault data, target system operation data, and target data analysis results are identified as fine-tuning data for fine-tuning training of the large language model.

[0078] To further improve the accuracy of the obtained target data analysis results, the electronic device can also regenerate adjusted data analysis results when the target feedback results indicate that the target solution has not resolved the potential system problems. These adjusted results will then be provided to the chaos engineering platform. For example... Figure 7 As shown, after step 107, the electronic device may also perform the following process: Step 109: If the target feedback result indicates that the target solution has not solved the potential system problem, mark the target solution as a failed measure to obtain the marked target solution.

[0079] It should be noted that if the feedback from the adjusted data analysis results still indicates that the underlying system problem has not been resolved, the adjusted data analysis results can be regenerated until the feedback from the adjusted data analysis results indicates that the underlying system problem has been resolved. By providing the adjusted data analysis results that resolve the underlying system problem to the chaos engineering platform, and then adjusting the application system accordingly, the number of adjustments to the application system can be reduced, thereby improving the efficiency of application system adjustments.

[0080] Step 110: Based on the target fault data, target system operation data, and marked target solutions, call the API interface to obtain the adjusted data analysis results returned by the large language model; the adjusted data analysis results include the adjusted solutions.

[0081] Step 111: Provide the adjusted data analysis results to the chaos engineering platform.

[0082] In the chaos engineering experimental data analysis method based on AI large model of this embodiment, the target fault data and target system operation data of the chaos engineering platform are obtained; the target system operation data is the operation data of the application system after the chaos engineering platform injects the target fault data into the application system; the API interface of the large language model is called according to the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model; the target data analysis results are provided to the chaos engineering platform; wherein, the combination of large language model to determine the target data analysis results can avoid the analysis and processing by analysts, reducing the data analysis cost; and based on the large number of parameters and accuracy of the large language model, the data analysis efficiency can be improved.

[0083] With the above Figures 1 to 7 Corresponding to the method for analyzing chaotic engineering experimental data based on AI large-scale models provided in the embodiments, this disclosure also provides a device for analyzing chaotic engineering experimental data based on AI large-scale models. Because the device for analyzing chaotic engineering experimental data based on AI large-scale models provided in the embodiments of this disclosure is different from the above... Figures 1 to 7 The implementation method of the chaotic engineering experimental data analysis method based on the AI ​​large model provided in the embodiments corresponds to the chaotic engineering experimental data analysis device based on the AI ​​large model provided in the embodiments of this disclosure, and will not be described in detail in the embodiments of this disclosure.

[0084] Figure 8 This is a schematic diagram of the structure of the chaotic engineering experimental data analysis device based on an AI large model, as shown in the eighth embodiment of this disclosure.

[0085] Reference Figure 8The chaotic engineering experiment data analysis device 800 based on the AI ​​large model may include: a first acquisition module 810, a calling module 820 and a first providing module 830.

[0086] The first acquisition module 810 is used to acquire target fault data and target system operation data from the chaos engineering platform; the target system operation data is the operation data of the application system after the chaos engineering platform injects the target fault data into the application system; the invocation module 820 is used to invoke the API interface of the large language model according to the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model; the first provision module 830 is used to provide the target data analysis results to the chaos engineering platform.

[0087] Optionally, the calling module 820 is specifically used to: obtain the input data format of the API interface; the input data format includes the position of each fault parameter and the position of each system operating parameter; adjust the position of each fault parameter in the target fault data and each system operating parameter in the target system operating data according to the position of each fault parameter and the position of each system operating parameter to obtain structured data; and call the API interface according to the structured data to obtain the target data analysis results.

[0088] Optionally, the target data analysis result includes: a target analysis judgment result and a target solution measure; the target analysis judgment result is used to indicate whether there is a potential system problem in the application system; the target solution measure is a solution strategy for the potential system problem; the device further includes: a second acquisition module and a determination module; the second acquisition module is used to acquire the target feedback result corresponding to the target data analysis result; the target feedback result indicates whether the target solution measure solves the potential system problem indicated in the target analysis judgment result; the determination module is used to determine the target fault data, the target system operation data, and the target data analysis result as fine-tuning data when the target feedback result indicates that the target solution measure solves the potential system problem, for fine-tuning training processing of the large language model.

[0089] Optionally, the apparatus further includes: a marking processing module and a second providing module; the marking processing module is configured to mark the target solution as a failed measure when the target feedback result indicates that the target solution has not solved the potential system problem, thereby obtaining a marked target solution; the calling module 820 is further configured to call the API interface based on the target fault data, the target system operation data, and the marked target solution to obtain the adjusted data analysis results returned by the large language model; the adjusted data analysis results include the adjusted solution; the second providing module is configured to provide the adjusted data analysis results to the chaos engineering platform.

[0090] Optionally, the second acquisition module is specifically used to: acquire the processed system; the processed system is obtained by adjusting the application system according to the target solution; inject the target fault data into the processed system to obtain processed system operation data; call the API interface according to the target fault data and the processed system operation data to obtain the processed data analysis results returned by the large language model; and determine the target feedback result corresponding to the target data analysis result according to the processed analysis judgment result in the processed analysis result.

[0091] Optionally, the second acquisition module is further configured to, when the post-processing analysis and determination result indicates that the potential system problem does not exist in the post-processing system, determine the target feedback result as a first feedback result; the first feedback result indicates that the target solution addresses the potential system problem; when the post-processing analysis and determination result indicates that the potential system problem exists in the post-processing system, determine the target feedback result as a second feedback result; the second feedback result indicates that the target solution has not addressed the potential system problem.

[0092] Optionally, the device further includes: a third acquisition module, an extraction and processing module, and a third providing module; the third acquisition module is used to acquire log data from the chaos engineering platform during the experiment on the application system; the extraction and processing module is used to extract the historical fault data, the historical system operation data, and the historical data analysis results from the log data; the third providing module is used to provide the historical fault data, the historical system operation data, and the historical data analysis results to the large language model for training the large language model.

[0093] Optionally, the historical data analysis results include: historical analysis judgment results and historical solutions; the training method of the large language model includes: determining feedback results corresponding to the historical data analysis results based on the historical solutions; the feedback results include a first feedback result and a second feedback result; the first feedback result indicates that the historical solutions address the potential system problems indicated in the historical analysis judgment results; the second feedback result indicates that the historical solutions do not address the potential system problems indicated in the historical analysis judgment results; determining positive samples based on the first feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the first feedback result; determining negative samples based on the second feedback result, and the historical data analysis results, historical fault data, and historical system operation data corresponding to the second feedback result; and training the large language model based on the positive samples and the negative samples.

[0094] In the chaotic engineering experimental data analysis device based on an AI large model in this embodiment, target fault data and target system operation data of the chaotic engineering platform are acquired. The target system operation data is the operation data of the application system after the chaotic engineering platform injects the target fault data into the application system. The API interface of the large language model is called according to the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model. The target data analysis results are then provided to the chaotic engineering platform. In this way, determining the target data analysis results by combining the large language model can avoid the analysis and processing by analysts, reducing the data analysis cost. Moreover, based on the large number of parameters and accuracy of the large language model, the data analysis efficiency can be improved.

[0095] In an exemplary embodiment, an electronic device is also proposed.

[0096] The electronic devices include: processor; Memory used to store processor-executable instructions; The processor is configured to execute instructions to implement the chaotic engineering experiment data analysis method based on the AI ​​large model as proposed in any of the foregoing embodiments.

[0097] As an example, Figure 9 This is a schematic diagram of the structure of an electronic device 900 as shown in an exemplary embodiment of this disclosure, as follows: Figure 9 As shown, the aforementioned electronic device 900 may further include: The system includes a memory 910 and a processor 920, and a bus 930 connecting different components (including the memory 910 and the processor 920). The memory 910 stores a computer program, and when the processor 920 executes the program, it implements the chaotic engineering experimental data analysis method based on an AI large model as described in this embodiment of the present disclosure.

[0098] Bus 930 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0099] Electronic device 900 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 900, including volatile and non-volatile media, removable and non-removable media.

[0100] The memory 910 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 940 and / or cache memory 950. The electronic device 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 960 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 9 Not shown; usually referred to as a "hard drive"). Although Figure 9 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 930 via one or more data media interfaces. Memory 910 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0101] A program / utility 980 having a set (at least one) of program modules 970 may be stored in, for example, memory 910. Such program modules 970 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 970 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0102] Electronic device 900 can also communicate with one or more external devices 990 (e.g., keyboard, pointing device, display 991, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 992. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 993. As shown, network adapter 993 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0103] The processor 920 executes various functional applications and performs data analysis of chaotic engineering experiments based on large AI models by running programs stored in the memory 910.

[0104] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the chaotic engineering experimental data analysis method based on the AI ​​large model of this disclosure, and will not be repeated here.

[0105] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of an electronic device to perform the chaos engineering experimental data analysis method based on a large AI model proposed in any of the above embodiments. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0106] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, characterized in that, when the computer program / instruction is executed by a processor, it implements the chaotic engineering experiment data analysis method based on an AI large model proposed in any of the above embodiments.

[0107] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0108] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for analyzing chaotic engineering experimental data based on a large AI model, characterized in that, The method includes: Acquire target fault data and target system operation data from the chaos engineering platform; the target system operation data is the operation data of the application system after the chaos engineering platform injects the target fault data into the application system. The API interface of the large language model is called based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model. The target data analysis results are provided to the chaos engineering platform.

2. The method according to claim 1, characterized in that, The step of calling the API interface of the large language model based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model includes: Obtain the input data format of the API interface; the input data format includes the location of each fault parameter and the location of each system operating parameter; Based on the positions of the fault parameters and the system operation parameters, the positions of the fault parameters in the target fault data and the system operation parameters in the target system operation data are adjusted to obtain structured data. The API interface is invoked based on the structured data to obtain the analysis results of the target data.

3. The method according to claim 1, characterized in that, The target data analysis results include: target analysis and determination results and target solutions; the target analysis and determination results are used to indicate whether there are potential system problems in the application system; the target solutions are strategies for addressing the potential system problems; the method further includes: Obtain the target feedback result corresponding to the target data analysis result; the target feedback result indicates whether the target solution measures resolve the potential system problem indicated in the target analysis and judgment result. When the target feedback result indicates that the target solution addresses the potential system problem, the target fault data, the target system operation data, and the target data analysis results are identified as fine-tuning data for fine-tuning training of the large language model.

4. The method according to claim 3, characterized in that, The method further includes: If the target feedback result indicates that the target solution has not solved the potential system problem, the target solution is marked as a failed solution to obtain the marked target solution. The API interface is invoked based on the target fault data, the target system operation data, and the marked target solutions to obtain the adjusted data analysis results returned by the large language model; the adjusted data analysis results include the adjusted solutions. The adjusted data analysis results are provided to the chaos engineering platform.

5. The method according to claim 3, characterized in that, The step of obtaining the target feedback result corresponding to the target data analysis result includes: Obtain the processed system; the processed system is obtained by adjusting the application system according to the target solution. The target fault data is injected into the processed system to obtain the processed system's operating data. The API interface is called based on the target fault data and the processed system operation data to obtain the processed data analysis results returned by the large language model. Based on the post-processing analysis judgment result in the post-processing analysis results, the target feedback result corresponding to the target data analysis result is determined.

6. The method according to claim 5, characterized in that, The step of determining the target feedback result corresponding to the target data analysis result based on the post-processing analysis judgment result in the post-processing analysis result includes: If the post-processing analysis and determination result indicates that the potential system problem does not exist in the post-processing system, the target feedback result is determined as the first feedback result; the first feedback result indicates that the target solution measures resolve the potential system problem. If the post-processing analysis and determination result indicates that the potential system problem exists in the post-processing system, the target feedback result is determined as the second feedback result; the second feedback result indicates that the target solution has not solved the potential system problem.

7. The method according to claim 1, characterized in that, The method further includes: Obtain log data from the chaos engineering platform during the experimentation process on the application system; Extract historical fault data, historical system operation data, and historical data analysis results from the log data; The historical fault data, the historical system operation data, and the historical data analysis results are provided to the large language model for training.

8. The method according to claim 7, characterized in that, The historical data analysis results include: historical analysis and judgment results and historical solutions; the training methods of the large language model include: Based on the historical solutions, a feedback result corresponding to the historical data analysis results is determined; the feedback result includes a first feedback result and a second feedback result; the first feedback result indicates that the historical solutions resolved the potential system problem indicated in the historical analysis and judgment results; the second feedback result indicates that the historical solutions did not resolve the potential system problem indicated in the historical analysis and judgment results. Based on the first feedback result, as well as the historical data analysis results, historical fault data, and historical system operation data corresponding to the first feedback result, positive samples are determined. Based on the second feedback result, as well as the historical data analysis results, historical fault data, and historical system operation data corresponding to the second feedback result, negative samples are determined; The large language model is trained based on the positive and negative samples.

9. A chaotic engineering experiment data analysis device based on an AI large model, characterized in that, The device includes: The first acquisition module is used to acquire target fault data and target system operation data from the chaos engineering platform; the target system operation data is the operation data of the application system after the chaos engineering platform injects the target fault data into the application system. The calling module is used to call the API interface of the large language model based on the target fault data and the target system operation data to obtain the target data analysis results returned by the large language model; The first providing module is used to provide the target data analysis results to the chaos engineering platform.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method for analyzing chaotic engineering experimental data based on a large AI model as described in any one of claims 1 to 8.

Images