Data analysis method and device based on large language model and computer program product

Abstract

The invention provides a data analysis method and device based on a large language model, electronic equipment, a storage medium and a computer program product, relates to the technical field of computers, in particular to the technical field of artificial intelligence large models, natural language understanding and load analysis, and can be applied to load analysis scenes. According to the specific implementation scheme, the method comprises the steps of structuring operation data to obtain structured data of a plurality of preset dimensions; according to the structured data, generating multiple pieces of prompt data in one-to-one correspondence with the multiple preset dimensions; and generating a plurality of data analysis results in one-to-one correspondence with the plurality of preset dimensions according to the plurality of prompt data through the large language model. According to the load analysis method and device, by generating the multiple pieces of prompt data corresponding to the multiple preset dimensions one to one, the large language model is guided to carry out load analysis of the multiple preset dimensions in a targeted mode, and the comprehensiveness and efficiency of load analysis are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, specifically to the fields of large artificial intelligence models, natural language understanding, and load analysis technologies. In particular, it relates to a data analysis method, apparatus, electronic device, storage medium, and computer program product based on a large language model, which can be applied to load analysis scenarios. Background Art

[0002] Load analysis is a testing method for evaluating the behavior of a system under expected loads, and is usually used to determine the performance, stability, and scalability of the system under actual usage conditions. Current load analysis tools or performance detection tools can only perform load on one aspect of the load and reflect the load running state. Summary of the Invention

[0003] The present disclosure provides a data analysis method, apparatus, electronic device, storage medium, and computer program product based on a large language model.

[0004] According to a first aspect, there is provided a data analysis method based on a large language model, including: structuring running data to obtain structured data in a plurality of preset dimensions; generating a plurality of prompt data corresponding one-to-one to the plurality of preset dimensions according to the structured data; and generating a plurality of data analysis results corresponding one-to-one to the plurality of preset dimensions according to the plurality of prompt data through the large language model.

[0005] According to a second aspect, there is provided a data analysis apparatus based on a large language model, including: a structuring unit configured to structure running data to obtain structured data in a plurality of preset dimensions; a prompt generation unit configured to generate a plurality of prompt data corresponding one-to-one to the plurality of preset dimensions according to the structured data; and a data analysis unit configured to generate a plurality of data analysis results corresponding one-to-one to the plurality of preset dimensions according to the plurality of prompt data through the large language model.

[0006] According to a third aspect, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any implementation manner of the first aspect.

[0007] According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in any implementation manner of the first aspect.

[0008] According to a fifth aspect, there is provided a computer program product, including: a computer program which, when executed by a processor, implements the method described in any implementation manner of the first aspect.

[0009] According to the technology of the present disclosure, there is provided a data analysis method and apparatus based on a large language model. By structuring the operation data, structured data in multiple preset dimensions is obtained; according to the structured data, multiple prompt data corresponding one-to-one to the multiple preset dimensions are generated; through the large language model, according to the multiple prompt data, multiple data analysis results corresponding one-to-one to the multiple preset dimensions are generated, thereby providing a multi-dimensional load analysis method based on the large language model. By generating multiple prompt data corresponding one-to-one to the multiple preset dimensions, the large language model is guided to perform load analysis in multiple preset dimensions specifically, improving the comprehensiveness and efficiency of the load analysis.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0012] Figure 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

[0013] Figure 2 is a flowchart of an embodiment of the data analysis method based on a large language model according to the present disclosure;

[0014] Figure 3 is a schematic diagram of an application scenario of the data analysis method based on a large language model according to this embodiment;

[0015] Figure 4 is a flowchart of another embodiment of the data analysis method based on a large language model according to the present disclosure;

[0016] Figure 5 is a structural diagram of an embodiment of the data analysis apparatus based on a large language model according to the present disclosure;

[0017] Figure 6 is a schematic structural diagram of a computer system suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0019] In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0020] Figure 1 An exemplary architecture 100 is shown to which the data analysis method and apparatus based on a large language model of the present disclosure can be applied.

[0021] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The terminal devices 101, 102, 103 are communicatively connected to form a topology network, and the network 104 is used as a medium to provide a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0022] The terminal devices 101, 102, 103 may be hardware devices or software that support network connections for data interaction and data processing. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices that support network connections, information acquisition, interaction, display, processing, etc., including but not limited to smart phones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above-listed electronic devices. It may be implemented as, for example, multiple software or software modules for providing distributed services, or it may be implemented as a single software or software module. No specific limitation is made here.

[0023] The server 105 may be a server that provides various services. For example, it is a background processing server that obtains the running data of the running instances in the terminal devices 101, 102, 103 and performs load analysis in multiple preset dimensions through a large language model. Optionally, the server may feedback the data analysis result to the terminal device. As an example, the server 105 may be a cloud server.

[0024] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made here.

[0025] It should also be noted that the data analysis method based on the large language model provided by the embodiments of the present disclosure is generally executed by the server, but it does not exclude the possibility of being executed by the terminal device or by the server and the terminal device cooperating with each other. Correspondingly, each part (such as each unit) included in the data analysis device based on the large language model can be all set in the server, or all set in the terminal device, or can be respectively set in the server and the terminal device.

[0026] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0027] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. When the electronic device on which the data analysis method based on the large language model runs does not need to perform data transmission with other electronic devices, the system architecture can only include the electronic device (such as a terminal device or a server) on which the data analysis method based on the large language model runs. Figure 2 , Figure 2 Please refer to

[0028] which is a flowchart of a data analysis method based on the large language model provided by the embodiments of the present disclosure. The process 200 includes the following steps:

[0029] In this embodiment, the execution subject of the data analysis method based on the large language model (such as the Figure 1 server in

[0030] can obtain the running data from remote or local through a wired network connection or a wireless network connection, and structure the running data to obtain structured data in multiple preset dimensions.

[0031] Multiple preset dimensions can be determined specifically based on the running instances corresponding to the running data, or flexibly determined according to business requirements. Taking a software running instance as an example, the multiple preset dimensions are, for example, dimensions such as time-consuming distribution, CPU (Central Processing Unit) computing information, GPU (Graphics Processing Unit) computing information, disk IO (Input / Output), network IO information, and single-machine interconnection information.

[0032] As an example, the above-mentioned execution entity can perform structured processing on the running data based on rules and templates. First, obtain running data from sources such as devices, systems, and sensors. This data can be log files, performance metrics, status information, etc. Then, according to business requirements, clarify the dimensions that need to be structured, such as timestamp, device ID (Identity document), CPU usage, memory occupancy, network traffic, etc. Then, create rules and templates for each preset dimension to match and extract data. For example, define a regular expression to extract the timestamp, and define a field mapping rule to map the device identifier to the device ID. Then, remove noise and invalid data to ensure the quality of the data. For example, filter out null values, outliers, or records with incorrect formats. Then, use the rules and templates to extract features related to the preset dimensions from the original data. Then, map the extracted features to the preset dimensions to form structured data. Then, convert the mapped data into a structured data format, such as JSON (JavaScript Objectation Not), XML (Extensible Markup Language), or a table structure in a relational database. Finally, store the structured data in a database or data warehouse for subsequent querying and analysis.

[0033] After completing the structured processing, the above-mentioned execution entity can also check the integrity and accuracy of the structured data to ensure that there is corresponding data for each preset dimension. Optimize the rules and templates according to actual needs to improve the processing efficiency.

[0034] As another example, the above-mentioned execution entity can perform structured processing based on a machine learning model. First, obtain operation data from sources such as devices, systems, and sensors. Then, according to business requirements, clarify the dimensions that need to be structured. Then, annotate some of the data to create a training dataset, and the annotation content includes features related to the preset dimensions. Then, select a suitable machine learning model, such as a natural language processing model (for log parsing) or a time series model (for performance metric analysis). Then, use the annotated data to train the model so that it can automatically identify and extract features related to the preset dimensions. Then, remove noise and invalid data to ensure the quality of the data. Then, use the trained model to extract features related to the preset dimensions from the original data. Then, map the extracted features to the preset dimensions to form structured data. Then, convert the mapped data into a structured data format. Finally, store the structured data in a database or a data warehouse.

[0035] After completing the structured processing, the above-mentioned execution entity can also check the integrity and accuracy of the structured data. According to the actual effect, adjust the model parameters or retrain the model to improve the accuracy and efficiency of feature extraction.

[0036] Taking multiple preset dimensions such as time-consuming distribution, CPU computing information, GPU computing information, disk I / O information, network I / O information, and single-machine interconnection information as examples, the structured data is as follows:

[0037] Load time-consuming distribution:

[0038] The proportion of CPU computing time-consuming is 80%; the proportion of GPU computing time-consuming is 80%; the proportion of lock synchronization time-consuming is 15%; the proportion of H2D (Host to Device) time-consuming is 15%; the proportion of network time-consuming is 15%; the proportion of D2H (Host to Device) time-consuming is 15%; the proportion of disk time-consuming is 5%; the proportion of D2D (Device to Device) time-consuming is 5%.

[0039] CPU computing:

[0040] CPU utilization rate is 300%; the average utilization rate of a single core is 40%; the usage rate of the vector unit is 15%; the usage rate of the memory bandwidth is 15%.

[0041] Disk I / O:

[0042] Disk read bandwidth is 3 Gb (Gigabyte) / s; disk write bandwidth is 4 Gb / s; disk Util (Utilization) is 40%; disk queue saturation is 15%.

[0043] Network I / O:

[0044] Network transmission bandwidth: 2 Gb / s; network reception bandwidth: 15 Gb / s; transmission bandwidth utilization rate: 15%; reception bandwidth utilization rate: 15%.

[0045] GPU computing:

[0046] GPU utilization rate: 80%; SM (Streaming Multiprocessor) activation / thread resource utilization rate: 15% / 12%; TensorCore (Tensor Processing Core) utilization rate: 15%; HBM (High Bandwidth Memory) capacity / bandwidth utilization rate: 90% / 5%.

[0047] Single - machine interconnection:

[0048] PCIE (Peripheral Component Interconnect Express) bandwidth utilization rate: 80%; PCIE read / write throughput: xx / xx; NVLINK (NVIDIA NVLink) bandwidth utilization rate: 15%; NVLINK read / write throughput: xx / xx.

[0049] In some alternative implementation manners of this embodiment, the above - mentioned execution entity may execute the above - mentioned step 201 in the following manner: Through a large - language model, structure the operation data to obtain structured data in multiple preset dimensions.

[0050] As an example, the above - mentioned execution entity may combine the operation data and multiple preset dimensions to generate structured prompt data; input the structured prompt data into the large - language model, and the large - language model generates structured data in multiple preset dimensions based on its powerful natural - language understanding ability and logical - analysis ability.

[0051] In this implementation manner, performing the structuring operation based on the large - language model improves the efficiency and accuracy of the obtained structured data.

[0052] Step 202: Generate multiple prompt data corresponding one - to - one to multiple preset dimensions according to the structured data.

[0053] In this embodiment, the above - mentioned execution entity may generate multiple prompt data corresponding one - to - one to multiple preset dimensions according to the structured data.

[0054] As an example, the above-mentioned execution entity can generate prompt words (prompt data) based on a preset prompt word generation rule. Specifically, first, a prompt word template is defined for each of the multiple preset dimensions. For each of the multiple preset dimensions, the prompt word template corresponding to the preset dimension includes the structured data of the preset dimension and the expected analysis effect. For example, the prompt template for a preset dimension is:

[0055] The following data are the running data of the program. Please understand these data: {structured data}.

[0056] After analysis, give the running characteristics of the program and possible bottlenecks.

[0057] Among them, the specific content in {} is the structured data corresponding to the preset dimension.

[0058] Then, each dimension value in the structured data is filled into the corresponding prompt word template, and all the generated prompt words are combined into a complete set of prompt words to guide the large language model to perform multi-dimensional load analysis.

[0059] As another example, the above-mentioned execution entity can generate prompt words based on NLP (Natural Language Processing). Specifically, first, the structured data is annotated to mark each preset dimension and its corresponding value. Then, the annotated data is used to train a natural language processing model so that it can understand the semantics of the data and generate corresponding prompt words. Finally, the structured data is input into the trained model, and the model automatically identifies each dimension and its value and generates the corresponding prompt words. The generated prompt words are combined into a complete set of prompt words to guide the large language model to perform multi-dimensional load analysis.

[0060] In this implementation manner, the above-mentioned execution entity can also adjust the model parameters or retrain the model according to the actual effect to improve the accuracy and naturalness of prompt word generation.

[0061] In some optional implementation manners of this embodiment, the above-mentioned execution entity can execute the above step 202 in the following manner: for each of the multiple preset dimensions, according to the structured data corresponding to the preset dimension and the data analysis algorithm, generate the prompt data corresponding to the preset dimension.

[0062] Among them, the data analysis algorithm is used to represent the analysis logic and analysis direction for the structured data corresponding to the preset dimension.

[0063] In this implementation manner, for each of the multiple preset dimensions, its corresponding data analysis algorithm can be preset to obtain the corresponding relationship data between the preset dimension and the data analysis algorithm.

[0064] Continuing with examples in multiple dimensions such as time-consuming distribution, CPU computing information, GPU computing information, disk I / O information, network I / O information, and single-machine interconnection information, the respective data analysis algorithms are as follows:

[0065] 1. Time-consuming distribution analysis

[0066] Histogram analysis: Divide the time-consuming data into certain intervals, count the data frequencies in each interval, and visually display the distribution of time-consuming through a histogram to understand which intervals the time-consuming is concentrated in and whether there are long-tailed time-consuming, etc.

[0067] Kernel density estimation: Smooth the time-consuming data and estimate its probability density function to more accurately describe the distribution form of the time-consuming data and discover potential distribution patterns.

[0068] Box plot analysis: Display the five-number summary (minimum value, first quartile, median, third quartile, maximum value) of the time-consuming data through a box plot, identify outliers in the time-consuming data, and understand the dispersion degree and symmetry of the data.

[0069] 2. CPU computing information analysis

[0070] Performance counter analysis: Use the performance counters provided by the CPU to collect metrics such as the number of instruction executions, the number of clock cycles, and cache hit rate. By analyzing the values of these counters, evaluate the execution efficiency of the CPU and performance bottlenecks.

[0071] Flame graph analysis: Visualize the stack trace information of the CPU through a flame graph, display the execution time and call relationship of each function, and help quickly locate functions or code paths with high CPU consumption.

[0072] Heat map analysis: Display information such as the usage rate or temperature of the CPU in the form of a heat map to visually present the load distribution of the CPU at different time points or on different cores, and discover hot spots and load imbalance problems.

[0073] 3. GPU computing information analysis

[0074] GPU performance analysis tools: Use professional GPU performance analysis tools (such as NVIDIA's Nsight or AMD's CodeXL) to collect metrics such as the execution time, memory bandwidth utilization rate, and computing unit utilization rate of the GPU, and analyze the performance bottlenecks and optimization directions of the GPU.

[0075] CUDA (Compute Unified Device Architecture) Event and Counter Analysis: For GPU applications programmed with CUDA, CUDA events and counters are used to accurately measure the execution time of GPU kernels, memory transfer time, etc., and evaluate the execution efficiency and resource utilization of the GPU.

[0076] GPU Visualization Tools: Utilize GPU visualization tools (such as NVIDIA's Visual Profiler) to graphically display the performance data of the GPU, helping to intuitively understand the execution process and resource usage of the GPU.

[0077] 4. Disk I / O Information Analysis

[0078] I / O Wait Time Analysis: Statistically analyze the wait time distribution of disk I / O requests. By calculating metrics such as average wait time and maximum wait time, evaluate the response speed and performance of the disk.

[0079] Throughput Analysis: Calculate the data transfer volume of the disk per unit time, analyze the changing trend of the disk's read / write throughput over time, and evaluate the bandwidth utilization of the disk.

[0080] Queue Length Analysis: Monitor the length of the disk I / O request queue, analyze the changes in the queue length, and determine whether there is disk overload or performance bottleneck.

[0081] 5. Network I / O Information Analysis

[0082] Traffic Analysis: Statistically analyze the incoming and outgoing traffic of the network interface. By plotting the curve of traffic over time, analyze the busyness and traffic pattern of the network.

[0083] Packet Loss Rate Analysis: Calculate the packet loss rate of network packets, evaluate the reliability of the network, and discover problems in network transmission.

[0084] Latency Analysis: Measure the transmission latency of network packets, analyze the distribution and changing trend of latency, and evaluate the response speed and performance of the network.

[0085] 6. Standalone Interconnection Information Analysis

[0086] Network Topology Analysis: Analyze the network connection topology between a standalone device and other devices, understand the network layout and connection relationships, and evaluate the reliability and redundancy of the network.

[0087] Bandwidth Utilization Analysis: Calculate the bandwidth utilization of the communication link between a standalone device and other devices, analyze the bandwidth usage, and determine whether there is a bandwidth bottleneck.

[0088] Data transfer efficiency analysis: By measuring metrics such as the throughput and latency of data transfer, evaluate the data transfer efficiency between a single machine and other devices, and identify performance issues during the transfer process.

[0089] It should be noted that the preset data analysis algorithms corresponding to the preset dimensions can be either existing data analysis algorithms or data analysis algorithms obtained by improving or combining existing data analysis algorithms.

[0090] As an example, the prompt data corresponding to the preset dimension is as follows:

[0091] The following data is the running data of the program. Please understand these data: {structured data}.

[0092] Based on the understood content, analyze according to the following analysis method: {data analysis algorithm}.

[0093] After analysis, present the running characteristics of the program and possible bottlenecks.

[0094] In this implementation, during the generation process of the prompt data, in addition to considering the structured data corresponding to the preset dimension, the data analysis methods corresponding to the preset dimension are also considered, providing targeted data analysis methods for each preset dimension, improving the accuracy of the prompt data and the comprehensiveness of the information, and helping to improve the accuracy of the data analysis results of the large language model based on the prompt data.

[0095] In some alternative implementation manners of this embodiment, the above execution entity can execute the generation process of the prompt data in the following manner: Generate the prompt data corresponding to the preset dimension according to the structured data, data analysis algorithm, and knowledge data corresponding to the preset dimension.

[0096] For example, the knowledge data can be the knowledge data corresponding to the preset dimension in the database, such as the summary and record of the analysis of past loads. The analysis cases are the detailed records of the data analysis process for specific loads, including background information, data collection methods, analysis processes, discovered problems, solutions, and final analysis results, etc. These cases can provide reference and inspiration for subsequent load analysis.

[0097] The analysis cases have the following functions: Provide reference for load analysis in similar scenarios, help quickly locate and solve problems; Improve the analysis ability and efficiency by summarizing past analysis experiences; Provide learning and training materials for team members to improve the overall technical level of the team; Provide data support for management to help make more informed decisions.

[0098] As an example, the prompt data corresponding to the preset dimension is as follows:

[0099] The following data are the running data of the program. Please understand these data: {structured data}.

[0100] Based on the understood content, and referring to the corresponding analysis cases in the knowledge base, perform analysis according to the following analysis method: {data analysis algorithm}.

[0101] After analysis, give the running characteristics of the program and possible bottlenecks.

[0102] In this implementation, during the generation of the prompt data, in addition to considering the structured data and data analysis methods corresponding to the preset dimensions, the knowledge data corresponding to the preset dimensions should also be considered, which further improves the accuracy of the prompt data and the comprehensiveness of the information, and helps to improve the accuracy of the data analysis results of the large language model based on the prompt data.

[0103] Step 203: Through the large language model, generate multiple data analysis results corresponding one by one to multiple preset dimensions according to multiple prompt data.

[0104] In this embodiment, the above-mentioned execution subject can generate multiple data analysis results corresponding one by one to multiple preset dimensions through the large language model according to multiple prompt data.

[0105] The large language model (LLM, Large Language Model) is an artificial intelligence model based on deep learning, mainly used for processing and generating natural language text. The large language model is usually trained on hundreds of millions or even hundreds of billions of text data, so it can capture rich language patterns and semantic information, can generate fluent and natural text, and is applicable to a variety of application scenarios, such as text generation, question answering, translation, etc. Through fine-tuning, it can be adapted to specific tasks, such as sentiment analysis, summary generation, dialogue systems, etc. Moreover, it can understand context information and generate text that conforms to the context.

[0106] As an example, the above-mentioned execution subject can input multiple prompt data into the large language model at the same time, and with the help of the powerful natural language understanding ability and logical analysis ability of the large language model, generate multiple data analysis results corresponding one by one to multiple preset dimensions, or the above-mentioned execution subject can input multiple prompt data into the large language model in sequence, and with the help of the powerful natural language understanding ability and logical analysis ability of the large language model, generate multiple data analysis results corresponding one by one to multiple preset dimensions in sequence.

[0107] In some optional implementation manners of this embodiment, the above-mentioned execution subject can execute the above-mentioned step 203 in the following manner: Through the large language model, generate multiple data analysis results according to multiple prompt data and preset system prompt data.

[0108] Among them, the system prompt data is the guiding framework inside the model, which is used to ensure that the model follows specific rules and processes during the analysis. It is usually preset by the team that develops or maintains the model, helps the model understand the background and objectives of the task, and is used to guide the data analysis process of the large language model.

[0109] As an example, the system prompt words are:

[0110] # Role Task

[0111] As an application of the performance analysis Agent, your main task is to understand the information given by the user and query the knowledge base, and then answer the user's questions based on this information.

[0112] 1. For the case where the user uploads a document and requests an answer based on the uploaded document, you need to understand it deeply, combine the relevant knowledge in the knowledge base, and use the large model to give the best answer. For example, when the user uploads program monitoring data and hopes to get bottleneck conclusions and optimization suggestions, it is necessary to analyze the operating characteristics of the program according to the program monitoring data in the user's attachment, such as the usage rules of CPU / memory / IO / network resources, hot functions, etc., and then query the knowledge from the knowledge base. Based on the knowledge in the knowledge base and the knowledge of the program operating characteristics obtained from the previous analysis, give the program bottleneck points and optimization suggestions.

[0113] 2. For the case where the user does not upload a document or the user's prompt does not require an answer based on the uploaded document, you should give an accurate and targeted answer based on the user's question and the knowledge in the knowledge base, also based on the large model.

[0114] When answering the user's questions, ensure the accuracy and clarity of the answers to meet the user's needs. At the same time, you also need to pay attention to protecting the privacy of the content of the documents uploaded by the user.

[0115] # Requirements and Limitations

[0116] * Understand accurately and handle the content of the documents uploaded by the user confidentially.

[0117] * When answering the user's questions, you must provide clear and concise answers based on true and accurate knowledge, and avoid complex and redundant expressions.

[0118] * Make full use of the capabilities of the large model during the process of querying the knowledge base and answering questions to improve the accuracy and pertinence of the answers.

[0119] In this implementation, the large language model generates multiple data analysis results under the joint guidance of multiple prompt data and the preset system prompt data, improving the accuracy and comprehensiveness of the data analysis process.

[0120] Continue to refer toFigure 3 , Figure 3 It is a schematic diagram 300 of an application scenario of the data analysis method based on a large language model according to this embodiment. First, the server 301 collects operation data from the terminal device 303 of the user 302; then, the operation data is structured to obtain structured data in multiple preset dimensions, where the multiple preset dimensions include time-consuming distribution, CPU computing information, GPU computing information, disk I / O information, network I / O information, and single-machine interconnection information; then, according to the structured data, multiple prompt data corresponding one-to-one to the multiple preset dimensions are generated; finally, through the large language model, according to the multiple prompt data, multiple data analysis results corresponding one-to-one to the multiple preset dimensions are generated.

[0121] In this embodiment, a data analysis method and device based on a large language model are provided. By structuring operation data, structured data in multiple preset dimensions is obtained; according to the structured data, multiple prompt data corresponding one-to-one to the multiple preset dimensions are generated; through the large language model, according to the multiple prompt data, multiple data analysis results corresponding one-to-one to the multiple preset dimensions are generated, thereby providing a multi-dimensional load analysis method based on a large language model. By generating multiple prompt data corresponding one-to-one to multiple preset dimensions, it guides the large language model to perform load analysis in multiple preset dimensions, improving the comprehensiveness and efficiency of load analysis.

[0122] In some alternative implementation manners of this embodiment, the above execution subject can also perform the following operations: Through the large language model, a summary analysis operation is performed on multiple data analysis results to obtain a summary analysis result.

[0123] The summary analysis represents further performing load analysis on the operation data based on the multiple data analysis results as data basis.

[0124] As an example, the above execution subject can combine multiple data analysis results with guiding data representing the analysis direction and analysis logic of the summary analysis to generate summary analysis prompt data; and then input the summary analysis prompt data into the large language model to generate a summary analysis result.

[0125] The guiding data representing the analysis direction and analysis logic of the summary analysis can be preset guiding data or guiding data flexibly set according to the received setting operations in real time.

[0126] In this implementation manner, the above execution subject can perform summary load analysis on the basis of multi-dimensional load analysis through the large language model, improving the comprehensiveness and in-depthness of load analysis and expanding the user's information acquisition degree.

[0127] In some alternative implementation manners of this embodiment, the above-mentioned execution entity may execute the summary analysis process in the following manner: First, according to multiple data analysis results, as well as the data analysis method and knowledge data corresponding to the summary analysis operation, generate summary analysis prompt data; then, through a large language model, generate a summary analysis result according to the summary analysis prompt data.

[0128] The data analysis methods corresponding to the summary analysis operation include but are not limited to:

[0129] Comprehensive scoring method: By assigning weights to each dimension, calculating the score of each dimension, and then synthesizing the scores of each dimension to obtain a total score.

[0130] Cluster analysis: Cluster the data of multiple dimensions to identify similar load patterns or performance bottlenecks.

[0131] Association rule mining: Discover the association relationships between different dimensions through the association rule mining algorithm.

[0132] Principal component analysis (PCA): Through dimensionality reduction technology, extract the main features, simplify the data structure, and highlight the key factors.

[0133] Time series analysis: Analyze the change trends of data in multiple dimensions over time to identify anomalies and periodic patterns.

[0134] Decision tree analysis: Identify the key factors affecting system performance through the decision tree model.

[0135] Regression analysis: Establish a mathematical relationship between different dimensions through a regression model to predict system performance.

[0136] Anomaly detection: Identify abnormal load patterns or performance bottlenecks through anomaly detection algorithms.

[0137] The knowledge data corresponding to the summary analysis operation may be the knowledge data in the knowledge base for performing summary analysis, such as the summary and record of the summary analysis of past load data in the knowledge base.

[0138] In this implementation manner, a specific implementation process of summary analysis is provided, which further improves the accuracy and comprehensiveness of the summary analysis result.

[0139] In some alternative implementation manners of this embodiment, the above-mentioned execution entity may also perform the following operation: Through a large language model, generate operation optimization data for the operation instance corresponding to the operation data according to the summary analysis result.

[0140] As an example, the above-mentioned execution entity can combine the summary analysis result and the prompt words used to instruct the large language model to determine the operation optimization data to generate optimization prompt words; input the optimization prompt words into the large language model to generate operation optimization data for the operation instance corresponding to the operation data.

[0141] In this implementation manner, on the basis of determining the summary analysis result, operation optimization suggestions are further generated according to the summary analysis result, improving the richness of information acquisition.

[0142] In some optional implementation manners of this embodiment, before performing the above-mentioned step 201, the above-mentioned execution entity may further perform the following operations:

[0143] First, determine the operation instance according to the received selection operation; then, collect the operation data of the operation instance through a preset data collection component.

[0144] As an example, the above-mentioned execution entity can display a display interface for the user to perform selection operations to the user. Multiple candidate operation instances are displayed on the display interface; the user can select an operation instance from multiple candidate operation instances based on action instructions such as clicking and touching, voice instructions, etc.; the above-mentioned execution entity can collect the operation data of the operation instance through a preset data collection component.

[0145] The preset data collection component is, for example:

[0146] Flume: Used for the collection of large-scale log data.

[0147] Logstash: Used for the collection and processing of logs.

[0148] Kafka: Used for the collection of real-time data streams.

[0149] Telegraf: Used for collecting metric data of systems and applications.

[0150] Prometheus: Used for monitoring and collecting time series data.

[0151] In this implementation manner, the above-mentioned execution entity can automatically collect the operation instance selected by the user through a preset data collection component for load analysis, improving the convenience and automation degree of the load analysis process.

[0152] Continue to refer to Figure 4 , which shows a schematic flow 400 of another embodiment of the data analysis method based on a large language model according to the present disclosure. In the flow 400, the following steps are included:

[0153] Step 401, determine the operation instance according to the received selection operation.

[0154] Step 402: Collect the operation data of the operation instance through a preset data collection component.

[0155] Step 403: Structure the operation data to obtain structured data in multiple preset dimensions.

[0156] Step 404: For each preset dimension among the multiple preset dimensions, generate prompt data corresponding to the preset dimension according to the structured data, data analysis algorithm, and knowledge data corresponding to the preset dimension.

[0157] Among them, the data analysis algorithm is used to represent the analysis logic and analysis direction for the structured data corresponding to the preset dimension.

[0158] Step 405: Through a large language model, generate multiple data analysis results corresponding one-to-one to the multiple preset dimensions according to the multiple prompt data.

[0159] Step 406: Generate summary analysis prompt data according to the multiple data analysis results, as well as the data analysis method and knowledge data corresponding to the summary analysis operation.

[0160] Step 407: Through a large language model, generate a summary analysis result according to the summary analysis prompt data.

[0161] Step 408: Through a large language model, generate operation optimization data for the operation instance corresponding to the operation data according to the summary analysis result.

[0162] The process 400 of the data analysis method based on a large language model in this embodiment specifically illustrates the collection process of operation data, the load analysis process in multiple preset dimensions, the summary analysis process, and the optimization data generation process, further improving the comprehensiveness and efficiency of load analysis.

[0163] Continue to refer to Figure 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a data analysis device based on a large language model. This system embodiment corresponds to Figure 2 the method embodiment shown, and this system can be specifically applied to various electronic devices.

[0164] As shown in Figure 5 , the data analysis device 500 based on a large language model includes: a structuring unit 501 configured to structure operation data to obtain structured data in multiple preset dimensions; a prompt generation unit 502 configured to generate multiple prompt data corresponding one-to-one to the multiple preset dimensions according to the structured data; and a data analysis unit 503 configured to generate multiple data analysis results corresponding one-to-one to the multiple preset dimensions through a large language model according to the multiple prompt data.

[0165] In some alternative implementation manners of this embodiment, the prompt generation unit 502 is further configured to: for each of a plurality of preset dimensions, generate prompt data corresponding to the preset dimension according to the structured data corresponding to the preset dimension and a data analysis algorithm, where the data analysis algorithm is used to represent the analysis logic and analysis direction for the structured data corresponding to the preset dimension.

[0166] In some alternative implementation manners of this embodiment, the prompt generation unit 502 is further configured to: generate prompt data corresponding to the preset dimension according to the structured data corresponding to the preset dimension, a data analysis algorithm, and knowledge data.

[0167] In some alternative implementation manners of this embodiment, the above device further includes: a summary analysis unit (not shown in the figure), configured to: perform a summary analysis operation on a plurality of data analysis results through a large language model to obtain a summary analysis result.

[0168] In some alternative implementation manners of this embodiment, the summary analysis unit (not shown in the figure) is further configured to: generate summary analysis prompt data according to a plurality of data analysis results, and the data analysis method and knowledge data corresponding to the summary analysis operation; generate a summary analysis result through a large language model according to the summary analysis prompt data.

[0169] In some alternative implementation manners of this embodiment, the above device further includes: an optimization unit (not shown in the figure), configured to: generate operation optimization data for an operation instance corresponding to operation data through a large language model according to the summary analysis result.

[0170] In some alternative implementation manners of this embodiment, the data analysis unit 503 is further configured to: generate a plurality of data analysis results through a large language model according to a plurality of prompt data and preset system prompt data, where the system prompt data is used to guide the data analysis process of the large language model.

[0171] In some alternative implementation manners of this embodiment, the structuring unit 501 is further configured to: structure operation data through a large language model to obtain structured data of a plurality of preset dimensions.

[0172] In some alternative implementation manners of this embodiment, the above device further includes: a selection unit (not shown in the figure), configured to determine an operation instance according to a received selection operation; a data collection unit (not shown in the figure), configured to collect operation data of the operation instance through a preset data collection component.

[0173] In this embodiment, a data analysis device based on a large language model is provided. By structuring the operation data, structured data in multiple preset dimensions is obtained; according to the structured data, multiple prompt data corresponding one-to-one to the multiple preset dimensions are generated; through the large language model, according to the multiple prompt data, multiple data analysis results corresponding one-to-one to the multiple preset dimensions are generated, thus providing a multi-dimensional load analysis device based on a large language model. By generating multiple prompt data corresponding one-to-one to the multiple preset dimensions, the large language model is guided to perform load analysis in multiple preset dimensions, improving the comprehensiveness and efficiency of the load analysis.

[0174] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to implement the data analysis method based on a large language model described in any of the above embodiments.

[0175] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions for enabling a computer to implement the data analysis method based on a large language model described in any of the above embodiments when executed.

[0176] The embodiment of the present disclosure provides a computer program product, which can implement the data analysis method based on a large language model described in any of the above embodiments when executed by a processor.

[0177] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0178] As Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to computer programs stored in a read-only memory (ROM) 602 or computer programs loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0179] Multiple components in device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disc, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0180] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as a data analysis method based on a large language model. For example, in some embodiments, the data analysis method based on a large language model can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the data analysis method based on a large language model described above can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the data analysis method based on a large language model in any other appropriate manner (e.g., by means of firmware).

[0181] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0182] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable large language model-based data analysis devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0183] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0184] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0185] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0186] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to address the defects of difficult management and weak business scalability existing in traditional physical hosts and virtual private server (VPS) services; or it can be a server of a distributed system, or a server combined with blockchain.

[0187] According to the technical solution of the embodiment of the present disclosure, a data analysis method and device based on a large language model are provided. By structuring the operation data, structured data in multiple preset dimensions is obtained; according to the structured data, multiple prompt data corresponding one-to-one to the multiple preset dimensions are generated; through the large language model, according to the multiple prompt data, multiple data analysis results corresponding one-to-one to the multiple preset dimensions are generated, thereby providing a multi-dimensional load analysis method based on the large language model. By generating multiple prompt data corresponding one-to-one to the multiple preset dimensions, the large language model is guided to perform load analysis in multiple preset dimensions specifically, improving the comprehensiveness and efficiency of the load analysis.

[0188] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution provided by the present disclosure can be achieved, and no limitation is made herein.

[0189] The above specific implementation manners do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A data analysis method based on a large language model, comprising: Structuring the operation data to obtain structured data in multiple preset dimensions; Generating multiple pieces of prompt data corresponding one-to-one to the multiple preset dimensions according to the structured data; Generating multiple data analysis results corresponding one-to-one to the multiple preset dimensions through a large language model according to the multiple pieces of prompt data.

2. The method according to claim 1, wherein The generating multiple pieces of prompt data corresponding one-to-one to the multiple preset dimensions according to the structured data includes: For each of the multiple preset dimensions, generating the prompt data corresponding to the preset dimension according to the structured data corresponding to the preset dimension and a data analysis algorithm, where the data analysis algorithm is used to represent the analysis logic and analysis direction for the structured data corresponding to the preset dimension.

3. The method according to claim 2, wherein The generating the prompt data corresponding to the preset dimension according to the structured data corresponding to the preset dimension and the data analysis algorithm includes: Generating the prompt data corresponding to the preset dimension according to the structured data corresponding to the preset dimension, the data analysis algorithm, and knowledge data.

4. The method according to any one of claims 1-3, wherein, It further includes: Performing a summary analysis operation on the multiple data analysis results through the large language model to obtain a summary analysis result.

5. The method according to claim 4, wherein, The performing a summary analysis operation on the multiple data analysis results through the large language model to obtain a summary analysis result includes: Generating summary analysis prompt data according to the multiple data analysis results, the data analysis method corresponding to the summary analysis operation, and knowledge data; Generating the summary analysis result through the large language model according to the summary analysis prompt data.

6. The method according to claim 4, wherein It further includes: Generating operation optimization data for the operation instance corresponding to the operation data through the large language model according to the summary analysis result.

7. The method according to claim 1, wherein, The generating multiple data analysis results corresponding one-to-one to the multiple preset dimensions through the large language model according to the multiple pieces of prompt data includes: Generating the multiple data analysis results through the large language model according to the multiple pieces of prompt data and preset system prompt data, where the system prompt data is used to guide the data analysis process of the large language model.

8. The method according to claim 1, wherein The structuring the operation data to obtain structured data in multiple preset dimensions includes: Structuring the operation data through the large language model to obtain the structured data in the multiple preset dimensions.

9. The method according to claim 1, wherein Before the structuring the operation data to obtain structured data in multiple preset dimensions, it further includes: Determining an operation instance according to a received selection operation; Collecting the operation data of the operation instance through a preset data collection component.

10. A data analysis device based on a large language model, comprising: A structuring unit configured to structure operation data to obtain structured data in multiple preset dimensions; A prompt generation unit configured to generate multiple pieces of prompt data corresponding one-to-one to the multiple preset dimensions according to the structured data; A data analysis unit, configured to generate a plurality of data analysis results corresponding one-to-one to the plurality of preset dimensions according to the plurality of prompt data through a large language model.

11. An electronic device, characterized in that, Comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

13. A computer program product, comprising: A computer program, which when executed by a processor implements the method according to any one of claims 1-9.

Images