Data monitoring method and device for log analysis platform, equipment and medium

Abstract

The invention discloses a data monitoring method and device for a log analysis platform, equipment and a medium, and the method comprises the steps: receiving text description data inputted by a user through a large language model, carrying out the semantic understanding of the text description data, and generating key parameters corresponding to a monitoring rule of the log analysis platform according to a semantic understanding result; converting the key parameters into monitoring rule interface request parameters through a large language model application development platform; and creating a target monitoring rule according to the API interface of the log analysis platform and the interface request parameter, and monitoring the operation data of the log analysis platform by using the target monitoring rule. According to the technical scheme, the time cost and the labor cost consumed in the monitoring rule creating process can be saved, the technical difficulty in the monitoring process of the log analysis platform is reduced, and the updated monitoring requirement in the log analysis platform is responded in time.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data monitoring method, apparatus, device, and medium for a log analysis platform. Background Technology

[0002] Currently, with the development of enterprise digitalization and online business scale, it is particularly important to monitor and perform security analysis on the operational data in the log analysis platform.

[0003] In existing technologies, the creation of monitoring rules in log analysis platforms is mainly done manually. For example, the Search Processing Language (SPL) and alarm conditions are written manually, the syntax and field mapping relationships in the monitoring rules are corrected manually, and the monitoring rules that have been created are maintained manually.

[0004] However, existing methods for creating monitoring rules are time-consuming and require significant time and manpower; they also have a high technical threshold, and due to the level of SPL proficiency, junior engineers generally have a first-time configuration success rate of less than 55%, requiring repeated iterations; and they are slow to respond to data specifications updates in the platform, such as adding or renaming fields, adjusting thresholds, and expanding alarm levels, which typically require 24-72 hours to complete the update and retest. Summary of the Invention

[0005] This invention provides a data monitoring method, apparatus, device, and medium for a log analysis platform, which can save time and manpower costs in the process of creating monitoring rules, reduce the technical difficulty of the monitoring process of the log analysis platform, and respond promptly to the updated monitoring needs in the log analysis platform.

[0006] According to one aspect of the present invention, a data monitoring method for a log analysis platform is provided, the method comprising:

[0007] The system receives text description data input by the user through a large language model, performs semantic understanding on the text description data, and generates key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results.

[0008] The key parameters are converted into monitoring rule interface request parameters through the large language model application development platform.

[0009] Based on the application programming interface (API) of the log analysis platform and the interface request parameters, target monitoring rules are created, and the target monitoring rules are used to monitor the running data of the log analysis platform.

[0010] Optionally, key parameters corresponding to the monitoring rules of the log analysis platform are generated based on the semantic understanding results, including:

[0011] Based on pre-configured parameter constraints, generate prompt information corresponding to the large language model;

[0012] Based on the semantic understanding of the text description data and the prompt information, key parameters corresponding to the monitoring rules of the log analysis platform are generated.

[0013] Optionally, the pre-configured parameter constraints include at least one of the following:

[0014] Key JSON data templates corresponding to the monitoring rules of the log analysis platform;

[0015] The log analysis platform searches for usage guidelines for the corresponding languages.

[0016] The field specifications corresponding to the key JSON data;

[0017] Users select the target data source or target field for the monitoring scenario of the log analysis platform;

[0018] The standard data template corresponding to the output content of the large language model.

[0019] Optionally, the key parameters can be converted into monitoring rule interface request parameters through a large language model application development platform, including:

[0020] Generate a parameter conversion script based on the standard parameter format corresponding to the interface created by the monitoring rules in the log analysis platform.

[0021] Using the large language model application development platform, the conversion script is used to convert key parameters into monitoring rule interface request parameters.

[0022] Optionally, after creating the target monitoring rule, the following may also be included:

[0023] If the creation of the target monitoring rule fails, a failure reason corresponding to the target monitoring rule is generated and fed back to the user through a visual interface.

[0024] Optionally, before using the target monitoring rules to monitor the operational data of the log analysis platform, the following steps are also included:

[0025] The logic of the target monitoring rules is validated for reasonableness.

[0026] If the target monitoring rule is successfully verified, then the target monitoring rule is enabled;

[0027] If the target monitoring rule fails to be verified, the target monitoring rule is modified and then the modified target monitoring rule is enabled.

[0028] According to another aspect of the present invention, a data monitoring device for a log analysis platform is provided, the device comprising:

[0029] The parameter generation module is used to receive text description data input by the user through a large language model, perform semantic understanding on the text description data, and generate key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results.

[0030] The parameter conversion module is used to convert the key parameters into monitoring rule interface request parameters through the large language model application development platform.

[0031] The rule creation module is used to create target monitoring rules based on the API interface of the log analysis platform and the interface request parameters, and to use the target monitoring rules to monitor the operation data of the log analysis platform.

[0032] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0033] At least one processor; and

[0034] A memory communicatively connected to the at least one processor; wherein,

[0035] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to execute the data monitoring method of the log analysis platform according to any embodiment of the present invention.

[0036] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data monitoring method of the log analysis platform according to any embodiment of the present invention.

[0037] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the data monitoring method of the log analysis platform according to any embodiment of the present invention.

[0038] The technical solution provided by this invention receives text description data input by the user through a large language model, performs semantic understanding on the text description data, generates key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results, converts the key parameters into monitoring rule interface request parameters through a large language model application development platform, creates target monitoring rules based on the API interface and interface request parameters of the log analysis platform, and uses the target monitoring rules to monitor the running data of the log analysis platform. This technical means can save the time and manpower costs consumed in the monitoring rule creation process, reduce the technical difficulty of the log analysis platform monitoring process, and respond promptly to the updated monitoring needs in the log analysis platform.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a data monitoring method for a log analysis platform provided according to an embodiment of the present invention;

[0042] Figure 2 This is a flowchart of another data monitoring method for a log analysis platform provided according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the structure of a data monitoring device for a log analysis platform according to an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the data monitoring method of the log analysis platform in this embodiment of the invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] Figure 1 This is a flowchart illustrating a data monitoring method for a log analysis platform provided in an embodiment of the present invention. This embodiment is applicable to automatically creating monitoring rules for a log analysis platform and using automated monitoring rules to quickly locate platform anomalies. This method can be executed by a data monitoring device of the log analysis platform, which can be implemented in hardware and / or software and configured in an electronic device. Figure 1 As shown, the method includes:

[0048] Step 110: Receive text description data input by the user through a large language model, perform semantic understanding on the text description data, and generate key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results.

[0049] In this embodiment, the log analysis platform can specifically be LogEasy, used to implement functions such as centralized log management, security auditing, and business analysis. For the log analysis platform, a Large Language Model (LLM) can be used to perform semantic understanding on the text description data input by the user and extract the key parameters required to create monitoring rules.

[0050] In a specific embodiment, the text description data input by the user can be requirement data related to the log analysis platform monitoring, such as "intending to implement SPL statistical monitoring, paying attention to whether the number of Web application firewall blocking events suddenly increases, specifying the data source as appname:waftag:attack, counting the number of blocking times for each attack type in the last ten minutes, and grouping by attack_type, executing once every five minutes; if a certain type of attack is blocked 20 times in ten minutes, a medium alert is given, 50 times a high alert is given, and 100 times a critical alert is given." The large language model can perform semantic analysis on the above text description data, extract key elements such as task name, task description, scheduling plan, time range, and core query conditions, and then convert the key elements into a structured set of parameters.

[0051] Optionally, after identifying the data source and analysis target corresponding to the text description data, the large language model can automatically generate efficient SPL query statements that conform to the syntax specifications of the log analysis platform.

[0052] Step 120: Convert the key parameters into monitoring rule interface request parameters through the large language model application development platform.

[0053] In this embodiment, the large language model application development platform can specifically be a DefineModify (Dify) orchestration platform, which provides visual and controllable workflow orchestration capabilities and converts the key parameters output by the large language model in the above steps into JSON structure parameters required by the log analysis platform API interface.

[0054] Step 130: Based on the API interface of the log analysis platform and the interface request parameters, create target monitoring rules and use the target monitoring rules to monitor the running data of the log analysis platform.

[0055] In this step, specifically, the interface request parameters output by the large language model application development platform in step 120 can be used as a custom instruction ai_gen_alert. The log analysis platform can call the custom instruction to create a corresponding target monitoring rule, and then use the target monitoring rule to monitor the running data of the log analysis platform and quickly locate abnormal running data of the log analysis platform.

[0056] The technical solution provided by this invention receives user-input text description data through a large language model, performs semantic understanding on the text description data, generates key parameters corresponding to monitoring rules for a log analysis platform based on the semantic understanding results, converts the key parameters into monitoring rule interface request parameters through a large language model application development platform, creates target monitoring rules based on the API interface and interface request parameters of the log analysis platform, and uses the target monitoring rules to monitor the operational data of the log analysis platform. This provides a way to automatically create monitoring rules for a log analysis platform, supports interactive rule creation via natural language processing, and achieves automatic conversion from natural language processing to rule JSON data. Compared with the existing technology of manually creating monitoring rules, this method saves time and manpower costs in the monitoring rule creation process, reduces the technical difficulty of the log analysis platform monitoring process, and allows for timely response to updated monitoring needs in the log analysis platform.

[0057] Figure 2 A flowchart of another data monitoring method for a log analysis platform provided in an embodiment of the present invention is shown below. Figure 2 As shown, the method includes:

[0058] Step 210: Receive text description data input by the user through a large language model, and perform semantic understanding on the text description data.

[0059] Step 220: Generate prompt information corresponding to the large language model based on the pre-configured parameter constraints.

[0060] In this embodiment, the parameter constraints are strong constraints corresponding to the JSON structure parameters required by the log analysis platform API interface.

[0061] In one embodiment of this example, the pre-configured parameter constraints include at least one of the following: a key JSON data template corresponding to the log analysis platform monitoring rules; usage rules for the search processing language in the log analysis platform; field rules for the key JSON data; the target data source or target field selected by the user for the log analysis platform monitoring scenario; and a standard data template corresponding to the output content of the large language model.

[0062] In a specific embodiment, the key JSON data template corresponding to the monitoring rules of the log analysis platform can be:

[0063] {

[0064] "alert_rule_name": "(Chinese name of the monitoring rule)"

[0065] "alert_rule_desc": "(A concise Chinese description of the monitoring scenario)"

[0066] "alert_rule_crontab": "0 0...",

[0067] "alert_rule_query": "(Log SPL statistical query for this monitoring scenario)"

[0068] "check_condition": {

[0069] "timerange": "-10m"

[0070] "operator": ")"

[0071] "thresholds": [

[0072] {

[0073] "level": "info"

[0074] "expression": {

[0075] "operator_type": "logic_and",

[0076] "children": [

[0077] {

[0078] "field": "(statistical field name)"

[0079] "operator_type": ")=",

[0080] "value": "(threshold)"

[0081] } ]

[0083] }

[0084] } ]

[0086] }

[0087] }

[0088] In another specific embodiment, the usage rules information corresponding to SPL in the log analysis platform can be:

[0089] The statements before the first pipe symbol "|" are the "search criteria"; the "search criteria" use a colon ":" to separate field names and field values; fields in the "AND" relation are concatenated using "AND"; fields in the "OR" relation are concatenated using "OR"; fields in the "NOT" relation are concatenated using "NOT"; the "IN" relation is not supported (it needs to be converted to the corresponding "OR" relation); a newline character "\n" is added at the end of each line for easy maintenance;

[0090] The "Search criteria" support multiple search methods, including specifying fields, regular expressions, wildcards, numeric ranges, keyword search, phrase search, and grouping priority.

[0091] The search criteria must not directly use the relational operators == / != (if the user input contains these operators, they must be converted as required):

[0092] When a field value in the "Search criteria" contains a space, escape it with a backslash (\).

[0093] Splunk subqueries are enclosed in square brackets [...];

[0094] LogEasy SPL subqueries are enclosed in two pairs of square brackets [...];

[0095] LogEasy SPL allows you to specify a time range using start time and end time during the search phase (the time rounding symbol is / , and the supported rounding methods are s / m / h / d / w / M / y, which must also be followed by sub-searches).

[0096] If the user explicitly requests start and end times, write them at the beginning of the search criteria using this syntax; if the user does not request them, there is no need to force the addition of start time or end time.

[0097] The first pipe symbol | indicates "streaming language";

[0098] The function to count the number of occurrences is written as count()asxxx_cnt;

[0099] In stats, multiple statistical functions and grouping fields are separated by commas;

[0100] In stats, replace min() with early(); replace max() with latest().

[0101] Statistical functions are only allowed to include: count(), values(), dc(), earliest(), and latest().

[0102] Only the following commands are allowed: append, dedup, eval, join, limit, mvcombine, mvexpand, sort, stats, table, top, where;

[0103] In the WHERE clause: replace LIKE() with MATCH(); replace the logical operators AND with &&, OR with ||, NOT with !, and replace equality / inequality with == / !=; the rest follow Java operators.

[0104] Splunk defines data sources using sourcetype, while LogEasy defines data sources using appname and tag.

[0105] appname:nginxtag:access is equivalent to Splunk sourcetype=nginx:access;

[0106] The Splunk time field _time corresponds to the timestamp.

[0107] In another specific embodiment, the field specification information corresponding to the key JSON data can be:

[0108] Use strings in the format "second-minute-hour-day-month-week-year"; use "?" to indicate if week and hour are not specified; generate an executable log file (e.g., LogEasySPL); the `stats` function produces one or more "statistical fields" (for alarm condition reference); if alarm determination is required, it is recommended to filter the statistical fields using `where` at the end of the query; the output data is a JSON object with a fixed structure.

[0109] {

[0110] "timerange";

[0111] "operator": ")"

[0112] “thresholds”: [...];

[0113] }

[0114] timerange: Alarm judgment window, using relative time strings, such as "-5m", "-10m", "-1h";

[0115] operator: always use ")";

[0116] thresholds:

[0117] The array elements represent the triggering conditions for different alarm levels;

[0118] level is "info", "low", "mid", "high", "critical";

[0119] expression.operator_type is always "logic_and";

[0120] `expression.children` is an array of conditions, each with the following format:

[0121] {"field": (statistical field name), "operator_type": >= or > or < or <=, value: <threshold numeric string>};

[0122] It is not required to cover all levels; results are generated based on the levels specified by the user. If no levels are specified by the user, 1-3 levels are generated according to best practices (commonly mid / high / critical), with the threshold increasing progressively.

[0123] The children.field field name comes from the stats result field name of alert_rule_query and has the same meaning.

[0124] In another specific embodiment, if the user selects a target data source or target field for the monitoring scenario of the log analysis platform, JSON data is generated according to the user's requirements; otherwise, JSON data is generated according to the common field name or requirements.

[0125] In another specific embodiment, the standard data template corresponding to the output content of the large language model is limited to the following: the output content is a JSON format string, the output result follows the template fields and hierarchy, it is not recommended to add or delete fields, and it cannot contain any extra text, explanations, line breaks or markdown syntax.

[0126] Step 230: Based on the semantic understanding results of the text description data and the prompt information, generate the key parameters corresponding to the monitoring rules of the log analysis platform.

[0127] Step 240: Generate a parameter conversion script by creating the standard parameter format corresponding to the interface based on the monitoring rules in the log analysis platform.

[0128] In this step, specifically, the code for the parameter conversion script is as follows:

[0129] import json;

[0130] from copy import deepcopy;

[0131] from datetime import datetime;

[0132] import requests;

[0133] DEBUG_FLAG=False;

[0134] #=====LogEase API=====;

[0135] LOGEASE_BASE="http: / / 192.168.1.201 / api / v3";

[0136] LOGEASE_ALERT_API=f"{LOGEASE_BASE} / alerts / ";

[0137] LOGEASE_ACCOUNT_API=f"{LOGEASE_BASE} / accounts / ";

[0138] #=====Auth=====;

[0139] AUTH_USER="admin";

[0140] AUTH_KEY=“OtWDBcbQxabXTwRireQyIlrHCjaeUYVu”;

[0141] The output variable is:

[0142] logease_api_url;

[0143] logease_auth_info;

[0144] req_body_json.

[0145] Step 250: Using the large language model application development platform, the conversion script is used to convert the key parameters into monitoring rule interface request parameters.

[0146] Step 260: Based on the API interface of the log analysis platform and the interface request parameters, create target monitoring rules and use the target monitoring rules to monitor the running data of the log analysis platform.

[0147] In one embodiment of this example, after creating the target monitoring rule, the method further includes: if the creation of the target monitoring rule fails, generating a failure reason corresponding to the target monitoring rule, and feeding back the failure reason to the user through a visual interface.

[0148] Before using the target monitoring rule to monitor the operational data of the log analysis platform, the process further includes: performing a logic validity check on the target monitoring rule; if the target monitoring rule is successfully verified, then the target monitoring rule is enabled; if the target monitoring rule fails to verify, then the target monitoring rule is modified, and then the modified target monitoring rule is enabled.

[0149] The technical solution provided by this invention receives text description data input by the user through a large language model, performs semantic understanding on the text description data, generates prompt information corresponding to the large language model based on pre-configured parameter constraints, generates key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results of the text description data and the prompt information, converts the key parameters into monitoring rule interface request parameters through a conversion script on the large language model application development platform, creates target monitoring rules based on the API interface and interface request parameters of the log analysis platform, and uses the target monitoring rules to monitor the running data of the log analysis platform. This technical means can save the time and manpower costs consumed in the process of creating monitoring rules, reduce the technical difficulty of the log analysis platform monitoring process, and respond promptly to the updated monitoring needs in the log analysis platform.

[0150] Figure 3 This is a schematic diagram of the structure of a data monitoring device for a log analysis platform provided in an embodiment of the present invention. The device is applied in electronic devices, such as... Figure 3 As shown, the device includes: a parameter generation module 310, a parameter conversion module 320, and a rule creation module 330.

[0151] The parameter generation module 310 is used to receive text description data input by the user through a large language model, perform semantic understanding on the text description data, and generate key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results.

[0152] The parameter conversion module 320 is used to convert the key parameters into monitoring rule interface request parameters through the large language model application development platform.

[0153] The rule creation module 330 is used to create target monitoring rules based on the API interface of the log analysis platform and the interface request parameters, and to use the target monitoring rules to monitor the running data of the log analysis platform.

[0154] The technical solution provided by this invention receives text description data input by the user through a large language model, performs semantic understanding on the text description data, generates key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results, converts the key parameters into monitoring rule interface request parameters through a large language model application development platform, creates target monitoring rules based on the API interface and interface request parameters of the log analysis platform, and uses the target monitoring rules to monitor the running data of the log analysis platform. This technical means can save the time and manpower costs consumed in the monitoring rule creation process, reduce the technical difficulty of the log analysis platform monitoring process, and respond promptly to the updated monitoring needs in the log analysis platform.

[0155] Based on the above embodiments, the parameter generation module 310 includes:

[0156] The prompt generation unit is used to generate prompt information corresponding to the large language model based on pre-configured parameter constraints.

[0157] The key parameter generation unit is used to generate key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results of the text description data and the prompt information.

[0158] The pre-configured parameter constraints include at least one of the following:

[0159] Key JSON data templates corresponding to the monitoring rules of the log analysis platform;

[0160] The log analysis platform searches for usage guidelines for the corresponding languages.

[0161] The field specifications corresponding to the key JSON data;

[0162] Users select the target data source or target field for the monitoring scenario of the log analysis platform;

[0163] The standard data template corresponding to the output content of the large language model.

[0164] The parameter conversion module 320 includes:

[0165] The script generation unit is used to generate a parameter conversion script based on the standard parameter format corresponding to the interface created by the monitoring rules in the log analysis platform.

[0166] The script application unit is used to convert key parameters into monitoring rule interface request parameters using the conversion script through the large language model application development platform.

[0167] Rule creation module 330 includes:

[0168] The rule detection unit is used to generate a failure reason corresponding to the target monitoring rule if the creation of the target monitoring rule fails, and to feed back the failure reason to the user through a visual interface;

[0169] The rule verification unit is used to verify the rationality of the logic of the target monitoring rule; if the target monitoring rule verification is successful, the target monitoring rule is enabled; if the target monitoring rule verification fails, the target monitoring rule is modified and then the modified target monitoring rule is enabled.

[0170] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in the embodiments of the present invention can be found in the methods provided in all the foregoing embodiments of the present invention.

[0171] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0172] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0173] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0174] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data monitoring methods in a log analysis platform.

[0175] In some embodiments, the data monitoring method of the log analysis platform can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data monitoring method of the log analysis platform described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the data monitoring method of the log analysis platform by any other suitable means (e.g., by means of firmware).

[0176] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0177] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0178] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0179] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0180] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0181] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0182] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0183] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data monitoring method for a log analysis platform, characterized in that, The method includes: The system receives text description data input by the user through a large language model, performs semantic understanding on the text description data, and generates key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results. The key parameters are converted into monitoring rule interface request parameters through the large language model application development platform. Based on the application programming interface (API) of the log analysis platform and the interface request parameters, target monitoring rules are created, and the target monitoring rules are used to monitor the operating data of the log analysis platform.

2. The method according to claim 1, characterized in that, Based on the semantic understanding results, key parameters corresponding to the monitoring rules of the log analysis platform are generated, including: Based on pre-configured parameter constraints, generate prompt information corresponding to the large language model; Based on the semantic understanding of the text description data and the prompt information, key parameters corresponding to the monitoring rules of the log analysis platform are generated.

3. The method according to claim 2, characterized in that, The pre-configured parameter constraints include at least one of the following: Key JSON data templates corresponding to the monitoring rules of the log analysis platform; The log analysis platform searches for usage guidelines for the corresponding languages. The field specifications corresponding to the key JSON data; Users select the target data source or target field for the monitoring scenario of the log analysis platform; The standard data template corresponding to the output content of the large language model.

4. The method according to claim 1, characterized in that, Through the large language model application development platform, the key parameters are converted into monitoring rule interface request parameters, including: Generate a parameter conversion script based on the standard parameter format corresponding to the interface created by the monitoring rules in the log analysis platform. Using the large language model application development platform, the conversion script is used to convert key parameters into monitoring rule interface request parameters.

5. The method according to claim 1, characterized in that, After creating the target monitoring rules, the following is also included: If the creation of the target monitoring rule fails, a failure reason corresponding to the target monitoring rule is generated and fed back to the user through a visual interface.

6. The method according to claim 1, characterized in that, Before using the target monitoring rules to monitor the operational data of the log analysis platform, the following steps are also included: The logic of the target monitoring rules is validated for reasonableness. If the target monitoring rule is successfully verified, then the target monitoring rule is enabled; If the target monitoring rule fails to be verified, the target monitoring rule is modified and then the modified target monitoring rule is enabled.

7. A data monitoring device for a log analysis platform, characterized in that, The device includes: The parameter generation module is used to receive text description data input by the user through a large language model, perform semantic understanding on the text description data, and generate key parameters corresponding to the monitoring rules of the log analysis platform based on the semantic understanding results. The parameter conversion module is used to convert the key parameters into monitoring rule interface request parameters through the large language model application development platform. The rule creation module is used to create target monitoring rules based on the application programming interface (API) of the log analysis platform and the interface request parameters, and to use the target monitoring rules to monitor the operation data of the log analysis platform.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data monitoring method of the log analysis platform according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the data monitoring method of the log analysis platform according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data monitoring method of the log analysis platform according to any one of claims 1-6.

Images