Monitoring method, device, medium and program product for an application system

Abstract

This application provides a monitoring method for an application system, applicable to the fields of artificial intelligence and fintech, involving the application of large-scale models in information security and fintech. The method includes: analyzing an information knowledge base and the business requirements document of the target application system using a pre-trained large-scale model to obtain hierarchical monitoring indicator data; wherein the hierarchical monitoring indicator data includes data of indicators to be monitored; based on the hierarchical monitoring indicator data, using a pre-trained multilayer perceptron regression model to predict the alarm thresholds corresponding to the data of indicators to be monitored; configuring the alarm threshold range of application indicators for the target interface of the target application system based on the alarm thresholds corresponding to the data of indicators to be monitored; and monitoring the target application system in real time based on the alarm threshold range of the application indicators for the target interface. This application also provides a monitoring device, storage medium, and program product for the application system.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and fintech, to the application of large models in information security and fintech, and more specifically to a monitoring method, device, medium and program product for an application system. Background Technology

[0002] In the process of production environment operation and management, it is necessary to establish a comprehensive monitoring mechanism for various operational indicators of various business application systems, and to promptly trigger alarms when abnormal fluctuations occur in the indicators, ensuring that the operation and maintenance team can respond and handle the situation quickly. However, improper configuration of indicators may also lead to a large number of invalid alarms, drowning out truly important abnormal signals. Therefore, the key to achieving accurate fault early warning and ensuring stable system operation lies in the reasonable design of monitoring indicators and the setting of alarm thresholds, and the accurate monitoring of application indicators of target interfaces.

[0003] Due to the diverse types of business application systems, current mainstream monitoring solutions heavily rely on manual intervention. This requires operations and maintenance personnel to manually configure indicator types and alarm thresholds based on current business characteristics and historical experience. Consequently, the monitoring of application indicators for the target interfaces in the existing configuration becomes unstable and inaccurate, leading to frequent missed and false alarms, delayed monitoring responses, and an inability to efficiently monitor the system. Summary of the Invention

[0004] In view of the above problems, this application provides a monitoring method, device, medium and program product for application systems that monitor the accuracy of application indicators.

[0005] According to a first aspect of this application, a monitoring method for an application system is provided. The method includes: obtaining hierarchical monitoring indicator data by analyzing an information knowledge base and a business requirements document of a target application system using a pre-trained large model; wherein the hierarchical monitoring indicator data includes indicator data to be monitored; predicting alarm thresholds corresponding to the indicator data to be monitored using a pre-trained multilayer perceptron regression model based on the hierarchical monitoring indicator data; configuring alarm threshold ranges for application indicators of a target interface of the target application system based on the alarm thresholds corresponding to the indicator data to be monitored; and monitoring the target application system in real time based on the alarm threshold ranges for the application indicators of the target interface.

[0006] According to an embodiment of this application, the training process of the large model includes: local deployment of the large model; and training the deployed large model using low-rank adaptive fine-tuning technology to adapt it to the application domain of the target application system; wherein the output format and output content of the deployed large model are controlled by a prompting engineering strategy.

[0007] According to an embodiment of this application, the method further includes: constructing the information knowledge base based on application type information, interface type information, and indicator type information.

[0008] According to an embodiment of this application, the hierarchical monitoring indicator data further includes the application type and the interface type corresponding to the application. The step of obtaining the hierarchical monitoring indicator data by analyzing the information knowledge base and the business requirements document of the target application system through a pre-trained large model includes: inputting the information knowledge base and the business requirements document of the target application system into the pre-trained large model, and outputting the application type, the interface type corresponding to the application, and the indicator data to be monitored; wherein, the indicator data to be monitored includes the indicator type to be monitored corresponding to the interface of the application.

[0009] According to an embodiment of this application, configuring the alarm threshold range of the application metrics of the target interface of the target application system based on the alarm threshold corresponding to the monitoring metric data includes: obtaining the application name and interface name corresponding to the target interface in the target application system; querying the monitoring metrics in the monitoring metric data based on the application name and the interface name to determine the monitoring metrics corresponding to the target interface; and importing the monitoring metrics corresponding to the target interface and the alarm threshold corresponding to the monitoring metrics as recommendation information into the management page, so that users can configure and adjust the alarm threshold range of the application metrics of the target interface of the target application system through the management page based on the imported recommendation information; wherein, the application metrics of the target interface are determined based on the monitoring metrics corresponding to the target interface.

[0010] According to an embodiment of this application, the method further includes: if the value of the application metric of the target interface is detected to be within the alarm threshold range of the application metric of the target interface, then generating alarm information corresponding to the application metric of the target interface, and pushing the alarm information to the operation and maintenance terminal of the target application system.

[0011] A second aspect of this application provides a monitoring device for an application system, the device comprising: a large-scale model analysis module, used to analyze an information knowledge base and a business requirement document of a target application system using a pre-trained large-scale model to obtain hierarchical monitoring indicator data; wherein the hierarchical monitoring indicator data includes indicator data to be monitored; a small-scale model prediction module, used to predict alarm thresholds corresponding to the indicator data to be monitored using a pre-trained multilayer perceptron regression model based on the hierarchical monitoring indicator data; a configuration module, used to configure alarm threshold ranges for application indicators of a target interface of the target application system based on the alarm thresholds corresponding to the indicator data to be monitored; and a monitoring module, used to monitor the target application system in real time based on the alarm threshold ranges for the application indicators of the target interface.

[0012] According to an embodiment of this application, the apparatus further includes: a large model training module for local deployment of the large model; and training the deployed large model using low-rank adaptive fine-tuning technology to adapt it to the application domain of the target application system; wherein the output format and output content of the deployed large model are controlled by a prompting engineering strategy.

[0013] According to an embodiment of this application, the apparatus further includes: a knowledge base construction module, used to construct the information knowledge base based on application type information, interface type information, and indicator type information.

[0014] According to an embodiment of this application, the hierarchical monitoring indicator data further includes application type and the interface type corresponding to the application. The large model analysis module includes: the model analysis unit, which is used to input the information knowledge base and the business requirements of the target application system into the pre-trained large model, and output the application type, the interface type corresponding to the application, and the indicator data to be monitored; wherein, the indicator data to be monitored includes the indicator type to be monitored corresponding to the interface of the application.

[0015] According to an embodiment of this application, the configuration module includes: an acquisition unit, configured to acquire the application name and interface name corresponding to the target interface in the target application system; a query unit, configured to query the monitored indicators in the monitored indicator data based on the application name and the interface name, so as to determine the monitored indicators corresponding to the target interface; and a configuration unit, configured to import the monitored indicators corresponding to the target interface and the alarm thresholds corresponding to the monitored indicators as recommendation information into a management page, so that users can configure and adjust the alarm threshold range of the application indicators of the target interface of the target application system through the management page based on the imported recommendation information; wherein, the application indicators of the target interface are determined based on the monitored indicators corresponding to the target interface.

[0016] According to an embodiment of this application, the device further includes: an alarm module, configured to generate alarm information corresponding to the application metric of the target interface if the value of the application metric of the target interface is detected to be within the alarm threshold range of the application metric of the target interface, and push the alarm information to the operation and maintenance terminal of the target application system.

[0017] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0018] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0019] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0020] In the embodiments of this application, a collaborative architecture of large and small models is adopted. The large model is responsible for semantic understanding, while the multilayer perceptron regression model (small model) focuses on numerical prediction. This fully leverages the advantages of each model and combines a full-scale category information knowledge base with business requirements. This avoids biases caused by subjective human factors and insufficient data sampling, ensuring that the alarm threshold range configured for the application indicators of the interface is stable and accurate. This enables efficient monitoring of the application system and improves the accuracy of monitoring the application indicators of the interface. Attached Figure Description

[0021] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 This illustration schematically depicts application scenarios of monitoring methods, apparatus, media, and program products of an application system according to embodiments of this application.

[0023] Figure 2 A flowchart illustrating a monitoring method for an application system according to an embodiment of this application is shown schematically.

[0024] Figure 3 This illustration schematically shows a flowchart of configuring alarm threshold ranges for application metrics of a target interface in an application system monitoring method according to an embodiment of this application.

[0025] Figure 4 This illustration shows a flowchart of the indicator configuration implementation of the monitoring method of the application system according to an embodiment of this application;

[0026] Figure 5 This illustration schematically shows another flowchart of a monitoring method for an application system according to an embodiment of this application;

[0027] Figure 6 This schematically illustrates a structural block diagram of a monitoring device for an application system according to an embodiment of this application; and

[0028] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a monitoring method for an application system according to an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0032] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0033] Monitoring financial application systems is a core line of defense for ensuring the continuity of financial operations, data security, and transaction compliance. Real-time detection of system anomalies (such as transaction delays, data tampering, and interface failures) can quickly provide early warnings of financial risks, prevent operational vulnerabilities, and avoid financial losses, customer trust crises, or regulatory compliance risks caused by system failures. It is a crucial support for maintaining the stable operation of the financial market. Monitoring application metrics of target interfaces within the system, and ensuring that the metrics of important target interfaces are within normal ranges, achieves the purpose of monitoring the application system.

[0034] The focus of application system monitoring lies in monitoring interface application metrics. Monitoring interface application metrics can be effectively monitored by setting thresholds. Therefore, the difficulty in monitoring interface application metrics lies in how to configure the relevant interface application metrics and their thresholds. However, existing methods for configuring application monitoring metrics have shortcomings: on the one hand, the manual configuration process is too subjective, with significant differences in configuration results between different operators, resulting in low stability of interface application metric configuration results; on the other hand, monitoring configuration experience across multiple services is difficult to transfer, making it impossible to establish reusable configuration recommendation standards, increasing the cost of configuration decisions, and resulting in low configuration efficiency.

[0035] This application provides a method for monitoring an application system, comprising: obtaining hierarchical monitoring indicator data by analyzing a pre-trained large model analysis information knowledge base and the business requirements document of the target application system; wherein the hierarchical monitoring indicator data includes indicator data to be monitored; predicting alarm thresholds corresponding to the indicator data to be monitored using a pre-trained multilayer perceptron regression model based on the hierarchical monitoring indicator data; configuring alarm threshold ranges for application indicators of the target interface of the target application system based on the alarm thresholds corresponding to the indicator data to be monitored; and monitoring the target application system in real time based on the alarm threshold ranges for the application indicators of the target interface. By adopting a collaborative architecture of large and small models, with the large model responsible for semantic understanding and the multilayer perceptron regression model (small model) focusing on numerical prediction, the advantages of each are fully utilized. Combined with a full-scale category information knowledge base and business requirements document, deviations caused by subjective human factors and insufficient data sampling are avoided, making the alarm threshold ranges configured for the application indicators of the interface stable and accurate, achieving efficient monitoring of the application system, and improving the accuracy of application indicator monitoring of the interface.

[0036] Figure 1 The illustration shows an application scenario diagram of the monitoring method, apparatus, medium, and program product of the application system according to embodiments of this application.

[0037] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0041] It should be noted that the application system monitoring method provided in this application embodiment can generally be executed by server 105. Correspondingly, the application system monitoring device provided in this application embodiment can generally be located in server 105. The application system monitoring method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the application system monitoring device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] It should be noted that the monitoring method and device of the application system in this application can be used in the fields of fintech and artificial intelligence, as well as in any field other than fintech. The application fields of the monitoring method and device of the application system in this application are not limited.

[0044] The following will be based on Figure 1 The described scene, through Figures 2-5The monitoring method of the application system according to the embodiments of this application will be described in detail.

[0045] Figure 2 A flowchart illustrating a monitoring method for an application system according to an embodiment of this application is shown.

[0046] like Figure 2 As shown, the monitoring method of the application system in this embodiment includes operations S210 to S240. This monitoring method does not limit the specific executing entity. The executing entity can be any electronic device, such as a terminal device or a server device, etc. The executing entity can also be any software application or client.

[0047] When operating S210, the hierarchical monitoring indicator data is obtained by analyzing the information knowledge base and the business requirements of the target application system through a pre-trained large model; the hierarchical monitoring indicator data includes the indicator data to be monitored.

[0048] The Large Language Model (LLM) in this application embodiment is an artificial intelligence model trained on massive amounts of data and a large number of parameters. It is capable of understanding and generating natural language, code, and other content, and learns complex language rules and knowledge from text data through deep learning technology.

[0049] The pre-trained large model can analyze the business requirements document based on the information knowledge base to conduct monitoring requirements analysis of the application indicators of the target interface. The information knowledge base and business requirements document are input into the pre-trained large model, and the output is hierarchical monitoring indicator data, which is used as the monitoring requirements.

[0050] The information knowledge base contains category information such as application type, interface type, and indicator type, as well as textual descriptions related to the category information.

[0051] The target application system is the application system to be monitored. Taking the financial field as an example, the target application system can be a banking business system, a securities trading system, a payment and settlement system, an online banking system, a mobile banking APP, a counter system, a credit reporting system, an asset management system, an identity authentication and security system, a loan business system, etc.

[0052] The business requirements document is the business requirements document for the specific application system to be configured for monitoring, including textual descriptions of the application system and its various interfaces.

[0053] According to an embodiment of this application, before operating S210 to analyze the information knowledge base through a pre-trained large model, the method further includes: constructing an information knowledge base based on application type information, interface type information, and indicator type information.

[0054] Application type information includes application-level information such as personal business, corporate business, regulatory reporting, and internal systems; interface type information includes interface-level information such as query, transaction, identity authentication, and system configuration; indicator type information includes indicator-level information such as interface call volume per second, data volume, average response time, response time variance, and failure rate.

[0055] Methods for building information knowledge bases include, but are not limited to, rule definition, machine learning, knowledge graphs, and natural language processing.

[0056] Rule definition: This involves manually writing explicit logical rules (such as conditional logic control statements, regular expressions, classification conditions, etc.) to define the classification, association, or processing logic of information. It is suitable for structured data with clear domain knowledge and simple logic, such as interface types matched according to Uniform Resource Locator (URL) path rules.

[0057] Machine learning: Automatically learns from historical application, interface, and metric data related to application systems by training models (such as classification and clustering models) to extract or classify information.

[0058] Knowledge graphs: Construct entity-relationship networks (e.g., using applications, interfaces, and metrics as nodes, connected by relational edges) to support semantic reasoning. Complex relationships can be expressed through knowledge graphs, such as application A calling interface B, where interface B depends on metric C.

[0059] Natural Language Processing (NLP) extracts knowledge from unstructured data (documents, logs) through text parsing techniques (such as keyword extraction, entity recognition, and text summarization). It can process unstructured text and automatically extract application concepts, interface descriptions, or metric definitions from documents.

[0060] In the embodiments of this application, the information knowledge base is constructed from three aspects: application, interface, and indicator, which helps the large model understand the relationship between application, interface, and indicator, and facilitates the accurate extraction of the indicator to be monitored.

[0061] According to an embodiment of this application, the hierarchical monitoring indicator data also includes the application type and the interface type corresponding to the application. In operation S210, the hierarchical monitoring indicator data is obtained by analyzing the information knowledge base and the business requirements document of the target application system through a pre-trained large model. This includes: inputting the information knowledge base and the business requirements document of the target application system into the pre-trained large model, and outputting the application type, the interface type corresponding to the application, and the indicator data to be monitored; wherein, the indicator data to be monitored includes the indicator type to be monitored corresponding to the interface of the application.

[0062] The information knowledge base and the business requirements of the target application system are input into the pre-trained large model. The pre-trained large model performs monitoring requirements analysis and outputs hierarchical monitoring indicator data. This data includes application type (representing the application to be monitored), interface type corresponding to the application (representing each interface to be monitored in each application), and monitoring indicator data (i.e., the monitoring indicator type of each target interface in each application).

[0063] In the embodiments of this application, hierarchical monitoring indicator data is output through a large model, from the application to the corresponding interface, and then to the indicator corresponding to the interface, to accurately determine the indicator to be monitored layer by layer, thereby improving the accuracy of application indicator monitoring of the target interface.

[0064] When operating S220, based on hierarchical monitoring indicator data, a pre-trained multilayer perceptron regression model is used to predict the alarm threshold corresponding to the monitored indicator data.

[0065] For each monitored indicator, a multilayer perceptron regression model (MLP model) is used to predict the alarm threshold. The input of the MLP model is the hierarchical monitoring indicator data, and the output is the alarm threshold corresponding to each monitored indicator.

[0066] For MLP model training, the alarm threshold configurations of application metrics for the target interfaces of the existing full application are used as the dataset. Metric features (such as average / peak CPU, request latency, error rate, etc.) are extracted and combined with information such as application type and interface type. Existing alarm thresholds (such as triggering an alarm when CPU > 80%) are used as supervision labels. The MLP model is constructed using a multilayer perceptron, including an input layer (feature dimension), a hidden layer (Rectified Linear Unit (ReLU) activation function), and an output layer (alarm threshold prediction), with a regression task as the loss function. The MLP model is trained by adjusting weights through backpropagation, aiming to minimize the error between the predicted alarm threshold and the actual configuration. Regularization can be added to prevent overfitting, or dynamic weights can be used to handle the sample imbalance problem.

[0067] Since the amount of data for training and running the alarm threshold prediction is not large, the MLP model can meet the prediction requirements, that is, predict reasonable alarm thresholds, reduce manual intervention, and save deployment and maintenance costs.

[0068] In operation S230, based on the alarm threshold corresponding to the monitored indicator data, configure the alarm threshold range of the application indicators of the target interface of the target application system.

[0069] The monitoring metrics and application metrics of the target interface are used as the smallest granularity, i.e., the objects being monitored. For example, the application metric "error rate" of the "transfer" interface in the "mobile banking" application is the target interface, and the error rate of mobile banking transfers is the monitoring object. The target interface is the interface within the configurable metric alarm range. It can be selected from the hierarchical monitoring metric data using the corresponding interface type, or it can be determined based on expert experience. The application metric of the target interface represents the metric selected from the monitoring metrics corresponding to the target interface; this application metric is used to monitor the target interface.

[0070] When operating S240, the alarm threshold range of the target application system is monitored in real time based on the application metrics of the target interface.

[0071] Application metrics (such as CPU, memory, request latency, error rate, etc.) of the target interfaces of the application system are continuously collected through agents / probes and reported to the monitoring service at a fixed frequency (e.g., per second). The monitoring service compares the real-time application metrics of the target interfaces with the pre-configured alarm threshold ranges for the application metrics of the target interfaces one by one. Normal range: If the value of an application metric for an interface is not within the alarm threshold range for that interface, it is only recorded and no alarm is triggered. Anomaly trigger: If the value of an application metric for an interface is within the alarm threshold range for that interface, the alarm process is initiated.

[0072] For example, the target interface application metrics for the corporate business application of the loan business system can be configured as response time, transactions per second, and transaction success rate. Alarm threshold ranges can be configured for target interface application metrics such as "response time > 2 seconds", "transactions per second < 100 (indicating low load) or > 5000 (indicating overload risk)", and "transaction success rate < 99.9%". These target interface application metrics are continuously monitored. Once the response time becomes as long as 5 seconds, or the transaction volume per second decreases to 50 or increases to 8000, or the transaction success rate drops to 90%, an alarm will be triggered to facilitate timely troubleshooting (such as detecting interface processing efficiency, abnormal traffic, abnormal server pressure, transaction link failure, etc.).

[0073] In the embodiments of this application, a collaborative architecture of large and small models is adopted. The large model is responsible for semantic understanding, while the multilayer perceptron regression model (small model) focuses on numerical prediction. This fully leverages the advantages of each model and combines a full-scale category information knowledge base with business requirements. This avoids biases caused by subjective human factors and insufficient data sampling, ensuring that the alarm threshold range configured for the application indicators of the interface is stable and accurate. This enables efficient monitoring of the application system and improves the accuracy of monitoring the application indicators of the interface.

[0074] According to an embodiment of this application, the training process of the large model includes: local deployment of the large model; and training the deployed large model using low-rank adaptive fine-tuning technology to adapt to the application domain of the target application system; wherein the output format and output content of the deployed large model are controlled by prompting engineering strategies.

[0075] By deploying large models locally, publicly available pre-trained weights can be used, and low-rank adaptation (LoRA) can be used to fine-tune them for text understanding in the financial domain, meeting the comprehensive requirements of local confidentiality, text understanding capabilities, and deployment costs. The large model's input consists of an information knowledge base and business requirements documents. Based on cue engineering strategies, the model outputs in a structured format (such as JSON), with the output content being hierarchical monitoring indicator data. Cue engineering strategies can be used to optimize the large model's input to improve output quality. LoRA fine-tuning involves inserting a small number of trainable parameters into the locally deployed large model for training. Low-rank decomposition (approximating the parameter update matrix as the product of two low-rank matrices) significantly reduces the number of trainable parameters, thereby significantly reducing computational and storage costs while maintaining model performance. This approach is suitable for personalized adjustments of large models in local scenarios.

[0076] For example, the prompt message in the engineering strategy is: Please output a structured JSON format according to the JSON format requirements. Based on the information knowledge base and the business requirements of the application system, describe the application interface and output the application metrics of the target interface that needs to be monitored.

[0077] In the embodiments of this application, a large model with localized deployment is adopted, and low-rank adaptive fine-tuning is used to adapt to the text understanding of the application domain in order to meet the comprehensive requirements of local confidentiality, text understanding capability, and deployment cost.

[0078] According to an embodiment of this application, localized deployment of a large model includes: configuring a software environment for the large model in a local hardware environment, wherein the local hardware environment is used to run the large model; obtaining the model file of the large model; performing localization processing on the model file based on the software environment; and deploying the localized model file to the local hardware environment.

[0079] Set up a local hardware environment to run the large model. This local hardware environment must at least meet the computational power, storage, and network configuration requirements for running the large model. When deploying the software environment (such as the operating system, drivers, deep learning frameworks, and basic tool libraries) on the local hardware environment, first deploy the operating system, then install the drivers, deep learning frameworks, and basic tool libraries that the large model depends on. Obtain the model file of the large model, which includes pre-training parameters, network structure configuration, and inference interface definitions. Perform localization processing on the model file, including model format conversion, hardware acceleration optimization, and inference accuracy adjustment, to make the large model adaptable to the local hardware and software environment. Deploy the processed model file to the local hardware environment, start the model service process, and configure the access interface. Functional verification and performance testing of the deployed large model can be performed using preset test cases to confirm that it meets the localization service requirements.

[0080] In the embodiments of this application, based on the local hardware and software environment, the model files of the large model are deployed to complete the local deployment of the large model, so that the internal data of the target application system does not flow out of the local area, enabling the large model to provide secure inference services, ensuring privacy and security, and the localized large model reduces network dependence and improves service response speed.

[0081] Figure 3 The diagram illustrates a flowchart of configuring alarm threshold ranges for application metrics of a target interface in a monitoring method for an application system according to an embodiment of this application.

[0082] like Figure 3 As shown, in operation 230, the alarm threshold range of the application indicators of the target interface of the target application system is configured based on the alarm threshold corresponding to the data of the indicator to be monitored, including operations S310 to S330.

[0083] In operation S310, obtain the application name and interface name corresponding to the target interface in the target application system.

[0084] When a user needs to configure monitoring of a specific target interface of an application, the core identifier of that target interface is first obtained, namely the application name (e.g., "Mobile Banking") and the interface name (e.g., "Transfer"). The target interface refers to the interface to be configured in the target application system.

[0085] When operating S320, based on the application name and interface name, query the monitoring metrics data to determine the monitoring metrics corresponding to the target interface.

[0086] Based on the application name and interface name, search for matching items in the pre-generated data of metrics to be monitored. For example, using the application name (e.g., "Mobile Banking") and the interface name (e.g., "Transfer"), find the metrics to be monitored as "Error Rate" and "Response Time", and use "Error Rate" and "Response Time" as the metrics to be monitored for that interface.

[0087] The metrics to be monitored include the metrics to be monitored for each interface (such as response time, error rate, call frequency, success rate, etc.) and the alarm thresholds for the corresponding metrics. For example, if the response time is greater than 2 seconds, an alarm will be triggered if the response time exceeds 2 seconds.

[0088] When operating S330, the monitoring metrics and alarm thresholds corresponding to the target interface are imported into the management page as recommended information. This allows users to configure and adjust the alarm threshold range of the application metrics of the target interface of the target application system through the management page based on the imported recommended information. The application metrics of the target interface are determined based on the monitoring metrics corresponding to the target interface.

[0089] By accurately querying the most suitable monitoring metrics and corresponding alarm thresholds for the target interface, the queried monitoring metrics can be used as the application metrics that the target interface in the target application system needs to monitor. Alternatively, the queried monitoring metrics can be filtered, and some of the monitoring metrics can be used as the application metrics that the target interface in the target application system needs to monitor.

[0090] The retrieved monitoring metrics and alarm thresholds are directly imported into the monitoring configuration items of the current target interface in the management page and presented to the user as recommended configuration information. The user can verify the application metrics of the target interface with the recommended configuration information, or directly adopt the recommended configuration information, or modify it according to actual needs (such as adjusting the alarm threshold range and filtering high-quality application metrics) to finally complete the configuration of target interface monitoring.

[0091] The management page serves as a visual monitoring and management interface for user interaction. Users can input configuration information and modify the alarm threshold range of application metrics for target interfaces through this management page.

[0092] Figure 4 This illustration schematically shows a flowchart of the indicator configuration implementation of the monitoring method of the application system according to an embodiment of this application, such as... Figure 4As shown, the knowledge base and application business requirements are input into the monitoring requirements analysis module (which embeds a large model) to obtain hierarchical monitoring indicator data, i.e., monitoring requirements. Each item (application-interface-indicator) in the monitoring requirements is input into the alarm threshold analysis module (which embeds a multilayer perceptron regression model), continuously outputting the corresponding alarm threshold. The output alarm thresholds of the monitored indicators are input into the monitoring management interaction module (which includes a management page for user interaction) as recommended configuration items. Users can optimize the application indicators and alarm threshold ranges of the target interface through the management page. Based on the configured application indicators and alarm threshold ranges of the target interface, the monitoring application system is implemented.

[0093] In the embodiments of this application, by automatically recommending information to the management page, the hassle of manually setting each item is eliminated. By automatically querying and matching the monitoring indicators corresponding to the target interface, alarm thresholds corresponding to the monitoring indicators that meet the characteristics of the interface are quickly generated. This achieves end-to-end fully automated processing from business needs to monitoring configuration, avoiding the communication costs and repetitive work caused by manual configuration in traditional methods, improving configuration efficiency and reducing costs.

[0094] Figure 5 Another flowchart illustrating a monitoring method for an application system according to an embodiment of this application is shown.

[0095] like Figure 5 As shown, the monitoring method of the application system in this embodiment includes operations S510 to S560.

[0096] In operating S510, a pre-trained large model is used to analyze the information knowledge base and the business requirements document of the target application system to obtain hierarchical monitoring indicator data; the hierarchical monitoring indicator data includes the indicator data to be monitored. The method of operating S510 is the same as that of operating S210 described above, and will not be repeated here.

[0097] In operating S520, based on hierarchical monitoring indicator data, a pre-trained multilayer perceptron regression model is used to predict the alarm threshold corresponding to the monitored indicator data. The operation of S520 is the same as the operation of S220 described above, and will not be repeated here.

[0098] In operation S530, based on the alarm threshold corresponding to the monitored metric data, configure the alarm threshold range for the application metrics of the target interface of the target application system. The operation of S530 is the same as the operation of S230 described above, and will not be repeated here.

[0099] When operating S540, the target application system is monitored in real time based on the alarm threshold range of the application metrics of the target interface. The method of operating S540 is the same as that of operating S240 described above, and will not be repeated here.

[0100] In operation S550, monitor in real time whether the value of the application metric of the target interface is within the alarm threshold range of the application metric of the target interface. If not, continue to execute operation S540; if yes, execute operation S560.

[0101] When operating S560, if the value of the application metric of the target interface is detected to be within the alarm threshold range of the application metric of the target interface, an alarm message corresponding to the application metric of the target interface is generated and pushed to the operation and maintenance terminal of the target application system.

[0102] During the monitoring of an application system, if the value of an application metric for a target interface is detected to be within the corresponding alarm threshold range, it indicates an anomaly in the target interface's application metric value. An alarm will be automatically generated and pushed to the target application system's operations and maintenance (O&M) server to notify relevant O&M personnel for timely handling of the anomaly. The alarm information includes, but is not limited to, the target interface name, the application name to which the target interface belongs, the application metric name, the application metric value, the alarm threshold range, and the alarm level (e.g., high, medium, low).

[0103] In the embodiments of this application, by detecting the value of the application metric of the interface and the threshold range of the configured application metric of the interface, alarm information is generated when an anomaly occurs, which can quickly respond to the anomaly, shorten the fault detection and handling time, and improve the operation and maintenance efficiency and the stability of the application system.

[0104] Based on the monitoring method of the above-mentioned application system, this application also provides a monitoring device for the application system. The following will be combined with... Figure 6 The device is described in detail.

[0105] Figure 6 The diagram illustrates the structural block diagram of a monitoring device for an application system according to an embodiment of this application.

[0106] like Figure 6 As shown, the monitoring device 600 of the application system in this embodiment includes a large model analysis module 610, a small model prediction module 620, a configuration module 630, and a monitoring module 640.

[0107] The large model analysis module 610 is used to analyze the information knowledge base and the business requirements document of the target application system through a pre-trained large model to obtain hierarchical monitoring indicator data; wherein, the hierarchical monitoring indicator data includes the indicator data to be monitored. In one embodiment, the large model analysis module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0108] The small model prediction module 620 is used to predict the alarm threshold corresponding to the monitored indicator data based on the hierarchical monitoring indicator data using a pre-trained multilayer perceptron regression model. In one embodiment, the small model prediction module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0109] The configuration module 630 is used to configure the alarm threshold range of the application metrics of the target interface of the target application system based on the alarm threshold corresponding to the monitored metric data. In one embodiment, the configuration module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0110] The monitoring module 640 is used to monitor the target application system in real time based on the alarm threshold range of the application metrics of the target interface. In one embodiment, the monitoring module 640 can be used to perform the operation S240 described above, which will not be repeated here.

[0111] According to an embodiment of this application, the apparatus 600 further includes: a large model training module for local deployment of a large model; and a low-rank adaptive fine-tuning technique for training the deployed large model to adapt to the application domain of the target application system; wherein the output format and output content of the deployed large model are controlled by a prompting engineering strategy.

[0112] According to an embodiment of this application, the device 600 further includes: a knowledge base construction module, used to construct an information knowledge base based on application type information, interface type information, and indicator type information.

[0113] According to an embodiment of this application, the hierarchical monitoring indicator data also includes the application type and the interface type corresponding to the application. The large model analysis module 610 includes: a model analysis unit, which is used to input the information knowledge base and the business requirements of the target application system into the pre-trained large model and output the application type, the interface type corresponding to the application, and the monitoring indicator data; wherein, the monitoring indicator data includes the monitoring indicator type corresponding to the interface of the application.

[0114] According to an embodiment of this application, the configuration module 630 includes: an acquisition unit, used to acquire the application name and interface name corresponding to the target interface in the target application system; a query unit, used to query the monitoring indicators in the monitoring indicator data according to the application name and interface name, so as to determine the monitoring indicators corresponding to the target interface; and a configuration unit, used to import the monitoring indicators corresponding to the target interface and the alarm thresholds corresponding to the monitoring indicators as recommendation information into the management page, so that users can configure and adjust the alarm threshold range of the application indicators of the target interface of the target application system through the management page based on the imported recommendation information; wherein, the application indicators of the target interface are determined based on the monitoring indicators corresponding to the target interface.

[0115] According to an embodiment of this application, the device 600 further includes: an alarm module, configured to generate alarm information corresponding to the application metric of the target interface and push the alarm information to the operation and maintenance terminal of the target application system if the value of the application metric of the target interface is detected to be within the alarm threshold range of the application metric of the target interface.

[0116] According to embodiments of this application, any multiple modules among the large model analysis module 610, small model prediction module 620, configuration module 630, and monitoring module 640 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the large model analysis module 610, small model prediction module 620, configuration module 630, and monitoring module 640 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the large model analysis module 610, the small model prediction module 620, the configuration module 630, and the monitoring module 640 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0117] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a monitoring method for an application system according to an embodiment of this application.

[0118] like Figure 7 As shown, an electronic device 900 according to an embodiment of this application includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0119] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in one or more memories.

[0120] According to embodiments of this application, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0121] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0122] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0123] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the monitoring method of the application system provided in the embodiments of this application.

[0124] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0125] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0126] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0127] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A monitoring method for an application system, characterized in that, The method includes: By analyzing the information knowledge base and the business requirements document of the target application system through a pre-trained large model, hierarchical monitoring indicator data is obtained; wherein, the hierarchical monitoring indicator data includes the indicator data to be monitored; Based on the hierarchical monitoring index data, a pre-trained multilayer perceptron regression model is used to predict the alarm threshold corresponding to the monitored index data. Based on the alarm thresholds corresponding to the monitored metric data, configure the alarm threshold range for the application metrics of the target interface of the target application system; and Based on the alarm threshold range of the application metrics of the target interface, the target application system is monitored in real time.

2. The method according to claim 1, characterized in that, The training process of the large model includes: Local deployment of the large model; and The large model after deployment is trained using low-rank adaptive fine-tuning technology to adapt to the application domain of the target application system; wherein the output format and output content of the large model after deployment are controlled by prompting engineering strategies.

3. The method according to claim 2, characterized in that, The localized deployment of the large model includes: Configure the software environment of the large model in a local hardware environment, wherein the local hardware environment is used to run the large model; Obtain the model file of the large model, and perform localization processing on the model file based on the software environment; and The localized model files are deployed to the local hardware environment.

4. The method according to claim 1, characterized in that, The method further includes: The information knowledge base is constructed based on application type information, interface type information, and indicator type information.

5. The method according to claim 1, wherein the hierarchical monitoring indicator data further includes application type and the interface type corresponding to the application, characterized in that, The process involves analyzing a pre-trained large model to obtain hierarchical monitoring indicator data from an information knowledge base and the business requirements document of the target application system, including: The information knowledge base and the business requirements document of the target application system are input into the pre-trained large model, and the application type, the corresponding interface type of the application, and the monitoring indicator data are output; wherein, the monitoring indicator data includes the monitoring indicator type corresponding to the interface of the application.

6. The method according to claim 1, characterized in that, The step of configuring the alarm threshold range for the application metrics of the target interface of the target application system based on the alarm threshold corresponding to the monitored metric data includes: Obtain the application name and interface name corresponding to the target interface in the target application system; Based on the application name and the interface name, query the monitored metrics in the monitored metrics data to determine the monitored metrics corresponding to the target interface; and The monitoring metrics and alarm thresholds corresponding to the target interface are imported into the management page as recommended information, so that users can configure and adjust the alarm threshold range of the application metrics of the target interface of the target application system through the management page based on the imported recommended information; wherein, the application metrics of the target interface are determined based on the monitoring metrics corresponding to the target interface.

7. The method according to claim 6, characterized in that, The method further includes: If the value of the application metric of the target interface is detected to be within the alarm threshold range of the application metric of the target interface, then an alarm message corresponding to the application metric of the target interface is generated and the alarm message is pushed to the operation and maintenance terminal of the target application system.

8. A monitoring device for an application system, characterized in that, The device includes: The large model analysis module is used to analyze the information knowledge base and the business requirements document of the target application system through a pre-trained large model to obtain hierarchical monitoring indicator data; wherein, the hierarchical monitoring indicator data includes the indicator data to be monitored; The small model prediction module is used to predict the alarm threshold corresponding to the monitored indicator data based on the hierarchical monitoring indicator data using a pre-trained multilayer perceptron regression model. The configuration module is used to configure the alarm threshold range of the application metrics of the target interface of the target application system based on the alarm threshold corresponding to the monitored metric data; and The monitoring module is used to monitor the target application system in real time based on the alarm threshold range of the application metrics of the target interface.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images