A method, device, equipment, medium, and product for full-link monitoring of large-scale models.
By introducing a multi-dimensional monitoring agent and a pluggable plugin system, the lack of internal logic and security in large model monitoring is solved, enabling fine-grained monitoring of the entire lifecycle of large models and improving interpretability and security.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIACHE INFORMATION TECH CO LTD
- Filing Date
- 2026-03-16
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies for monitoring large models lack in-depth insights into their internal operating logic, inference paths, security, and compliance. They are difficult to effectively monitor the decision-making basis of models when handling sensitive requests, and lack the ability to detect in real time whether models produce hallucinations or biased outputs.
By introducing a multi-dimensional monitoring agent and a pluggable monitoring plugin system, user requests are intercepted and analyzed, data during the inference process of large models is captured, and structured monitoring reports are generated, enabling fine-grained monitoring of the entire lifecycle of large models.
It improves the interpretability, security, and reliability of large models, enables real-time monitoring of model inputs, internal states, and outputs, provides fine-grained monitoring, and reduces maintenance costs.
Smart Images
Figure CN122137756A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of artificial intelligence security protection technology, and in particular to a method, device, equipment, medium and product for full-link monitoring of large models. Background Technology
[0002] With the widespread application of LLMs (Large Language Models) in natural language processing tasks, the need for monitoring during their deployment and operation is becoming increasingly prominent. Current technologies for monitoring large models typically focus on computational resource consumption (such as GPU utilization, memory usage, and inference latency) or simple input / output log recording. However, these monitoring methods often lack in-depth insights into the internal operational logic, inference paths, security, and compliance of large models. For example, existing technologies struggle to effectively monitor the decision-making process of models when handling sensitive requests, or to detect in real-time whether the model is generating hallucinations or biased outputs. Summary of the Invention
[0003] This invention provides a method, apparatus, device, medium, and product for full-link monitoring of large models, enabling fine-grained monitoring of the entire lifecycle of large model operation by introducing multi-dimensional monitoring agents and a pluggable monitoring plugin system.
[0004] According to one aspect of the present invention, a method for end-to-end monitoring of a large model is provided, comprising: Obtain the monitoring policy and load the corresponding monitoring plugin from the plugin library according to the monitoring policy; the monitoring plugin includes: input-side plugin and output-side plugin; Intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; Data capture is triggered at a specified layer during the large model inference process by using a pre-registered objective function, and metrics are calculated on the captured raw data. The original output of the large model is intercepted, and the output-side plugin is called to analyze the original output to obtain a second analysis result; Based on the first analysis results, the calculated indicators, and the second analysis results, a structured monitoring report is generated.
[0005] According to another aspect of the present invention, a large-scale model end-to-end monitoring device is provided, the device comprising: The acquisition and recording module is used to acquire the monitoring strategy and load the corresponding monitoring plugin from the plugin library according to the monitoring strategy; the monitoring plugin includes: input-side plugin and output-side plugin; The first analysis module is used to intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; The capture and calculation module is used to trigger data capture at a specified layer during the large model inference process through a pre-registered objective function, and to perform index calculations on the captured raw data. The second analysis module is used to intercept the original output of the large model and call the output-side plugin to analyze the original output to obtain the second analysis result; The generation module is used to generate a structured monitoring report based on the first analysis results, the calculated indicators, and the second analysis results.
[0006] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the large-scale model end-to-end monitoring method according to any embodiment of the present invention.
[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the large-model full-link monitoring method according to any embodiment of the present invention.
[0008] According to another aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the large-model full-link monitoring method described in any embodiment of the present invention.
[0009] This invention, through its embodiments, acquires a monitoring strategy and loads corresponding monitoring plugins from a plugin library based on that strategy. These monitoring plugins include input-side and output-side plugins. The system intercepts user requests to a large model, calls the input-side plugin to analyze the requests, and obtains a first analysis result. It then triggers data capture at a specified layer during the large model's inference process using a pre-registered objective function, calculates metrics on the captured raw data, intercepts the large model's raw output, and calls the output-side plugin to analyze the raw output, obtaining a second analysis result. Based on the first analysis result, the calculated metrics, and the second analysis result, a structured monitoring report is generated. This invention's technical solution, by introducing a multi-dimensional monitoring agent and a pluggable monitoring plugin system, enables fine-grained monitoring of the entire lifecycle of a large model. This solution not only monitors hardware metrics but also deeply monitors the large model's input, internal state, output, and context, thereby improving the large model's interpretability, security, and reliability.
[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a flowchart of a large-scale model end-to-end monitoring method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a large-scale model end-to-end monitoring device in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the large-scale full-link monitoring method of this invention. Detailed Implementation
[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and their derivatives, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0015] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0016] Example 1 The objective drawbacks of the background technology are as follows: Lack of interpretable monitoring: It is difficult to track which parts of the input text the large model’s attention mechanism focuses on when generating a specific output, making the model’s decision-making process like a “black box”.
[0017] Lack of security and compliance monitoring: There is a lack of real-time detection mechanisms for whether the output content of large models contains sensitive information, inappropriate remarks, or leaks of privacy data.
[0018] Performance monitoring is limited to a single dimension: most existing monitoring is limited to hardware metrics and lacks monitoring of business metrics such as model inference quality and accuracy drift.
[0019] Delayed response: Problem detection often relies on manual review after the fact, lacking the ability to intercept and correct erroneous outputs in real time.
[0020] Figure 1 This is a flowchart of a large-scale model end-to-end monitoring method according to an embodiment of the present invention. This embodiment is applicable to the end-to-end monitoring of large models. The method can be executed by the large-scale model end-to-end monitoring device according to the present invention. This device can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps: S101. Obtain the monitoring policy and load the corresponding monitoring plugin from the plugin library according to the monitoring policy.
[0021] In this embodiment, the monitoring strategy can be a monitoring strategy for large models set by the system administrator or an automated configuration tool.
[0022] The monitoring plugins include input-side plugins and output-side plugins.
[0023] It should be noted that the input-side plugin can be used to monitor the input information of a large model, and the output-side plugin can be used to monitor the output information of a large model.
[0024] Specifically, system administrators or automated configuration tools set up a monitoring blueprint for large models. The monitoring strategy mainly includes monitoring the large model ID / name, monitoring mode, sampling rate parameters, and enabling / disabling metrics. Then, based on the above strategy, corresponding monitoring function modules are dynamically loaded from the plugin library. Monitoring plugins can include model input-side plugins and model output-side plugins to monitor the entire lifecycle of the large model.
[0025] S102. Intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result.
[0026] The first analysis result can be the result of monitoring and analysis on the input side of a large model.
[0027] Specifically, before the request reaches the large model inference service, the monitoring agent intercepts it, calls the input-side plugin to analyze the request sent by the user to the large model, and obtains the first analysis result.
[0028] S103. Trigger data capture at a specified layer during the large model inference process using a pre-registered objective function, and calculate the metrics on the captured raw data.
[0029] In this embodiment, the target function can be a Hook function, i.e., a hook function.
[0030] Specifically, during model inference (forward propagation), pre-registered hook functions are triggered at specific layers to capture intermediate data. The captured raw data is then used to calculate interpretability metrics and generate intermediate visualization files, such as heatmap images.
[0031] S104. Intercept the raw output of the large model and call the output-side plugin to analyze the raw output to obtain the second analysis result.
[0032] The original output can be the output information provided by the large model in response to the user's request. The second analysis result can be the result of monitoring and analysis of the output side of the large model.
[0033] Specifically, the original output of the large model is intercepted, and multiple output analysis plugins are called in parallel to perform analysis to obtain a second analysis result.
[0034] S105. Based on the first analysis results, the calculated indicators, and the second analysis results, generate a structured monitoring report.
[0035] It should be noted that a structured monitoring report can be a structured JSON object that aggregates all monitoring results, containing key parameters and analysis results from all the above steps.
[0036] Specifically, all monitoring results are aggregated, input into the rules engine for final adjudication, and a structured JSON object is obtained, containing key parameters and analysis results from all the above steps, and the entire data chain is persistently stored.
[0037] This invention, through its embodiments, acquires a monitoring strategy and loads corresponding monitoring plugins from a plugin library based on that strategy. These monitoring plugins include input-side and output-side plugins. The system intercepts user requests to a large model, calls the input-side plugin to analyze the requests, and obtains a first analysis result. It then triggers data capture at a specified layer during the large model's inference process using a pre-registered objective function, calculates metrics on the captured raw data, intercepts the large model's raw output, and calls the output-side plugin to analyze the raw output, obtaining a second analysis result. Based on the first analysis result, the calculated metrics, and the second analysis result, a structured monitoring report is generated. This invention's technical solution, by introducing a multi-dimensional monitoring agent and a pluggable monitoring plugin system, enables fine-grained monitoring of the entire lifecycle of a large model. This solution not only monitors hardware metrics but also deeply monitors the large model's input, internal state, output, and context, thereby improving the large model's interpretability, security, and reliability.
[0038] Optionally, the monitoring strategy includes: monitoring target model identifier, monitoring mode, sampling rate parameter, and monitoring indicator switch set.
[0039] The identifier can be, for example, an ID / name. Furthermore, the target model ID / name can, for example, identify the specific model instance being monitored.
[0040] The monitoring modes include full monitoring (intercepting all requests) and sampling monitoring (conducting in-depth analysis of a proportion, such as 10% of requests).
[0041] For example, the sampling rate parameter (if applicable) could be 0.1.
[0042] The monitoring metric set includes at least one of the following dimensions: security, accuracy, latency, and resource consumption. Monitoring plugins include at least two of the following: sensitive word filtering plugin, attention visualization plugin, hallucination detection plugin, compliance check plugin, and malicious command detector. The sensitive word filtering plugin configures the sensitive pattern file path and matching mode. The matching mode includes: exact matching and fuzzy matching. The hallucination detection plugin configures the knowledge base endpoint address and confidence threshold parameter (e.g., 0.85).
[0043] Optionally, the request may include: session identifier, original user input, and request metadata.
[0044] For example, a session identifier could be a session ID, used to connect the entire process. The original user input could be, for example, "Please explain how to make XX?". Request metadata could include information such as user ID and timestamps.
[0045] Before calling the input-side plugin to analyze the request and obtain the first analysis result, the process also includes: The original user input is segmented and vectorized preprocessed to extract the segmentation list, input text length, and semantic vector.
[0046] Specifically, the original user input undergoes preprocessing such as word segmentation and vectorization, and key features are extracted. Specific parameters may include, for example: a token list: ["please", "explain", "how", "make", "XX", "?"]. Input text length: input_length: 6 (number of tokens). Semantic vector: embedding_vector: [0.12, -0.05, ..., 0.78] (an example of a 768-dimensional vector).
[0047] The input-side plugin is invoked to analyze the request, yielding the first analysis result, including: The input-side plugin is invoked to analyze the preprocessed input and obtain the first analysis result.
[0048] The first analysis results include: risk level, risk type, and labeling information.
[0049] Specifically, the loaded input-side plugin (such as a malicious command detector) is invoked to analyze the processed input and obtain the plugin analysis results. For example, the security plugin returns: Risk level: Level 5; Risk type: Dangerous goods manufacturing; Marking information: ["manufacturing", "XX"].
[0050] After calling the input-side plugin to analyze the request and obtain the first analysis result, it also includes: The decision to block the request is determined based on the risk level and the preset blocking threshold.
[0051] The preset blocking threshold can be a threshold that should be blocked when there is a risk in the original user input information, set in advance based on actual conditions or experience. For example, it can be set according to the risk level. For instance, the risk level can be divided into five levels, with level one being the lowest risk and the subsequent levels increasing sequentially. When the risk level reaches level three, blocking should be performed. In this case, the preset blocking threshold can be level three.
[0052] Specifically, the model determines whether to block user input requests based on risk level and preset blocking threshold. It also performs question analysis and generates answers for user input requests.
[0053] Optionally, data capture can be triggered at a specified layer during the large model inference process via a pre-registered objective function, and metrics can be calculated on the captured raw data, including: Data capture is triggered at a specified layer during the large model inference process via pre-registered hook functions.
[0054] The specified layers include a Transformer attention layer and a multilayer perception layer. The captured raw data includes attention weight tensors and hidden state tensors. The shape of the attention weight tensor is: batch size, number of attention heads, sequence length, and sequence length. The shape of the hidden state tensor is: batch size, sequence length, and hidden layer dimension.
[0055] In the specific implementation, the captured attention weights (attention_weights) are a four-dimensional tensor with the shape (batch_size, num_heads, sequence_length, sequence_length). Example parameters: batch=1, num_heads=12, seq_len=6. Specifically, the data consists of the association score matrix of each output token with each input token in each attention head. The captured hidden states (hidden_states) are a three-dimensional tensor with the shape (batch_size, sequence_length, hidden_dim). Example parameters: at the output of a certain layer, each of the 6 tokens corresponds to a 768-dimensional feature vector.
[0056] Here, `batch_size` is the batch size, representing the number of samples processed in a single forward propagation. `num_heads` is the number of attention heads, representing the number of parallel attention branches in the multi-head attention mechanism. `sequence_length` is the sequence length, representing the number of tokens in the input text after word segmentation. `hidden_dim` is the hidden layer dimension, representing the dimensionality of the feature vectors within the model, i.e., the dimension of the representation vector corresponding to each token.
[0057] The captured raw data is used to calculate metrics, including the aggregation of multi-head attention weights, to generate an association matrix of sequence length × sequence length.
[0058] Specifically, the multi-head attention is averaged (or specified for a few heads) to obtain an association matrix of (sequence_length, sequence_length).
[0059] After calculating metrics from the captured raw data, the process also includes: A visualization file is generated based on the calculated metrics.
[0060] The visualization files include an attention heatmap.
[0061] Specifically, heatmap images are generated based on the calculated indicators.
[0062] The technical solution of this invention obtains the attention weights and intermediate layer features in the model inference process through a specific interface (Hook) and transforms them into a human-understandable visual representation (such as a heatmap).
[0063] Optionally, the output-side plugin can be invoked to analyze the raw output and obtain a second analysis result, including: The output-side plugin is invoked to perform multi-dimensional analysis on the original output, resulting in a second analysis result.
[0064] The multi-dimensional analysis includes at least two of the following: security testing, accuracy (illusion) testing, and compliance testing.
[0065] The second analysis result includes the detection score and result label.
[0066] For example, the model's original output is: "Making XX is an illegal and dangerous act...". The model's own confidence (if any): 0.92.
[0067] Specifically, the raw output of the large model is intercepted, and multiple output analysis plugins are called in parallel for analysis. For example: Security detection: The security plugin is called, returning a security score of 0.95 (the higher the score, the safer), and a hazard category identifier: [] (an empty list indicates no specific hazard category was found). Accuracy / illusion detection: The illusion detection plugin is called, comparing the output with the knowledge base. Parameters: Fact detection score: 0.88, Potential illusion: []. Compliance detection: The compliance plugin is called, checking according to industry rules. Parameters: Compliance detection: Passed, Applicable standards: ["Advertising Law", "Privacy Protection Regulations"].
[0068] Optionally, after generating the structured monitoring report, the following may also be included: Input the monitoring report into the rules engine, and trigger the corresponding action according to the predefined rules.
[0069] The actions include at least one of the following: blocking requests, sending alarms, and logging only. The rule engine's predefined rules include: if the input risk level is greater than or equal to a preset high-risk threshold, a blocking action is triggered, and a preset alternative response is returned; if the output accuracy score is lower than a preset accuracy threshold, an alarm action is triggered, and alarm information is sent.
[0070] Specifically, all monitoring results are aggregated and input into the rules engine for final decision-making. For example, the rules engine inputs key metrics from the reports (such as risk level, safety score, and fact detection score) as input variables for rule judgment, triggering actions based on the rules.
[0071] In the specific implementation process, the process and parameters can be, for example, the rule engine determines the final action based on predefined rules (e.g., if the input risk level is greater than or equal to the preset high-risk threshold, a blocking action is triggered and a preset alternative response is returned; if the output accuracy score is lower than the preset accuracy threshold, an alarm action is triggered and an alarm message is sent).
[0072] Finally, the entire data chain is persistently stored. The specific storage data structure includes: session ID, timestamp, original user input, raw data captured by the hook (which can be stored as a link to a file), raw output, analysis results of each plugin, final action, and reason, etc., all of the above parameters are fully recorded.
[0073] The technical solution of this invention proposes a unified monitoring agent method that can simultaneously handle input interception, internal state capture, and output detection, and uniformly manage monitoring rules and response actions. By monitoring the internal attention mechanism and intermediate states of a large model, it transforms the "black box" model into a "gray box," helping developers understand the model's decision-making logic and facilitating debugging and optimization. By adopting a plug-in architecture, different monitoring functions (such as security detection and performance analysis) are designed as independent plug-ins, supporting hot-swapping and dynamic configuration. This makes the monitoring system easy to expand and customize. Users can dynamically load or unload monitoring functions according to their needs without modifying the core model code, thus reducing maintenance costs.
[0074] Example 2 Figure 2 This is a schematic diagram of a large-scale model end-to-end monitoring device according to an embodiment of the present invention. This embodiment is applicable to the end-to-end monitoring of large-scale models. The device can be implemented using software and / or hardware methods, and can be integrated into any device that provides end-to-end monitoring functionality for large-scale models, such as... Figure 2As shown, the large-scale model full-link monitoring device specifically includes: an acquisition and recording module 201, a first analysis module 202, a capture and calculation module 203, a second analysis module 204, and a generation module 205.
[0075] The acquisition and recording module 201 is used to acquire the monitoring strategy and load the corresponding monitoring plugin from the plugin library according to the monitoring strategy; the monitoring plugin includes: input-side plugin and output-side plugin; The first analysis module 202 is used to intercept the request sent by the user to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; The capture and calculation module 203 is used to trigger data capture at a specified layer during the large model inference process through a pre-registered objective function, and to perform index calculation on the captured raw data. The second analysis module 204 is used to intercept the original output of the large model and call the output-side plugin to analyze the original output to obtain the second analysis result; The generation module 205 is used to generate a structured monitoring report based on the first analysis result, the calculated indicators, and the second analysis result.
[0076] Optionally, the monitoring strategy includes: a monitoring target model identifier, a monitoring mode, a sampling rate parameter, and a set of monitoring indicator switches; The monitoring modes include full monitoring and sampling monitoring; The set of monitoring indicator switches includes at least one of the following dimensions: security, accuracy, latency, and resource consumption. The monitoring plugins include at least two of the following: sensitive word filtering plugin, attention visualization plugin, hallucination detection plugin, compliance check plugin, and malicious instruction detector.
[0077] Optionally, the request includes: a session identifier, original user input, and request metadata; The device further includes: The preprocessing module is used to perform word segmentation and vectorization preprocessing on the original user input, and extract the word segmentation list, input text length and semantic vector; The first analysis module 202 is specifically used for: The input-side plugin is invoked to analyze the preprocessed input and obtain a first analysis result; the first analysis result includes: risk level, risk type, and labeling information; The device further includes: The determination module is used to determine the request blocking judgment result based on the risk level and the preset blocking threshold.
[0078] Optionally, the capture and calculation module 203 is specifically used for: Data capture is triggered at a specified layer during the large model inference process by pre-registered hook functions; the specified layer includes a Transformer attention layer and a multilayer perceptron layer, and the captured raw data includes attention weight tensors and hidden state tensors; the shape of the attention weight tensor is: batch size, number of attention heads, sequence length, sequence length; the shape of the hidden state tensor is: batch size, sequence length, hidden layer dimension; The captured raw data is used to calculate metrics, including the aggregation of multi-head attention weights, to generate an association matrix of sequence length × sequence length. After calculating metrics from the captured raw data, the process also includes: A visualization file is generated based on the calculated metrics, and the visualization file includes an attention heatmap.
[0079] Optionally, the second analysis module 204 is specifically used for: The output-side plugin is invoked to perform multi-dimensional analysis on the original output to obtain a second analysis result; the multi-dimensional analysis includes at least two of the following: security detection, accuracy detection, and compliance detection; the second analysis result includes a detection score and a result identifier.
[0080] Optionally, the device further includes: The input and trigger module is used to input the monitoring report into the rule engine and trigger corresponding actions according to predefined rules; the actions include at least one of: blocking requests, sending alarms, and logging only. The predefined rules of the rule engine include: if the input risk level is greater than or equal to a preset high-risk threshold, a blocking action is triggered and a preset alternative response is returned; if the output accuracy score is lower than a preset accuracy threshold, an alarm action is triggered and an alarm message is sent.
[0081] The above-mentioned products can execute the large-model full-link monitoring method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects of the execution method.
[0082] Example 3 Figure 3A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0083] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.
[0084] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0085] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as the large-model end-to-end monitoring method: Obtain the monitoring policy and load the corresponding monitoring plugin from the plugin library according to the monitoring policy; the monitoring plugin includes: input-side plugin and output-side plugin; Intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; Data capture is triggered at a specified layer during the large model inference process by using a pre-registered objective function, and metrics are calculated on the captured raw data. The original output of the large model is intercepted, and the output-side plugin is called to analyze the original output to obtain a second analysis result; Based on the first analysis results, the calculated indicators, and the second analysis results, a structured monitoring report is generated.
[0086] In some embodiments, the large-scale model end-to-end monitoring method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the large-scale model end-to-end monitoring method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the large-scale model end-to-end monitoring method by any other suitable means (e.g., by means of firmware).
[0087] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0088] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0089] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0091] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0092] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0093] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the large-model end-to-end monitoring method of any embodiment of the present invention.
[0094] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A large-scale model end-to-end monitoring method, characterized in that, include: Obtain the monitoring policy and load the corresponding monitoring plugin from the plugin library according to the monitoring policy; The monitoring plugin includes: an input-side plugin and an output-side plugin; Intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; Data capture is triggered at a specified layer during the large model inference process by using a pre-registered objective function, and metrics are calculated on the captured raw data. The original output of the large model is intercepted, and the output-side plugin is called to analyze the original output to obtain a second analysis result; Based on the first analysis results, the calculated indicators, and the second analysis results, a structured monitoring report is generated.
2. The method according to claim 1, characterized in that, The monitoring strategy includes: monitoring target model identifier, monitoring mode, sampling rate parameter, and a set of monitoring indicator switches; The monitoring modes include full monitoring and sampling monitoring; The set of monitoring indicator switches includes at least one of the following dimensions: security, accuracy, latency, and resource consumption. The monitoring plugins include at least two of the following: sensitive word filtering plugin, attention visualization plugin, hallucination detection plugin, compliance check plugin, and malicious instruction detector.
3. The method according to claim 1, characterized in that, The request includes: session identifier, original user input, and request metadata; Before calling the input-side plugin to analyze the request and obtain the first analysis result, the process also includes: The original user input is segmented and vectorized preprocessed to extract the segmentation list, input text length, and semantic vector. The input-side plugin is invoked to analyze the request, and a first analysis result is obtained, including: The input-side plugin is invoked to analyze the preprocessed input and obtain a first analysis result; the first analysis result includes: risk level, risk type, and labeling information; After calling the input-side plugin to analyze the request and obtain the first analysis result, the process further includes: The request blocking judgment result is determined based on the risk level and the preset blocking threshold.
4. The method according to claim 1, characterized in that, Data capture is triggered at a specified layer during the large model inference process using a pre-registered objective function, and metrics are calculated on the captured raw data, including: Data capture is triggered at a specified layer during the large model inference process by pre-registered hook functions; the specified layer includes a Transformer attention layer and a multilayer perceptron layer, and the captured raw data includes attention weight tensors and hidden state tensors; the shape of the attention weight tensor is: batch size, number of attention heads, sequence length, sequence length; the shape of the hidden state tensor is: batch size, sequence length, hidden layer dimension; The captured raw data is used to calculate metrics, including the aggregation of multi-head attention weights, to generate an association matrix of sequence length × sequence length. After calculating metrics from the captured raw data, the process also includes: A visualization file is generated based on the calculated metrics, and the visualization file includes an attention heatmap.
5. The method according to claim 1, characterized in that, The output-side plugin is invoked to analyze the original output, resulting in a second analysis result, including: The output-side plugin is invoked to perform multi-dimensional analysis on the original output to obtain a second analysis result; the multi-dimensional analysis includes at least two of the following: security detection, accuracy detection, and compliance detection; the second analysis result includes a detection score and a result identifier.
6. The method according to claim 1, characterized in that, After generating the structured monitoring report, the following is also included: The monitoring report is input into the rule engine, and corresponding actions are triggered according to predefined rules; the actions include at least one of the following: blocking requests, sending alarms, and logging only. The predefined rules of the rule engine include: if the input risk level is greater than or equal to a preset high-risk threshold, a blocking action is triggered and a preset alternative response is returned; if the output accuracy score is lower than a preset accuracy threshold, an alarm action is triggered and an alarm message is sent.
7. A large-scale model end-to-end monitoring device, characterized in that, include: The acquisition and recording module is used to acquire the monitoring strategy and load the corresponding monitoring plugin from the plugin library according to the monitoring strategy; the monitoring plugin includes: input-side plugin and output-side plugin; The first analysis module is used to intercept the user's request to the large model, call the input-side plugin to analyze the request, and obtain the first analysis result; The capture and calculation module is used to trigger data capture at a specified layer during the large model inference process through a pre-registered objective function, and to perform index calculations on the captured raw data. The second analysis module is used to intercept the original output of the large model and call the output-side plugin to analyze the original output to obtain the second analysis result; The generation module is used to generate a structured monitoring report based on the first analysis results, the calculated indicators, and the second analysis results.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the large model full-link monitoring method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the large model full-link monitoring method according to any one of claims 1-6.
10. A computer program product comprising a computer program that, when executed by a processor, implements the large-scale model end-to-end monitoring method according to any one of claims 1-6.