Large model application service operation tracking method and device

By integrating tracking services in large-model application services, real-time monitoring and tracking of operation links, the problem of difficult monitoring of the operating status of large-model application services in private cloud environments is solved, and the stability and operation efficiency of the business system are improved.

CN119094599APending Publication Date: 2024-12-06CHINA ELECTRONICS JINXIN DIGITAL TECH GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411190113.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When a large-model application service is running in a private cloud environment, it is difficult to understand the operating status and performance in real time, which makes it difficult to discover and solve potential problems in a timely manner, affecting the stability of the business system.

Method used

By obtaining the tracking service associated with the big model application, sending service registration requests to the server, authenticating and registering the tracking service, integrating the tracking service into the big model application service, and using the link tracking callback function to monitor and track various link nodes in real time , realize real-time monitoring and tracking of the operating link.

Benefits of technology

Improve the accuracy and observability of operation tracking, enhance the security and flexibility of tracking services, realize real-time monitoring and tracking of operation links without intruding into business logic, ensuring long-term storage and security of data helps developers quickly locate problems and analyze performance bottlenecks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094599A_ABST
    Figure CN119094599A_ABST
Patent Text Reader

Abstract

The invention provides a large model application service operation tracking method and device. The method comprises the steps of obtaining a tracking service associated with a large model application end; sending a service registration request to a server; wherein the service registration request carries a tracking service and is used for the server to authenticate the large-model application side and registering the tracking service under the condition that the authentication is passed; receiving a service registration response sent by the server; under the condition that the service registration response indicates that the tracking service is successfully registered, integrating the tracking service into a to-be-tracked large-model application service of the large-model application end; and tracking each link node in the operation link of the large-model application service by adopting the link tracking callback function associated with the tracking service, thereby realizing real-time monitoring and tracking of the operation information of each link node in the operation link of the large-model application service under the condition of not invading the business logic of the large-model application service. And the observability and the high efficiency of operation tracking are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for tracking the operation of a large model application service. Background Art

[0002] At present, the localization technology of large model application services is becoming increasingly popular, and more and more large model application services are running in the enterprise private cloud environment. The operation of large model application services in the private cloud environment not only improves the security and privacy protection capabilities of the business system, but also reduces external dependence and enhances autonomous controllability. It brings more flexible and efficient data solutions to enterprises.

[0003] However, when the large model application service runs in a private cloud environment, it may be affected by various factors, such as hardware resource limitations, network fluctuations, data input anomalies, etc. It is difficult to understand the status and performance of the large model application service at each operation stage in real time, making it difficult to discover and solve potential problems in time, affecting the stability of the business system. By real-time tracking of each link node in the large model application service operation link, potential problems can be discovered and solved in time, improving the stability of the business system. Therefore, how to realize the operation tracking of the large model application service is very important. Summary of the invention

[0004] The present disclosure provides a large model application service operation tracking method and device to at least solve one of the technical problems in the related art to a certain extent. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a large model application service operation tracking method is provided, which is applied to a large model application end, and includes: obtaining a tracking service associated with the large model application end; sending a service registration request to a server end, wherein the service registration request carries the tracking service, and is used by the server end to authenticate the large model application end, and register the tracking service if the authentication is successful; receiving a service registration response sent by the server end; if the service registration response indicates that the tracking service is successfully registered, integrating the tracking service into the large model application service to be tracked of the large model application end; and using a link tracking callback function associated with the tracking service to track each link node in the operation link of the large model application service.

[0006] According to a second aspect of an embodiment of the present disclosure, a large model application service operation tracking device is provided, which is applied to a large model application end, and includes: an acquisition module, used to acquire a tracking service associated with the large model application end; a registration module, used to send a service registration request to a server end, wherein the service registration request carries the tracking service, and is used by the server end to authenticate the large model application end, and register the tracking service if the authentication is successful; a receiving module, used to receive a service registration response sent by the server end; an integration module, used to integrate the tracking service into the large model application service to be tracked of the large model application end if the service registration response indicates that the tracking service is successfully registered; a tracking module, used to track each link node in the operation link of the large model application service using a link tracking callback function associated with the tracking service.

[0007] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the large model application service operation tracking method as described in the embodiment of the first aspect of the present disclosure.

[0008] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the large model application service operation tracking method as described in the embodiment of the first aspect of the present disclosure.

[0009] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including: a computer program, which, when executed by a processor, implements the large model application service operation tracking method as described in the embodiment of the first aspect of the present disclosure.

[0010] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0011] In this technical solution, by acquiring the tracking service associated with the large model application end, the accuracy and pertinence of the operation tracking are ensured, making the operation tracking more in line with actual needs; then, a service registration request containing the tracking service is sent to the server end, the server end authenticates the large model application end, and after the authentication is passed and the tracking service is registered, the security of the tracking service is improved. Then, the large model application end receives the service registration response, and when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked, and by adopting the link tracking callback function closely associated with the tracking service, each link node in the operation link of the large model application service is tracked, thereby realizing real-time monitoring and tracking of the operation information of each link node in the operation link of the large model application service without intruding the business logic of the large model application service, thereby improving the observability and efficiency of the operation tracking; wherein, when the tracking service is sent to the server end, the tracking service is sent to the server end. When the service is integrated into the large model application service to be tracked on the large model application side, by creating a tracking object in the large model application service to carry the tracking service through the tracking object, the flexibility of the large model application service operation tracking is improved, and the decoupling from the business logic of the large model application service is realized, and then the operation link of the large model application service is obtained; the tracking object is added to the configuration information of the operation link as a tracking callback function, and the real-time monitoring and tracking of the operation information of each link node in the operation link of the large model application service is realized without invading the business logic of the large model application service; in addition, after tracking each link node in the operation link of the large model application service, the link tracking data of each link node is obtained, and each link tracking data is sent to the server side to store and display each link tracking data, so as to realize the long-term preservation and security of each link tracking data and avoid the risk of data loss or damage. In addition, through the display function provided by the server, the link tracking data can be presented intuitively, helping developers and operation and maintenance personnel to quickly locate problems and analyze performance bottlenecks.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0014] Figure 1 It is a flowchart of a large model application service operation tracking method shown in the first embodiment of the present disclosure;

[0015] Figure 2It is a flowchart of a large model application service operation tracking method shown in the second embodiment of the present disclosure;

[0016] Figure 3 It is a flowchart of a large model application service operation tracking method shown in the third embodiment of the present disclosure;

[0017] Figure 4 It is a flowchart of a large model application service operation tracking method shown in the fourth embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of a large model application service operation tracking method shown in the fifth embodiment of the present disclosure;

[0019] Figure 6 It is a flowchart of a large model application service operation tracking method shown in an embodiment of the present disclosure;

[0020] Figure 7 It is a structural diagram of a large model application service operation tracking method shown in an embodiment of the present disclosure;

[0021] Figure 8 It is a structural diagram of a large model application service operation tracking device shown in the sixth embodiment of the present disclosure;

[0022] Fig. 9 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0026] At present, the trend of big model technology is more centered around the optimization of the model itself and the application of the model (such as the Prompt project). There is a lack of technical means for full-link tracking of big model application services during operation, which poses challenges to the stability, performance, and maintainability of big model application services during operation.

[0027] Therefore, in order to solve the above problems, the present disclosure proposes a method and device for tracking the operation of a large model application service.

[0028] The following describes the large model application service operation tracking method and device of the embodiment of the present disclosure with reference to the accompanying drawings.

[0029] Figure 1 It is a flow chart of the large model application service operation tracking method shown in the first embodiment of the present disclosure. It should be noted that the embodiment of the present disclosure is applied to the large model application end, and the large model application end accesses and uses the large model application service.

[0030] like Figure 1 As shown, the large model application service operation tracking method includes the following steps:

[0031] Step 101, obtain the tracking service associated with the large model application end.

[0032] In order to track the big model application service in real time, as a possible implementation method, the big model application service is tracked based on the tracking service associated with the big model application end. Among them, the tracking service contains the tracking logic code, and the tracking logic is used to monitor, record and analyze the various link nodes in the operation process of the big model application service in real time, so as to understand the operation status of the big model application service, find potential problems and optimize service performance.

[0033] As an example, in response to the tracing logic code development operation, a tracing service associated with the large model application end is generated.

[0034] As another example, obtain an open source tracking framework, introduce the open source tracking framework into the large model application end, and configure the corresponding tracking logic to obtain the tracking service.

[0035] Step 102: Send a service registration request to the server.

[0036] The service registration request carries the tracking service, which is used by the server to authenticate the large model application, and register the tracking service if the authentication is successful.

[0037] In order to improve the security and legitimacy of the large model application, as a possible implementation method, the large model application is authenticated on the server side, and if the authentication is passed, the tracking service associated with the large model application is registered.

[0038] As an example, the large model application sends a service registration request to the server, wherein the registration request carries the tracking service. When the server receives the registration request, it authenticates the large model application and registers the tracking service if the authentication passes.

[0039] Step 103: Receive a service registration response sent by the server.

[0040] It should be understood that when the registration tracking service is completed, a service registration response is sent to the large model application end, and then the large model application end receives the service registration response sent by the server end, wherein the service registration can indicate that the tracking registration is successful or unsuccessful.

[0041] Step 104 , when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the big model application service to be tracked at the big model application end.

[0042] It should be understood that when the service registration response indicates that the tracking service registration is successful, it means that the tracking service associated with the large model application end has been accepted by the server, allowing tracking of each link node running the large model application service.

[0043] In order to achieve effective tracking of each link node of the large model application service, as a possible implementation method, when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked.

[0044] As an example, the tracking service is directly configured in the large model application service to be tracked.

[0045] As another example, the tracking logic of the tracking service is integrated into the large model application service code to be tracked.

[0046] In order to avoid unsuccessful registration of tracking service due to network problems, as a possible implementation method, when the service registration response indicates that the authentication of the large model application end is passed and the tracking service registration is unsuccessful, the service registration request is re-sent to the server end; when the re-received service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked of the large model application end.

[0047] In addition, in order to let relevant personnel know the reasons for the unsuccessful registration of the tracking service in a timely manner, as an example, when the re-received service registration response indicates that the tracking service registration is unsuccessful, the target information associated with the unsuccessful registration of the tracking service is obtained from the log; based on the target information, a prompt message is generated; wherein the prompt message is used to prompt that the large model application end authentication has passed and the tracking service registration is unsuccessful.

[0048] Step 105, using the link tracking callback function associated with the tracking service to track each link node in the running link of the large model application service.

[0049] In order to timely and effectively track each link node in the running link of the large model application service without intruding the business logic of the large model application service, as an example, a link tracking callback function associated with the tracking service is used to perform real-time tracking on each node in the running link of the large model application service.

[0050] The large model application service operation tracking method of the disclosed embodiment is applied to the large model application end, and by obtaining the tracking service associated with the large model application end, the accuracy and pertinence of the operation tracking are ensured, so that the operation tracking is more in line with the actual needs; then, a service registration request including the tracking service is sent to the server end, and the server end authenticates the large model application end, and registers the tracking service after the authentication is passed, thereby improving the security of the tracking service; further, the large model application end receives the service registration response, and when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked; by adopting the link tracking callback function closely associated with the tracking service, each link node in the operation link of the large model application service is tracked, thereby realizing real-time monitoring and tracking of the operation information of each link node in the operation link of the large model application service without intruding the business logic of the large model application service, thereby improving the observability and efficiency of the operation tracking.

[0051] In order to clearly illustrate how in the above embodiment, when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the big model application service to be tracked on the big model application side, the present disclosure proposes another big model application service operation tracking method.

[0052] Figure 2 It is a flow chart of a large model application service operation tracking method shown in the second embodiment of the present disclosure. It should be noted that the embodiment of the present disclosure is applied to the large model application end.

[0053] like Figure 2 As shown, the large model application service operation tracking method includes the following steps:

[0054] Step 201, obtain the tracking service associated with the large model application end.

[0055] Step 202: Send a service registration request to the server.

[0056] The service registration request carries the tracking service, which is used by the server to authenticate the large model application, and register the tracking service if the authentication is successful.

[0057] Step 203: Receive the service registration response sent by the server.

[0058] Step 204: When the service registration response indicates that the trace service registration is successful, create a trace object in the large model application service to carry the trace service through the trace object.

[0059] To improve the flexibility of the large model application service operation tracing and achieve decoupling from the business logic of the large model application service, as an example, create a trace object in the large model application service and use the trace object to carry the trace service.

[0060] Step 205: Obtain the operation link of the large model application service.

[0061] As a possible implementation, based on the business source code associated with the large model application service, determine the operation link of the large model application service, where each link node in the operation link is used to indicate the processing operations of the large model application service at different operation stages.

[0062] As an example, the business source code is the source code of the business logic of the pre-written large model application service, and the source code indicates the processing operations of the large model application service at different operation stages.

[0063] Step 206: Add the trace object as a trace callback function to the configuration information of the operation link.

[0064] To achieve real-time monitoring and tracing of the operation information of each link node in the operation link of the large model application service without intruding into the business logic of the large model application service, as an example, add the trace object as a trace callback function to the configuration information of the operation link.

[0065] For example, if the operation link is chain2, the trace object can be added as a trace callback function to the configuration information of the operation link in the following way:

[0066] chain2.invoke({"person":"So-and-so","language":"Chinese"},config={"callbacks":[traceclient_handler]})。

[0067] Step 207: Use the link trace callback function associated with the trace service to trace each link node in the operation link of the large model application service.

[0068] Furthermore, when the large model application service executes to a link node in the link, the link tracking callback function associated with the link node will be automatically triggered. The tracking callback function is responsible for collecting information about the current node execution status, such as execution time, input data, output data, exception information, etc.

[0069] It should be noted that the execution process of steps 201 to 203 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0070] In summary, by creating a tracking object in the big model application service when the service registration response indicates that the tracking service registration is successful, and using the tracking object to carry the tracking service, the flexibility of the big model application service operation tracking is improved, and decoupling from the business logic of the big model application service is achieved, thereby obtaining the operation link of the big model application service; wherein, the nodes in the operation link are used to indicate the processing operations of the big model application service at different operation stages; the tracking object is added as a tracking callback function to the configuration information of the operation link, thereby achieving real-time monitoring and tracking of the operation information of each link node in the operation link of the big model application service without intruding the business logic of the big model application service.

[0071] In order to clearly describe how to obtain the tracking service associated with the large model application end in the above embodiment, the present disclosure proposes another large model application service operation tracking method.

[0072] Figure 3 It is a flow chart of a large model application service operation tracking method shown in the third embodiment of the present disclosure. It should be noted that the embodiment of the present disclosure is applied to the large model application end.

[0073] like Figure 3 As shown, the large model application service operation tracking method includes the following steps:

[0074] Step 301, obtain the link start calling function.

[0075] The link start calling function is used to initialize the tracking information when each link node starts running.

[0076] As an example, in response to the encoding operation, a link start calling function is generated, wherein the link start calling function is used to initialize the tracking information when each link node starts running.

[0077] As another example, a link start calling function for initializing tracking information when each link node starts running is obtained from a source code library.

[0078] For example, taking the link start calling function as on_chain_start(), the link start calling function receives the configuration information (serialized), input data (inputs), unique identifier (run_id), parent identifier (parent_run_id), tags (tags), metadata (metadata) and other tracking information about the new chain, and initializes the tracking information when each link node starts running.

[0079] Step 302, obtaining a link tracking information generation function.

[0080] Among them, the link tracking information generation function is used to generate the tracking information of the current link node, and the tracking information of the current link node is generated according to the initialized tracking information of the current link node.

[0081] As an example, in response to an encoding operation, a link tracking information generation function is generated; wherein the link tracking information generation function is used to generate tracking information of a current link node, and the tracking information of the current link node is generated based on the initialized tracking information of the current link node.

[0082] As another example, a link tracking information generation function for generating tracking information of a current link node is obtained from a source code library.

[0083] For example, taking the link tracking information generation function self.__generate_trace_and_parent() as an example, the self.__generate_trace_and_parent() method receives multiple parameters, such as serialized configuration information (serialized), input data inputs, run ID (run_id), parent run ID (parent_run_id), tags (tags), metadata (metadata) and version information (version).

[0084] Step 303: Obtain a link tracking content construction dictionary.

[0085] Among them, the link tracking content construction dictionary is used to store the tracking information of the current link node.

[0086] For example, the link tracking content is content, which is used to store the tracking information of the current link node. Content includes multiple fields associated with multiple parameters in the link tracking information generation function (such as "id", "trace_id") and context information of large model application services (such as "metadata", "input" and "version", etc.).

[0087] Step 304, construct a dictionary according to the link start calling function, the link tracking information generating function and the link tracking content to generate a tracking service.

[0088] In order to improve the maintainability and scalability of the tracking service, the link start calling function, the link tracking information generating function and the link tracking content construction dictionary are further encapsulated to obtain the tracking service.

[0089] It should be noted that, in order to facilitate troubleshooting during operation, the tracking service also includes: abnormal recording and processing during operation tracking.

[0090] Step 305: Send a service registration request to the server.

[0091] The service registration request carries the tracking service, which is used by the server to authenticate the large model application end, and register the tracking service if the authentication is successful.

[0092] Step 306: Receive a service registration response sent by the server.

[0093] Step 307: When the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked at the large model application end.

[0094] Step 308: Use the link tracking callback function associated with the tracking service to track each link node in the running link of the large model application service.

[0095] It should be noted that the execution process of steps 305 to 308 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0096] In summary, the tracking service is generated by obtaining the link start calling function; obtaining the link tracking information generating function; obtaining the link tracking content constructing dictionary; constructing the dictionary according to the link start calling function, the link tracking information generating function and the link tracking content. Thus, by defining the link start calling function, the tracking information is initialized when each link node is started, providing a basis for subsequent tracking; the link tracking information generating function is used to dynamically generate detailed tracking information according to the status of the current node to ensure the real-time and accuracy of the information; constructing a dictionary based on the link tracking content used to store the tracking information can provide rich context for analysis; finally, the link start calling function, the link tracking information generating function and the link tracking content are integrated into the tracking service to improve the maintainability and scalability of the tracking service.

[0097] In order to facilitate the storage and display of the tracking data of each link node on the running link of the large model application service, as an example, the large model application end sends the tracking data of each link node to the server end, and the server end stores and displays the tracking data of each link. Figure 4 The embodiments are described in detail.

[0098] Figure 4 It is a flow chart of a large model application service operation tracking method shown in the fourth embodiment of the present disclosure. It should be noted that the embodiment of the present disclosure is applied to the large model application end.

[0099] like Figure 4 As shown, the large model application service operation tracking method includes the following steps:

[0100] Step 401, obtaining link tracking data of each link node.

[0101] Among them, the link tracking data is obtained by tracking the corresponding link nodes in the running link of the large model application service.

[0102] In order to facilitate real-time monitoring and in-depth analysis of the entire operation process of the large model application service, so as to quickly locate and solve performance problems, as an example, the link tracking data obtained by tracking the corresponding link nodes in the operation link of the large model application service is obtained to store and display the link tracking data of each link node.

[0103] Step 402, sending each link tracking data to the server to store and display each link tracking data.

[0104] As an example, each link tracking data is sent to a server, which stores each link tracking data and processes each link tracking data for display.

[0105] In summary, by obtaining the link tracking data of each link node and sending each link tracking data to the server to store and display each link tracking data, the long-term preservation and security of each link tracking data is achieved, avoiding the risk of data loss or damage. In addition, through the display function provided by the server, the link tracking data can be presented intuitively, helping developers and operation and maintenance personnel to quickly locate problems and analyze performance bottlenecks.

[0106] Based on the above embodiment, after receiving the tracking data of each link, the server Figure 5 As shown, the following steps are used to store and display the tracking data of each link:

[0107] Step 501, using a collector to receive the link tracking data sent by the large model application end.

[0108] It should be understood that, since the collector is a receiver of link tracking data, it can receive data from different large model application terminals for subsequent storage and display. Therefore, as an example, a collector is used to receive each link tracking data sent by the large model application terminal.

[0109] Step 502: Send each link tracking data to the storage through the collector.

[0110] Furthermore, in order to facilitate the storage of each link tracking data, each link tracking data is sent to the storage through the collector.

[0111] Step 503: store each link tracking data in a specified storage format through a memory.

[0112] To ensure data consistency and readability, as an example, the memory stores each link tracking data in a specified storage format.

[0113] Step 504 , for any link tracking data, parse any link tracking data to obtain structure tracking data matching the specified storage format.

[0114] In order to facilitate the display of link tracking data in a unified structured format, as a possible implementation method, the link tracking data is parsed and converted into a unified structured format.

[0115] As an example, any link tracking data is converted into structure tracking data that complies with a specified storage format.

[0116] Step 505: Display the structure tracking data corresponding to any link tracking data.

[0117] As an example, the display logic of the structure tracking data corresponding to any link tracking data is encapsulated to obtain the encapsulated display logic; the encapsulated display logic and the structure tracking data corresponding to any link tracking data are sent to a display; and the structure tracking data corresponding to any link tracking data is displayed through the display using the encapsulated display logic.

[0118] It is important to understand that link tracking data usually has a complex structure, such as nested objects, arrays, etc. In order to effectively display this information, relevant developers write one or more display logics according to the display requirements. The display logic can parse these complex data structures and convert them into easy-to-understand formats (such as charts, tree diagrams, timelines, etc.). In order to facilitate the reuse of the display logic when needed, the display logic is encapsulated, and the encapsulated display logic and the structure tracking data corresponding to any link tracking data are sent to the display together. Finally, the display uses the encapsulated display logic to display the structure tracking data corresponding to each link tracking data.

[0119] In summary, a collector is used to receive each link tracking data sent by the large model application end; each link tracking data is sent to a storage device through the collector; each link tracking data is stored in a specified storage format through the storage device; for any link tracking data, any link tracking data is parsed to obtain structure tracking data that matches the specified storage format; the structure tracking data corresponding to any link tracking data is displayed, thereby achieving efficient and complete storage and display of each link tracking data.

[0120] Based on any embodiment of the present disclosure, Figure 6 As shown, the large model application service operation tracking method of the embodiment of the present disclosure can also be implemented based on the following steps:

[0121] S01. Design and build the overall tracking and observation service system, and the overall observable service system architecture Figure 7 As shown, among which, for Figure 7 The TraceClient and TraceServer in are explained separately;

[0122] 1. TraceClient (Large Model Application)

[0123] That is, the client of link tracking. Since the current mainstream large model applications are almost all developed in python (a high-level programming language) and JS (JavaScript, a high-level programming language), the client provides two language versions, python and JS, and can support seamless integration into various mainstream large model application frameworks, including traditional python / pytorch, langchain, Llama, Open-AI, etc. The client provides a series of pluggable tracking link monitoring indicator collection functions. Before use, it must first register with the authentication (Auth) module on the server. During the operation, the link data accessed by the large model application is collected and sent back to the collection (collector) module on the server for link tracking data storage;

[0124] 2. TraceServer (server)

[0125] The link tracking server mainly includes the following modules:

[0126] Collector module: monitors data collection, is responsible for communicating with TraceClient, and collects relevant tracking link data;

[0127] Storage module: responsible for storing the collected tracking link data. In general scenarios, relational databases can be used for related storage. In massive data scenarios, distributed search engines such as Elasticsearch can be used to expand performance.

[0128] MonitorService module: It is the logical encapsulation of the core service for running tracking. It is responsible for assembling the display logic of the collected monitoring data, connecting the collection (collector) module and the authentication (auth) module, and providing specific service implementation for the external monitoring interface (MonitorAPI) module.

[0129] Monitoring interface (MonitorAPI) module: external interface layer, providing service interface to the external and UI (user interface) layers;

[0130] Authentication (Auth) module: provides TraceClient identity registration and authentication services;

[0131] Dashboard UI (Dashboard User Interface): It is a display that provides a dashboard for link tracking data and is responsible for the front-end interface display;

[0132] S02. Integrate TraceClient on the LLM application side and collect relevant link monitoring data. The following takes the more popular Langchain framework as an example to show the main integration process:

[0133] The first step is to obtain the tracking service associated with the large model application;

[0134] The second step is to send a service registration request to the server to authenticate and register the TraceClient remotely.

[0135] Among them, the service registration request carries the tracking service (CallbackHandler);

[0136] The third step is to integrate relevant tracking services into the large model application service;

[0137] As an example, a tracking object is created in the large model application service to carry the tracking service through the tracking object; the running link of the large model application service is obtained; the tracking object is added as a tracking callback function to the configuration information of the running link; for example, taking the running link of the large model application service as chain2 and the tracking object as traceclient_handler as an example, the non-invasive collection of the tracking link data of the large model LLM application is realized through the tracking callback function callback, that is, the link tracking data of the large model LLM application is collected through the code chain2.invoke({"person":"某某","language":"中文"},config={"callbacks":[trace client_handler]});

[0138] It should be noted that the tracking service will collect the ID of the running large model application service, the metadata during the running process (such as, including the request timestamp, the source IP address of the request, user authentication information, service version, etc.), and the context information of the actual large model application service, and encapsulate the large model application service ID, the metadata during the running process (such as, including the request timestamp, the source IP address of the request, user authentication information, service version, etc.), and the context information of the actual large model application service into the link tracking content (content), and then perform overall transmission;

[0139] S03. TraceServer uniformly stores the link tracking data collected by TraceClient;

[0140] The underlying storage structure needs to be pre-designed, and the storage structure of the link tracking data can be shown in Table 1:

[0141] Table 1 Table fields and corresponding explanations of table fields

[0142]

[0143] The collection module of TraceServer will synchronously store it in the corresponding database service after receiving the message passed from the large model application side;

[0144] S04. TraceServer parses based on the already stored link tracking data, designs relevant functional modules, and provides relevant application programming interfaces (Application Programming Interface, abbreviated as API);

[0145] It should be noted that the core services of TraceServer include the following modules:

[0146] TraceQueryDAO: used for delivery to the database and provides data access persistence layer services;

[0147] TraceQueryService: used to encapsulate complex display logic;

[0148] Trace Query API: provides external service API;

[0149] S05. Based on the data returned by TraceQueryAPI, information is displayed in DashboardUI. It should be noted that the current mainstream front-end VUE and React (front-end JavaScript framework) both support DashboardUI.

[0150] Corresponding to the large model application service operation tracking method provided in the above-mentioned embodiment, the present disclosure also provides a large model application service operation tracking device. Since the large model application service operation tracking device provided in the embodiment of the present disclosure corresponds to the large model application service operation tracking method provided in the above-mentioned embodiment, the implementation method of the large model application service operation tracking method is also applicable to the large model application service operation tracking device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0151] Figure 8 It is a structural diagram of a large model application service operation tracking device shown in the sixth embodiment of the present disclosure.

[0152] like Figure 8 As shown, the large model application service operation tracking device 800 includes: an acquisition module 810, a registration module 820, a receiving module 830, an integration module 840 and a tracking module 850.

[0153] Among them, the acquisition module 810 is used to obtain the tracking service associated with the large model application end; the registration module 820 is used to send a service registration request to the server end, wherein the service registration request carries the tracking service, and the server end authenticates the large model application end, and registers the tracking service when the authentication is passed; the receiving module 830 is used to receive the service registration response sent by the server end; the integration module 840 is used to integrate the tracking service into the large model application service to be tracked on the large model application end when the service registration response indicates that the tracking service registration is successful; the tracking module 850 is used to use the link tracking callback function associated with the tracking service to track each link node in the running link of the large model application service.

[0154] As a possible implementation method of the embodiment of the present disclosure, the integration module 840 is used to create a tracking object in the big model application service when the service registration response indicates that the tracking service registration is successful, so as to carry the tracking service through the tracking object; obtain the running link of the big model application service; wherein the link nodes in the running link are used to indicate the processing operations of the big model application service in different running stages; and add the tracking object as a tracking callback function to the configuration information of the running link.

[0155] As a possible implementation method of the embodiment of the present disclosure, the acquisition module 810 is used to obtain a link start calling function; wherein the link start calling function is used to initialize tracking information when each link node starts to run; obtain a link tracking information generation function; wherein the link tracking information generation function is used to generate tracking information of the current link node, and the tracking information of the current link node is generated according to the initialized tracking information of the current link node; obtain a link tracking content construction dictionary; wherein the link tracking content construction dictionary is used to store the tracking information of the current link node; generate a tracking service according to the link start calling function, the link tracking information generation function and the link tracking content construction dictionary.

[0156] As a possible implementation of the embodiment of the present disclosure, the large model application service operation tracking device 800 also includes: a sending module.

[0157] Among them, the sending module is used to obtain the link tracking data of each link node; wherein the link tracking data is obtained by tracking the corresponding link nodes in the running link of the large model application service; each link tracking data is sent to the server to store and display each link tracking data.

[0158] As a possible implementation method of the embodiment of the present disclosure, a storage module is used to store each link tracking data, wherein the storage module is used to use a collector to receive each link tracking data sent by the large model application end; send each link tracking data to the memory through the collector; and store each link tracking data in a specified storage format through the memory.

[0159] As a possible implementation method of the embodiment of the present disclosure, a display module is used to display each link tracking data, wherein the display module is used to parse any link tracking data to obtain structural tracking data matching a specified storage format; and to display the structural tracking data corresponding to any link tracking data.

[0160] As a possible implementation method of the embodiment of the present disclosure, a display module is used to encapsulate the display logic of the structure tracking data corresponding to any link tracking data to obtain the encapsulated display logic; send the encapsulated display logic and the structure tracking data corresponding to any link tracking data to a display; and display the structure tracking data corresponding to any link tracking data through the display using the encapsulated display logic.

[0161] As a possible implementation method of the embodiment of the present disclosure, the registration module 820 is also used to re-send the service registration request to the server side when the service registration response indicates that the authentication of the large model application side is passed and the tracking service registration is unsuccessful; re-receive the service registration response sent by the server side; and integrate the tracking service into the large model application service to be tracked on the large model application side when the re-received service registration response indicates that the tracking service registration is successful.

[0162] As a possible implementation of the embodiment of the present disclosure, the large model application service operation tracking device 800 also includes: a generation module.

[0163] Among them, the generation module is used to obtain target information associated with the unsuccessful tracking service registration from the log when the re-received service registration response indicates that the tracking service registration is unsuccessful; generate prompt information based on the target information; wherein the prompt information is used to prompt that the large model application end authentication has passed and the tracking service registration is unsuccessful.

[0164] The large model application service operation tracking device of the disclosed embodiment ensures the accuracy and pertinence of operation tracking by acquiring the tracking service associated with the large model application end, so that the operation tracking is more in line with actual needs; then, a service registration request including the tracking service is sent to the server end, the server end authenticates the large model application end, and after the authentication is passed and the tracking service is registered, the security of the tracking service is improved; then, the large model application end receives the service registration response, and when the service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked; by adopting the link tracking callback function closely associated with the tracking service, each link node in the operation link of the large model application service is tracked, thereby realizing real-time monitoring and tracking of the operation information of each link node in the operation link of the large model application service without intruding the business logic of the large model application service, thereby improving the observability and efficiency of operation tracking.

[0165] In an exemplary embodiment, an electronic device is also provided.

[0166] Among them, electronic equipment includes:

[0167] processor;

[0168] a memory for storing processor-executable instructions;

[0169] The processor is configured to execute instructions to implement the large model application service operation tracking method proposed in any of the aforementioned embodiments.

[0170] As an example, Fig. 9 is a schematic diagram of the structure of an electronic device 900 shown in an exemplary embodiment of the present disclosure, such as Fig. 9 As shown, the electronic device 900 may further include:

[0171] The memory 910 and the processor 920, a bus 930 connecting different components (including the memory 910 and the processor 920), the memory 910 stores a computer program, and when the processor 920 executes the program, the large model application service operation tracking method described in the embodiment of the present disclosure is implemented.

[0172] Bus 930 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0173] The electronic device 900 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 900, including volatile and non-volatile media, removable and non-removable media.

[0174] The memory 910 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 940 and / or cache memory 950. The server 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 960 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 9 not shown, usually called a "hard drive"). Although Fig. 9Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 930 via one or more data medium interfaces. The memory 910 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0175] A program / utility 980 having a set (at least one) of program modules 970 may be stored, for example, in the memory 910, such program modules 970 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 970 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0176] The electronic device 900 may also communicate with one or more external devices 990 (e.g., keyboard, pointing device, display 991, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 992. In addition, the electronic device 900 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 993. As shown, the network adapter 993 communicates with other modules of the electronic device 900 through a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0177] The processor 920 executes various functional applications and data processing by running the programs stored in the memory 910 .

[0178] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the large model application service operation tracking method of the embodiment of the present disclosure, and will not be repeated here.

[0179] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the large model application service operation tracking method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0180] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the large model application service operation tracking method proposed in any of the above embodiments is implemented.

[0181] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0182] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A large model application service operation tracking method, characterized in that: Applied to large model applications, including: Get the tracking service associated with the big model application; Sending a service registration request to the server; wherein the service registration request carries the tracking service, and is used by the server to authenticate the large model application end, and register the tracking service if the authentication is passed; Receiving a service registration response sent by the server; In a case where the service registration response indicates that the tracking service is successfully registered, integrating the tracking service into the big model application service to be tracked on the big model application end; The link tracking callback function associated with the tracking service is used to track each link node in the running link of the large model application service.

2. The method according to claim 1, characterized in that In the case where the service registration response indicates that the tracking service is successfully registered, integrating the tracking service into the large model application service to be tracked on the large model application end includes: When the service registration response indicates that the tracking service is successfully registered, creating a tracking object in the large model application service to carry the tracking service through the tracking object; Acquire the operation link of the large model application service; wherein the link nodes in the operation link are used to indicate the processing operations of the large model application service at different operation stages; The tracking object is added as the tracking callback function to the configuration information of the running link.

3. The method according to claim 1, characterized in that The obtaining of the tracking service associated with the large model application terminal includes: Obtaining a link start calling function; wherein the link start calling function is used to initialize tracking information when each link node starts running; Obtaining a link tracking information generation function; wherein the link tracking information generation function is used to generate tracking information of a current link node, and the tracking information of the current link node is generated according to the initialized tracking information of the current link node; Obtaining a link tracking content construction dictionary; wherein the link tracking content construction dictionary is used to store tracking information of the current link node; The tracking service is generated according to the link start calling function, the link tracking information generating function and the link tracking content construction dictionary.

4. The method according to claim 1, characterized in that: The method further comprises: Acquire link tracking data of each link node; wherein the link tracking data is obtained by tracking the corresponding link node in the running link of the large model application service; Each link tracking data is sent to the server to store and display each link tracking data.

5. The method according to claim 4, characterized in that The storing of each link tracking data includes: Using a collector to receive each link tracking data sent by the large model application end; Sending each of the link tracking data to a storage device through the collector; Each link tracking data is stored in a specified storage format by the memory.

6. The method according to claim 5, characterized in that The link tracking data is displayed, including: For any link tracking data, the link tracking data is parsed to obtain structure tracking data matching the specified storage format; The structure tracking data corresponding to any of the link tracking data is displayed.

7. The method according to claim 6, characterized in that The displaying of the structure tracking data corresponding to any link tracking data includes: Encapsulating the display logic of the structure tracking data corresponding to any link tracking data to obtain the encapsulated display logic; Sending the packaged presentation logic and the structure tracking data corresponding to any one of the link tracking data to a display; The display uses the packaged display logic to display the structure tracking data corresponding to any link tracking data.

8. The method according to claim 1, characterized in that The method further comprises: When the service registration response indicates that the large model application end authentication is successful and the tracking service registration is unsuccessful, resending the service registration request to the server end; Re-receive the service registration response sent by the server; When the re-received service registration response indicates that the tracking service registration is successful, the tracking service is integrated into the large model application service to be tracked on the large model application end.

9. The method according to claim 8, characterized in that The method further comprises: When the re-received service registration response indicates that the tracking service registration is unsuccessful, obtaining target information associated with the unsuccessful tracking service registration from the log; Generate prompt information based on the target information; wherein, the prompt information is used to prompt that the large model application end authentication is passed and the tracking service registration is unsuccessful.

10. A large model application service operation tracking device, characterized in that: Applied to large model applications, including: The acquisition module is used to obtain the tracking service associated with the large model application end; A registration module, used for sending a service registration request to a server, wherein the service registration request carries the tracking service, and is used for the server to authenticate the large model application end, and register the tracking service if the authentication is passed; A receiving module, used to receive a service registration response sent by the server; An integration module, configured to integrate the tracking service into the large model application service to be tracked of the large model application end when the service registration response indicates that the tracking service is successfully registered; The tracking module is used to track each link node in the running link of the large model application service by using the link tracking callback function associated with the tracking service.