Log processing method, apparatus, device, medium, and program product

Abstract

The application embodiment provides a log processing method, device, equipment, medium and program product, relates to natural language processing technology in artificial intelligence technology, and the method comprises the following steps: through an acquisition module on an application framework of an application server, in an application server running process, according to a configured log acquisition period, the value of a basic configuration type field is acquired from an operating system of the application server, the value of a framework type field is acquired from the application framework, the value of a business logic type field is acquired from business logic code, a structured log is formed, and the structured log is stored in a structured manner. The acquisition module on the application framework is used for directly acquiring the structured log, so that no intermediate log file is generated, storage resources of log data are saved, a complex log analysis process and a structured extraction process are avoided, the efficiency of log analysis is improved, the collected log data is more comprehensive and accurate, and the accuracy of log analysis is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI), and more particularly to a log processing method, apparatus, device, medium, and program product. Background Technology

[0002] Logs are an important tool for analyzing and handling problems and anomalies in applications or operating systems. Logs record information such as dates, times, event types, and messages related to the application or system. During game operation, it is often necessary to obtain log files from the background game server, using these log files for game development, management, and optimization.

[0003] Traditional log processing methods include the following steps: collecting log data from the game server in text file format, and then performing operations such as storage, transmission, and data analysis on the collected log data. Log data can also be stored in a database. When analysis is needed, the text-based log data is read from the database, and a pre-written log analysis program parses and extracts data to obtain structured information, which is then analyzed.

[0004] Traditional log processing methods involve large amounts of data storage and transmission, which consumes a lot of storage and transmission resources. In addition, the parsing and extraction processes in log analysis also consume a lot of processing resources, resulting in low log processing efficiency and high processing costs. Summary of the Invention

[0005] This application provides a log processing method, apparatus, device, medium, and program product, which improves the efficiency and accuracy of log processing.

[0006] In a first aspect, embodiments of this application provide a log processing method, the method comprising: collecting values ​​of a basic configuration type field from the operating system of the application server, collecting values ​​of a framework type field from the application framework, and collecting values ​​of a business logic type field from the business logic code, according to a configured log collection period, through a collection module on the application framework of an application server; combining the values ​​of the basic configuration type field, the framework type field, and the business logic type field collected in the same collection period into a structured log; preprocessing the collected structured log through a preprocessing module on the application framework; and storing the preprocessed structured log in a structured manner through a writing module on the application framework.

[0007] Secondly, embodiments of this application provide a log processing apparatus, comprising: a collection module configured on the application framework of an application server, configured to collect values ​​of a basic configuration type field from the operating system of the application server, values ​​of a framework type field from the application framework, and values ​​of a business logic type field from the business logic code during the operation of the application server, according to a configured log collection period; the collection module is further configured to combine the values ​​of the basic configuration type field, the frame type field, and the business logic type field collected in the same collection period into a structured log; a preprocessing module configured on the application framework, configured to preprocess the structured log collected by the collection module; and a writing module configured on the application framework, configured to perform structured storage on the preprocessed structured log.

[0008] Thirdly, embodiments of this application provide an electronic device, the electronic device comprising: a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in the first aspect above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the method described in the first aspect above.

[0010] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0011] The technical solution provided in this application embodiment directly collects structured logs using the collection module on the application server's application framework during application server operation. This avoids generating any intermediate log files, saving log data storage resources and eliminating complex log parsing and structure extraction processes, thus conserving log processing resources and improving log analysis efficiency. Furthermore, using an application framework for log collection allows for seamless integration of log processing with the application server during application service development, eliminating the need for additional software and hardware. This reduces the log processing cost for the application server. The structured logs collected by the application framework include basic configuration type fields, framework type fields, and business logic type fields, resulting in more comprehensive and accurate log data, thereby improving the accuracy of log analysis. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram illustrating an implementation environment of the method provided in this application embodiment; Figure 2 A diagram illustrating the generation of logs in a game scenario; Figure 3 This is a flowchart of the log processing method provided in Embodiment 1 of this application; Figure 4 This is a block diagram illustrating the principle of the log processing method provided in Embodiment 2 of this application. Figure 5 A diagram illustrating the writing method used by the writing module to write structured logs to a remote server; Figure 6 This diagram illustrates the writing process of the writing module writing structured logs to a remote server. Figure 7 A flowchart of the log processing method provided in Embodiment 2 of this application; Figure 8 This is a schematic diagram of the log processing device provided in Embodiment 3 of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0016] This application relates to artificial intelligence (AI) technology, which refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines capable of reacting in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0017] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0018] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, knowledge graphs, and search-related technologies (such as ranking, keyword analysis, and recommendation).

[0019] To facilitate understanding of the embodiments of this application, before describing the various embodiments of this application, some concepts involved in all embodiments of this application will be appropriately explained.

[0020] This application provides a log processing method. Figure 1 A schematic diagram of an implementation environment for the method provided in this application embodiment, such as... Figure 1 As shown, the implementation environment includes: terminals 110, 120, and 130, application server 140, and remote server 150. Terminals 110, 120, and 130 communicate with application server 140 via wired or wireless network. Application server 140 can also communicate with remote server 150 via wired or wireless network.

[0021] The terminal is not limited to the device shown in the diagram. It can be a mobile phone, tablet, desktop computer, laptop, smart voice interaction device, smart home appliance, or other similar devices. An application runs on the terminal. For example, this application can be traditional application software, cloud application software, a mini-program or application module within a host application, or a web platform; no limitation is made here. The application connects to an application server and receives services provided by the application server.

[0022] Optionally, the aforementioned applications may be game applications, e-commerce applications, short video applications, audio applications, novel applications, content interaction applications, instant messaging applications, etc., without specific restrictions.

[0023] Application server 140 is used to provide service content for applications. Different application servers 140 provide different service content. Application server 140 refers to a logical device that can independently provide at least one service. It can be understood that application server 140 does not correspond to a physical server. One application server 140 can be deployed on multiple physical servers, and multiple application servers can also be deployed on one physical server.

[0024] During operation, application server 140 actively collects structured logs through the collection module on the application framework and writes the structured logs to a remote server. Optionally, application server 140 can also perform preprocessing, local storage, or provide the structured logs to other interfaces.

[0025] Structured logs consist of meaningful fields such as user accounts, timestamps, and Internet Protocol (IP) addresses. These fields have physical meaning, facilitating user viewing and analysis of the logs. Traditional methods typically acquire unstructured logs, which are unformatted strings. Users usually cannot directly use unstructured logs and need to employ log analysis methods to extract structured information for further analysis. In this embodiment, the application server 140 can directly collect structured log information, avoiding the collection, storage, and extraction of structured information from unstructured logs, thereby improving log processing efficiency and reducing log processing costs.

[0026] The remote server 150 is used to store structured logs in a structured manner and provides a query interface. Subsequently, the application server 140 or other devices can query the structured logs from the remote server 150 and further analyze and process the structured logs.

[0027] Optionally, the remote server can use Elasticsearch, a distributed, highly scalable, and real-time search and data analysis engine. Through its distributed architecture and index optimization technology, Elasticsearch can efficiently store and query log data.

[0028] Optionally, the remote server may also use a database to store structured log data. This application embodiment does not limit the specific storage architecture used by the remote server.

[0029] In this embodiment, the application server 140 and the remote server 150 can be deployed on independent physical servers, or on server clusters or distributed systems composed of multiple physical servers. They can also be deployed on cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud security, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0030] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0031] Figure 1 In the implementation environment shown, the operation of writing structured logs to the remote server 150 and other processing operations can be performed by the application server 140. It can be understood that in this embodiment, the application server 140 may only be responsible for collecting structured logs, and the operation of writing structured logs to the remote server 150 and other processing operations may be performed by a separate log processing server.

[0032] Application server 140 generates logs during operation. Figure 2 To generate a diagram of the logs in the game scene, such as... Figure 2 As shown, the game terminal establishes a connection with the game server, and the game program runs on the game terminal. During the operation of the game server, multiple business processes are established. The business processes are used to handle specific game business. For example, the business processes include: room service process, game service process, directory service process, chat service process, etc.

[0033] During operation, the game service requires not only the support of the game server but also the support of certain common components. These common components provide public services, which are developed independently of the game service. The game server calls these common components through interfaces. These common components can be understood as part of the game application, and they also generate log data. For example, common components include, but are not limited to, access components, distribution components, and authentication components. The access component handles game requests, the distribution component handles message forwarding for the game service, and the authentication component provides account login, authentication, and management during the game service access process.

[0034] During game operation, not only do the game server's business processes generate log data, but the aforementioned common components also generate log data. The log data generated by the game server's business processes and the log data generated by the common components are output to the game server's log processing module. The log processing module provides the following processing functions: preprocessing, local storage, and local output.

[0035] In this embodiment, the log data collected by the log processing module is structured log. The preprocessing of the log data refers to the preprocessing of the structured log. The preprocessing includes at least one of the following processes: log format normalization, deduplication, filtering, classification, etc.

[0036] Preprocessed log data needs to be persisted for subsequent querying and analysis. Traditionally, log data is stored on the local hard drive or in a database, also known as local storage. Local storage stores log data locally as unformatted strings, but this method presents security and stability issues. A better approach is to use remote storage, storing log data on one or more remote servers.

[0037] Remote log storage writes logs to a remote server via local output, ensuring log security and reliability. Furthermore, remote storage enables cross-regional and cross-platform log sharing, facilitating collaboration and analysis across teams.

[0038] The local output function also provides storage options for various paths, such as storing log data to remote servers, providing it to other data statistics devices, or data security devices. After the logs are stored, the stored log data can be viewed on remote servers or application servers via a terminal.

[0039] After introducing some concepts involved in the embodiments of this application, the following describes in detail a log processing method provided by the embodiments of this application with reference to the accompanying drawings.

[0040] Figure 3 This is a flowchart of the log processing method provided in Embodiment 1 of this application. The method in this embodiment can be executed by an application server, such as... Figure 3 As shown, the method in this embodiment includes the following steps.

[0041] S101. Through the collection module on the application framework of the application server, during the operation of the application server, according to the configured log collection period, collect the values ​​of the basic configuration type field from the operating system of the application server, collect the values ​​of the framework type field from the application framework, and collect the values ​​of the business logic type field from the business logic code.

[0042] An application framework, also known as an application development framework or program framework, is the foundation and core of application development. It consists of conventions used and followed by all developers. Developers can extend these conventions as necessary, but the program always maintains the consistency of its main structure. An application framework can be understood as a piece of code that runs on the operating system. It provides many Application Programming Interfaces (APIs) to the outside world, allowing developers to create specific applications within the framework.

[0043] The method in this embodiment extends the application framework of the application server to achieve log collection. Extending the application framework means adding programs and interfaces to the existing application framework to implement the method of this embodiment. By extending the application framework, a collection module is added to it. This collection module is used to collect structured logs, thus enabling the application framework to have log collection functionality.

[0044] In this embodiment, the application framework's data collection module can proactively collect structured logs periodically according to a configured log collection cycle during application server operation, unaffected by user requests or specific business events (such as payment events or game interruption events), thereby enabling the collection of more comprehensive log data. This log collection cycle can be, for example, 30 seconds, one minute, or five minutes; this embodiment does not limit this. Optionally, the log collection cycle can be flexibly configured by the administrator, allowing for the setting of an appropriate log collection cycle based on the application scenario of the application server.

[0045] The data collection module can collect structured logs from the application server's operating system, application framework, and business logic code, making the collected structured logs more comprehensive and complete, thus meeting subsequent log analysis needs. Specifically, the module collects different log data from the operating system, application framework, and business logic code. Specifically, it collects values ​​of basic configuration type fields from the application server's operating system, framework type fields from the application framework, and business logic type fields from the business logic code.

[0046] S102. Combine the values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period into a structured log.

[0047] The data collection module collects a large amount of structured logs. A single structured log entry is also called a structured log record. A structured log entry includes the following type fields: basic configuration type fields, framework type fields, and business logic type fields. Each type of field includes at least one field, and the types of fields may vary depending on the application server.

[0048] The Basic Configuration Type field is used to record the hardware and / or software configuration information of the application server. For example, the Basic Configuration Type field includes at least one of the following fields: Deployment Environment field and IP address. The Deployment Environment field describes the environment in which the application server is deployed. The application server can be deployed in different data centers; therefore, the Deployment Environment field can be the name of the data center. In game applications, the application server can also be deployed on different partitions; therefore, the Deployment Environment field can also include the name of the partition where the application server is located.

[0049] The framework type field is used to record some information in the application framework of the application server. For example, the framework type field includes at least one of the following fields: trace ID field, thread ID field, file name field, file line number field, method name field, and log level field.

[0050] The trace ID is used to identify a specific request. When a user's request enters the system, a globally unique trace ID is generated. During the request transmission process, the trace ID can completely connect the paths called by the request in the system.

[0051] Alternatively, lightweight threads, also known as coroutines, can be used in the application server. Therefore, the thread ID can be replaced with the coroutine ID.

[0052] The filename field and the text line number field refer to the filename of the source file that was called when this structured log was generated, and the line number in the file.

[0053] The method name field refers to the name of the method called when this structured log is generated. A method can be understood as a special kind of function. Therefore, the method here can also be replaced with a function. Alternatively, the frame type field can include both the method name field and the function name field.

[0054] For example, the log levels include the following five levels: debug, info, warning, error, and fatal. Different log levels print different log information. Optionally, this embodiment uses the debug level.

[0055] The business logic type field is used to record information related to the specific business logic of the application server.

[0056] For example, the business logic type field includes at least one of the following fields: time field, user information field, identifier field of first request type, identifier field of second request type, and custom field. The first request type is the request type between the application server and the client, and the second request type is the request type between the application server and other servers.

[0057] The time field is used to record the collection timestamp or printing timestamp of each structured log. The timestamp recorded in the time field can be at the microsecond level.

[0058] The user information field is used to record information about users who access the application server. The user information field may include user account, user level, user type, etc.

[0059] Communication between the client and the application server uses requests. Requests can be categorized by protocol type, such as Hypertext Transfer Protocol (HTTP) requests and Transmission Control Protocol (TCP) requests. Even within the same protocol type, requests can be further divided into different types based on their content. For example, HTTP requests can be categorized into GET requests, POST requests, PUT requests, DELET requests, and PATCH requests.

[0060] The first request type identifier field is used to record the identifier of the request type between the application server and the client. The first request type can be a type based on the transport protocol or a type based on the request content. This embodiment does not specify the request type.

[0061] During application server operation, in addition to communicating with clients, it also communicates with other servers. For example, the application server can communicate with... Figure 2 The first request type communicates with the server where the common components are located. The second request type is the request type between the application server and other servers. The division of the second request type is similar to that of the first request type, so it will not be repeated here.

[0062] A custom field is a field reserved for users, who can define the content of the field according to their own needs. For example, a custom field can be the name of a game item.

[0063] Optionally, the collection module includes multiple collection units, each of which is used to collect at least one field. During the operation of the application server, the multiple collection units collect the structured logs of the application server and populate the collected structured logs into a predefined log structure, which is used to temporarily store the collected structured logs.

[0064] For example, the collection unit includes three collection units: a first collection unit, a second collection unit, and a third collection unit. The first collection unit is used to, according to the log collection cycle, invoke the operating system to collect the values ​​of basic configuration type fields from the operating system and fill the collected values ​​of the basic configuration type fields into a predefined log structure. The second collection unit is used to, according to the log collection cycle, collect the values ​​of framework type fields from the application framework and fill the collected values ​​of the framework type fields into the log structure. The third collection unit is used to, according to the log collection cycle, collect business logic type fields from the business logic code of the application server and fill the collected values ​​of the business logic type fields into the log structure. The values ​​of the basic configuration type fields, the collected framework type fields, and the business logic type fields collected in the same collection cycle, stored in the log structure, constitute a structured log entry.

[0065] In some common log processing methods, only some business logic fields can be obtained for analysis. However, in this embodiment, the collection module not only collects business logic type fields, but also basic configuration type fields and framework type fields, making the collected log data more complete and comprehensive. Based on the log data, a comprehensive analysis of the application server's basic configuration, framework, and business logic can be performed, allowing for better management and optimization of the application server.

[0066] In this embodiment, structured logs are directly collected using the application framework's collection module during application server operation, without generating any intermediate log files. This saves log data storage resources and avoids complex log parsing and structure extraction processes, reducing parsing and extraction costs. Furthermore, using an application framework for log collection allows for seamless integration with the application server during application service development, eliminating the need for additional software or hardware for log processing and lowering the application server's log processing costs.

[0067] The application framework proactively collects structured logs, resulting in more complete log content that is independent of passive triggering conditions. For example, existing solutions trigger log collection based on user requests, only collecting logs when a user request is detected, leading to limited log data collection.

[0068] S103. The collected structured logs are preprocessed using the preprocessing module on the application framework.

[0069] Optionally, before storing the structured logs, the structured logs can be preprocessed by a preprocessing module. This preprocessing includes at least one of the following: log format normalization, deduplication, filtering, and classification.

[0070] The preprocessing module reads structured logs from the log structure, preprocesses the structured logs, and sends the preprocessed structured logs to the writing module, which then writes the structured logs to the remote server.

[0071] In this embodiment, the preprocessing module can be a module independent of the application framework within the application server, or it can be a module on the application framework. By preprocessing the structured logs through a preprocessing module obtained by extending the application framework, no additional software modules are required, achieving seamless integration of log processing with the application server and reducing the cost of log processing.

[0072] Log format standardization refers to processing the values ​​of each field in a structured log to ensure that the values ​​of each field conform to preset standards. These preset standards include, but are not limited to, preset format standards and value standards. For example, if the values ​​of some fields exceed the preset value range threshold, the values ​​of those fields need to be modified to the correct values. Or, if the format of some field values ​​does not meet the requirements, the format of those field values ​​needs to be converted.

[0073] During the log collection process, some duplicate logs may be collected. Deduplication refers to removing duplicate fields or duplicate log entries to reduce the storage and transmission resources of subsequent logs.

[0074] Filtering refers to filtering out unnecessary fields, fields with incorrect values, or illegal fields.

[0075] Categorization processing is based on the values ​​of fields in the collected structured logs to categorize the logs. For example, based on the type of item in the business logic type field, the user level can be determined and the user level information can be added to the structured logs. A categorization field can be added to the existing fields in the collected structured logs.

[0076] Optionally, when the business logic type field includes a custom field, the preprocessing of structured logs can also include word segmentation of the custom field value. The custom field value may be a piece of text, and the text content is usually very long. Word segmentation divides the custom field into multiple terms. The purpose of word segmentation is to adapt to the inverted index search method during the log search stage and avoid using prefix matching to match the custom field value during the log search stage.

[0077] For example, based on the configured word segmentation rules, the values ​​of custom fields in the structured log are segmented to obtain at least one term corresponding to the value of the custom field. When the text of the self-locating field is very long, multiple terms can be obtained through word segmentation.

[0078] S104. The preprocessed structured logs are stored in a structured manner through the writing module on the application framework.

[0079] Structured storage refers to storing logs according to their fields, such as in a table format or key-value pair format.

[0080] In one implementation, the structured logs are stored locally, i.e., on the application server. In another implementation, the structured logs are stored on a remote server, which is used to store and manage the structured logs from the application server. Alternatively, the application server may also provide other interfaces to store the structured logs on statistical devices, security devices, etc.

[0081] When storing structured logs on a remote server, the efficiency and reliability of log data transmission need to be considered.

[0082] In this embodiment, structured logs are directly collected using the collection module on the application server's application framework during application server operation. This avoids generating any intermediate log files, saving log data storage resources and eliminating complex log parsing and structure extraction processes, thus conserving log processing resources and improving log analysis efficiency. Furthermore, using an application framework for log collection allows for seamless integration of log processing with the application server during application service development, eliminating the need for additional software or hardware. This reduces the log processing cost for the application server. The structured logs collected by the application framework include basic configuration type fields, framework type fields, and business logic type fields, resulting in more comprehensive and accurate log data, thereby improving the accuracy of log analysis.

[0083] Based on Embodiment 1, Embodiment 2 of this application provides a log processing method, which describes the complete process of storing structured logs to a remote server. Figure 4This is a block diagram illustrating the principle of the log processing method provided in Embodiment 2 of this application, as follows: Figure 4 As shown, the log processing method includes four processing stages: field definition, log collection, log preprocessing, and log writing.

[0084] Field definition refers to the process by which developers define the fields to be collected based on log processing requirements. Business logic type fields can be defined by business developers, while basic configuration type fields and framework type fields can be defined by the framework provider to ensure log availability. Defined fields include... Figure 4 As shown, in the business logic type field, cs cmd refers to the identifier field of the first request type, and ss cmd refers to the identifier field of the second request type.

[0085] After defining the log fields, the application framework of the application server is extended to form a collection module. Log collection refers to the process of capturing the values ​​of the log fields defined in the field definition stage. The collection module is used to collect data during the field collection stage. When the application server starts, the collection module collects basic configuration type fields based on the application server's resources and configuration. During the running process after the application server starts, it collects framework type fields and business logic type fields and writes the values ​​of the collected fields into a structure in memory.

[0086] Log preprocessing refers to the preprocessing of structured logs collected by the acquisition module. This preprocessing includes, but is not limited to, log format standardization, deduplication, filtering, and classification.

[0087] Log writing refers to the process of storing preprocessed structured logs. The log writing diagram shows that multi-path writing is used, which includes local log writing, remote log writing, and pipelined writing. Local log writing refers to writing structured logs to the local log file of the application server, remote log writing refers to writing structured logs to a remote server, and pipelined writing refers to providing structured logs to other devices that need log data.

[0088] In this embodiment, structured logs are written to a remote server via a writing module on the application server. This writing module can be a module within the application server that is independent of the application framework, or it can be a module within the application framework. By extending the application framework to obtain the writing module, structured logs are written without the need for additional software modules, achieving seamless integration of log processing with the application server and reducing the cost of log processing.

[0089] When writing structured logs to a remote server, the writing module needs to consider the writing efficiency and reliability of the structured logs. Reliability can ensure the integrity and security of the structured log data.

[0090] Figure 5 A diagram illustrating the writing method used by the writing module to write structured logs to a remote server is shown below. Figure 5 As shown, write efficiency is achieved through asynchronous processing, data merging, and multi-threaded concurrent writes. Write reliability is achieved through failure retries, graceful shutdown, and at least-once retry success.

[0091] Figure 6 This diagram illustrates the writing process of the writing module writing structured logs to a remote server. (Refer to...) Figure 5 and Figure 6 The writing module receives structured logs from the application server's business log interface and then... Figure 5 The write method writes to a remote server. This business log interface is an interface on the application framework. The structured logs received by the write module may be pre-processed logs or unprocessed logs.

[0092] During the operation of the application server, a large amount of logs are generated. To avoid the impact of processing large amounts of log data on business operations, this embodiment adopts asynchronous log writing. That is, the log writing thread and the business thread use different threads and run independently, so that log writing will not affect business operations. To achieve asynchronous processing, a log data queue is set up. The log data generated by the application server is stored in the log data queue, and the writing thread consumes data from the log data queue, that is, reads the structured logs and writes them to the remote server.

[0093] When the write thread writes structured logs to the remote server, it does so via network input / output (IO). Therefore, the write thread is also called the network IO write thread. The write thread can write one structured log entry at a time or multiple structured log entries at a time when writing structured logs to the remote server.

[0094] Writing a single structured log entry means writing one structured log entry with a single write request, while writing multiple structured log entries means writing multiple structured log entries with a single write request. The write module and the remote server communicate via requests. Each time data is written, a request needs to be established, and this request needs to be released after the write is complete. Establishing and releasing requests takes time. If each write request only carries one structured log entry, the log writing efficiency is very low, with a significant amount of time wasted on request establishment and release, and the log writing latency is unacceptable.

[0095] In this embodiment, multiple structured log entries are read from the log data queue and written to the remote server in a single write request. This means that a single request can write multiple structured log entries, thereby improving log writing efficiency.

[0096] Figure 5 and Figure 6 Data merging in this context refers to combining multiple structured logs into a single write request and writing it to a remote server. Optionally, data merging also includes business logic-based merging of multiple structured logs. Before writing multiple structured logs to the remote server through a single write request, the write thread performs business logic-based merging of the multiple structured logs, and then writes the merged structured log to the remote server through the write request.

[0097] Multiple structured logs can be merged into one structured log based on business logic. The purpose of merging business logic is to reduce the amount of data transmitted in write requests. For example, a data move operation may generate two structured logs. Most of the content of the two structured logs is the same, only the operation address is different. The operation address can be a custom field in the logical field. By merging, the two structured logs are merged into one structured log. The merged structured log includes the two operation addresses and the corresponding operation time.

[0098] Data merging reduces the number of write requests processed per unit of time and also lowers log write latency. For example, if the remote server uses Elasticsearch, the write thread writes structured logs to Elasticsearch by calling Elasticsearch's Platform as a Service (PaaS) interface. Before writing data, the write thread needs to establish a PaaS request with the PaaS interface. To reduce the queries per second (QPS) of the PaaS request, multiple structured logs are merged into a single PaaS request, thus reducing the number of PaaS requests processed per unit of time.

[0099] Multi-threaded concurrent writing refers to using multiple concurrent writing threads to write structured logs to a remote server. Multi-threaded concurrent writing provides a certain data processing capability, thereby ensuring the real-time reporting of log data.

[0100] During the process of writing structured logs to a remote server, the write thread may fail due to network instability or other reasons. To prevent the loss of structured logs that failed to be written, this embodiment employs a retry mechanism to ensure that the structured logs that failed to be written can be rewritten to the remote server.

[0101] Specifically, when multiple structured log entries fail to be written, these failed entries are stored in allocated storage space, and a rewrite thread places them into a log data queue. By re-adding the failed structured log entries to the log data queue, the write thread rewrites them to the remote server. It's understandable that when the write thread reads structured logs from the log data queue, it doesn't distinguish between failed and initial write entries.

[0102] This storage space is a pre-allocated address space, which can reside in the operating system's cache. This storage space can be used to store structured logs that have failed to be written. It can be accessed by the rewrite thread, which reads the log data from this storage space into a log data queue.

[0103] Optionally, this storage space is also used to store structured logs that overflow the log data queue. When the log data queue receives structured logs from the log interface at a high rate, it may cause the log data queue to overflow, meaning the log data queue is full and cannot hold structured logs. To avoid data loss due to log data queue overflow, in this embodiment, when the log data queue overflows, the overflowing structured logs are stored in the aforementioned storage space; the overflowing structured logs are then put into the log data queue by a rewrite thread.

[0104] During the operation of the application server, the device may be shut down due to system upgrades, expansions or other reasons. In this embodiment, the application server is shut down using the graceful shutdown method.

[0105] Graceful shutdown, simply put, involves sending a stop command to an application process while ensuring that ongoing business operations remain unaffected. Existing requests can continue to be processed, but new requests are no longer accepted. When a running application or process is stopped, the operating system sends a termination signal to the process. Without any graceful shutdown mechanism enabled, the application will terminate immediately upon receiving the signal. At this point, some unfinished programs may exit directly, potentially causing business logic to fail. In some business scenarios, this can lead to data inconsistencies, and transaction logic may not be rolled back.

[0106] In this embodiment, if graceful shutdown is not used, the device will shut down immediately upon receiving a shutdown command. At this time, the logs in the log data queue have not yet been processed. When the device is powered on again, the unprocessed logs in the log data queue will be cleared, resulting in the loss of unprocessed logs. With graceful shutdown, the device will shut down after the data in the log data queue has been processed, and the log data queue will no longer receive new structured logs.

[0107] Specifically, after the graceful shutdown module sends a shutdown command to the log data queue, the log data queue no longer receives new structured logs, and the write thread continues to consume data from the log data queue. When all the data in the log data queue has been consumed, the device is shut down, thus ensuring that log data is not lost when the application server is shut down.

[0108] Optionally, in this embodiment, the retry thread uses the at least-once success method, which means that at least one data retry is guaranteed to succeed. The retry thread may fail when writing logs from storage to the log data queue. If the at least-once success method is not used, the data would typically be considered abnormal after multiple attempts and discarded. This embodiment, by using the at least-once success method, continuously attempts to write logs to the log data queue, avoiding data loss and thus increasing the reliability of the log data.

[0109] based on Figure 5 and Figure 6 The writing method and writing process are shown below. Figure 7 This is a flowchart of a log processing method provided in Embodiment 2 of this application. The method provided in this embodiment includes the following steps.

[0110] S201. During the operation of the application server, the application framework's collection module collects the values ​​of basic configuration type fields from the application server's operating system, the values ​​of framework type fields from the application framework, and the values ​​of business logic type fields from the business logic code, according to the configured log collection cycle.

[0111] S202. Combine the values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period into a structured log.

[0112] S203. The preprocessing module of the application framework collects structured logs from the collection module and preprocesses the structured logs.

[0113] The preprocessing includes at least one of the following: log format normalization, deduplication, filtering, and classification.

[0114] S204. The application framework's writing module puts the preprocessed structured log data into the log data queue.

[0115] Optionally, when the log data queue overflows, the overflowing structured logs are stored in a storage space that stores both the overflowing structured logs and the structured logs that failed to be written.

[0116] S205. Read structured logs from the log data queue through multiple parallel write threads, wherein each write thread reads multiple structured logs from the log data queue.

[0117] S206. Each write thread writes multiple structured logs it reads to the remote server through a single write request.

[0118] S207. When the write thread fails to write, the write thread stores the multiple structured logs that failed to be written into the allocated storage space.

[0119] S208. The structured logs in the storage space are put into the log data queue by rewriting the thread.

[0120] S209. When a device shutdown command is received, a graceful shutdown method is adopted. After the data in the log data queue is processed, the device is shut down and the log data queue no longer receives new structured logs.

[0121] Steps S204-S209 are executed by the application framework's write module. It can be understood that steps S205-S207 and step S208 are not executed in any order. The write thread and the rewrite thread are two different threads. That is, the retry mechanism adopts asynchronous retry. The two threads execute independently and do not affect each other. They may execute at the same time or one after the other.

[0122] After the application server writes structured logs to a remote server, the remote server stores these logs using structured storage methods, such as Elasticsearch or a database. Log analysts can then query the log data from the remote server. The structured storage improves query efficiency, allows users to input fields from the structured logs for queries, and provides highly readable results that facilitate log analysis.

[0123] Elasticsearch, through its distributed architecture and index optimization technology, can efficiently store and query log data.

[0124] In this embodiment, the application framework of the application server is extended by adding a collection module, a preprocessing module, and a writing module. The collection module collects structured logs from the application server during its operation. The preprocessing module preprocesses the structured logs collected by the collection module. The writing module writes the preprocessed structured logs to a remote server. The writing module uses asynchronous processing, data merging, and multi-threaded concurrent writing methods to ensure log writing efficiency, while employing failure retries, graceful shutdown, and at least one successful retry method to ensure log writing reliability.

[0125] In the above method embodiments, the acquisition module, preprocessing module, and writing module on the application server's application framework for processing log data are functional modules defined from a functional perspective. It should be understood that the acquisition module, preprocessing module, and writing module can also be implemented using a single functional module. For example, a log processing module can be added to the application framework. This log processing module is used to acquire structured logs from the application server during its operation, preprocess the structured logs acquired by the acquisition module, and write the preprocessed structured logs to a remote server.

[0126] This application also provides a log processing apparatus. Figure 8 This is a schematic diagram of the log processing device provided in Embodiment 3 of this application, as shown below. Figure 8 As shown, the log processing device 100 may include: The data collection module 11 is configured on the application framework of the application server and is used to collect the values ​​of the basic configuration type field from the operating system of the application server, the values ​​of the framework type field from the application framework, and the values ​​of the business logic type field from the business logic code during the operation of the application server, according to the configured log collection period. The collection module 11 is also used to combine the values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period into a structured log; Preprocessing module 12, configured on the application framework, is used to preprocess the structured logs collected by the acquisition module; The writing module 13 is configured on the application framework for structured storage of the preprocessed structured logs.

[0127] In some possible implementations, the acquisition module 11 includes a first acquisition unit, a second acquisition unit, and a third acquisition unit; The first collection unit is used to call the operating system according to the log collection period, collect the value of the basic configuration type field from the operating system, and fill the collected value of the basic configuration type field into a predefined log structure; The second collection unit is used to collect the value of the frame type field from the application framework according to the log collection period, and fill the collected value of the frame type field into the log structure; The third collection unit is used to collect the value of the business logic type field from the business logic code according to the log collection period, and fill the collected value of the business logic type field into the log structure; The values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period, stored in the log structure, constitute a structured log.

[0128] In some possible implementations, the basic configuration type field includes at least one of the following fields: deployment environment field and IP address; the framework type field includes at least one of the following fields: trace identifier field, thread identifier field, file name field, file line number field, method name field, and log level field; the business logic type field includes at least one of the following fields: time field, user information field, identifier field of first request type, identifier field of second request type, and custom field, wherein the first request type is the request type between the application server and the client, and the second request type is the request type between the application server and other servers.

[0129] In some possible implementations, the preprocessing includes at least one of the following processes: log format normalization, deduplication, filtering, and classification.

[0130] In some feasible implementations, when the business logic type field includes a custom field, the preprocessing also includes word segmentation; the preprocessing module 12 is specifically used to: segment the value of the custom field in the structured log according to the configured word segmentation rules to obtain at least one term corresponding to the value of the custom field.

[0131] In some possible implementations, the structured logs are stored on a remote server; the writing module 13 is specifically used to: put the preprocessed structured log data into a log data queue; read multiple structured logs from the log data queue through a writing thread, and write the multiple structured logs to the remote server through a single write request.

[0132] In some possible implementations, the writing module 13 is further configured to: before writing the multiple structured logs to the remote server through a single write request, perform business merging on the multiple structured logs according to business logic through the writing thread, and write the merged structured logs to the remote server through the write request.

[0133] In some possible implementations, the writing module 13 is further configured to: when the writing of the multiple structured logs to the remote server fails, store the multiple structured logs that failed to be written into an allocated storage space through the writing thread, the storage space being used to store the structured logs that failed to be written; and put the structured logs in the storage space into the log data queue through the rewrite thread.

[0134] In some possible implementations, the writing module 13 is also used to: store the overflowing structured logs into the storage space when the log data queue overflows.

[0135] In some possible implementations, the writing module 13 is specifically used to: read structured logs from the log data queue through multiple parallel writing threads, with each writing thread reading multiple structured logs from the log data queue; and each writing thread writing the multiple structured logs read to the remote server through a single write request.

[0136] In some feasible implementations, a shutdown module is also included, which, when a device shutdown command is received, uses a graceful shutdown method to shut down the device after processing the data in the log data queue, and the log data queue no longer receives new structured logs.

[0137] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here.

[0138] The apparatus 100 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly manifested as execution by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0139] This application also provides an electronic device. Figure 9 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application, as shown below. Figure 9 As shown, the electronic device 200 may include: The system includes a memory 21 and a processor 22. The memory 21 stores computer programs and transfers the program code to the processor 22. In other words, the processor 22 can retrieve and run the computer programs from the memory 21 to implement the methods described in the embodiments of this application.

[0140] For example, the processor 22 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0141] In some embodiments of this application, the processor 22 may include, but is not limited to: General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0142] In some embodiments of this application, the memory 21 includes, but is not limited to: Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0143] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 22 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device 200.

[0144] like Figure 9 As shown, the electronic device 200 also includes a transceiver 23, which can be connected to the processor 22 or the memory 21.

[0145] The processor 22 can control the transceiver 23 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 23 may include a transmitter and a receiver. The transceiver 23 may further include antennas, and the number of antennas may be one or more.

[0146] Understandable, although Figure 9 As not shown in the diagram, the electronic device 200 may also include other modules such as a positioning module, a Bluetooth module, a display, and a controller, which will not be described in detail here.

[0147] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0148] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0149] This application also provides a computer program product comprising a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the corresponding processes in the above method embodiments; for brevity, these will not be elaborated further here.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0151] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0152] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A log processing method, characterized in that, include: During the operation of the application server, the collection module on the application framework periodically collects the values ​​of the basic configuration type field from the operating system of the application server, the values ​​of the framework type field from the application framework, and the values ​​of the business logic type field from the business logic code, according to the configured log collection period. The values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period are combined into a structured log. The collected structured logs are preprocessed using the preprocessing module on the application framework. The preprocessed structured logs are stored in a structured manner through the writing module on the application framework, including: putting the preprocessed structured log data into a log data queue; reading multiple structured logs from the log data queue through the writing thread; and writing the multiple structured logs to a remote server through a single write request. When multiple structured log entries fail to be written to the remote server, the write thread stores the failed structured log entries into an allocated storage space, which is used to store the failed structured log entries; the rewrite thread then places the structured log entries in the storage space into the log data queue. When a device shutdown command is received, a graceful shutdown method is adopted, in which the log data queue no longer receives new structured logs, the writing thread continues to consume data from the log data queue, and the device is shut down when all the data in the log data queue has been consumed.

2. The method according to claim 1, characterized in that, The acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit; The first collection unit is used to call the operating system according to the log collection period, collect the value of the basic configuration type field from the operating system, and fill the collected value of the basic configuration type field into a predefined log structure; The second collection unit is used to collect the value of the frame type field from the application framework according to the log collection period, and fill the collected value of the frame type field into the log structure; The third collection unit is used to collect the value of the business logic type field from the business logic code according to the log collection period, and fill the collected value of the business logic type field into the log structure; The values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period, stored in the log structure, constitute a structured log.

3. The method according to claim 1, characterized in that, The basic configuration type field includes at least one of the following fields: deployment environment field and Internet Protocol IP address; The framework type field includes at least one of the following fields: trace identifier field, thread identifier field, file name field, file line number field, method name field, and log level field; The business logic type field includes at least one of the following fields: time field, user information field, identifier field of first request type, identifier field of second request type, and custom field. The first request type is the request type between the application server and the client, and the second request type is the request type between the application server and other servers.

4. The method according to any one of claims 1-3, characterized in that, The preprocessing includes at least one of the following processes: log format normalization, deduplication, filtering, and classification.

5. The method according to claim 4, characterized in that, When the business logic type field includes a custom field, the preprocessing also includes word segmentation. The preprocessing of the collected structured logs includes: According to the configured word segmentation rules, the value of the custom field in the structured log is segmented to obtain at least one word corresponding to the value of the custom field.

6. The method according to claim 1, characterized in that, Before writing the multiple structured logs to the remote server via a single write request, the process also includes: The write thread performs business merging on the multiple structured logs according to business logic, and writes the merged structured logs to the remote server through the write request.

7. The method according to claim 6, characterized in that, The step of placing the preprocessed structured log data into the log data queue includes: When the log data queue overflows, the overflowing structured logs are stored in the storage space.

8. The method according to claim 1, characterized in that, The step of reading multiple structured log entries from the log data queue using a write thread and writing these multiple structured log entries to the remote server using a single write request includes: Structured logs are read from the log data queue through multiple parallel write threads, with each write thread reading multiple structured logs from the log data queue. Each write thread writes multiple read structured logs to the remote server through a single write request.

9. A log processing device, characterized in that, include: The data collection module is configured on the application framework of the application server. During the operation of the application server, it is used to periodically collect the values ​​of the basic configuration type field from the operating system of the application server, the values ​​of the framework type field from the application framework, and the values ​​of the business logic type field from the business logic code according to the configured log collection period. The acquisition module is also used to combine the values ​​of the basic configuration type field, the acquisition framework type field, and the business logic type field collected in the same acquisition period into a structured log; A preprocessing module, configured on the application framework, is used to preprocess the structured logs collected by the acquisition module; The write module, configured on the application framework, is used to perform structured storage on the preprocessed structured logs, including: putting the preprocessed structured log data into a log data queue; reading multiple structured logs from the log data queue through a write thread; and writing the multiple structured logs to a remote server through a single write request. The writing module is also configured to, when the writing of the multiple structured logs to the remote server fails, store the multiple structured logs that failed to be written into the allocated storage space through the writing thread, the storage space being used to store the structured logs that failed to be written; and to put the structured logs in the storage space into the log data queue through the rewrite thread. The shutdown module is used to gracefully shut down the device when a shutdown command is received. In this mode, the log data queue no longer receives new structured logs, and the writing thread continues to consume data from the log data queue. When all the data in the log data queue has been consumed, the device is shut down.

10. The apparatus according to claim 9, characterized in that, The acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit; The first collection unit is used to call the operating system according to the log collection period, collect the value of the basic configuration type field from the operating system, and fill the collected value of the basic configuration type field into a predefined log structure; The second collection unit is used to collect the value of the frame type field from the application framework according to the log collection period, and fill the collected value of the frame type field into the log structure; The third collection unit is used to collect the value of the business logic type field from the business logic code according to the log collection period, and fill the collected value of the business logic type field into the log structure; The values ​​of the basic configuration type field, the collection framework type field, and the business logic type field collected in the same collection period, stored in the log structure, constitute a structured log.

11. The apparatus according to claim 9, characterized in that, The basic configuration type field includes at least one of the following fields: deployment environment field and Internet Protocol IP address; The framework type field includes at least one of the following fields: trace identifier field, thread identifier field, file name field, file line number field, method name field, and log level field; The business logic type field includes at least one of the following fields: time field, user information field, identifier field of first request type, identifier field of second request type, and custom field. The first request type is the request type between the application server and the client, and the second request type is the request type between the application server and other servers.

12. The apparatus according to any one of claims 9-11, characterized in that, The preprocessing includes at least one of the following processes: log format normalization, deduplication, filtering, and classification.

13. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1 to 8.

14. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 8.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images