Log processing method, device, and storage medium

Abstract

This application provides a log processing method, apparatus, and storage medium. The method includes: determining log recording categories configured for an application; each log recording category having a one-to-one corresponding feature code; the feature code being used to identify log data corresponding to the log recording category; obtaining feature information corresponding to the feature code generated by the application during its operation; and generating log data corresponding to the log recording category based on the feature code and the feature information corresponding to the feature code. This application can define multiple log recording categories for software logs, and all places where logs are recorded will have feature code information corresponding to the log recording category, thereby meeting the user's specific log recording needs and providing an effective and convenient way for later log analysis and querying.

Description

[Technical Field]

[0001] This application relates to the field of software logging technology, and in particular to a log processing method, apparatus and storage medium. [Previous Technology]

[0002] A log is a time-ordered collection of certain operations and their results on a specified object within a system. Each log file consists of log records, and each log record describes a single event. Log files can record necessary and valuable information for application software, thereby facilitating user monitoring of the system software or allowing the identification of specific problems when system software malfunctions.

[0003] Therefore, log file storage is an essential part of application development and operation, and is an important basis for developers to troubleshoot application faults and problems. However, faced with a large number and complex logs, it is undoubtedly a time-consuming and labor-intensive task for developers to find and locate problems from massive amounts of logs and to analyze the logs.

[0004] The existing log storage methods only perform simple classification and basic recording storage of software logs. The types of log records are limited, which cannot meet the specific recording needs of users, nor can they provide an effective and convenient way for later log analysis. [Summary of the Invention]

[0005] In order to solve the above problems, embodiments of this application disclose a log processing method, apparatus and storage medium.

[0006] In a first aspect, embodiments of this application provide a log processing method, the method comprising: determining a log recording category configured for an application; the log recording category having a one-to-one corresponding feature code; the feature code being used to identify log data corresponding to the log recording category; obtaining feature information generated by the application during operation that corresponds to the feature code; and generating log data corresponding to the log recording category based on the feature code corresponding to the log recording category and the feature information corresponding to the feature code.

[0007] In some embodiments, the log record category includes a first log record category, the first log record category indicating performance parameter information of the application's task execution process, the feature code corresponding to the first log record category is a first feature code, and obtaining the feature information corresponding to the feature code generated by the application during operation includes: intercepting a first request for the application, obtaining performance parameter information of a first key code fragment responding to the first request to execute the task; and using the performance parameter information as the feature information corresponding to the first feature code.

[0008] In some embodiments, the log record category includes a second log record category, the second log record category indicating thread lock information during the execution of a task by the application, and the feature code corresponding to the second log record category is a second feature code. Obtaining the feature information corresponding to the feature code generated by the application during its operation includes: intercepting a second request to the application, obtaining first and second thread lock information of a second key code segment responding to the second request to execute a task; the first thread lock information corresponds to the thread lock information when the thread state is locking, and the second thread lock information corresponds to the thread lock information when the thread state is locking; releasing the second request to the application, obtaining third and fourth thread lock information of the second key code segment responding to the second request to execute a task; the third thread lock information corresponds to the thread lock information when the thread state is unlocking, and the fourth thread lock information corresponds to the thread lock information when the thread state is unlocking; and using the first thread lock information, the second thread lock information, the third thread lock information, and the fourth thread lock information as feature information corresponding to the second feature code.

[0009] In some embodiments, generating log data corresponding to the log record category based on the feature code corresponding to the log record category and the feature information corresponding to the feature code includes: recording the feature code corresponding to the log record category and the feature information corresponding to the feature code according to a preset format corresponding to the log record category, and generating log data corresponding to the log record category.

[0010] In some embodiments, the method further includes: when it is necessary to write the log data to a UI control item, obtaining the display rules of the log data in the UI control item; the display rules include at least one of the following: displayable log record categories, maximum number of displayable items, and display order; mapping the log data to the UI control item, and displaying the log data in the UI control item according to the display rules.

[0011] In some embodiments, the method further includes: when it is necessary to write the log data into an instant-time memory (IMM) type memory, determining whether the log data matches the log record category that the IIM type memory can accept; if the log data matches the log record category that the IIM type memory can accept, then writing the log data into the memory queue of the IIM type memory; determining whether the log data already stored in the memory queue of the IIM type memory has reached the log data threshold of the memory queue; if the log data already stored in the memory queue has reached the log data threshold, then deleting the log data exceeding the log data threshold in the head of the log queue.

[0012] In some embodiments, the method further includes: when it is necessary to write the log data to the server, determining whether the communication connection with the server is normal; if the communication connection is abnormal, determining whether the cached log data has reached a preset cache threshold; if the cached log data has not reached the preset cache threshold, then caching the log data; if the cached log data has reached the preset cache threshold, determining whether to discard the log data; if it is determined that the log data should not be discarded, then storing the log data on a local disk.

[0013] In some embodiments, the method further includes: if the communication connection is normal, determining whether the log data is the latest data; if the log data is the latest data, sending the log data to the server so that the server performs distributed storage of the log data; if the log file is not the latest data, sending the log data to the server when the communication connection with the server is idle.

[0014] In a second aspect, embodiments of this application provide a log processing apparatus, the log processing apparatus comprising: a processor and a memory, the memory storing a computer program, and the processor executing the computer program performing the steps of any of the above-described log processing methods.

[0015] In a third aspect, embodiments of this application provide a storage medium storing a computer program that executes the steps of the log processing method described in any of the preceding claims.

[0016] The embodiments of this application include the following advantages: Before recording the feature information generated during application operation, the embodiments of this application can determine the log recording category configured for the application. Each log recording category has a one-to-one corresponding feature code, which can identify the log data corresponding to the log recording category. Then, based on the feature code and the feature information corresponding to the feature code, the log data corresponding to the log recording category is generated. Thus, complex log data can be distinguished according to multiple log recording categories, facilitating later analysis of the log data. Moreover, massive amounts of log data can be marked according to the feature codes corresponding to the log recording categories. When an application encounters an anomaly, relevant log data can be quickly queried based on the feature codes, enabling rapid and accurate location of the problem encountered by the application.

Implementation Method

[0018] In order to make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Referring to FIG1, a flowchart of a log processing method according to an embodiment of the present application is shown. The method may specifically include the following steps: Step 101, determining the log recording category configured for the application; the log recording category has a one-to-one corresponding feature code; the feature code is used to identify the log data corresponding to the log recording category; In this embodiment of the present application, in order to solve the problem that the existing technology has few log recording categories and cannot meet the specific recording needs of users, the present application redefines the log recording category, which includes regular log categories, predefined log categories and custom log categories.

[0020] Common log categories include Error, Warning, Information, and Debug. Predefined log categories include History, Callback, Threading, EntryExit, ChangeRecord, and OperatorAttention. Custom log categories allow for customized category characteristics, with multiple custom categories reserved for expansion and diverse use to meet various storage needs.

[0021] In this embodiment of the application, before recording application log data, the log recording category configured for the application can be determined first, and a one-to-one corresponding feature code can be set for the log recording category. The feature code can be used to identify the log data corresponding to the log recording category, and the feature code will exist in all places where log data is recorded.

[0022] By categorizing log records when recording data, technicians no longer need to spend a lot of time reviewing and organizing all log data. Instead, they can filter the log data of key interest by using the signature or other keyword information corresponding to the log record category, so as to quickly troubleshoot problems. For example, technicians can enter {signature} to query log data of the corresponding log record category; they can also enter {application group name}.{submodule name}.{signature}, etc., to query more log data related to the corresponding log record category.

[0023] Step 102: Obtain feature information corresponding to the signature code generated by the application during runtime; In this embodiment, the feature information corresponding to the signature code generated by the application during runtime can be obtained through a preset functional module. The preset functional module can be a module pre-written by a technician to obtain specific feature information. The feature information corresponding to a signature code can be obtained through a corresponding functional module. For example, if a technician needs to record the performance of a certain code segment of the application, the log recording category can be set to the code segment performance management category, and the signature code corresponding to the log recording category can be set to EntryExit. The performance parameter information of the key code segment function during the application's runtime can be obtained through the preset functional module LogEntryExit (performance statistics module), and this performance parameter information can be used as the feature information corresponding to the signature code EntryExit.

[0024] This embodiment of the application reduces the time spent manually organizing feature information and lowers labor costs by pre-setting log recording categories and feature codes, and then obtaining application feature information based on the functional modules corresponding to the feature codes. Simultaneously, since the feature information is already marked with feature codes when recorded as log data, the efficiency of technicians in screening feature information can be improved.

[0025] Step 103: Generate log data corresponding to the log record category based on the feature code and feature information corresponding to the feature code.

[0026] In this embodiment of the application, log data corresponding to a log record category can be generated based on the feature code corresponding to the log record category and the feature information corresponding to the feature code. The feature information may include, but is not limited to, the timestamp of the execution of a code segment of the application, the thread lock information of a code segment of the application, the thread ID of the application, the name of the application, the log source, and a description of the application's access actions. Log data corresponding to different log record categories may have different combinations and different arrangements of the feature information listed above.

[0027] This embodiment of the application can determine the log recording category configured for the application before recording the feature information generated during the application's operation. Each log recording category has a unique feature code, which identifies the log data corresponding to that category. Then, based on the feature code and the feature information corresponding to the feature code, log data corresponding to that category is generated. Thus, complex log data can be distinguished according to multiple log recording categories, facilitating later analysis. Furthermore, massive amounts of log data can be marked according to the feature codes corresponding to log recording categories. When an application encounters an anomaly, relevant log data can be quickly queried based on the feature codes, enabling rapid and accurate location of the problem encountered by the application.

[0028] Referring to Figure 2, a flowchart of another log processing method according to an embodiment of this application is shown. The method may specifically include the following steps: Step 201, determining the log recording category configured for the application; the log recording category has a one-to-one corresponding feature code; the feature code is used to identify the log data corresponding to the log recording category; Step 202, obtaining feature information corresponding to the feature code generated by the application during operation; This embodiment of the application can obtain feature information corresponding to the feature code generated by the application during operation through a preset functional module. The preset functional module may include LogEntryExit (performance statistics module), AutoLogLock (log lock module), etc. Among them, the LogEntryExit module can perform dynamic statistics on the task time of key code segment functions and perform performance statistical analysis, such as the number of times and duration of calling a certain function or functional module, and can reflect the logical relationship between business applications; the AutoLogLock module can record the log lock information of key code segments so as to analyze deadlock problems in a multi-threaded environment through log analysis. The preset functional module may be a module for obtaining specific feature information written in advance by technicians. The feature information corresponding to a feature code can be obtained through a one-to-one corresponding functional module.

[0029] In one embodiment, the log record category includes a first log record category, which indicates performance parameter information of the application's task execution process, and the feature code corresponding to the first log record category is a first feature code. The performance parameter information of the application's task execution process may include: the start and end times of the application's task execution, the number of times a function or function block is called, and the calling relationships between functions.

[0030] Referring to Figure 3, a flowchart illustrating the steps of performance statistics for key code segments in an embodiment of this application is shown. This embodiment uses the LogEntryExit module to obtain feature information corresponding to the first feature code of the first log record category using the using() method. For example, the function using (new LogEntryExit(source)) can be called to count the execution time of the first key code segment, the number of times the application calls a function or functional block, and the calling relationships between functions. Here, source is the log source information that the requester uses to mark its own features, such as the name of a function or functional block in the first key code segment. A functional block is a pre-coded collection of programs in the application, similar to a function, but capable of storing variables compared to a function.

[0031] Specifically, obtaining the feature information corresponding to the first feature code generated by the application during operation may include the following sub-steps: Sub-step S11, setting category enable control; In this embodiment of the application, EnabledCategories (category enable control) is used to determine which log data should be ignored and which log data should be retained when outputting log data generated based on the feature information corresponding to the first feature code. This category enable control can be determined by external system configuration parameters, or it can be configured by technicians themselves.

[0032] Sub-step S12: Intercept the first request for the application and obtain the performance parameter information of the first key code segment that responds to the first request to execute the task; When the requester requests the application, the LogEntryExit module can intercept the request and dynamically obtain the application's performance parameter information, such as the start and end times of the application's task execution, the number of times a function or function block is called, and the calling relationship between functions, and mark the above performance parameter information.

[0033] Sub-step S13: Dynamically acquire the key features of the functional block of the first key code segment; the key features of the functional block include the thread ID of the first key code segment, the name of the application, the log source, and the description of the application's access action; the LogEntryExit module can also dynamically acquire the key features of the functional block of the first key code segment, such as the thread ID of the first key code segment, the name of the application, the log source, the description of the application's access action, etc., and use the above key features as feature information corresponding to the first feature code. At the same time, it can also trace and mark the calling process of the method function or functional block by acquiring the method function of the calling layer.

[0034] Sub-step S14: Use the performance parameter information as the feature information corresponding to the first feature code; Combining sub-steps S13 and S14, the feature information corresponding to the first feature code can be obtained, including: performance parameter information of the first key code segment execution task, thread ID of the first key code segment, application name, log source, and application access action description.

[0035] Sub-step S15: Execute sub-steps S12 to S14 in a loop until all business request calls are completed, thus completing the recording of all feature information of the first critical code segment. This application embodiment can record performance parameter information of the first critical code segment execution task according to predefined rules. The predefined rules define the information recording format of the performance parameter information indicated by the first log recording category. For example, if the first feature code corresponding to the first log recording category is EntryExit, the recording format of the start execution time and end execution time in the performance parameter information of the first critical code segment execution task, as well as the time statistics method, are as follows: Start execution time: TID {threadName} Entering {methodName}; End execution time: TID {this.threadName} Exiting {this.methodName} after {num} ms; Time statistics method: When the task ends, the execution time of the task can be statistically analyzed using the Dispose method until the log information recording is complete.

[0036] In this embodiment of the application, the feature information and the first signature of the first key code segment can also be structured, that is, the feature information and the first signature are recorded in a specific order, which may be as follows: UtcTimestamp (performance parameter information related to execution time), ThreadId (thread ID), SourceProcess (name of the application software), Source (log source), Type (first signature), Message (description of the application's access action). Among them, the format of ThreadId is: 0x + unique identifier of the currently managed thread, such as 0x000a.

[0037] The above describes the specific process of obtaining feature information generated by the application during operation for the first log record category according to the embodiments of this application. The embodiments of this application use the performance of application key code segments as feature information of the first feature code. This allows for quick querying of information such as the execution time and call count of method functions or function blocks of application key code segments using the first feature code, thereby improving the efficiency of application performance information filtering.

[0038] In one embodiment, the log record category further includes a second log record category, which indicates thread lock information during the application's task execution process, and the feature code corresponding to the second log record category is a second feature code.

[0039] Referring to Figure 4, a flowchart illustrating the steps for obtaining thread lock information according to an embodiment of this application is shown. This embodiment uses the AutoLogLock module to obtain feature information corresponding to the second feature code of the second log record category. For example, thread lock information during the application's task execution process can be statistically analyzed by calling the function using (new AutoLogLock(this.LogLock, this.Source)).

[0040] Referring to Figure 5, a flowchart illustrating the steps of thread state transition in a controlled module according to an embodiment of this application is shown. The controlled module is used to lock the thread executing the function block. In this embodiment, thread lock information is obtained starting from NoState (asynchronous state). After entering the thread, it is locked, preventing other programs from accessing the thread and ensuring the security of the information acquisition process. This embodiment uses the AutoLogLock module to manage the state of the controlled module and marks the thread state executing the function block. The thread states are: locking (locking in progress) → locked (locking complete) → unlocking (unlocking in progress) → unlocked (unlocking complete).

[0041] Specifically, obtaining the feature information corresponding to the second feature code generated by the application during its operation may include the following sub-steps: Sub-step S21, setting category enable control; Sub-step S22, intercepting the second request for the application, obtaining the first thread lock information and the second thread lock information of the second key code segment that responds to the second request to execute the task; the first thread lock information corresponds to the thread lock information when the thread state is in the process of locking, and the second thread lock information corresponds to the thread lock information when the thread state is in the process of locking; In this embodiment of the application, the AutoLogLock module intercepts the business request, marks the thread state of the thread executing the function block as locking → locked, and records the first thread lock information and the second thread lock information according to the predefined rules. The message recording formats for the first and second thread lock information are as follows: When the controlled module's state is locking, the first thread lock information is recorded, with the message format: TID {threadId} locking {name}; When the controlled module's state is locked, the second thread lock information is recorded, with the message format: TID {threadId} locked {ththis.name} ({number of calls}). Sub-step S23: Execute the task of this function block; After the thread of the function block is locked, other programs cannot access the thread, allowing the function block to continue executing its own task in the thread-locked state, in order to obtain key characteristics during the execution of the function block, such as thread ID, application name, log source, application access action description, etc. The above key characteristics can be used as feature information corresponding to the second feature code. Sub-step S24: Release the second request for the application, and obtain the third and fourth thread lock information of the second key code segment that responded to the second request to execute the task; the third thread lock information corresponds to the thread lock information when the thread state is unlocking, and the fourth thread lock information corresponds to the thread lock information when the thread state is unlocking. In this embodiment of the application, the AutoLogLock module releases the business request, marks the thread state executing the function block as unlocking→unlocked, and records the third and fourth thread lock information according to predefined rules. The message recording format of the third and fourth thread lock information is as follows: When the state of the controlled module is unlocking, the third thread lock information is recorded, and the message format is: TID {lockedBy} unlocking {this.name} ({number of calls}); When the state of the controlled module is unlocked, the fourth thread lock information is recorded, and the message format is: TID {lockedBy} unlocked {this.name} ({num}).

[0042] Sub-step S25: The first thread lock information, the second thread lock information, the third thread lock information, and the fourth thread lock information are used as feature information corresponding to the second feature code.

[0043] Combining sub-steps S22 to S25, the feature information corresponding to the second feature code can be obtained, including: first thread lock information, second thread lock information, third thread lock information, fourth thread lock information, thread ID, application name, log source, and application access action description. Furthermore, this embodiment can also structure the second feature code and the feature information corresponding to it according to the method of sub-step S15, thereby completing the recording of thread lock information during the application's task execution process.

[0044] This application embodiment uses the second log record category to statistically analyze the thread lock information of the application's key code fragments. The log can be used to conveniently analyze deadlock problems in a multi-threaded environment, and facilitates the monitoring of information and behavior of each log source in the later stage using the second feature code corresponding to the second log record category.

[0045] Step 203: Record the feature code and feature information corresponding to the log record category according to the preset format corresponding to the log record category, and generate log data corresponding to the log record category.

[0046] In this embodiment of the application, the feature code and feature information corresponding to the log record category can be recorded according to a preset format corresponding to the log record category, thereby generating log data corresponding to the log record category. For example, sub-step S15 describes the process of recording the first feature code and the feature information corresponding to the first feature code according to its corresponding preset format for the first log record category. Sub-steps S22 to S25 describe the process of recording the second feature code and the feature information corresponding to the second feature code according to its corresponding preset format for the second log record category. After the feature code and the feature information corresponding to the feature code are recorded according to the preset format, the log data corresponding to the log record category is obtained.

[0047] This application provides a wide variety of log recording categories and corresponding feature codes for each category. It also allows for performance analysis of key code segments and management of resource locks for key functional modules using preset functional modules corresponding to these feature codes. This results in feature information that is then structured to obtain log data. This application simplifies the process of log recording for developers, automates log recording, provides a simple interface, and shortens the development cycle. Furthermore, by using feature codes corresponding to log recording categories, it significantly reduces the time required for analyzing, troubleshooting, and locating logs when system software issues arise. This allows developers to quickly trace the source of problems or anomalies, saving human resources and time.

[0048] In addition to the log data generation method described above, this application embodiment also provides a log data storage method for storing and managing log data. This application embodiment supports multi-target output of log data. Besides traditional log data output functions, such as output to a console, file, or database, this application also supports output to a UI control item list, real-time memory, remote service center, etc. UI control items refer to visual elements used to construct the user interface, such as buttons, text boxes, function menus, scrolls, and various operation panels.

[0049] Referring to Figure 6, a schematic diagram of storing log data in a UI control item according to an embodiment of this application is shown. This embodiment of the application can graphically display the required log data in the UI control item according to log record categories. The specific steps are as follows: When log data needs to be written to the UI control item, obtain the display rules for the log data in the UI control item; the display rules include at least one of the displayable log record categories, the maximum number of items to display, and the display order; map the log data to the UI control item, and display the log data in the UI control item according to the display rules.

[0050] The log data in this embodiment can be log data from a semiconductor manufacturing device. The semiconductor manufacturing device can have a UI control item list to facilitate the display of log data. Specifically, LogSinks (log data output pool) contains a log output target ListViewLogSink (UI control item list). ListViewLogSink is a wrapper for the standard UI control item Windows.Forms.ListView, which can match the output information of log data with the UI control item according to rules to complete the mapping of log information to the UI control item list. The semiconductor manufacturing device binds the control item list to the properties of ListViewLogSink through the application layer, and sets the log record categories that can be displayed, the maximum number of log data items to be displayed, and the display order, and finally completes the graphical display of log data. The display order of the output information of log data in the UI control item is as follows: defined by columns as Time, Source, Type, and Message.

[0051] In one embodiment, the method further includes: when it is necessary to write log data to a real-time memory type memory, determining whether the log data matches the log record category that the real-time memory type memory can accept; if the log data matches the log record category that the real-time memory type memory can accept, then writing the log data to the memory queue of the real-time memory type memory; determining whether the log data already stored in the memory queue of the real-time memory type memory has reached the log data threshold of the memory queue; if the log data already stored in the memory queue has reached the log data threshold, then deleting the log data exceeding the log data threshold in the log queue head.

[0052] Referring to FIG7, a flowchart of the steps of storing log data into real-time memory type memory according to an embodiment of the present application is shown. The process of storing log data using MemoryLogSink (real-time memory) may include the following sub-steps: Sub-step S31: Pre-set the acceptable log record categories for real-time memory and the maximum number of log data entries that the memory queue can store; Sub-step S32: Record the feature code and feature information corresponding to the log record category according to the preset format corresponding to the log record category, and generate log data corresponding to the log record category; Sub-step S33: Determine whether the log data to be written matches the preset log record category. If they do not match, the process ends and the log data is not stored; if they match, proceed to sub-step S34; Sub-step S34: Add the log data to the real-time memory management queue; Sub-step S35: Determine whether the real-time memory queue entries exceed the maximum preset capacity of the memory management queue; if they exceed the maximum preset capacity, proceed to sub-steps S36 and S37 in sequence; if they do not exceed the maximum preset capacity, proceed to sub-step S37; Sub-step S36: Delete the log data with the earliest recording time. Sub-step S37: Store the log data into MemoryLogSink.

[0053] The process of reading log data from real-time memory type memory may include the following sub-steps: Sub-step S38, obtaining log data from MemoryLogSink; Sub-step S39, retrieving log data from the real-time memory management queue; Sub-step S310, transmitting the log data to the application layer.

[0054] The LogSinks of this application include the log output target MemoryLogSink. The queue management module in MemoryLogSink is responsible for managing the reading and writing of log data, which saves the time lost by reading and writing log data through the intermediate layer and achieves the purpose of fast response.

[0055] In one embodiment, the log data in this application embodiment can also be written to a LogServer (remote central log server) so that the server can centrally manage each log data and provide storage and query services for managing each log data. Simultaneously, log data distributed across various nodes can be synchronized to the central node. In a distributed deployment environment, log data read and write operations are performed on the remote central log server using API functions via data communication, thereby reducing the system pressure caused by the increasing number of physical devices.

[0056] Referring to FIG8, a flowchart of the steps of distributed log data deployment according to an embodiment of the present application is shown. The steps of distributed log data deployment may include the following sub-steps: Sub-step S41, sending log storage requests from multiple log sources of multiple user terminals to a remote central log server; Sub-step S42, intercepting service requests and centrally managing the log data of each user terminal through the server; Sub-step S43, recording the feature code and feature information corresponding to the log record category according to the preset format corresponding to the log record category by the server, generating log data corresponding to the log record category, and storing the log data.

[0057] The LogSinks of this application also include the log output target SpoolingClientLogSink (multiplexing buffer processing client), which can transmit log data to a remote central log server for unified management when log data needs to be deployed in a distributed manner.

[0058] Referring to FIG9, a flowchart of the steps for requesting remote storage of log data according to an embodiment of the present application is shown. The steps for requesting remote storage of log data may include the following sub-steps: Sub-step S51, initiating a log data storage request to the log output target SpoolingClientLogSink; Sub-step S52, determining whether to perform distributed deployment; if distributed deployment is performed, executing sub-step S53; if distributed deployment is not performed, the process ends; Sub-step S53, determining whether a remote connection server exists; if no remote connection server exists, then executing sub-steps S54 and S55 in sequence; if a remote connection server exists, then executing sub-step S55; Sub-step S54, creating a remote connection; Sub-step S55, determining whether the type of log record written matches the preset log record type; if the log record type matches, executing sub-step S56; if the log record type does not match, the process ends; Sub-step S56, initiating a remote log data storage request to the LogServer through SpoolingClientLogSink.

[0059] Referring to FIG10, a flowchart of the log data transmission steps according to an embodiment of the present application is shown. Before SpoolingClientLogSink transmits the log data to the remote central log server, it is necessary to determine whether the communication connection is normal; if the communication connection is abnormal, it is determined whether the data has been cached; if not, the log data is cached; if so, the step of determining whether the communication connection is normal is returned; if the communication connection is normal, the log data is sent to the server so that the server performs distributed storage of the log data. The step of caching log data may include the following sub-steps: Sub-step S61: Determine whether the cached log data has reached a preset caching threshold; if the cached log data has not reached the preset caching threshold, proceed to sub-step S62; if the cached log data has reached the preset caching threshold, proceed to sub-step S65; Sub-step S62: Determine whether the log data is cached data that has not yet been queued; if the log data is cached data that has not yet been queued, proceed to sub-steps S63 and S64 in sequence; if the log data is not cached data that has not yet been queued, proceed to sub-step S64; Sub-step S63: Cache the log data; Sub-step S64: Perform a network query according to preset rules, and then return to the step of determining whether the communication connection is normal; Sub-step S65: Issue a notification warning and determine whether to discard the log data according to preset conditions; if it is determined not to discard the log data, proceed to sub-step S66; if it is determined to discard the log data, proceed to sub-step S67. Sub-step S66: Store the log data to the local disk, and then return to the step of determining whether the communication connection is normal; Sub-step S67: Discard the log data in the buffer.

[0060] The step of sending log data to the server may include the following sub-steps: Sub-step S68, determining whether the log data is the latest data; if the log file is not the latest data, proceeding to sub-step S69; if the log data is the latest data, proceeding to sub-step S611; where, the latest data refers to the log data generated closest to the current time. If the log data is the latest data, it indicates that the log data has a higher storage priority and can be stored in a distributed manner, taking precedence over important data interactions; if the log data is not the latest data, i.e., it is outdated historical data, it indicates that the log data has a lower storage priority and can only be stored in a distributed manner when there is no important data interaction on the network.

[0061] Sub-step S69: Determine if there is important data exchange on the network; if there is important data exchange on the network, proceed to sub-step S610; if there is no important data exchange on the network, i.e., the communication connection is idle, proceed to sub-step S611; Sub-step S610: Prioritize important data exchange, and then return to the step of determining whether the communication connection is normal; Sub-step S611: Send log data to the server so that the server can perform distributed storage of log data.

[0062] In one embodiment, the method further includes: writing log data into a log file; determining whether the log file meets the deletion conditions; the deletion conditions include: whether the number of files in the file group where the log file is located exceeds a preset threshold, whether the storage time of the files in the file group where the log file is located exceeds a preset threshold, whether the disk space occupied by the log file exceeds a preset threshold, whether the file group where the log file is located has preset conditions and whether the conditions exceed the preset threshold, at least one of the following; when the log file meets the deletion conditions, the log file is deleted.

[0063] In this embodiment of the application, log data can also be stored in a log file. Referring to FIG11, a flowchart of the steps for writing log data to a log file according to an embodiment of the application is shown. The steps for writing log data to a log file may include the following sub-steps: Sub-step S71, initiating a log data recording request; Sub-step S72, receiving log data; After receiving the log file at the server of the semiconductor process equipment, the log file can be automatically managed according to preset file management rules, preset file compression format, and preset file creation format, and then the log file is structured into a specified format and stored in a preset directory according to preset rules. Among them, the automatic management of log files refers to operations such as standardized automatic compression of log files and deletion of expired files.

[0064] Sub-step S73: Determine whether the log data needs to be stored based on the log category feature code. If storage is not required, proceed to sub-step S74. If storage is required, proceed to sub-step S75. Sub-step S74: Discard the log data. Sub-step S75: Determine the log file group to which each log data belongs. Sub-step S76: Determine whether there is an available log file in the currently available path. If an available log file exists, proceed to sub-step S77. If no available log file exists, proceed to sub-step S78. Sub-step S77: Structure the log information into a specified format according to preset rules and write it into the log file. In this embodiment, log file structuring refers to naming the file according to a preset format, such as [email protected]. Log information structuring refers to recording and storing the log data's attribute information in a pre-specified format, such as SourceProcess (name of the application software that generates the log file), Source (log source, log unit), Type (log category), UtcTimestamp (timestamp), ThreadId (thread ID), etc.

[0065] Sub-step S78: Delete currently expired files; In this embodiment of the application, the deletion of expired files means that when the number of files under the specified path reaches the preset maximum storage number or the number of days the log files under the specified path are stored exceeds the preset number of days, the system automatically backs up or deletes the historical log files. For example, the preset maximum storage number can be 100 log files and the preset number of days can be 30 days. This application does not impose any restrictions on this.

[0066] Sub-step S79: Compress the expired log files in the log file group according to the compression rules and file compression format preset in the log file group; In this embodiment of the application, standardized automated compression means that when the file size reaches the preset capacity, the expired log files in the log file group are uniformly compressed according to the compression rules and file compression format preset in the log file group.

[0067] Sub-step S710: Create a new log file according to the preset rules.

[0068] Referring to FIG12, a flowchart of the log file deletion steps according to an embodiment of the present application is shown. In one embodiment, when the log file meets the deletion conditions, the step of deleting the log file (i.e., the above sub-step S78) may include the following sub-steps: Sub-step S81, delete files that exceed the threshold according to the deletion conditions; Sub-step S82, the deletion conditions include: whether the number of files in the file group where the log file is located exceeds a preset threshold, whether the storage time of the files in the file group where the log file is located exceeds a preset threshold, whether the disk space occupied by the log file exceeds a preset threshold, whether the file group where the log file is located has preset conditions and whether the conditions exceed the preset threshold, at least one of the following; Sub-step S83, determine whether the deletion conditions are met; if the deletion conditions are met, then execute sub-step S84; if the deletion conditions are not met, then the process ends; Sub-step S84, issue a warning and determine whether to automatically delete the log file; if not automatically deleted, then execute sub-step S85 and sub-step S86 in sequence; if it is determined to automatically delete, then execute sub-step S86; The specific steps for issuing warnings are as follows: A first warning is issued when the number of files in the log file's group reaches a first threshold, or when the disk space occupied by the log file reaches a first threshold. The first threshold indicates that the normal operation of the system is not affected, and the first warning is used to indicate that the log file does not necessarily need to be deleted. A second warning is issued when the number of files in the log file's group reaches a second threshold, or when the disk space occupied by the log file reaches a second threshold. The second threshold indicates that the normal operation of the system is affected, and the second warning is used to indicate that the log file must be deleted. Sub-step S85 notifies the user to manually initiate a deletion command; sub-step S86 automatically executes the deletion operation according to the pre-set deletion rules.

[0069] This application embodiment provides a rich variety of log recording categories, supporting both standardized log category recording and user-customized log category recording. It can also trace and mark the calling process of method functions or functional blocks by obtaining the calling layer's method functions, statistically analyze the performance of method functions or functional blocks of key code segments, manage resource locks of key functional modules, and perform structured processing and storage of log data. It also provides rich log data storage methods, supports caching and distributed deployment, and centrally manages log files, possessing a comprehensive log file management mechanism. This application embodiment simplifies the process of developers recording log data, achieves automated log data recording, has a simple interface, shortens the development cycle, and meets diverse log storage needs. Furthermore, the rich log recording categories can significantly shorten the time for analyzing, troubleshooting, and locating logs when system software problems occur, allowing developers to quickly trace the source of problems or anomalies, saving human resources and time costs.

[0070] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for the embodiments of this application.

[0071] Referring to FIG13, a structural block diagram of a log processing device provided in an embodiment of the present application is shown. The log processing device 1301 includes: a processor 13011 and a memory 13012. The memory 13012 stores a computer program. When the processor executes the computer program, it executes the steps of any of the above-mentioned log processing methods.

[0072] Referring to FIG14, a structural block diagram of a storage medium provided in an embodiment of the present application is shown. The storage medium 1401 stores a computer program 14011, which executes the steps of the log processing method described above.

[0073] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0074] Those skilled in the art should understand that embodiments of the present application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, embodiments of the present application can take the form of a computer program product implemented on one or more machine-readable media (including but not limited to magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0075] Embodiments of this application are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute by the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0076] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce computer-implemented processing, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0078] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0079] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0080] The above provides a detailed description of a log processing method, apparatus, and storage medium provided by this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application. [Simplified Explanation of the Diagram]

[0017] In order to more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 is a flowchart of a log processing method according to an embodiment of this application; Figure 2 is a flowchart of another log processing method according to an embodiment of this application; Figure 3 is a flowchart of performance statistics of key code fragments according to an embodiment of this application; Figure 4 is a flowchart of thread lock information acquisition according to an embodiment of this application; Figure 5 is a flowchart of thread state transition of a controlled module according to an embodiment of this application; Figure 6 is a schematic diagram of storing log data into UI control items according to an embodiment of this application; Figure 7 is a flowchart of storing log data into real-time memory according to an embodiment of this application; Figure 8 is a flowchart of distributed deployment of log data according to an embodiment of this application; Figure 9 is a flowchart of requesting remote storage of log data according to an embodiment of this application; Figure 10 is a flowchart of sending log data according to an embodiment of this application; Figure 11 is a flowchart of writing log data into a log file according to an embodiment of this application; Figure 12 is a flowchart of deleting a log file according to an embodiment of this application; Figure 13 is a structural block diagram of a log processing device according to an embodiment of this application; Figure 14 is a structural block diagram of a storage medium according to an embodiment of this application.

Claims

1. A log processing method applied to a semiconductor manufacturing equipment, the method comprising: Determine the logging categories configured for the application; This log record category has a one-to-one corresponding feature code; The signature is used to identify the log data corresponding to the log record category; the feature information corresponding to the signature generated by the application during its operation is obtained; and the log data corresponding to the log record category is generated based on the signature and the feature information, wherein the log record category includes a second log record category, the second log record category indicates the thread lock information of the application's task execution process, the signature corresponding to the second log record category is the second signature, and obtaining the feature information corresponding to the signature generated by the application during its operation includes: intercepting a second request for the application, obtaining first and second thread lock information of the second key code fragment responding to the second request to execute the task; the first thread lock information corresponds to the thread lock information when the thread state is in the process of locking, and the second thread lock information corresponds to the thread lock information when the thread state is in the process of locking; Release the second request for the application, obtain the third thread lock information and the fourth thread lock information of the second key code fragment that responded to the second request to execute the task; the third thread lock information corresponds to the thread lock information when the thread state is unlocking, and the fourth thread lock information corresponds to the thread lock information when the thread state is unlocking completed; and use the first thread lock information, the second thread lock information, the third thread lock information and the fourth thread lock information as feature information corresponding to the second feature code.

2. The method as described in claim 1, wherein the log record category further includes a first log record category, the first log record category indicating performance parameter information during the application's task execution process, the signature corresponding to the first log record category being a first signature, and obtaining the feature information corresponding to the signature generated by the application during runtime includes: Intercept the first request to the application and obtain performance parameter information of the first critical code fragment that responds to the first request to execute the task. And use the performance parameter information as the feature information corresponding to the first feature code.

3. The method as described in request item 1, wherein, Generating log data corresponding to a log record category based on the feature code and the feature information corresponding to the feature code includes: recording the feature code and the feature information corresponding to the feature code according to the preset format corresponding to the log record category, and generating the log data corresponding to the log record category.

4. The method as described in claim 1, further comprising: When it is necessary to write the log data to a UI control, obtain the display rules of the log data in that UI control. The display rules include at least one of the following: the types of log records that can be displayed, the maximum number of records that can be displayed, and the display order; and map the log data to the UI control and display the log data in the UI control according to the display rules.

5. The method as described in claim 1, further comprising: When it is necessary to write the log data to the instant-time memory (JTM), it is determined whether the log data matches the log record category that the JTM can accept. If the log data matches the log record category that the JTM can accept, the log data is written to the memory queue of the JTM. It is then determined whether the log data already stored in the memory queue of the JTM has reached the log data threshold of the memory queue. If the log data already stored in the memory queue has reached the log data threshold, the log data exceeding the log data threshold in the head of the log queue is deleted.

6. The method as described in claim 1, further comprising: When it is necessary to write the log data to the server, check whether the communication connection with the server is normal. If the communication connection is abnormal, determine whether the cached log data has reached the preset cache threshold; if the cached log data has not reached the preset cache threshold, cache the log data; if the cached log data has reached the preset cache threshold, determine whether to discard the log data; and if it is determined not to discard the log data, store the log data on the local disk.

7. The method as described in claim 6, further comprising: If the communication connection is normal, then determine whether the log data is the latest data; If the log data is the latest data, then the log data is sent to the server so that the server can perform distributed storage of the log data; and if the log file is not the latest data, then the log data is sent to the server when the communication connection with the server is idle.

8. A log processing apparatus, comprising: A processor and memory, the memory storing a computer program, the processor executing the computer program to perform the steps of the log processing method as described in any one of claims 1 to 7.

9. A storage medium storing a computer program that performs the steps of the log processing method as described in any one of claims 1 to 7.

Images