Log processing method and apparatus, and storage medium

Abstract

A log processing method and apparatus, and a storage medium. The method comprises: determining log record categories configured for an application program, wherein the log record categories have respective feature codes, and the feature codes are used for identifying log data corresponding to the log record categories; obtaining feature information, corresponding to the feature codes, generated by the application program during operating; and on the basis of the feature codes corresponding to the log record categories and the feature information corresponding to the feature codes, generating the log data corresponding to the log record categories. In the present application, multiple log record categories can be defined for software logs, and all log record locations have feature code information corresponding to the log record categories, so that specific log record requirements of users can be satisfied, and an effective and convenient method is provided for later analysis and log query.

Description

Log processing method, device and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of software logs, in particular to a log processing method, device and storage medium. BACKGROUND

[0002] A log refers to a time-ordered collection of certain operations of a specified object of a system and operation results thereof, each log file is composed of log records, and each log record describes a single event. A log file can record necessary and valuable information for application software, thereby facilitating user monitoring of system software, or when a problem occurs in the system software, the log file can be used to locate the specific problem.

[0003] Therefore, log file storage is an essential part of application development and operation, and is an important basis for developers to troubleshoot application problems and issues. However, in the face of a large number of and complex logs, it is undoubtedly a time-consuming and laborious task for developers to find and locate problems from the massive logs, as well as to analyze the logs.

[0004] The existing log storage method only performs simple classification and basic record storage on software logs, the log record categories are few, and cannot meet the specific recording needs of users, nor can it provide an effective and convenient way for later log analysis. SUMMARY

[0005] To solve the above problems, the embodiments of the present application disclose a log processing method, device and storage medium.

[0006] In a first aspect, the embodiments of the present application provide a log processing method, which comprises:

[0007] determining a log record category configured for an application program; the log record category has a one-to-one corresponding feature code; the feature code is used to identify log data corresponding to the log record category;

[0008] obtaining feature information corresponding to the feature code generated by the application program in a running process;

[0009] generating log data corresponding to the log record category according to the feature code corresponding to the log record category and the feature information corresponding to the feature code.

[0010] In some embodiments, the log record category includes a first log record category, the first log record category indicating performance parameter information of the application executing a task process, the first log record category corresponding to a first feature code, the obtaining of the feature information corresponding to the feature code generated by the application during the running process including:

[0011] intercepting a first request for the application, and obtaining performance parameter information of a first key code segment executing a task in response to the first request;

[0012] the performance parameter information being taken as the feature information corresponding to the first feature code.

[0013] In some embodiments, the log record category includes a second log record category, the second log record category indicating thread lock information of the application executing a task process, the second log record category corresponding to a second feature code, the obtaining of the feature information corresponding to the feature code generated by the application during the running process including:

[0014] intercepting a second request for the application, and obtaining first thread lock information and second thread lock information of a second key code segment executing a task in response to the second request; the first thread lock information corresponding to thread state being the thread lock information when locking, and the second thread lock information corresponding to the thread state being the thread lock information when completing locking;

[0015] releasing the second request for the application, and obtaining third thread lock information and fourth thread lock information of the second key code segment executing a task in response to the second request; the third thread lock information corresponding to thread state being the thread lock information when unlocking, and the fourth thread lock information corresponding to the thread state being the thread lock information when completing unlocking;

[0016] the first thread lock information, the second thread lock information, the third thread lock information, and the fourth thread lock information being taken as the feature information corresponding to the second feature code.

[0017] In some embodiments, the generating of the log data corresponding to the log record category according to the feature code corresponding to the log record category and the feature information corresponding to the feature code includes:

[0018] the feature code corresponding to the log record category and the feature information corresponding to the feature code being recorded in a preset format corresponding to the log record category, to generate the log data corresponding to the log record category.

[0019] In some embodiments, the method further includes:

[0020] obtaining a display rule of the log data in the UI control when it is required to write the log data into the UI control; the display rule comprises at least one of a displayable log record category, a maximum display entry, and a display order;

[0021] mapping the log data to the UI control, and displaying the log data in the UI control according to the display rule.

[0022] In some embodiments, the method further comprises:

[0023] judging whether the log data matches a log record category acceptable by the real-time memory type storage when it is required to write the log data into the real-time memory type storage;

[0024] writing the log data into a memory queue of the real-time memory type storage if the log data matches the log record category acceptable by the real-time memory type storage;

[0025] judging whether stored log data of the memory queue of the real-time memory type storage reaches a log data threshold of the memory queue;

[0026] deleting log data exceeding the log data threshold from a head of the log queue if the stored log data of the memory queue reaches the log data threshold.

[0027] In some embodiments, the method further comprises:

[0028] judging whether a communication connection with the server is normal when it is required to write the log data into the server;

[0029] judging whether cached log data reaches a preset cache threshold if the communication connection is not normal;

[0030] caching the log data if the cached log data does not reach the preset cache threshold;

[0031] judging whether to discard the log data if the cached log data reaches the preset cache threshold;

[0032] storing the log data to a local disk if it is determined not to discard the log data.

[0033] In some embodiments, the method further comprises:

[0034] judging whether the log data is latest data if the communication connection is normal;

[0035] If the log data is the latest data, the log data is sent to the server to enable the server to store the log data in a distributed manner.

[0036] If the log file is not the latest data, the log data is sent to the server in a case where a communication connection with the server is idle.

[0037] In a second aspect, an embodiment of the present application provides a log processing apparatus, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to perform the steps of the log processing method in any of the preceding embodiments.

[0038] In a third aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and the computer program performs the steps of the log processing method in any of the preceding embodiments.

[0039] Embodiments of the present application have the following advantages:

[0040] Embodiments of the present application can determine a log recording category configured for an application before generating feature information generated in a running process of the application, the log recording category has a one-to-one corresponding feature code, and the feature code can identify log data corresponding to the log recording category; then, according to the feature code corresponding to the log recording category and the feature information corresponding to the feature code, the log data corresponding to the log recording category is generated. Thus, complex log data can be distinguished according to multiple log recording categories, so as to facilitate analysis of the log data in the later period. Moreover, massive log data can be marked according to the feature code corresponding to the log recording category, and when the application is abnormal, relevant log data can be quickly queried according to the feature code, so as to quickly and accurately locate problems encountered by the application. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0042] FIG. 1 is a step flowchart of a log processing method according to an embodiment of the present application;

[0043] FIG. 2 is a step flowchart of another log processing method according to an embodiment of the present application;

[0044] FIG. 3 is a step flowchart of key code segment performance statistics according to an embodiment of the present application;

[0045] Figure 4 is a flow chart of the steps of thread lock information acquisition according to an embodiment of the present application;

[0046] Figure 5 is a flow chart of the steps of thread state conversion of a controlled module according to an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of log data storage to a UI control according to an embodiment of the present application;

[0048] Figure 7 is a flow chart of the steps of log data storage to a real-time memory type storage according to an embodiment of the present application;

[0049] Figure 8 is a flow chart of the steps of distributed deployment of log data according to an embodiment of the present application;

[0050] Figure 9 is a flow chart of the steps of requesting remote storage of log data according to an embodiment of the present application;

[0051] Figure 10 is a flow chart of the steps of log data sending according to an embodiment of the present application;

[0052] Figure 11 is a flow chart of the steps of log data writing to a log file according to an embodiment of the present application;

[0053] Figure 12 is a flow chart of the steps of log file deletion according to an embodiment of the present application;

[0054] Figure 13 is a structural block diagram of a log processing apparatus according to an embodiment of the present application;

[0055] Figure 14 is a structural block diagram of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the above objectives, features and advantages of the present application more apparent, further specific embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Referring to Figure 1, a flow chart of the steps of a log processing method according to an embodiment of the present application is shown, which can specifically include the following steps:

[0058] Step 101, determine a log record category configured for an application; the log record category has a one-to-one corresponding feature code; the feature code is used to identify log data corresponding to the log record category;

[0059] In the present embodiment, in order to solve the problem of the few log record categories in the prior art, which cannot meet the specific recording requirements of users, the log record category is redefined, and the log record category includes a regular log category, a predefined log category and a custom log category.

[0060] The conventional log categories include Error (error), Warning (warning), Information (message), Debug (debug), and the like. The predefined log categories include History (history type), Callback (callback type), Threading (thread type), EntryExit (code segment entry and exit management type), ChangeRecord (storage record change type), OperatorAttention (operator attention type), and the like. The custom log category can customize the category features, reserve various custom categories for expansion, and diversify to meet the needs of the storage type.

[0061] The embodiment of the present application can determine the log record category configured for the application program before recording the log data of the application program, and set a one-to-one corresponding feature code for the log record category. The feature code can be used to identify the log data corresponding to the log record category, and the feature code exists in all places where the log data is recorded.

[0062] By dividing the log record category when recording the log data, the technical personnel do not need to spend a lot of time reviewing and sorting all the log data, but can filter the log data of interest through the feature code or other keyword information corresponding to the log record category, so as to quickly troubleshoot problems. For example, the technical personnel can input {feature code} to query the log data corresponding to the log record category, or input {application group name}.{submodule name}.{feature code} and the like to query more log data related to the log record category.

[0063] Step 102, obtaining the feature information corresponding to the feature code generated by the application program in the running process;

[0064] In the embodiment of the present application, the feature information corresponding to the feature code generated by the application program in the running process can be obtained through a preset function module. The preset function module can be a module for obtaining specific feature information written by the technical personnel in advance, and the feature information corresponding to one feature code can be obtained through a one-to-one corresponding function module. For example, if the technical personnel need to record the performance of a code segment of the application program, the log record category can be set as the code segment performance management category, the feature code corresponding to the log record category can be set as EntryExit, and the performance parameter information of the key code segment function of the application program in the running process can be obtained through the preset function module LogEntryExit (performance statistics module), and the performance parameter information is taken as the feature information corresponding to the feature code EntryExit.

[0065] The application embodiment can reduce the time for manually sorting feature information and reduce labor costs by setting log record categories and feature codes in advance and obtaining feature information of an application according to a function module corresponding to the feature codes. Meanwhile, the feature information is marked by the feature codes when being recorded as log data, so that the screening efficiency of the feature information by technical personnel can be improved.

[0066] In step 103, log data corresponding to the log record category is generated according to the feature code corresponding to the log record category and the feature information corresponding to the feature code.

[0067] In the application embodiment, log data corresponding to the log record category can be generated according to the feature code corresponding to the log record category and the feature information corresponding to the feature code. The feature information can include, but is not limited to, a timestamp of execution of a code segment of the application, thread lock information of the code segment of the application, a thread ID of the application, a name of the application, a log source, and a description of an access action of the application. The log data corresponding to different log record categories can have different combinations and different arrangements of the above-mentioned feature information.

[0068] The application embodiment can determine the log record category configured for the application before recording the feature information generated in the running process of the application. The log record category has a one-to-one corresponding feature code, and the feature code can identify log data corresponding to the log record category. Then, log data corresponding to the log record category is generated according to the feature code corresponding to the log record category and the feature information corresponding to the feature code. Therefore, complex log data can be distinguished according to various log record categories, so as to facilitate analysis of the log data in the later period. Moreover, massive log data can be marked according to the feature code corresponding to the log record category. When the application has an exception, relevant log data can be quickly queried according to the feature code, so as to quickly and accurately locate the problem encountered by the application.

[0069] Referring to FIG. 2, a step flowchart of another log processing method of the application embodiment is shown. The method can specifically include the following steps:

[0070] In step 201, a log record category configured for an application is determined. The log record category has a one-to-one corresponding feature code. The feature code is used to identify log data corresponding to the log record category.

[0071] In step 202, feature information corresponding to the feature code generated in a running process of the application is obtained.

[0072] The embodiment of the present application can acquire the feature information corresponding to the feature code generated by the application during the running process through the preset function module. The preset function module can include LogEntryExit (performance statistics module), AutoLogLock (log lock module), etc. The LogEntryExit module can perform dynamic statistics on the task time of the key code segment function, and perform performance statistics analysis, such as the number of times and the time length statistics of calling a function or a function module, and can reflect the logical relationship between business applications. The AutoLogLock module can record the log lock information of the key code segment, so as to analyze the deadlock problem in the multi-threaded environment through the log. The preset function module can be a module for acquiring specific feature information written in advance by a technician. The feature information corresponding to one feature code can be acquired through a one-to-one function module.

[0073] In one embodiment, the log record category includes a first log record category, the first log record category indicates the performance parameter information of the task process of the application, and the feature code corresponding to the first log record category is a first feature code. The performance parameter information of the task process of the application can include the start execution time and the end execution time of the task of the application, the number of times of calling a function or a function block, and the calling relationship between functions.

[0074] Referring to FIG. 3, a step flowchart of the key code segment performance statistics of the embodiment of the present application is shown. The embodiment of the present application uses the LogEntryExit module to acquire the feature information corresponding to the first feature code of the first log record category through the using () mode. For example, the time of the first key code segment to execute the task, the number of times of calling a function or a function block of the application, and the calling relationship between functions can be counted by calling the function using (new LogEntryExit (source)). The source is the log source information marked by the requester to indicate its own features, such as the name of a function or a function block in the first key code segment, etc. The function block is a collection of programs pre-coded in the application, similar to a function, but can store variables compared with a function.

[0075] Specifically, acquiring the feature information corresponding to the first feature code generated by the application during the running process can include the following sub-steps:

[0076] Sub-step S11, setting category enable control;

[0077] The embodiment of the present application determines which log data generated according to the feature information corresponding to the first feature code should be ignored and which log data should be retained when the log data is output through EnabledCategories (category enablement control), which can be determined by system external configuration parameters or configured by technicians.

[0078] In substep S12, the first request for the application program is intercepted, and the performance parameter information of the first key code segment performing the task in response to the first request is obtained.

[0079] When the requester requests the application program, the request can be intercepted through the LogEntryExit module, and the performance parameter information of the application program is dynamically obtained, for example, the start execution time and the end execution time of the application program performing the task, the number of times of calling a function or a function block, and the calling relationship between functions, and the above performance parameter information is marked.

[0080] In substep S13, the key features of the function block of the first key code segment are dynamically obtained, including the thread ID of the first key code segment, the name of the application program, the log source, and the access action description of the application program.

[0081] The LogEntryExit module can also dynamically obtain the key features of the function block of the first key code segment, such as the thread ID of the first key code segment, the name of the application program, the log source, and the access action description of the application program, and the above key features are taken as the feature information corresponding to the first feature code. At the same time, the calling process of the method function or the function block can also be traced and marked through the method function of the calling layer.

[0082] In substep S14, the performance parameter information is taken as the feature information corresponding to the first feature code.

[0083] In combination with substep S13 and substep S14, the feature information corresponding to the first feature code can be obtained, including the performance parameter information of the first key code segment performing the task, the thread ID of the first key code segment, the name of the application program, the log source, and the access action description of the application program.

[0084] In substep S15, substep S12 to substep S14 are cyclically executed until all business requests are called, and the recording of all feature information of the first key code segment is completed.

[0085] The embodiment of the present application can record the performance parameter information of the first key code segment executing a task according to a predefined rule. The predefined rule defines an information record format of the performance parameter information indicated by the first log record category. For example, if the first feature code corresponding to the first log record category is EntryExit, the record format of the start execution time and the end execution time in the performance parameter information of the first key code segment executing a task, and the time counting manner are as follows:

[0086] Start execution time: TID{threadName}Entering{methodName};

[0087] End execution time: TID{this.threadName}Exiting{this.methodName}after{num}ms;

[0088] Time counting manner: the execution time of the task can be counted by using the Dispose method until the log information record is completed.

[0089] The embodiment of the present application can also structure the feature information of the first key code segment and the first feature code, that is, record the feature information and the first feature code in a specific order. The order can be as follows:

[0090] UtcTimestamp (performance parameter information related to execution time), ThreadId (thread ID), SourceProcess (name of application software), Source (log source), Type (first feature code), Message (description of access action of application program). The format of ThreadId is: 0x+obtains the unique identifier of the current managed thread, such as 0x000a.

[0091] The above describes the specific process of the embodiment of the present application for obtaining the feature information generated by the application program in the running process according to the first log record category. The embodiment of the present application counts the performance of the key code segment of the application program as the feature information of the first feature code, so that the execution time, the number of calls and other information of the method function or the function block of the key code segment of the application program can be quickly queried through the first feature code, and the information screening efficiency of the application program performance is improved.

[0092] In one embodiment, the log record category further includes a second log record category, the second log record category indicates thread lock information of the application program executing a task process, and the feature code corresponding to the second log record category is a second feature code.

[0093] Referring to FIG. 4, a flow chart of steps of thread lock information acquisition according to an embodiment of the present application is shown. According to the embodiment of the present application, the AutoLogLock module is used to acquire the feature information corresponding to the second feature code of the second log record category. For example, the function using (newAutoLogLock (this.LogLock, this.Source)) can be called to count the thread lock information of the application during the execution of a task.

[0094] Referring to FIG. 5, a flow chart of steps of thread state conversion of a controlled module according to an embodiment of the present application is shown. The controlled module is used to lock the thread of an execution function block. According to the embodiment of the present application, the thread lock information is acquired from the NoState (unsynchronized state) and the thread is locked after entering the thread to prevent other programs from accessing the thread, thereby ensuring the security of the information acquisition process. According to the embodiment of the present application, the AutoLogLock module is used to manage the state of the controlled module and mark the thread state of the thread executing the function block. The thread state is: locking (being locked) → locked (locking completed) → unlocking (being unlocked) → unlocked (unlocked completed).

[0095] Specifically, the feature information corresponding to the second feature code generated by the application during the execution of a task can include the following sub-steps:

[0096] In sub-step S21, the category enable control is set.

[0097] In sub-step S22, the second request for the application is intercepted and the first thread lock information and the second thread lock information of the second key code segment executing the task in response to the second request are acquired. The first thread lock information corresponds to the thread lock information when the thread state is being locked and the second thread lock information corresponds to the thread lock information when the thread state is locked.

[0098] According to the embodiment of the present application, the AutoLogLock module is used to intercept the business request, mark the thread state of the thread executing the function block as locking → locked, and record the first thread lock information and the second thread lock information according to the predefined rules. The message record format of the first thread lock information and the second thread lock information is as follows:

[0099] When the state of the controlled module is locking, the first thread lock information is recorded and the message format is: TID{threadId}locking{name}.

[0100] When the state of the controlled module is locked, the second thread lock information is recorded and the message format is: TID{threadId}locked{this.name}({number of calls}).

[0101] Sub-step S23, the function block task is executed.

[0102] After the thread lock of the function block, other programs cannot access the thread, and the function block continues to execute its own task in the thread lock state, so as to obtain the key features in the execution process of the function block, such as the thread ID, the name of the application program, the log source, the access action description of the application program, and the like, which can be used as the feature information corresponding to the second feature code.

[0103] Sub-step S24, the second request for the application program is released, and third thread lock information and fourth thread lock information for executing the task of the second key code segment responding to the second request are obtained; the third thread lock information corresponds to the thread lock information when the thread state is unlocking, and the fourth thread lock information corresponds to the thread lock information when the thread state is unlocked.

[0104] The embodiment of the application releases the business request through the AutoLogLock module, marks the thread state of executing the function block as unlocking→unlocked, and records the third thread lock information and the fourth thread lock information according to the pre-defined rules. The message recording format of the third thread lock information and the fourth thread lock information is as follows:

[0105] When the state of the controlled module is unlocking, the third thread lock information is recorded, and the message format is: TID{lockedBy}unlocking{this.name}({call times});

[0106] When the state of the controlled module is unlocked, the fourth thread lock information is recorded, and the message format is: TID{lockedBy}unlocked{this.name}({num}).

[0107] Sub-step S25, the first thread lock information, the second thread lock information, the third thread lock information, and the fourth thread lock information are used as the feature information corresponding to the second feature code.

[0108] In combination with the sub-step S22 to the sub-step S25, the feature information corresponding to the second feature code can include the first thread lock information, the second thread lock information, the third thread lock information, the fourth thread lock information, the thread ID, the name of the application program, the log source, and the access action description of the application program. In addition, the embodiment of the application can also structure the second feature code and the feature information corresponding to the second feature code according to the method of the sub-step S15, and further complete the recording of the thread lock information in the execution process of the application program.

[0109] The second log record category can be used to analyze the thread lock information of the key code segment of the application program, and the deadlock problem in the multi-thread environment can be conveniently analyzed through the log. The second feature code corresponding to the second log record category can be used to monitor the information and behavior of each log source in the later period.

[0110] In step 203, the feature code corresponding to the log record category and the feature information corresponding to the feature code are recorded according to the preset format corresponding to the log record category, and log data corresponding to the log record category is generated.

[0111] In the embodiments of the present application, the feature code corresponding to the log record category and the feature information corresponding to the feature code can be recorded according to the preset format corresponding to the log record category, so as to generate the log data corresponding to the log record category. For example, sub-step S15 introduces the process of recording the first feature code and the feature information corresponding to the first feature code according to the preset format corresponding to the first log record category. Sub-steps S22 to S25 introduce the process of recording the second feature code and the feature information corresponding to the second feature code according to the preset format corresponding to the second log record category. After the feature code corresponding to the log record category and the feature information corresponding to the feature code are recorded according to the preset format, the log data corresponding to the log record category is obtained.

[0112] The embodiments of the present application provide a variety of log record categories, feature codes corresponding to the log record categories, and can also perform statistics on the performance of the key code segment through the preset function module corresponding to the feature code, manage the resource lock of the key function module, obtain the feature information, and perform structured processing on the feature information to obtain the log data. The embodiments of the present application can simplify the process of recording the log data by the developer, realize the automatic recording of the log data, have simple interfaces, shorten the development period, and at the same time, through the feature code corresponding to the log record category, the time for analyzing, troubleshooting and positioning the log when the system software has a problem can be greatly shortened, the developer is helped to quickly trace the place where the problem or exception occurs, and the human resources and time cost are saved.

[0113] In addition to the above-mentioned method for generating log data, the embodiments of the present application also provide a method for storing log data, so as to store and manage the log data. The embodiments of the present application support multi-target output of the log data. In addition to the traditional log data output functions such as outputting to the console, file, database and the like, the present application also supports outputting to the UI control list, real-time memory type storage, remote service center and the like. The UI control refers to visual elements used to build user interfaces, such as buttons, text boxes, menus, scroll bars, and various operation panels.

[0114] Referring to FIG. 6, a schematic diagram of storing log data to a UI control according to an embodiment of the present application is shown. The embodiment of the present application can graphically display the required log data to the UI control according to log record categories. The specific steps are as follows:

[0115] When it is required to write log data to the UI control, the display rule of the log data in the UI control is obtained; the display rule includes at least one of the displayable log record category, the maximum display entry and the display order;

[0116] The log data is mapped to the UI control, and the log data is displayed in the UI control according to the display rule.

[0117] The log data in the embodiment of the present application can be log data on a semiconductor process equipment, and the semiconductor process equipment can have a UI control list to facilitate the display of the log data. Specifically, LogSinks (log data output pool) contains a log output target ListViewLogSink (UI control list). ListViewLogSink is a package of standard UI control Windows.Forms.ListView, which can match the output information of the log data with the UI control according to the rule, and complete the mapping of the log information to the UI control list. The semiconductor process equipment binds the control list to the property of ListViewLogSink through the application layer, and sets the displayable log record category, the maximum display entry of the log data and the display order, and finally completes the graphical display of the log data. The display order of the output information of the log data in the UI control is: Time (time), Source (log source information), Type (log category feature code), and Message (detailed information) in turn according to column.

[0118] In one embodiment, the method further comprises: when it is required to write log data to a real-time memory type storage, judging whether the log data matches the log record category acceptable by the real-time memory type storage; if the log data matches the log record category acceptable by the real-time memory type storage, writing the log data to the memory queue of the real-time memory type storage; judging whether the stored log data of the memory queue of the real-time memory type storage reaches the log data threshold of the memory queue; if the stored log data of the memory queue reaches the log data threshold, deleting the log data exceeding the log data threshold in the head of the log queue.

[0119] Referring to FIG. 7, a flowchart of the steps of storing log data to a real-time memory type storage according to an embodiment of the present application is shown. The flow of MemoryLogSink (real-time memory type storage) storing log data can include the following sub-steps:

[0120] Sub-step S31, presetting the log record category acceptable by the real-time memory type storage and the maximum log data entries storable by the memory queue;

[0121] Sub-step S32, recording the feature code corresponding to the log record category and the feature information corresponding to the feature code according to the preset format corresponding to the log record category, and generating the log data corresponding to the log record category;

[0122] Sub-step S33, judging whether the log data requested to be written is consistent with the preset log record category, if not, ending the process and not storing the log data; if yes, executing sub-step S34;

[0123] Sub-step S34, adding the log data to the real-time memory management queue;

[0124] Sub-step S35, judging whether the entries of the real-time memory queue exceed the maximum preset capacity of the memory management queue; if yes, executing sub-step S36 and sub-step S37 in turn; if not, executing sub-step S37;

[0125] Sub-step S36, deleting the log data with the earliest record time;

[0126] Sub-step S37, storing the log data into the MemoryLogSink.

[0127] The process of reading the log data by the real-time memory type storage can include the following sub-steps:

[0128] Sub-step S38, obtaining the log data in the MemoryLogSink;

[0129] Sub-step S39, taking out the log data from the real-time memory management queue;

[0130] Sub-step S310, transmitting the log data to the application layer.

[0131] The LogSinks of the present application contains the log output target MemoryLogSink, and the queue management module in the MemoryLogSink is responsible for managing the reading and writing of the log data, thereby saving the time loss of reading and writing the log data through the intermediate layer and achieving the purpose of fast response.

[0132] In an embodiment, the log data in the embodiments of the present application can also be written to a LogServer (remote central log server) so that the server centrally manages each log data and provides storage and query services for managing each log data. Meanwhile, the log data stored in each node can also be synchronized to the central node. In a distributed deployment environment, the API function is used in a data communication manner to read and write log data to the remote central log server, so as to reduce the system pressure caused by the increasing number of physical devices.

[0133] Referring to FIG. 8, a step flowchart of distributed deployment of log data in the embodiments of the present application is shown, and the steps of distributed deployment of log data can include the following sub-steps:

[0134] Sub-step S41, sending a log storage request sent by a plurality of log sources of a plurality of clients to a remote central log server;

[0135] Sub-step S42, intercepting a service request and centrally managing log data of each client by the server;

[0136] Sub-step S43, recording a feature code corresponding to a log record category and feature information corresponding to the feature code according to a preset format corresponding to the log record category by the server, generating log data corresponding to the log record category, and storing the log data.

[0137] The LogSinks of the present application also include a log output target SpoolingClientLogSink (multi-task buffer processing client), which can transmit log data to a remote central log server for unified management when the log data needs to be distributed.

[0138] Referring to FIG. 9, a step flowchart of requesting remote storage of log data in the embodiments of the present application is shown, and the steps of requesting remote storage of log data can include the following sub-steps:

[0139] Sub-step S51, initiating a log data storage request to a log output target SpoolingClientLogSink;

[0140] Sub-step S52, determining whether to perform distributed deployment; in the case of performing distributed deployment, sub-step S53 is executed; in the case of not performing distributed deployment, the flow ends;

[0141] Sub-step S53, determining whether there is a remote connection server; if there is no remote connection server, sub-step S54 and sub-step S55 are sequentially executed; if there is a remote connection server, sub-step S55 is executed;

[0142] Sub-step S54, creating a remote connection;

[0143] Sub-step S55, judging whether the log record category of the write-in is consistent with the preset log record category; in the case of consistent log record category, executing sub-step S56; in the case of inconsistent log record category, ending the flow;

[0144] Sub-step S56, initiating a remote log data storage request to the Log Server through the SpoolingClientLogSink.

[0145] Referring to FIG. 10, a flow chart of the log data sending steps of the embodiment of the present application is shown. Before the SpoolingClientLogSink transmits the log data to the remote central log server, it is necessary to judge whether the communication connection is normal; if the communication connection is not normal, it is necessary to judge whether the data has been buffered; if not, the log data is buffered; if yes, it is necessary to return to the step of judging whether the communication connection is normal; if the communication connection is normal, the log data is sent to the server so that the server stores the log data in a distributed manner. The step of buffering the log data can include the following sub-steps:

[0146] Sub-step S61, judging whether the buffered log data reaches the preset buffer threshold; if the buffered log data does not reach the preset buffer threshold, executing sub-step S62; if the buffered log data reaches the preset buffer threshold, executing sub-step S65;

[0147] Sub-step S62, judging whether the log data is the data not enqueued in the buffer queue; if the log data is the data not enqueued in the buffer queue, executing sub-step S63 and sub-step S64 in turn; if the log data is not the data not enqueued in the buffer queue, executing sub-step S64;

[0148] Sub-step S63, buffering the log data;

[0149] Sub-step S64, making a network inquiry according to the preset rule, and then returning to the step of judging whether the communication connection is normal;

[0150] Sub-step S65, issuing a notification warning, and judging whether to discard the log data according to the preset condition; if it is determined not to discard the log data, executing sub-step S66; if it is determined to discard the log data, executing sub-step S67;

[0151] Sub-step S66, storing the log data to the local disk, and then returning to the step of judging whether the communication connection is normal;

[0152] Sub-step S67, discarding the log data in the buffer area.

[0153] The step of sending the log data to the server can include the following sub-steps:

[0154] In sub-step S68, it is determined whether the log data is the latest data. If the log data is not the latest data, sub-step S69 is entered. If the log data is the latest data, sub-step S611 is entered.

[0155] The latest data refers to the log data that is generated closest to the current time. If the log data is the latest data, it means that the log data has a high storage priority and can be stored in a distributed manner in priority to the important data interaction. If the log data is not the latest data, i.e., the log data is historical data that is out of date, it means that the log data has a low storage priority and can be stored in a distributed manner only when there is no important data interaction in the network.

[0156] In sub-step S69, it is determined whether there is important data interaction in the network. If there is important data interaction in the network, sub-step S610 is entered. If there is no important data interaction in the network, i.e., the communication connection is idle, sub-step S611 is entered.

[0157] In sub-step S610, the important data interaction is performed in priority, and then the step of determining whether the communication connection is normal is returned.

[0158] In sub-step S611, the log data is sent to the server, so that the server stores the log data in a distributed manner.

[0159] In an embodiment, the method further includes: writing the log data into a log file; determining whether the log file meets a deletion condition; the deletion condition includes at least one of whether the number of files in a file group in which the log file is located exceeds a pre-set threshold, whether the storage time of the files in the file group in which the log file is located exceeds a pre-set threshold, whether the disk space occupied by the log file exceeds a pre-set threshold, whether a pre-set condition exists in the file group in which the log file is located, and whether the condition exceeds a pre-set threshold; and deleting the log file when the log file meets the deletion condition.

[0160] In the embodiment, the log data can also be stored into a log file. Referring to FIG. 11, a flowchart of a step of writing log data into a log file according to the embodiment is shown. The step of writing the log data into the log file can include the following sub-steps:

[0161] In sub-step S71, a log data recording request is initiated.

[0162] In sub-step S72, the log data is received.

[0163] After the log file is received at the service end of the semiconductor process equipment, the log file can be automatically managed according to a preset file management rule, a preset file compression format and a preset file creation format, and then the log file is structured into a specified format according to a preset rule and stored in a preset directory. The automatic management of the log file refers to the operations such as the standardized automatic compression of the log file and the deletion of expired files.

[0164] In substep S73, it is determined whether the log data needs to be stored according to the mark of the log category characteristic code to which the log data belongs. If the log data does not need to be stored, substep S74 is performed. If the log data needs to be stored, substep S75 is performed.

[0165] In substep S74, the log data is discarded.

[0166] In substep S75, the log file group to which each piece of log data belongs is determined.

[0167] In substep S76, it is determined whether there is a usable log file in the current available path. If there is a usable log file, substep S77 is performed. If there is no usable log file, substep S78 is performed.

[0168] In substep S77, the log information is structured into a specified format according to a preset rule, and written into a log file.

[0169] In the embodiments of the present application, the structuring of the log file refers to naming the file according to a preset format, for example, Tool.20230717@064617.log.gz. The structuring of the log information refers to recording and storing the attribute information of the log data according to a pre-specified format, for example, SourceProcess (the name of the application software generating the log file), Source (the log source, the unit to which the log belongs), Type (the log category), UtcTimestamp (the time stamp), ThreadId (the thread ID) and the like.

[0170] In substep S78, the current expired file is deleted.

[0171] In the embodiments of the present application, the deletion of the expired file refers to that when the number of files in the specified path reaches a preset maximum storage number or the storage days of the log files in the specified path exceed a preset number of days, the system automatically backs up or deletes the historical log files. For example, the preset maximum storage number can be 100 log files, and the preset number of days can be 30 days, which are not limited in the present application.

[0172] In substep S79, the expired log files in the log file group are compressed according to a preset compression rule and a preset file compression format of the log file group.

[0173] In the embodiments of the present application, the normalized automatic compression refers to that when the file size reaches the preset capacity, the expired log files in the log file group are uniformly compressed according to the pre-set compression rule and the pre-set file compression format of the log file group.

[0174] In the sub-step S710, a new log file is created according to the pre-set rule.

[0175] Referring to FIG. 12, a flowchart of the log file deletion in the embodiments of the present application is shown. In one embodiment, the step of deleting the log file (i.e., the sub-step S78) when the log file meets the deletion condition can include the following sub-steps:

[0176] In the sub-step S81, the files exceeding the threshold are deleted according to the deletion condition.

[0177] In the sub-step S82, the deletion condition includes at least one of the following: whether the number of files in the file group where the log file is located exceeds the pre-set threshold, whether the storage time of the files in the file group where the log file is located exceeds the pre-set threshold, whether the disk space occupied by the log file exceeds the pre-set threshold, whether the pre-set condition exists in the file group where the log file is located, and whether the condition exceeds the pre-set threshold.

[0178] In the sub-step S83, it is determined whether the deletion condition is met. If the deletion condition is met, the sub-step S84 is executed. If the deletion condition is not met, the flow ends.

[0179] In the sub-step S84, a warning is issued, and it is determined whether the log file is automatically deleted. If the log file is not automatically deleted, the sub-step S85 and the sub-step S86 are executed in sequence. If it is determined that the log file is automatically deleted, the sub-step S86 is executed.

[0180] The operation of issuing the warning is specifically as follows: when the number of files in the file group where the log file is located reaches a first threshold, or when the disk space occupied by the log file reaches the first threshold, a first warning is issued. The first threshold represents that the system is not affected in normal operation, and the first warning is used to prompt that the log file is not necessarily deleted. When the number of files in the file group where the log file is located reaches a second threshold, or when the disk space occupied by the log file reaches the second threshold, a second warning is issued. The second threshold represents that the system is affected in normal operation, and the second warning is used to prompt that the log file must be deleted.

[0181] In the sub-step S85, the user is notified to manually initiate the deletion instruction.

[0182] In the sub-step S86, the deletion operation is automatically executed according to the pre-set deletion rule.

[0183] The embodiments of the present application provide a variety of log record categories, support standardized log category recording and user customized log category recording, can trace and mark the calling process of a method function or a function block by acquiring the method function of the calling layer, perform statistics on the performance of a method function or a function block of a key code segment, manage the resource lock of a key function module, and perform structured processing and storage on log data, meanwhile, a variety of log data storage methods are provided, cache and distributed deployment are supported, log files are centrally managed, and a perfect log file management mechanism is provided. The embodiments of the present application can simplify the process of log data recording by a developer, realize automatic log data recording, have simple interfaces, shorten the development cycle, and meet diversified log storage requirements, meanwhile, through the variety of log record categories, the time for analyzing, troubleshooting and locating logs when a problem occurs in system software can be greatly shortened, the developer can quickly trace the place where the problem or exception occurs, and human resources and time cost are saved.

[0184] It should be noted that, for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited to the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily the necessary of the embodiments of the present application.

[0185] Referring to FIG. 13, a structural block diagram of a log processing apparatus provided by the embodiments of the present application is shown, the log processing apparatus 1301 includes a processor 13011 and a memory 13012, the memory 13012 stores a computer program, and the processor executes the steps of the log processing method of any one of the above embodiments when executing the computer program.

[0186] Referring to FIG. 14, a structural block diagram of a storage medium provided by the embodiments of the present application is shown, the storage medium 1401 stores a computer program 14011, and the computer program 14011 executes the steps of the log processing method of any one of the above embodiments.

[0187] Each of the embodiments in the specification is described in a progressive manner, and each embodiment mainly describes the difference from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0188] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable medium (including, but not limited to, diskette, CD-ROM, optical storage device, etc.) having computer usable program code embodied in the medium.

[0189] The present application is described in reference to the flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0190] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions means which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0192] Although preferred embodiments of the present application have been described, those skilled in the art will appreciate that additional modifications and changes can be made to these embodiments without departing from the scope of the present application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the present application.

[0193] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0194] The log processing method, device and storage medium provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A log processing method, characterized in that: Applied to semiconductor process equipment, the method includes: Determine a log record category configured for the application; the log record category has a one-to-one corresponding feature code; the feature code is used to identify log data corresponding to the log record category; Acquire feature information corresponding to the feature code generated during the operation of the application; Log data corresponding to the log record category is generated according to the feature code corresponding to the log record category and feature information corresponding to the feature code.

2. The method according to claim 1, characterized in that The log record category includes a first log record category, the first log record category indicates performance parameter information of the application program during task execution, a feature code corresponding to the first log record category is a first feature code, and obtaining feature information corresponding to the feature code generated by the application program during operation includes: intercepting a first request for the application and obtaining performance parameter information of a task executed by a first key code segment in response to the first request; The performance parameter information is used as feature information corresponding to the first feature code.

3. The method according to claim 1, characterized in that The log record category includes a second log record category, the second log record category indicates thread lock information of the application program during task execution, the feature code corresponding to the second log record category is a second feature code, and obtaining feature information corresponding to the feature code generated by the application program during operation includes: intercepting a second request for the application, and obtaining first thread lock information and second thread lock information of a second critical code segment executing a task in response to the second request; the first thread lock information corresponding to the thread lock information when the thread state is locking, and the second thread lock information corresponding to the thread lock information when the thread state is locking completed; releasing a second request for the application, and obtaining third and fourth thread lock information for a second critical code segment executing a task in response to the second request; the third thread lock information corresponding to the thread lock information when the thread state is unlocking, and the fourth thread lock information corresponding to the thread lock information when the thread state is unlocked; The first thread lock information, the second thread lock information, the third thread lock information, and the fourth thread lock information are used as feature information corresponding to the second feature code.

4. The method according to claim 1, wherein Generating log data corresponding to the log record category according to the feature code corresponding to the log record category and feature information corresponding to the feature code includes: According to a preset format corresponding to the log record category, a feature code corresponding to the log record category and feature information corresponding to the feature code are recorded to generate log data corresponding to the log record category.

5. The method according to claim 1, characterized in that The method further comprises: When the log data needs to be written into the UI control, obtaining a display rule of the log data in the UI control; the display rule includes at least one of a displayable log record category, a maximum display entry, and a display order; The log data is mapped to the UI control, and the log data is displayed on the UI control according to the display rule.

6. The method according to claim 1, characterized in that The method further comprises: When the log data needs to be written into the real-time memory type storage, determining whether the log data matches a log record category that can be received by the real-time memory type storage; If the log data matches a log record category acceptable to the real-time memory type storage, writing the log data to a memory queue of the real-time memory type storage; Determining whether the stored log data of the memory queue of the real-time memory type storage reaches the log data threshold of the memory queue; If the stored log data in the memory queue reaches the log data threshold, the log data at the head of the log queue that exceeds the log data threshold is deleted.

7. The method according to claim 1, characterized in that The method further comprises: When the log data needs to be written to the server, determining whether the communication connection with the server is normal; If the communication connection is abnormal, it is determined whether the cached log data has reached the preset cache threshold; If the cached log data does not reach a preset cache threshold, cache the log data; If the cached log data reaches a preset cache threshold, determining whether to discard the log data; If it is determined not to discard the log data, the log data is stored in a local disk.

8. The method according to claim 7, characterized in that The method further comprises: If the communication connection is normal, determining whether the log data is the latest data; If the log data is the latest data, sending the log data to the server so that the server performs distributed storage on the log data; If the log file is not the latest data, the log data is sent to the server when the communication connection with the server is idle.

9. A log processing device, characterized in that: include: A processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the log processing method according to any one of claims 1 to 8 are performed.

10. A storage medium, characterized in that: The storage medium stores a computer program, and the computer program executes the steps of the log processing method according to any one of claims 1 to 8.

Images