Log file generation method and device
The pre-generated hash table parses the service request information and generates a log file that meets the log center template file, which solves the problem of frequent code modifications when the Web server sends out logs, improving stability and flexibility.
Patent Information
- Application Number
- CN202111601184.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-24
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2041-12-24
AI Technical Summary
Existing web servers need to frequently modify the code when sending out logs to adapt to the log formats of different log centers, which increases development workload and leads to instability of the platform.
The pre-generated hash table analyzes the processing information of the service request, and generates a log file that conforms to the log center template file, avoiding code modification and increasing flexibility.
It realizes the flexibility and stability of log file generation, reduces the need for code modification, and improves the flexibility and convenience of docking between the Web server and the log center.
Smart Images

Figure CN114661682B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a log file generation method and device. Background Art
[0002] Openresty is a high-performance, easily scalable Web server framework that is loved and used by a large number of users. With the development of network scale and the increase in business volume, as well as the need for daily operation and maintenance, the demand for Syslog system logs is increasing. System logs can be used to record various events that occur on the device every day, and can be configured to be analyzed and processed by an external log center to uniformly monitor and manage the device. When we use a Web server to send logs externally, we need to connect to different log centers, and different log centers belong to different companies and have different log formats defined. When the existing Web server sends logs externally, it adapts to the log format of different log centers by modifying the relevant code for log outbound transmission. This will cause frequent code modifications in the Web server, which will not only increase the workload of Web server development, but also cause platform instability. Summary of the invention
[0003] In view of the problems in the prior art, an embodiment of the present invention provides a method and device for generating a log file.
[0004] Specifically, the embodiment of the present invention provides the following technical solutions:
[0005] In a first aspect, an embodiment of the present invention provides a log file generation method, which is applied to a Web server, comprising:
[0006] In response to receiving a service request from a client, processing the service request to obtain processing information corresponding to the service request;
[0007] Based on a pre-generated hash table, data parsing is performed on the processing information corresponding to the service request to generate a log file corresponding to the service request; wherein the hash table is generated based on template parsing of a template file of a log center in advance;
[0008] The generated log file is sent to the log center.
[0009] Furthermore, the generating of the hash table based on pre-template parsing of the template file of the log center includes:
[0010] Obtain the template file of the log center;
[0011] The template file is parsed to generate the hash table, wherein the hash table includes a Key field and a corresponding Value field, wherein the Key field in the hash table is determined based on the log component items in the template file.
[0012] The step of performing data parsing on the processing information corresponding to the service request based on the pre-generated hash table to generate a log file corresponding to the service request includes:
[0013] Based on the hash table, data parsing is performed on the processing information corresponding to the service request to determine the Value field in the hash table, and a log file corresponding to the service request is generated based on the hash table in which the Value field is determined.
[0014] Furthermore, before performing template parsing on the template file and generating the hash table, the process further includes:
[0015] Receiving a template file sent by the log center;
[0016] The template file of the log center stored locally is updated according to the received template file; wherein the template file sent by the log center is obtained by modifying the template file of the log center stored locally.
[0017] Furthermore, the performing template parsing on the template file to generate the hash table includes:
[0018] Performing template parsing on the template file, and generating the hash table according to the data type defined in the template file;
[0019] The performing data parsing on the processing information corresponding to the service request based on the hash table to determine the Value field in the hash table includes:
[0020] Based on the hash table, data parsing is performed on the processing information corresponding to the service request according to the data type defined in the template file to determine the Value field in the hash table.
[0021] Furthermore, the data types defined in the template file include: standard variable types, special variable types and fixed information types.
[0022] Further, performing template parsing on the template file to generate the hash table includes:
[0023] The template file is parsed by a plug-in to generate the hash table.
[0024] Furthermore, the template file is a Json file; the template file is parsed by a plug-in to generate the hash table, including:
[0025] Based on the hierarchical structure of the Json file, the Json file is template parsed by a plug-in to generate the hash table, wherein the hierarchical structure of the hash table corresponds to the hierarchical structure of the Json file;
[0026] The log file is a Json file; generating a log file corresponding to the service request based on the hash table includes:
[0027] The hash table is converted into a Json file as a log file corresponding to the service request.
[0028] Furthermore, the Web server adopts the Openresty framework, and the plug-in is written in the Lua scripting language.
[0029] In a second aspect, an embodiment of the present invention further provides a log file generating device, which is applied to a Web server, comprising:
[0030] A request processing module, used to process the service request in response to receiving the service request from the client, and obtain processing information corresponding to the service request;
[0031] A data parsing module, used to perform data parsing on the processing information corresponding to the service request based on a pre-generated hash table, and generate a log file corresponding to the service request; wherein the hash table is generated based on pre-template parsing of a template file of a log center;
[0032] The log sending module is used to send the generated log files to the log center.
[0033] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the log file generation method described in the first aspect are implemented.
[0034] In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the log file generation method described in the first aspect.
[0035] In a fifth aspect, an embodiment of the present invention further provides a computer program product having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the steps of the log file generation method described in the first aspect.
[0036] The log file generation method and device provided by the embodiment of the present invention can generate a log file that conforms to the template file of the log center by performing template parsing on the template file in the log and generating a hash table. According to the generated hash table, data parsing is performed on the processing information corresponding to the service request, so that the log file of the outbound log center does not need to adapt to the log format of the log center by modifying the relevant code for outbound log sending. As long as the log information is obtained and the generation of the log file is controlled by parsing the template file of the log center, the log file that conforms to the template file of the log center can be obtained. When it is necessary to change or add a field in the log format, it is only necessary to change or add the corresponding field in the template file to realize the modification of the log file format without modifying any relevant code. The stability of the Web server can be enhanced, the flexibility of the connection between the Web server and the log center can be enhanced, and it is more convenient. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 It is a flow chart of the log file generation method provided by the present invention;
[0039] Figure 2 It is a schematic diagram of a process of generating a hash table through template parsing provided by the present invention;
[0040] Figure 3 It is another schematic diagram of a process of generating a hash table by template parsing provided by the present invention;
[0041] Figure 4 It is a schematic diagram of an application scenario of the log file generation method provided by the present invention;
[0042] Figure 5 It is a schematic diagram of the composition structure of the log file generating device provided by the present invention;
[0043] Figure 6 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] Combine the following Figure 1-Figure 4 The log file generation method of the present invention is described.
[0046] See also Figure 1 , Figure 1 is a flow chart of the log file generation method provided by the present invention, Figure 1 The log file generation method shown can be applied to a Web server, such as Figure 1 As shown, the log file generation method at least includes:
[0047] 101. In response to receiving a service request from a client, the service request is processed to obtain processing information corresponding to the service request.
[0048] In an embodiment of the present invention, a Web server is used to connect with a client, receive a service request sent by a client through a Web browser, process the service request, and feed back the processing result to the Web browser. Usually, a Web server uses Hyper Text Transfer Protocol (HTTP) to exchange information with a client, so the Web server can also be called an HTTP server. A Web server can be a physical server including an independent host, a virtual server carried by a host cluster, a cloud server, etc. The embodiment of the present invention does not limit the implementation method of the Web server. A Web server can be a server applied to different scenarios, and the embodiment of the present invention does not limit the application scenario of the Web server. After receiving the service request from the client, the Web server will process the received service request and obtain the processing information corresponding to the service request. The processing information corresponding to the service request can be recorded by the system as log information.
[0049] 102. Based on a pre-generated hash table, perform data analysis on the processing information corresponding to the service request to generate a log file corresponding to the service request; wherein the hash table is generated based on pre-analysis of a template file of a log center.
[0050] In an embodiment of the present invention, after the Web server processes the received service request and obtains the processing information corresponding to the service request, the processing information corresponding to the service request can be parsed according to the hash table generated by pre-templating the template file of the log center, and a log file that meets the requirements of the template file of the log center can be generated. Among them, the template file is a file configured according to the log format required by the log center. The log center may include different log centers connected to the Web server. Since the log formats defined by different log centers are different, different log centers have different template files. By parsing the different template files of different log centers, hash tables corresponding to different log centers can be generated. According to the hash tables corresponding to different log centers, the processing information corresponding to the service request can be parsed, and log files that meet the requirements of the template files of different log centers can be generated respectively.
[0051] 103, sending the generated log file to the log center.
[0052] In an embodiment of the present invention, after generating a log file that meets the requirements of the template file of the log center, the Web server can send the generated log file that meets the requirements of the template file of the log center to the corresponding log center, and the log center will perform corresponding analysis and processing on the log file, and uniformly monitor and manage the equipment.
[0053] The log file generation method provided by the embodiment of the present invention can generate a log file that conforms to the template file of the log center by performing template parsing on the template file in the log and generating a hash table. According to the generated hash table, data parsing is performed on the processing information corresponding to the service request, so that the log file of the outbound log center does not need to adapt to the log format of the log center by modifying the relevant code for outbound log sending. As long as the log information is obtained and the generation of the log file is controlled by parsing the template file of the log center, the log file that conforms to the template file of the log center can be obtained. When it is necessary to change or add a field in the log format, it is only necessary to change or add the corresponding field in the template file to realize the modification of the log file format without modifying any relevant code. The stability of the Web server can be enhanced, the flexibility of the connection between the Web server and the log center can be enhanced, and it is more convenient.
[0054] See also Figure 2 , Figure 2 It is a schematic diagram of the process of generating a hash table by template parsing provided by the present invention, such as Figure 2 As shown, generating a hash table based on pre-template parsing of the template file of the log center includes at least:
[0055] 201, obtain the template file of the log center.
[0056] 202 , parse the template file to generate a hash table, where the hash table includes a Key field and a Value field, wherein the Key field in the hash table is determined based on the log component items in the template file.
[0057] In an embodiment of the present invention, when the Web server is started, a template file of the log center can be obtained from the log center, and the obtained template file of the log center can be template parsed to generate a corresponding hash table; or, when the Web server is connected to the log center, the template file of the log center can be obtained from the log center, the obtained template file of the log center can be stored, and when the Web server is started, the stored template file of the log center can be template parsed to generate a corresponding hash table; or, when the Web server is connected to the log center, the template file of the log center can be obtained from the log center, and the obtained template file of the log center can be template parsed to generate a corresponding hash table; the embodiment of the present invention does not limit the implementation method of the Web server obtaining the template file of the log center and performing template parsing on the template file to generate a hash table.
[0058] In the embodiment of the present invention, the hash table is a data structure that is directly accessed according to the key code value, that is, the Key-value pair. That is to say, it can speed up the search by mapping the key code value to a position in the table to access the record; therefore, the hash table includes a Key field and a Value field, wherein the Key field in the hash table can be determined according to the log component items in the template file when the template file is parsed, for example, the log component items in the template file can be host, uri, user name, device name, etc.; the Value field in the hash table can be determined according to the real data value obtained by the data parsing when the processing information corresponding to the service request is parsed according to the hash table. For example, the real data value obtained by parsing the processing information is qax.com, / syslog / log, huang, ID19865, etc., wherein the host is qax.com, the uri is / syslog / log, the user name is huang, and the device name is ID19865; after determining the Key field and the Value field in the hash table, the log file corresponding to the service request can be generated according to the hash table at this time.
[0059] In some optional examples, if the log center needs to modify the original log format, for example, add some new structures and data to the original log format, it only needs to add the corresponding structure and the corresponding variable value position in the template file, and send the modified template file to the Web server. The Web server replaces the original template file stored in the log center according to the received modified template file, and performs template parsing on the modified template file to generate a hash table. The log file in the required log format can be generated according to the hash table generated at this time. There is no need to make any modifications to the Web server code, and the log format can be modified at will. therefore Before parsing the template file and generating a hash table, the Web server can also receive a template file sent by the log center and update the template file of the log center stored locally according to the received template file; wherein the template file sent by the log center is obtained by modifying the template file of the log center stored locally.
[0060] In some optional examples, the template file can define the data types in the log file, for example, the defined data types include: standard variable types, special variable types and fixed information types; the standard variable types can be standard variables in HTTP requests, such as the requested host, uri, etc.; the special variable types can be special variables generated in the Web server, such as the user name, application name, device information, etc. in the login request; fixed information types, such as the device name where the identification platform is located, etc. These three data types can almost cover all data types reported by the web server log.
[0061] When performing template parsing on a template file to generate a hash table, a hash table can be generated according to the data type defined in the template file by performing template parsing on the template file. When performing data parsing on processing information corresponding to a service request according to the hash table to determine the Value field in the hash table, data parsing on processing information corresponding to the service request can be performed according to the hash table according to the data type defined in the template file to determine the Value field in the hash table.
[0062] For example, when performing template parsing to generate a hash table for a template file, a hash table can be generated for standard variable types and a hash table can be generated for special variable types, so that all standard variables in HTTP requests obtained by data parsing are stored in a hash table, such as the hash table of ngx.var, and all special variables generated in the Web server obtained by data parsing are also stored in a hash table, such as the hash table of ctx.var. When performing data parsing on processing information corresponding to a service request, different types of data can be distinguished according to the prefix of the data type defined in the template file; for example, the prefix "ngx-" represents standard variable data in HTTP requests, the prefix "ctx-" represents special variable data generated in the Web server, and the prefix "v-" represents fixed information; the data level can be identified by ".", such as "ab" represents the value of key b in hash table a, that is, a[b]; in this way, ngx-host in the template file will be parsed as the value of ngx.var[host], ctx-user.name will be parsed as the value of ctx.var[user][name]; v-ID15672 will be directly parsed as ID15672.
[0063] In some optional examples, template parsing of template files can be performed through plug-ins to generate hash tables, so as to utilize the high extensibility of the plug-in and facilitate adapting to the needs of template parsing of different template files.
[0064] See also Figure 3 , Figure 3 FIG. 1 is another flow chart of generating a hash table by template parsing provided by the present invention, such as Figure 3 As shown, the hash table is generated based on the template parsing of the template file of the log center in advance, which at least includes:
[0065] 301, get the Json file of the log center.
[0066] 302. Based on the hierarchical structure of the Json file, the Json file is template parsed through the plug-in to generate a hash table, where the hash table includes a Key field and a Value field, wherein the Key field in the hash table is determined based on the log component items in the Json file, and the hierarchical structure of the hash table corresponds to the hierarchical structure of the Json file.
[0067] In the embodiment of the present invention, since the Json format has good versatility and readability, the template file can be a Json file. When the template file is parsed, the Json file can be parsed through a plug-in according to the hierarchical structure of the Json file to generate a hash table. At this time, the hash table structure is the structure of the Json file, and the Key field in the hash table is determined according to the Key field in the Json file. At this time, the log file is also a Json file. When generating a log file corresponding to a service request based on the hash table, the hash table can be converted into a Json file as the log file corresponding to the service request.
[0068] In some optional examples, the Web server can use the Openresty framework, which is a high-performance service framework based on Nginx and Lua. It integrates a large number of sophisticated Lua libraries, third-party modules, and most dependencies. It is used to easily build dynamic Web applications, Web services, and dynamic gateways that can handle ultra-high concurrency and high scalability. The Web server can be secondary developed based on the Openresty framework to obtain a Web server that meets the requirements of specific application scenarios. The template parsing and hash table generation plug-in for the template file is written in Lua scripting language.
[0069] See also Figure 4 , Figure 4 is a schematic diagram of an application scenario of the log file generation method provided by the present invention, such as Figure 4 As shown, when the Web server is started, the template file of the log center is obtained from the log center connected to the Web server. Only one template file is shown in the figure, and the dotted line indicates that the template file here can be a template file of different log formats of multiple log centers connected to the Web server. The obtained log file is parsed by template to generate a corresponding hash table, which is stored in the memory. When the Web server receives the HTTP service request from the client, it will process the service request to obtain the processing information corresponding to the service request. The processing information corresponding to the service request is transmitted to the back-end service of the Web server, and the corresponding service will be provided for the service request through the back-end service.
[0070] During the transmission of the processing information corresponding to the service request to the back-end service, the processing information corresponding to the service request can be obtained, and the processing information corresponding to the service request can be parsed through the hash table stored in the memory of the Web server to generate a log file corresponding to the service request. A log file is generated for each service request, and the generated log file is sent to the corresponding log center for analysis and processing.
[0071] The log center needs to analyze the logs reported by the Web server, so the reported logs need to have a specific format and corresponding information. The following is an example of the basic information of an application being accessed and the log of the accessing user that the log center needs to report:
[0072]
[0073] After the Web server is connected to the log center, you need to configure the corresponding template file and restart the Web server to generate corresponding log files according to different requests. The following is an example of the corresponding template file configured:
[0074]
[0075]
[0076] The log file generating device provided by the present invention is described below. The log file generating device described below and the log file generating method described above can be referred to each other.
[0077] See also Figure 5 , Figure 5 is a schematic diagram of the composition structure of the log file generating device provided by the present invention, Figure 5 The log file generating device shown is applied to a Web server and can be used to execute Figure 1 Log file generation method, such as Figure 5 As shown, the log file generating device at least includes:
[0078] The request processing module 510 is used to process the service request in response to receiving the service request from the client and obtain processing information corresponding to the service request.
[0079] The data analysis module 520 is used to perform data analysis on the processing information corresponding to the service request based on a pre-generated hash table, and generate a log file corresponding to the service request; wherein the hash table is generated based on pre-template analysis of a template file of the log center.
[0080] The log sending module 530 is used to send the generated log file to the log center.
[0081] Optionally, the log file generating device further includes:
[0082] The file acquisition module is used to obtain the template file of the log center.
[0083] The template parsing module is used to parse the template file and generate a hash table. The hash table includes a Key field and a corresponding Value field, wherein the Key field in the hash table is determined based on the log component items in the template file.
[0084] The data analysis module 520 is used to perform data analysis on the processing information corresponding to the service request based on the hash table, determine the Value field in the hash table, and generate a log file corresponding to the service request based on the hash table of the determined Value field.
[0085] Optionally, the log file generating device further includes:
[0086] The file receiving module is used to receive the template file sent by the log center.
[0087] The file updating module is used to update the template file of the locally stored log center according to the received template file; wherein the template file sent by the log center is obtained by modifying the template file of the locally stored log center.
[0088] Optionally, the template parsing module is used to perform template parsing on the template file and generate a hash table according to the data type defined in the template file.
[0089] The data analysis module 520 is used to perform data analysis on the processing information corresponding to the service request based on the hash table and according to the data type defined in the template file, and determine the Value field in the hash table.
[0090] Optionally, the data types defined in the template file include: standard variable types, special variable types and fixed information types.
[0091] Optionally, the template parsing module is used to perform template parsing on the template file through a plug-in to generate a hash table.
[0092] Optionally, the template file is a Json file.
[0093] The template parsing module is used to perform template parsing on the Json file through a plug-in based on the hierarchical structure of the Json file to generate a hash table, where the hierarchical structure of the hash table corresponds to the hierarchical structure of the Json file.
[0094] The log file is a Json file.
[0095] The data parsing module 520 is used to convert the hash table into a Json file as a log file corresponding to the service request.
[0096] Optionally, the web server adopts the Openresty framework and the plug-in is written in the Lua scripting language.
[0097] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the following method: in response to receiving a service request from a client, the service request is processed to obtain processing information corresponding to the service request; based on a pre-generated hash table, the processing information corresponding to the service request is parsed to generate a log file corresponding to the service request; wherein the hash table is generated based on a template parsing of a template file of a log center in advance; and the generated log file is sent to the log center.
[0098] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0099] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the methods provided in the above embodiments, for example, including: in response to receiving a service request from a client, processing the service request to obtain processing information corresponding to the service request; based on a pre-generated hash table, performing data parsing on the processing information corresponding to the service request to generate a log file corresponding to the service request; wherein the hash table is generated based on pre-template parsing of a template file of a log center; and sending the generated log file to the log center.
[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A log file generation method, characterized in that: Applicable to Web servers, including: In response to receiving a service request from a client, processing the service request to obtain processing information corresponding to the service request; Based on a pre-generated hash table, data parsing is performed on the processing information corresponding to the service request to generate a log file corresponding to the service request; wherein the hash table is generated based on template parsing of a template file of a log center in advance; The generating of the hash table based on pre-template parsing of the template file of the log center includes: acquiring the template file of the log center; performing template parsing on the template file, and generating the hash table according to the data type defined in the template file, wherein the hash table includes a Key field and a corresponding Value field, wherein the Key field in the hash table is determined based on the log component item in the template file; The performing data parsing on the processing information corresponding to the service request based on the pre-generated hash table to generate a log file corresponding to the service request includes: performing data parsing on the processing information corresponding to the service request according to the data type defined in the template file based on the hash table, determining the Value field in the hash table, and generating the log file corresponding to the service request based on the hash table determining the Value field; The generated log file is sent to the log center.
2. The log file generation method according to claim 1, characterized in that: Before performing template parsing on the template file and generating the hash table, the method further includes: Receiving a template file sent by the log center; The template file of the log center stored locally is updated according to the received template file; wherein the template file sent by the log center is obtained by modifying the template file of the log center stored locally.
3. The log file generation method according to claim 1, characterized in that: The data types defined in the template file include: standard variable types, special variable types and fixed information types.
4. The log file generation method according to claim 2, characterized in that: Performing template parsing on the template file to generate the hash table includes: The template file is parsed by a plug-in to generate the hash table.
5. The log file generation method according to claim 4, characterized in that: The template file is a Json file; Performing template parsing on the template file through a plug-in to generate the hash table includes: Based on the hierarchical structure of the Json file, the Json file is template parsed by a plug-in to generate the hash table, wherein the hierarchical structure of the hash table corresponds to the hierarchical structure of the Json file; The log file is a Json file; generating a log file corresponding to the service request based on the hash table includes: The hash table is converted into a Json file as a log file corresponding to the service request.
6. The log file generation method according to claim 5, characterized in that: The Web server adopts the Openresty framework, and the plug-in is written in the Lua scripting language.
7. A log file generating device, characterized in that: Applicable to Web servers, including: A request processing module, used to process the service request in response to receiving the service request from the client, and obtain processing information corresponding to the service request; A data parsing module, used to perform data parsing on the processing information corresponding to the service request based on a pre-generated hash table, and generate a log file corresponding to the service request; wherein the hash table is generated based on pre-template parsing of a template file of a log center; The log file generating device further includes: a file acquiring module, used to acquire a template file of a log center; a template parsing module, used to parse the template file, and generate a hash table according to the data type defined in the template file, wherein the hash table includes a Key field and a corresponding Value field, wherein the Key field in the hash table is determined based on the log component item in the template file; A data parsing module, used to parse the processing information corresponding to the service request based on the hash table and the data type defined in the template file, determine the Value field in the hash table, and generate a log file corresponding to the service request based on the hash table of the determined Value field; The log sending module is used to send the generated log files to the log center.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the log file generation method according to any one of claims 1 to 6 are implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the log file generation method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Cross-platform and cross-application log collecting system and collecting managing method thereof
CN103425750A
Log processing based numerical control system security processing method
CN105975376A
Stream log collection method, device and equipment and storage medium
CN111786973A
Business system monitoring method and device, storage medium and equipment
CN112579387A
Log data processing method and system and storage medium
CN112866319A
