Log generation method and device, electronic device, and storage medium

Abstract

Embodiments of the present application provide a log generation method and device, electronic equipment and storage medium, belonging to the technical field of artificial intelligence. The method comprises: obtaining log configuration information, the log configuration information comprising format information and correction information, the correction information being used to determine a to-be-corrected object for correction processing. Log data items are obtained from the format information, and a format template is constructed for the log data items. An interceptor is constructed according to the log configuration information. The interceptor is used to intercept a monitoring object, and first processing data of the monitoring object is read according to the log data items. According to the correction information, a to-be-corrected object is identified from the first processing data, and correction processing is performed on the to-be-corrected object to obtain second processing data, the second processing data comprising first target data corresponding to the log data items. The first target data is imported into the format template to generate a target log file. It can be seen that the embodiments of the present application can improve the log generation quality, thereby improving the accuracy of log positioning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a log generation method and apparatus, electronic device, and storage medium. Background Technology

[0002] System log printing is a common method to assist in diagnosing whether a system is functioning properly, facilitating quick location and analysis of system problems by relevant personnel. Currently, system log printing typically requires recording large amounts of processed data into a database. In practice, this method has been found to be labor-intensive and prone to issues such as missing or incorrect keywords, thus affecting the accuracy of subsequent log-based problem identification. Therefore, improving the accuracy of log-based problem identification has become a pressing technical issue that needs to be addressed. Summary of the Invention

[0003] The main objective of this application is to provide a log generation method, apparatus, electronic device, and storage medium, which aims to improve the accuracy of log location.

[0004] To achieve the above objectives, a first aspect of this application proposes a log generation method, the method comprising:

[0005] Obtain log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing.

[0006] Obtain log data items from the format information and construct a format template for the log data items;

[0007] Based on the log configuration information, construct the interceptor;

[0008] The interceptor is used to intercept the monitored object, and the first processing data of the monitored object is read according to the log data item;

[0009] Based on the correction information, the object to be corrected is identified from the first processed data, and the object to be corrected is corrected to obtain second processed data, the second processed data including the first target data corresponding to the log data item;

[0010] Import the first target data into the format template to generate the target log file.

[0011] In some embodiments, the object to be corrected includes an object to be removed; the step of correcting the object to be corrected to obtain second processed data includes:

[0012] The object to be removed is removed from the first processed data to obtain the second processed data.

[0013] In some embodiments, the log configuration information further includes print-free information; the step of reading the first processing data of the monitored object based on the log data items includes:

[0014] Obtain the object configuration items and the preset object information corresponding to the object configuration items from the print-free information;

[0015] Based on the object configuration item, read the target object information of the monitored object, whereby the target object information is used to uniquely identify the monitored object;

[0016] If the target object information is not included in the preset object information, then the first processing data of the monitored object is read according to the log data item;

[0017] The method further includes:

[0018] If the preset object information includes the target object information, then the monitoring object continues to run.

[0019] In some embodiments, the object to be corrected includes a de-identified object; the correction process on the object to be corrected to obtain second processed data includes:

[0020] Obtain the desensitization rule corresponding to the desensitized object from the correction information;

[0021] According to the desensitization rules, the desensitized object is desensitized in the first processed data to obtain the second processed data.

[0022] In some embodiments, the method further includes:

[0023] Collect log files generated within a specified time period as reference log files;

[0024] Matching information is obtained from the correction information, and the matching information is used to determine the de-identified object;

[0025] Based on the matching information, data matching processing is performed in the reference log file to obtain log data related to the matching information, and reference keywords are obtained from the log data;

[0026] If the matching information does not include the reference keyword, then the reference keyword is added to the matching information to obtain new matching information.

[0027] In some embodiments, the matching information includes matching keywords, regular expressions, and association information between the matching keywords and the regular expressions; the step of performing data matching processing in the reference log file according to the matching information to obtain log data related to the matching information, and obtaining reference keywords from the log data, includes:

[0028] Based on the regular expression and the association information, data matching processing is performed in the reference log file to obtain log data related to the matching information;

[0029] Keyword information other than the regular expression and the association information is obtained from the log data and used as reference keywords.

[0030] In some embodiments, the step of intercepting the monitored object using the interceptor and reading the first processing data of the monitored object based on the log data item includes:

[0031] Before running the monitored object, the interceptor is used to intercept the monitored object and read the input parameters of the monitored object;

[0032] If a first parameter included in the log data item is identified from the input parameters, then the parameter value of the first parameter is read from the monitored object;

[0033] After running the monitored object, the interceptor is used to intercept the monitored object and read the output parameters of the monitored object;

[0034] If a second parameter is identified from the output parameters, the parameter value of the second parameter is read from the monitored object;

[0035] The parameter values ​​of the first parameter and the second parameter are taken as the first processing data.

[0036] To achieve the above objectives, a second aspect of this application provides a log generation apparatus, the apparatus comprising:

[0037] The acquisition module is used to acquire log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing.

[0038] The first construction module is used to obtain log data items from the format information and construct a format template for the log data items;

[0039] The second construction module is used to construct an interceptor based on the log configuration information;

[0040] The reading module is used to intercept the monitored object using the interceptor and read the first processing data of the monitored object according to the log data item;

[0041] The correction module is used to identify the object to be corrected from the first processed data according to the correction information, and to perform correction processing on the object to be corrected to obtain second processed data, wherein the second processed data includes the first target data corresponding to the log data item;

[0042] The import module is used to import the first target data into the format template to generate the target log file.

[0043] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0044] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0045] The log generation method, apparatus, electronic device, and storage medium proposed in this application construct an interceptor by acquiring log configuration information and then use the interceptor to intercept monitored objects. This allows for the identification of objects to be corrected within the first processed data of the monitored objects, enabling correction of log generation content that does not meet requirements and reducing data instability caused by excessive log volume. Furthermore, by obtaining log data items from format information, constructing format templates for these log data items, and then importing the first target data of the log data items into the appropriate format template to generate the target log file, the application also achieves log data formatting, facilitating accurate subsequent location of log content and improving log generation quality. Attached Figure Description

[0046] Figure 1 This is a schematic flowchart of a log generation method provided in an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of a process for correcting an object to be corrected in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of a process for updating matching information in an embodiment of this application;

[0049] Figure 4 yes Figure 1 A specific flowchart of step S140;

[0050] Figure 5 A schematic diagram of the log generation device provided in this application embodiment;

[0051] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] First, let's analyze some of the terms used in this application:

[0056] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0057] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0058] Information extraction is a text processing technique that extracts factual information such as entities, relationships, and events from natural language text and outputs it as structured data. Information extraction is a technique for extracting specific information from text data. Text data is composed of specific units, such as sentences, paragraphs, and chapters. Text information is composed of smaller, specific units, such as characters, words, phrases, sentences, paragraphs, or combinations of these units. Extracting noun phrases, names of people, and place names from text data is an example of information extraction. Of course, text information extraction techniques can extract information of various types.

[0059] System log printing is a common method to assist in diagnosing whether a system is functioning properly, facilitating quick location and analysis of system problems by relevant personnel. Currently, system log printing typically requires recording large amounts of processed data into a database. In practice, this method has been found to be labor-intensive and prone to issues such as missing or incorrect keywords, thus affecting the accuracy of subsequent log-based problem identification. Therefore, improving the accuracy of log-based problem identification has become a pressing technical issue that needs to be addressed.

[0060] Based on this, embodiments of this application provide a log generation method and apparatus, electronic device, and storage medium, aiming to improve the accuracy of log location.

[0061] The log generation method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the log generation method in this application is described.

[0062] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0063] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0064] The log generation method provided in this application relates to the field of artificial intelligence technology. The log generation method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the log generation method, but is not limited to the above forms. The following description uses a terminal as an example.

[0065] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0066] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0067] Figure 1 This is a flowchart illustrating a log generation method provided in an embodiment of this application. Figure 1 The method may include, but is not limited to, steps S110 to S160.

[0068] Step S110: Obtain log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing.

[0069] In this embodiment, log configuration information can be used to configure information such as the log file generation process, field names printed in the log file, object attributes, or log format. It is understood that the log configuration information can be adjusted according to actual needs and does not constitute a specific limitation. The information storage format of the log configuration information includes, but is not limited to, key-value pairs, dictionaries, or tuples. For example, a "data item: value" storage format can be used for quick location. Specifically, the format information is used to determine the log data item to be printed and the corresponding format template for the log data item.

[0070] In practical applications, the log generation method in this application embodiment can be implemented in the application environment of existing log components. The log components can be logback, log4j, slf4j, etc., and there are no specific limitations on them.

[0071] Step S120: Obtain log data items from the format information and construct a format template for the log data items.

[0072] In this embodiment, log data items are used to specify data items recorded in the log file. By constructing corresponding format templates for log data items, the correctness and uniqueness of log records are ensured, which facilitates accurate location of log data items in the log file and avoids problems such as missing or duplicate keywords when directly printing data. It also enables quick querying of the data content corresponding to log data items. A log data item may include one or more data items, without specific limitations.

[0073] For example, if the log data items include a function class name, an execution method name, and a parameter name, the constructed format template can be: "Enter——{A}.{B}——parameter{C}={IN}", where A is used to import the data Da corresponding to the function class name, B is used to import the data Db corresponding to the execution method name, C is used to import the data Dc corresponding to the parameter name, and IN is used to import the parameter value Din corresponding to parameter Dc. In other words, entering at least one of the keywords "Enter", "Da", "Db", "parameter Dc", and "Din" will quickly retrieve the corresponding log records.

[0074] Step S130: Build the interceptor based on the log configuration information.

[0075] Interceptors are used to intercept requests and execute predefined processing methods, such as performing permission verification, logging request information, and determining whether a user is logged in. In this embodiment, the interceptor is used to intercept requests from the monitored object and execute the log generation method. The interceptor can be a HandlerInterceptor or MethodInterceptor from Spring MVC, without specific limitations. The number of interceptors can be one or more, also without limitation. Optionally, by constructing an interceptor chain (i.e., at least two interceptors linked in the defined order), when the corresponding monitored object is accessed, the terminal can execute each interceptor in the defined order.

[0076] Step S140: Intercept the monitored object using an interceptor and read the first processing data of the monitored object based on the log data items.

[0077] In this embodiment, the monitored object can be any object that can be intercepted by an interceptor, including controller methods, service methods, or DAO methods in the Spring AOP framework, without specific limitations. The first processed data includes the initial target data corresponding to the log data items.

[0078] Step S150: Based on the correction information, identify the object to be corrected from the first processing data, and perform correction processing on the object to be corrected to obtain the second processing data, which includes the first target data corresponding to the log data item.

[0079] In the embodiments of this application, the object to be corrected can be adjusted according to actual needs, and the object to be corrected can at least include the object to be removed (such as the data item to be removed) and the object to be desensitized (such as the data item to be desensitized), etc., without specific limitations.

[0080] In some optional implementations, the terminal uses the Logback component. By inheriting the LOGBACKMessageConverter class and defining a custom convert method, a converter can be built. This converter can then identify and correct the object to be corrected from the first processed data to obtain the second processed data. More specifically, the converter can be used to load the correction information first, thereby updating the correction information, before executing the correction processing flow in step S150 above.

[0081] In practical applications, by adding a converter to the configuration file of the log component, the log generation strategy can be flexibly adjusted. Therefore, after the terminal collects and processes data through the log component, it will first use the converter to correct the collected and processed data, and then write the corrected processed data into the log file.

[0082] Step S160: Import the first target data into the format template to generate the target log file.

[0083] In this embodiment, the target log file may include one or more log records, without limitation. Specifically, in step S160, the first target data can be imported into the format template to obtain the log record corresponding to the monitored object. Then, in one approach, the log record can be written to a separately constructed log file to obtain the target log file; in another approach, the log record can also be written to a historical log file to obtain the target log file, and the historical log file can be the latest log file collected by accessing the local log storage path.

[0084] For example, combining the above format template "Enter——{A}.{B}——parameter{C}={IN}", assuming that in the first target data, Da=UserInfoController, Db=getPersonInfo, parameter Dc=[commonReqDTO], Din=[CommonReqDTO(appId=elife-cmall-mkt, uuid=100)], then after importing the first target data into the format template, the log record corresponding to the monitored object is obtained, that is:

[0085] Enter——UserInfoController.getPersonInfo——Parameter [commonReqDTO] = [CommonReqDTO (appId=elife-cmall-mkt, uuid=100)].

[0086] As can be seen, the log generation method provided in this application constructs an interceptor by obtaining log configuration information and uses the interceptor to intercept monitored objects. This allows for the identification of objects to be corrected within the first processed data of the monitored objects, enabling correction of log generation content that does not meet requirements and reducing data instability caused by excessive log volume. Furthermore, by obtaining log data items from format information, constructing format templates for the log data items, and then importing the first target data of the log data items into a suitable format template to generate a target log file, the method also achieves formatted processing of log data, facilitating accurate subsequent location of log content and improving log generation quality.

[0087] In step S150 of some embodiments, the object to be corrected includes an object to be removed. The object to be removed can represent data items that do not need to be printed to the log file, such as parameters unrelated to business requirements. Based on this, the object to be corrected is processed to obtain second processed data. Specifically, this can be done by removing the object to be removed from the first processed data to obtain the second processed data. It is evident that by removing the object to be removed from the first processed data, the log file can be reduced in size, thus reducing the workload of log analysis.

[0088] In step S140 of some embodiments, the log configuration information also includes print-free information. Based on this, the first processing data of the monitored object is read according to the log data items, including but not limited to the following steps:

[0089] First, retrieve the object configuration items and their corresponding preset object information from the non-printing information. The preset object information represents the value information corresponding to the object configuration items, used to filter objects that do not require reading or processing data and do not need to be written to the log file. Examples include relatively stable interfaces with low error rates, or interfaces whose input / output parameters are unrelated to business requirements. Object configuration items can be data items related to the object attributes of a specific object (such as class name or method name), but unrelated to the processing data of that specific object (such as input or output parameters when executing a method).

[0090] Next, based on the object configuration items, the target object information of the monitored object is read. This target object information uniquely identifies the monitored object and represents the corresponding value of the object configuration item within the monitored object.

[0091] Finally, if the target object information is not included in the preset object information, it means that the monitored object belongs to the object written to the log file. Then, the first processing data of the monitored object is read according to the log data item.

[0092] Correspondingly, if the preset object information includes the target object information, the monitoring object will continue to run, and therefore the processing data of the monitoring object will no longer be read or written to the log file. For example, if the preset object information includes the UserRecognizeController class, the terminal will not write log records related to the UserRecognizeController class to the target log file.

[0093] It is evident that by pre-filtering monitoring objects that do not need to be written to log files, the amount of log data can be further reduced, as well as repetitive and tedious data reading and processing procedures can be decreased.

[0094] It is understandable that in some optional implementations, the terminal may first detect whether the preset object information includes the target object information. If the preset object information does not include the target object information, the terminal may read the first processing data of the monitored object according to the log data item, and then remove the object to be removed from the first processing data to obtain the second processing data, thereby achieving two-level data filtering.

[0095] In step S150 of some embodiments, the object to be corrected includes a desensitized object. A desensitized object refers to the object whose information needs to be desensitized. Information desensitization refers to transforming certain sensitive information using desensitization rules to achieve reliable protection of sensitive privacy data. The desensitized object may include personal information such as ID card number, mobile phone number, card number, and customer number, or other information types set according to actual needs, without specific limitations.

[0096] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating a process for correcting an object to be corrected, as described in an embodiment of this application. Figure 2 As shown, the object to be corrected is subjected to correction processing to obtain second processing data, including but not limited to the following steps S210 to S220.

[0097] Step S210: Obtain the desensitization rules corresponding to the desensitized object from the correction information.

[0098] Specifically, the correction information can include at least matching information and de-identification rules. Matching information is used to identify the de-identified object, and this information includes, but is not limited to, regular expressions, or combinations of regular expressions and matching keywords. By adding matching keywords, any object that meets the same matching conditions as the de-identified object but is not substantially the same as the de-identified object can be further excluded, thus avoiding erroneous de-identification. For example, the regular expressions ([0-9]{1})([0-9]{4,})([0-9]{2}) can match a string of numbers, but cannot distinguish whether this string of numbers is a mobile phone number or some kind of identification number. Furthermore, adding matching keywords can reduce the performance overhead of performing full-text matching of log content using pure wildcards.

[0099] To give a further example, assuming the anonymized object is a mobile phone number, parentheses can be used as delimiters for different fields in the matching information, and the corresponding matching information can include the following six fields:

[0100] ([mM]obile|[pP]hone|[pP]honeNum|[pP]honeNumber)(\":\"|=|]\\s=\\s\\[)([^0-9]{0,})([0-9]{1})([0-9]{4,})([0-9]{2}).

[0101] In this context, ([0-9]{1})([0-9]{4,})([0-9]{2}) is a regular expression, and [mM]obile|[pP]hone|[pP]honeNum|[pP]honeNumber)(\":\"|=|]\\s=\\s\\[)([^0-9]{0,}) is a matching keyword. Based on this, for string A "mobile=19999999999", the terminal can determine that string A satisfies both the matching keyword and the regular expression, and therefore selects string A as the target for de-identification. However, for string B "IDcard=20000000000", the terminal can determine that string B satisfies the regular expression but not the matching keyword, and therefore will not select string B as the target for de-identification.

[0102] Step S220: According to the desensitization rules, the desensitized objects are desensitized in the first processed data to obtain the second processed data.

[0103] In this embodiment, the de-identification rule may include first data to be replaced in the matching information and second data for replacing the first field. For example, for the matching information in the following example, a corresponding de-identification rule can be set: $1$2$3$4******$6. This means that the data corresponding to the fifth field in the matching information is used as the first data, and the first data is replaced with "******". For instance, if the de-identified object identified from the first processed data is "mobile=19999999999", then the de-identified object is de-identified to obtain the second processed data, namely "mobile=1******99".

[0104] As can be seen, through the data desensitization processing of steps S210 to S220 above, privacy fields can be located and desensitized by flexibly configuring correction information. This can reduce the workload of developing desensitization programs for privacy information, improve development efficiency, and facilitate maintenance.

[0105] Furthermore, in some alternative implementations, please refer to Figure 3 , Figure 3 This is a schematic diagram of a process for updating matching information in an embodiment of this application. For example... Figure 3 As shown, the terminal can also execute steps S310 to S340. Step S310: Collect log files generated within a specified time period as reference log files.

[0106] In this embodiment, the specified time period can be set and adjusted manually, such as the last 7 days, the last two weeks, or the last three weeks, without specific limitations. Specifically, the reference log file can be obtained by accessing the local log storage path.

[0107] Step S320: Obtain matching information from the correction information.

[0108] Step S330: Perform data matching processing in the reference log file based on the matching information to obtain log data related to the matching information, and extract reference keywords from the log data.

[0109] In this embodiment of the application, the reference keywords can optionally be obtained by extracting information from the log data.

[0110] Step S340: If the matching information does not include the reference keyword, then add the reference keyword to the matching information to obtain new matching information.

[0111] For example, suppose the matching information is:

[0112] ([mM]obile|[pP]hone|[pP]honeNum|[pP]honeNumber)(\":\"|=|]\\s=\\s\\[)([^0-9]{0,})([0-9]{1})([0-9]{4,})([0-9]{2}).

[0113] Using the matching information, if the log data identified from the reference log file includes "name1=19999999999" and "name2":"19999999999", then the reference keywords "name1" and "name2" can be obtained from the log data. Since the above matching information does not include the reference keywords "name1" and "name2", they can be added to the matching information to obtain new matching information, i.e.:

[0114] ([mM]obile|[pP]hone|[pP]honeNum|[pP]honeNumber|name1|name2)(\":\"|=|]\\s=\\s\\[)([^0-9]{0,})([0-9]{1})([0-9]{4,})([0-9]{2}).

[0115] As can be seen, by implementing the above steps S310 to S340, the de-identification configuration can be updated through data analysis, and the accuracy of matching information can be continuously improved, thereby improving the accuracy of de-identification object identification.

[0116] In some optional implementations, the matching information includes matching keywords, regular expressions, and association information between the matching keywords and the regular expressions. The association information can be characters or symbols used to associate the matching keywords with the regular expressions, such as the symbol "&", and is not limited thereto. Therefore, step S340 specifically includes, but is not limited to, the following steps: performing data matching processing in the reference log file based on the regular expressions and association information to obtain log data related to the matching information; and retrieving keyword information other than the regular expressions and association information from the log data as reference keywords.

[0117] In other words, combining regular expressions with related information can further limit the structural relationship between regular expressions and reference keywords. This not only allows for flexible matching of new reference keywords but also ensures that the new reference keywords meet the actual matching requirements, thereby improving the accuracy of keyword updates.

[0118] Specifically, in step S340, if the matching information does not include the reference keyword, as an optional implementation, the terminal can output the reference keyword, for example, display the reference keyword in the terminal's log management interface, and then add the reference keyword to the matching information when a confirmation instruction for the reference keyword is received. The confirmation instruction can be generated by detecting configuration operations, including but not limited to clicking the confirmation control corresponding to the reference keyword in the log management interface.

[0119] In other optional implementations, the terminal can also obtain the evaluation metrics corresponding to the reference keywords. If the evaluation metrics meet specified conditions, the reference keywords are added to the matching information. These evaluation metrics may include read counts, importance, or stability. Importance can be a manually recorded score for the reference keywords, and stability can be calculated using a specified formula (e.g., stability = 1 - number of times the reference keyword appears in bug records / number of reads of the reference keyword), without specific limitations. Correspondingly, the specified conditions can be adjusted according to the type of evaluation metric, such as read counts exceeding a specified number, or importance or stability exceeding a specified value.

[0120] As can be seen, by adopting the above optional implementation methods, and by updating the matching information of reference keywords that meet the review requirements through manual review or evaluation index review, the reliability of reference keywords can be improved.

[0121] In step S140 of some embodiments, please refer to Figure 4 , Figure 4 yes Figure 1 A schematic diagram of a specific process for step S140. For example... Figure 4 As shown, step S140 specifically includes, but is not limited to, the following steps S141 to S145.

[0122] Step S141: Before running the monitored object, use an interceptor to intercept the monitored object and read the input parameters of the monitored object.

[0123] Step S142: If the first parameter included in the log data item is identified from the input parameters, then the parameter value of the first parameter is read from the monitoring object.

[0124] Step S143: After running the monitored object, use an interceptor to intercept and process the monitored object, and read the output parameters of the monitored object.

[0125] Step S144: If the second parameter included in the log data item is identified from the output parameters, then the parameter value of the second parameter is read from the monitoring object.

[0126] Step S145: Take the parameter values ​​of the first parameter and the second parameter as the first processing data.

[0127] As can be seen, by using steps S141 to S145 to intercept the monitored objects before and after operation, the input and output data of the monitored objects can be collected. This allows for more comprehensive coverage of log records based on different operating nodes, resulting in better data comparison, analysis, and tracing.

[0128] Optionally, in some implementations, if the data length corresponding to the value of the first parameter is greater than a preset first data length, the value of the first parameter can be truncated to obtain a first parameter value whose corresponding data length is less than or equal to the first data length. Then, the first parameter value is added to the first processed data, thus avoiding overflow errors due to excessive data length. Similarly, if the data length corresponding to the value of the second parameter is greater than a preset second data length, the value of the second parameter can also be truncated to obtain a second parameter value whose corresponding data length is less than or equal to the first data length. Then, the second parameter value is added to the first processed data. Both the first and second data lengths can be set and adjusted according to requirements.

[0129] In other implementations, if the data length corresponding to the first parameter's value is greater than the first data length, the first parameter's value can be first stored in an extended file located at a specified storage path, and a first index can be added to the first parameter. This first index is used to guide the user to the extended file. Then, the first parameter's value is truncated to obtain the final value. Based on this, in practical applications, if the first parameter's value is read from the target log file, the first index can be called to read the extended file and match the first parameter's value from it. Therefore, this avoids burdening the target log file with excessive data volume and allows for the complete reading of the parameter value, offering greater flexibility. Similarly, if the data length corresponding to the second parameter's value is greater than the second data length, the second parameter's value can also be first stored in an extended file located at a specified storage path, and a second index can be added to the second parameter. This second index is used to guide the user to the extended file, and then the second parameter's value is truncated to obtain the final value.

[0130] Please see Figure 5 This application also provides a log generation apparatus that can implement the above-described log generation method. The apparatus includes an acquisition module 510, a first construction module 520, a second construction module 530, a reading module 540, a correction module 550, and an import module 560, wherein:

[0131] The acquisition module 510 is used to acquire log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing.

[0132] The first construction module 520 is used to obtain log data items from format information and construct format templates for log data items;

[0133] The second building module 530 is used to build an interceptor based on the log configuration information;

[0134] The reading module 540 is used to intercept the monitored object using an interceptor and read the first processing data of the monitored object based on the log data items;

[0135] The correction module 550 is used to identify the object to be corrected from the first processed data according to the correction information, and to perform correction processing on the object to be corrected to obtain the second processed data, which includes the first target data corresponding to the log data item.

[0136] Import module 560 is used to import the first target data into the format template to generate the target log file.

[0137] The specific implementation of this log generation device is basically the same as the specific embodiment of the log generation method described above, and will not be repeated here.

[0138] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described log generation method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0139] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0140] The processor 601 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0141] The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the log generation method of the embodiments of this application.

[0142] The input / output interface 603 is used to implement information input and output;

[0143] The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0144] Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0145] The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0146] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described log generation method.

[0147] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0148] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0149] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0150] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0152] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0153] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0155] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0157] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0158] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A log generation method, characterized in that, The method includes: Obtain log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing. Obtain log data items from the format information and construct a format template for the log data items; Based on the log configuration information, construct the interceptor; The interceptor is used to intercept the monitored object, and the first processing data of the monitored object is read according to the log data item; Based on the correction information, the object to be corrected is identified from the first processed data, and the object to be corrected is corrected to obtain second processed data, the second processed data including the first target data corresponding to the log data item; Import the first target data into the format template to generate the target log file; The step of intercepting the monitored object using the interceptor and reading the first processing data of the monitored object based on the log data items includes: Before running the monitored object, the interceptor is used to intercept the monitored object and read the input parameters of the monitored object; If a first parameter included in the log data item is identified from the input parameters, then the parameter value of the first parameter is read from the monitored object; After running the monitored object, the interceptor is used to intercept the monitored object and read the output parameters of the monitored object; If a second parameter is identified from the output parameters, the parameter value of the second parameter is read from the monitored object; The parameter values ​​of the first parameter and the second parameter are taken as the first processing data.

2. The method according to claim 1, characterized in that, The object to be corrected includes an object to be removed; the correction process on the object to be corrected to obtain second processing data includes: The object to be removed is removed from the first processed data to obtain the second processed data.

3. The method according to claim 1, characterized in that, The log configuration information also includes print-free information; the step of reading the first processing data of the monitored object based on the log data items includes: Obtain the object configuration items and the preset object information corresponding to the object configuration items from the print-free information; Based on the object configuration item, read the target object information of the monitored object, whereby the target object information is used to uniquely identify the monitored object; If the target object information is not included in the preset object information, then the first processing data of the monitored object is read according to the log data item; The method further includes: If the preset object information includes the target object information, then the monitoring object continues to run.

4. The method according to claim 1, characterized in that, The object to be corrected includes de-identified objects; the correction process on the object to be corrected to obtain second processed data includes: Obtain the desensitization rule corresponding to the desensitized object from the correction information; According to the desensitization rules, the desensitized object is desensitized in the first processed data to obtain the second processed data.

5. The method according to claim 4, characterized in that, The method further includes: Collect log files generated within a specified time period as reference log files; Matching information is obtained from the correction information, and the matching information is used to determine the de-identified object; Based on the matching information, data matching processing is performed in the reference log file to obtain log data related to the matching information, and reference keywords are obtained from the log data; If the matching information does not include the reference keyword, then the reference keyword is added to the matching information to obtain new matching information.

6. The method according to claim 5, characterized in that, The matching information includes matching keywords, regular expressions, and association information between the matching keywords and the regular expressions; the step of performing data matching processing in the reference log file based on the matching information to obtain log data related to the matching information, and obtaining reference keywords from the log data, includes: Based on the regular expression and the association information, data matching processing is performed in the reference log file to obtain log data related to the matching information; Keyword information other than the regular expression and the association information is obtained from the log data and used as reference keywords.

7. A log generation device, characterized in that, The device includes: The acquisition module is used to acquire log configuration information, which includes format information and correction information. The correction information is used to determine the object to be corrected for correction processing. The first construction module is used to obtain log data items from the format information and construct a format template for the log data items; The second construction module is used to construct an interceptor based on the log configuration information; The reading module is used to intercept the monitored object using the interceptor and read the first processing data of the monitored object according to the log data item; The correction module is used to identify the object to be corrected from the first processed data according to the correction information, and to perform correction processing on the object to be corrected to obtain second processed data, wherein the second processed data includes the first target data corresponding to the log data item; The import module is used to import the first target data into the format template to generate a target log file; The reading module is configured to intercept the monitored object using the interceptor and read the first processing data of the monitored object based on the log data items, including: Before running the monitored object, the interceptor is used to intercept the monitored object and read the input parameters of the monitored object; If a first parameter included in the log data item is identified from the input parameters, then the parameter value of the first parameter is read from the monitored object; After running the monitored object, the interceptor is used to intercept the monitored object and read the output parameters of the monitored object; If a second parameter is identified from the output parameters, the parameter value of the second parameter is read from the monitored object; The parameter values ​​of the first parameter and the second parameter are taken as the first processing data.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the log generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the log generation method according to any one of claims 1 to 6.

Images