Training method and device of log detection model, equipment and medium

Abstract

The invention discloses a log detection model training method and device, equipment and a medium. The method comprises the following steps: acquiring collected client log data; according to the collected client log data and local training conditions, training to obtain a first log detection model; sending the first log detection model to at least one client to enable each client to train the first log detection model again; fusing the received local model parameter sent by each client and the model parameter of the first log detection model to obtain a fused parameter, and updating the first log detection model to obtain a second log detection model; and determining a target log model according to the second log detection model and the global training condition. According to the embodiment of the invention, the recognition accuracy of the log detection model can be improved.

Description

Technical Field

[0001] This invention relates to the field of data testing technology, and in particular to a training method, apparatus, device, and medium for a log detection model. Background Technology

[0002] With the rapid development of cloud computing technology, more and more application software is being migrated and deployed on cloud servers and clients. If the application software deployed on cloud servers and clients exhibits abnormal behavior, it will threaten the stable operation of the application software. Therefore, it is necessary to detect the logs generated by the application software deployed on cloud servers and clients.

[0003] Currently, existing log detection methods mainly use traditional machine learning models to identify log data and determine abnormal data in the logs.

[0004] However, different models need to be trained for different clients, resulting in low model generalization ability. Summary of the Invention

[0005] This invention provides a training method, apparatus, device, and medium for a log detection model to improve the accuracy of log detection model training.

[0006] In a first aspect, embodiments of the present invention provide a method for training a log detection model, the method comprising:

[0007] Retrieve collected client log data;

[0008] Based on the collected client log data and local training conditions, the first log detection model is trained.

[0009] The first log detection model is sent to at least one client so that each client can retrain the first log detection model.

[0010] The local model parameters sent by each client and the model parameters of the first log detection model are fused to obtain fused parameters, and the first log detection model is updated to obtain the second log detection model.

[0011] The target log model is determined based on the second log detection model and global training conditions.

[0012] Secondly, embodiments of the present invention also provide a training apparatus for a log detection model, the apparatus comprising:

[0013] The log data acquisition module is used to acquire collected client log data;

[0014] The first model acquisition module is used to train the first log detection model based on the collected client log data and local training conditions.

[0015] The model retraining module is used to send the first log detection model to at least one client so that each client can retrain the first log detection model.

[0016] The model update module is used to fuse the local model parameters sent by each client and the model parameters of the first log detection model to obtain the fused parameters, and then update the first log detection model to obtain the second log detection model.

[0017] The target model determination module is used to determine the target log model based on the second log detection model and global training conditions.

[0018] Thirdly, embodiments of the present invention also provide a training device for a log detection model, the training device for the log detection model comprising:

[0019] At least one processor; and

[0020] A memory that is communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the training method of the log detection model of any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute a training method for a log detection model according to any embodiment of the present invention.

[0023] The technical solution of this invention allows the log data of each client to be stored on the local device. Only the parameters of the client's model need to be uploaded to the server, avoiding the transmission of sensitive data, preventing data leakage, reducing the amount of data to be transmitted, and improving the model training efficiency. Furthermore, the client and server train the model alternately, so that the model can recognize the log data corresponding to each client and server, improving the generalization ability of the model and the training accuracy of the log detection model.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a training method for a log detection model according to Embodiment 1 of the present invention;

[0027] Figure 2 This is a flowchart of a training method for a log detection model according to Embodiment 2 of the present invention;

[0028] Figure 3 This is a structural diagram of a training device for a log detection model according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of a training device for implementing a log detection model provided in an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] The acquisition, storage, and application of client log data, etc., involved in the technical solutions of this invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a training method for a log detection model according to Embodiment 1 of the present invention. This embodiment of the invention is applicable to the training of log detection models. The method can be executed by a log detection model training device, which can be implemented in hardware and / or software.

[0035] See Figure 1 The training method for the log detection model shown includes:

[0036] S101. Obtain the collected client log data.

[0037] Among them, client log data can be a collection of feature data extracted from logs on the client.

[0038] Specifically, client logs refer to the record files generated by client devices (such as mobile phones or computers) during runtime. They contain information about application operation, user actions, and system status. Client logs have various types. For example, operation logs mainly record application operation information, including debugging information, error messages, reasons for application crashes, response time, and resources used. Service logs mainly contain client behavior information, such as the time, frequency, preferences, or favorites of using a certain service, and may also include basic information such as client terminal type, operating system model, and version. There is at least one client and one server, with each client communicating with the server. The server can send log retrieval requests to each client. After receiving the request, each client uploads its local non-sensitive logs to the server. The server extracts features from the non-sensitive logs of each client to determine the client's log data. Sensitive data from each client does not need to be uploaded to the server.

[0039] S102. Based on the collected client log data and local training conditions, train the first log detection model.

[0040] The local training conditions can be the pre-set conditions for the model training to end on the server. The first log detection model can be the model trained on the server after meeting the local training conditions.

[0041] Specifically, the server has a local model to be trained. The server can train the model based on the collected client log data, adjust the model parameters corresponding to the model to be trained, so that the model output meets the local training conditions of the server, determine the first log detection model, and realize the initialization processing of each model parameter of the model to be trained. The local training conditions may be reaching a preset number of model training times or reaching a preset training accuracy threshold, etc., and this embodiment of the invention does not limit them.

[0042] S103. Send the first log detection model to at least one client so that each client can retrain the first log detection model.

[0043] Specifically, the first log detection model is sent to at least one client. After receiving the first log detection model, each client retrains the first log detection model using its local sensitive data until the client's preset training termination condition is met. The client's training termination condition can be reaching a preset number of training iterations or reaching a preset training accuracy. The retrained first log detection model for each client is then determined.

[0044] S104. The local model parameters sent by each client and the model parameters of the first log detection model are fused to obtain fused parameters, and the first log detection model is updated to obtain the second log detection model.

[0045] The local model parameters can be the model parameters after the client retrains the first log detection model.

[0046] Specifically, after each client retrains the first log detection model, it adjusts the training parameter values ​​corresponding to at least one training parameter to determine the retrained first log detection model. For each client, the parameter values ​​corresponding to the parameters adjusted during training in the retrained first log detection model are determined as the client's local model parameters. For example, parameter terms can be the first-order momentum coefficient, second-order momentum coefficient, neuron strength, or bias value of the optimizer. Each client sends its corresponding local model parameters to the server. The server fuses the received local model parameters from each client with the model parameters of its local first log detection model to obtain fused parameters. The server updates the model parameters in its local first log detection model according to the fused parameters to determine the second log detection model.

[0047] S105. Determine the target log model based on the second log detection model and the global training conditions.

[0048] The global training conditions can be preset training termination conditions for the second log detection model. The target log model can be the model obtained after training the second log detection model.

[0049] Specifically, the server trains the second log detection model based on the client log data corresponding to each client stored locally until the global training conditions are met, and the target log model is determined. The global training termination conditions may be that the accuracy error of the model in multiple consecutive training rounds is within a preset range, the training time reaches a preset threshold, or the number of training rounds reaches a preset threshold, etc. This embodiment of the invention does not impose any restrictions on these conditions.

[0050] The technical solution of this invention allows the log data of each client to be stored on the local device. Only the parameters of the client's model need to be uploaded to the server, avoiding the transmission of sensitive data, preventing data leakage, reducing the amount of data to be transmitted, and improving the model training efficiency. Furthermore, the client and server train the model alternately, so that the model can recognize the log data corresponding to each client and server, improving the generalization ability of the model and the training accuracy of the log detection model.

[0051] Example 2

[0052] Figure 2 This is a flowchart illustrating a training method for a log detection model according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the training operation of the log detection model.

[0053] Furthermore, the step of "determining the target log model based on the second log detection model and global training conditions" is refined to "when the second log detection model does not meet the global training conditions, the second log detection model is sent to each client so that each client can retrain the second log detection model; the local model parameters and the model parameters of the second log detection model sent by each client are fused to obtain fused parameters, and the second log detection model is updated; when the second log detection model meets the global training conditions, the current second log detection model is determined as the target log model," in order to improve the operation of training the log detection model.

[0054] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments.

[0055] See Figure 2 The training method for the log detection model shown includes:

[0056] S201. Obtain the collected client log data.

[0057] S202. Based on the collected client log data and local training conditions, train the first log detection model.

[0058] S203. Send the first log detection model to at least one client so that each client can retrain the first log detection model.

[0059] S204. The local model parameters sent by each client and the model parameters of the first log detection model are fused to obtain fused parameters, and the first log detection model is updated to obtain the second log detection model.

[0060] S205. When the second log detection model does not meet the global training conditions, the second log detection model is sent to each client so that each client can retrain the second log detection model.

[0061] Specifically, when the second log detection model does not meet the global training conditions, it indicates that the second log detection model has not yet reached the training termination condition and needs to continue training. The server then sends the second log detection model to each client, allowing each client to retrain the second log detection model based on its corresponding sensitive data, until the preset training termination condition for each client is met, thus determining the trained second log detection model. Each client then sends its local model parameters for its corresponding second log detection model to the server.

[0062] S206. The local model parameters and the model parameters of the second log detection model sent by each client are fused to obtain fused parameters, and the second log detection model is updated.

[0063] Specifically, the server merges the parameter values ​​corresponding to the same parameter items in the local model parameters sent by each client and the model parameters of the second log detection model to obtain fused parameters. The server then updates the parameter values ​​of the same parameter items in the second log detection model based on the fused parameters, so as to complete model training and aggregation without leaking sensitive data from each client.

[0064] S207. When the second log detection model meets the global training conditions, the current second log detection model is determined as the target log model.

[0065] Specifically, when the second log detection model meets the global training conditions, it indicates that the second log model has reached the training end condition. The current second log detection model is determined as the target log model, and each target log model can be sent to each client so that each client can detect its local log data according to the target log model it receives. If an anomaly is detected in the client's local log data, an alarm is issued. The alarm method includes, but is not limited to, pop-up prompts, email prompts, web page prompts, information prompts, or ringing prompts. This embodiment of the invention does not limit this.

[0066] This invention refines the model training method and improves the accuracy of model training by executing different model training steps for different second log detection models to determine whether they meet the global training conditions.

[0067] Optionally, the received local model parameters sent by each client and the model parameters of the first log detection model are fused to obtain fused parameters, and the first log detection model is updated. This includes: obtaining at least one parameter to be processed; for each parameter to be processed, obtaining the client parameter value corresponding to the parameter to be processed in the local model parameters sent by each client, and obtaining the server parameter value corresponding to the parameter to be processed in the first log detection model; calculating the parameter mean based on each client parameter value and the server parameter value; and replacing the server parameter value corresponding to the parameter to be processed in the first log detection model based on the parameter mean, in order to update the first log detection model.

[0068] Here, the parameters to be processed can be parameter items in the model that have been changed through training. Client parameter values ​​can be the parameter values ​​corresponding to the parameters to be processed in the first log detection model corresponding to the client. Server parameter values ​​can be the parameter values ​​corresponding to the parameters to be processed in the first log detection model corresponding to the server. The parameter mean can be the average value calculated from each client parameter value and the server parameter value.

[0069] Specifically, at least one parameter to be processed is obtained, and this parameter corresponds to the model type of the model to be trained on the server. The parameter to be processed can be obtained by querying the mapping relationship between the model to be trained and the parameter. The query method can be a depth-first search algorithm or a breadth-first search algorithm, etc., and this embodiment of the invention does not limit this. For each parameter to be processed, the client parameter value corresponding to the parameter to be processed in the local model parameters sent by each client is obtained, as well as the server parameter value corresponding to the parameter to be processed in the first log detection model is obtained; the average value of each client parameter value and the server parameter value is calculated to determine the mean value of the parameter to be processed; the server parameter value corresponding to the parameter to be processed in the first log detection model on the server is replaced according to the mean value of the parameter to be processed, resulting in an updated first log detection model.

[0070] The average parameter value is calculated by comparing the parameter values ​​of each client with those of the server. The server parameter value corresponding to the parameter to be processed in the first log detection model is replaced according to the average parameter value to update the first log detection model. Each client can transmit only the model parameters to the server without transmitting sensitive data, which improves the security of client data.

[0071] Optionally, the model parameters corresponding to the target log model can be updated at the fully connected layer via the client.

[0072] Specifically, the server sends the target log detection model to each client. Upon receiving the model, each client classifies its network layers into fully connected layers and non-fully connected layers. Each client then trains the fully connected layers of the target log model using locally collected sensitive data, adjusting the parameter value of at least one parameter in the fully connected layers to obtain an updated target log model. Each client can then use the updated target log model to detect its real-time client logs and monitor its operational status.

[0073] By using the model parameters corresponding to the target log model, the client can train and update the target log model using fully connected layers, which can reduce the amount of computation during training and improve the training efficiency of the model.

[0074] Optionally, the collected client log data is obtained, including: for each client, obtaining the client logs and at least one log template corresponding to the client; matching the client logs with each log template to determine at least one log feature; and concatenating the log features to determine the client log data corresponding to the client.

[0075] Client-side logs can be log files generated during client runtime. Log templates can be log recording framework data containing a fixed format and key information placeholders.

[0076] Specifically, for each client, obtain the corresponding client logs and at least one log template; different types of logs correspond to different log templates. Match the client logs with each log template to determine at least one log feature; concatenate the log features to determine the corresponding client log data. For example, the client log is: July 29, 2025, 11:05:30, an error occurred in the database module, error code: DB001, the reason is that the database server IP: 192.168.1.100 is not responding, located at UserDAO.java: line 156, the user login function is temporarily unavailable; the log template is: [time][log level = ERROR][template] An error occurred in the [block]: [Error type] (Error code: [code]), Reason: [Specific reason], Location: [Code location], Impact: [Affected scope]; Log characteristics are: [2025-07-29 11:05:30][ERROR][Database module] An error occurred: [Connection timeout] (Error code: DB001), Reason: [Database server IP: 192.168.1.100 not responding], Location: [UserDAO.java: line 156], Impact: [User login function is temporarily unavailable].

[0077] By obtaining the client logs and at least one log template corresponding to the client, matching the client logs with each log template to determine at least one log feature, and concatenating the log features to determine the client log data corresponding to the client, the content structure of the logs is standardized, ensuring the integrity and consistency of log information and improving the accuracy of log analysis.

[0078] Optionally, the log template is obtained through the following steps: obtaining at least one client historical log for each client; for each client, performing word segmentation on the client historical logs to determine at least one log string; and performing tree structure clustering on the log strings corresponding to the client historical logs to determine at least one log template for the client.

[0079] The client's historical logs can be logs of client operation data within a historical time period. The log string can be a string obtained by segmenting the client's historical logs.

[0080] Specifically, obtain at least one historical log entry for each client; for each client, perform word segmentation on the historical log entries to determine at least one log string; perform tree-structure clustering on the log strings corresponding to the historical log entries of each client to determine at least one log template for the client. The tree-structure clustering method can be the Drain algorithm, which can construct the log strings corresponding to the historical log entries of each client using a prefix tree method. Each node of the tree stores a log string, and strings with similarity within a preset range are clustered. The at least one log template for the client is determined according to the order of the tree nodes.

[0081] By segmenting the historical logs of each client, at least one log string is determined. Then, tree-structured clustering is performed on each log string corresponding to the historical logs of each client to determine at least one log template for each client. By using deep tree and hierarchical matching, full string comparison is avoided, thus improving the efficiency of log template extraction.

[0082] Optional log templates include: text semantic feature templates, event sequence feature templates, event count feature templates, and time interval feature templates.

[0083] Specifically, the text semantic feature template can be a template from which semantic features can be extracted from log data. The event sequence feature template can be a template from which the temporal order features of events can be extracted from log data. The event count feature template can be a template from which the number of events can be extracted from log data. The time interval feature template can be a template from which the time interval features of events can be extracted from log data.

[0084] Specifically, log templates include: text semantic feature templates, event sequence feature templates, event counting feature templates, and time interval feature templates. Text semantic feature templates record the meaning of the log text, for example, the event type or cause of occurrence, used to record user actions and reflect the operational status. Logs are usually generated in chronological order; events within the same client form a continuous sequence. Event sequence feature templates record the order of events, used to analyze whether the process is normal (e.g., system startup steps or fault trigger chains). The number of times the same type of event occurs within a unit of time often reflects the operational status (e.g., frequent errors may be a precursor to a fault). Event counting feature templates count the number of times a specific event occurs within a fixed time window, which can be used to detect frequency anomalies. The time intervals of consecutively occurring similar or related events often follow a fixed pattern (e.g., a scheduled task executes every 10 minutes). Time interval feature templates record the time difference between events; detecting interval anomalies can be used to identify abnormal execution rhythms.

[0085] By extracting log features using text semantic feature templates, event sequence feature templates, event count feature templates, and time interval feature templates, and determining log data based on multidimensional data, the comprehensiveness of the data and the accuracy of model training are improved.

[0086] Example 3

[0087] Figure 3 This is a schematic diagram of a training device for a log detection model provided in Embodiment 3 of the present invention. This embodiment of the present invention is applicable to the training of log detection models. The device can execute the training method for the log detection model and can be implemented in hardware and / or software.

[0088] See Figure 3 The training device for the log detection model shown includes: a log data acquisition module 301, a first model acquisition module 302, a model retraining module 303, a model update module 304, and a target model determination module 305, wherein...

[0089] Log data acquisition module 301 is used to acquire collected client log data;

[0090] The first model acquisition module 302 is used to train the first log detection model based on the collected client log data and local training conditions.

[0091] The model retraining module 303 is used to send the first log detection model to at least one client so that each client can retrain the first log detection model.

[0092] The model update module 304 is used to fuse the local model parameters sent by each client and the model parameters of the first log detection model to obtain the fused parameters, and update the first log detection model to obtain the second log detection model.

[0093] The target model determination module 305 is used to determine the target log model based on the second log detection model and the global training conditions.

[0094] The technical solution of this invention allows the log data of each client to be stored on the local device. Only the parameters of the client's model need to be uploaded to the server, avoiding the transmission of sensitive data, preventing data leakage, reducing the amount of data to be transmitted, and improving the model training efficiency. Furthermore, the client and server train the model alternately, so that the model can recognize the log data corresponding to each client and server, improving the generalization ability of the model and the training accuracy of the log detection model.

[0095] Optionally, the target model determination module 305 is specifically used for:

[0096] When the second log detection model does not meet the global training conditions, the second log detection model is sent to each client so that each client can retrain the second log detection model.

[0097] The local model parameters and the model parameters of the second log detection model sent by each client are fused to obtain fused parameters, and the second log detection model is updated.

[0098] When the second log detection model meets the global training conditions, the current second log detection model is determined as the target log model.

[0099] Optional, model update module 304, specifically used for:

[0100] Obtain at least one parameter to be processed;

[0101] For each parameter to be processed, obtain the client parameter value corresponding to the parameter to be processed in the local model parameters sent by each client, and obtain the server parameter value corresponding to the parameter to be processed in the first log detection model.

[0102] Calculate the average parameter value based on the parameter values ​​of each client and the server.

[0103] The server parameter values ​​corresponding to the parameters to be processed in the first log detection model are replaced based on the average parameter value, which is used to update the first log detection model.

[0104] Optionally, the model parameters corresponding to the target log model can be updated by training the target log model with fully connected layers through the client.

[0105] Optionally, the log data acquisition module 301 is specifically used for:

[0106] For each client, obtain the client logs and at least one log template corresponding to that client;

[0107] Match the client logs with each log template to determine at least one log characteristic;

[0108] By concatenating the various log features, the corresponding client log data can be determined.

[0109] Optional, a log template can be obtained through the following steps:

[0110] Retrieve at least one historical log entry for each client;

[0111] For each client, the historical logs of each client are segmented into words to determine at least one log string;

[0112] Tree-structured clustering is performed on each log string corresponding to the historical logs of each client to determine at least one log template for each client.

[0113] Optional log templates include: text semantic feature templates, event sequence feature templates, event count feature templates, and time interval feature templates.

[0114] The training apparatus for the log detection model provided in this embodiment of the invention can execute the training method for the log detection model provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the training method for the log detection model.

[0115] Example 4

[0116] Figure 4 A schematic diagram of the structure of a training device 400 for implementing an embodiment of the present invention is shown.

[0117] like Figure 4As shown, the log detection model training device 400 includes at least one processor 401 and a memory, such as a read-only memory (ROM) 402 and a random access memory (RAM) 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the ROM 402 or loaded from storage unit 408 into the RAM 403. The RAM 403 can also store various programs and data required for the operation of the log detection model training device 400. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0118] Multiple components in the log detection model training device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless transceiver, etc. The communication unit 409 allows the log detection model training device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0119] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as the training methods for log detection models.

[0120] In some embodiments, the log detection model training method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on the log detection model training device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the log detection model training method described above can be performed. Alternatively, in other embodiments, processor 401 can be configured to execute the log detection model training method by any other suitable means (e.g., by means of firmware).

[0121] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0123] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] To provide user interaction, the systems and techniques described herein can be implemented on a training device for the log detection model, which includes: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the training device. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0126] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0127] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0128] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A training method for a log detection model, characterized in that, Applied to a server, the method includes: Retrieve collected client log data; Based on the collected client log data and local training conditions, a first log detection model is trained. The first log detection model is sent to at least one client so that each client can retrain the first log detection model. The local model parameters sent by each of the clients and the model parameters of the first log detection model are fused to obtain fused parameters, and the first log detection model is updated to obtain the second log detection model. The target log model is determined based on the second log detection model and the global training conditions.

2. The method according to claim 1, characterized in that, The step of determining the target log model based on the second log detection model and global training conditions includes: When the second log detection model does not meet the global training conditions, the second log detection model is sent to each of the clients so that each of the clients can retrain the second log detection model. The local model parameters sent by each of the clients and the model parameters of the second log detection model are fused to obtain fused parameters, and the second log detection model is updated. When the second log detection model meets the global training conditions, the current second log detection model is determined as the target log model.

3. The method according to claim 1, characterized in that, The step of fusing the received local model parameters sent by each of the clients and the model parameters of the first log detection model to obtain fused parameters, and then updating the first log detection model, includes: Obtain at least one parameter to be processed; For each parameter to be processed, obtain the client parameter value corresponding to the parameter to be processed in the local model parameters sent by each client, and obtain the server parameter value corresponding to the parameter to be processed in the first log detection model. Calculate the average parameter value based on the client parameter value and the server parameter value; The server parameter values ​​corresponding to the parameters to be processed in the first log detection model are replaced according to the average value of the parameters, in order to update the first log detection model.

4. The method according to claim 1, characterized in that, The model parameters corresponding to the target log model can be updated by training the target log model with a fully connected layer through the client.

5. The method according to claim 1, characterized in that, The acquisition of collected client log data includes: For each of the aforementioned clients, obtain the client logs and at least one log template corresponding to that client; The client logs are matched with each of the log templates to determine at least one log feature; By concatenating the log features, the client log data corresponding to the client is determined.

6. The method according to claim 1, characterized in that, The log template is obtained through the following steps: Obtain at least one client history log corresponding to each of the aforementioned clients; For each of the aforementioned clients, the historical logs of each client are segmented to determine at least one log string; Tree-structured clustering is performed on the log strings corresponding to the historical logs of each client to determine at least one log template corresponding to each client.

7. The method according to claim 6, characterized in that, The log template includes: a text semantic feature template, an event sequence feature template, an event count feature template, and a time interval feature template.

8. A training device for a log detection model, characterized in that, The device includes: The log data acquisition module is used to acquire collected client log data; The first model acquisition module is used to train a first log detection model based on the collected client log data and local training conditions. The model retraining module is used to send the first log detection model to at least one client so that each client can retrain the first log detection model. The model update module is used to fuse the local model parameters sent by each of the clients and the model parameters of the first log detection model to obtain fused parameters, and update the first log detection model to obtain the second log detection model. The target model determination module is used to determine the target log model based on the second log detection model and the global training conditions.

9. A training device for a log detection model, characterized in that, The training equipment for the log detection model includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the training method of the log detection model according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the training method of the log detection model according to any one of claims 1-7.

Images