Model training method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of machine learning, and discloses a model training method and device, computer equipment and a storage medium, which are applied to a central node deployed with a global model. The method comprises the following steps: acquiring N traffic data transmitted by at least one network device in a first time period; sending at least part of the N traffic data to at least one edge node deployed with a replica model; receiving sub-model parameters included in at least one first training result output by at least one edge node; performing aggregation operation on the at least one sub-model parameter to generate a first target parameter, and updating the global model to obtain a first model; iteratively training the first model between the center node and the at least one edge node until a target model is obtained through training; and detecting the target traffic data by using the target model. According to the invention, the problem of false alarm or missing alarm of some attack behaviors caused by low detection accuracy, high false alarm rate and poor efficiency of the abnormal traffic data can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a model training method, device, computer equipment and storage medium. Background Art

[0002] With the large-scale application and popularization of new scenarios such as cloud computing, big data, and the Internet of Things, the amount of network traffic data has grown exponentially. Faced with such huge data, ensuring the security of network traffic data has become a necessary issue. Therefore, anomaly detection of the huge network traffic data generated by node transmission is the core technology to ensure network security.

[0003] In the current network traffic data anomaly detection methods, abnormal traffic data can be identified or detected through statistical data, log analysis or rule-based methods. However, these methods need to process traffic data one by one, and it is difficult to process traffic data with complex nonlinear relationships, resulting in low detection accuracy of abnormal traffic data, high false alarm rate and poor efficiency, causing false alarms or omissions of certain attack behaviors. Summary of the invention

[0004] In view of this, the present invention provides a model training method, apparatus, computer equipment and storage medium to solve the problem of low detection accuracy, high false alarm rate and poor efficiency of abnormal traffic data, resulting in false alarm or omission of certain attack behaviors.

[0005] In a first aspect, the present invention provides a model training method, which is applied to a central node, and a global model is deployed on the central node; the method includes: obtaining N traffic data transmitted by at least one network device in a first time period, N≥1 and is a positive integer; sending at least part of the N traffic data to at least one edge node managed by the central node, each edge node is deployed with a replica model, and the replica model is a replica of the global model; receiving at least one first training result output by at least one edge node, each first training result is generated by each edge node training at least part of the received traffic data in the replica model, and each first training result includes a sub-model parameter; performing an aggregation operation on at least one sub-model parameter in at least one first training result to generate a first target parameter, and using the first target parameter to update the global model to obtain a first model; if the first model does not meet the first preset condition, iteratively training the first model between the central node and at least one edge node until the trained model meets the first preset condition, thereby obtaining a target model; using the target model to detect the target traffic data to obtain a detection result, and the detection result is used to indicate whether the target traffic data is abnormal or normal.

[0006] Based on the method of the first aspect above, the N acquired traffic data can be sent to at least one edge node managed by the central node, and after training through the replica model of each edge node, the sub-model parameters are output, and an aggregation operation of at least one sub-model parameter is performed to obtain a global model deployed by the central node for updating to obtain a first model; and the first model is iteratively trained between the central node and at least one edge node until a target model that meets the first preset condition is obtained through training, so that the target model can detect whether the target traffic data is abnormal.

[0007] Since the training of the target model in the embodiment of the present invention is performed by the central node distributing data to each edge node, compared with the method of directly training the target model at the central node, the excessive workload caused by large-scale traffic data training at the central node can be avoided; in addition, since the node performance and adaptation scenarios of each edge node are different, N traffic data can be trained in a diversified manner to obtain sub-model parameters obtained by training different edge nodes, thereby improving the accuracy of the target model in detecting abnormal traffic data and reducing the false alarm rate.

[0008] In an optional embodiment, the N traffic data include normal traffic data and abnormal traffic data; the abnormal traffic data include M types of abnormal traffic data groups, N≥M≥1 and is a positive integer; at least part of the N traffic data is sent to at least one edge node managed by the central node, including: obtaining the computing power of each edge node; calculating a first ratio between the computing power of each edge node and all the computing powers of at least one edge node; sending the normal traffic data of the first ratio to each edge node, and sending the abnormal traffic data in each type of abnormal traffic data group of the first ratio to each edge node.

[0009] In this implementation, the traffic data to be sent to each edge node can be determined based on the computing power of each edge node, so that the amount of traffic data received by each edge node matches the computing power of the edge node, thereby avoiding excessive workload caused by the edge node training too much traffic data, or avoiding waste of resources caused by the edge node training too little traffic data.

[0010] In an optional embodiment, an aggregation operation is performed on at least one sub-model parameter in at least one first training result to generate a first target parameter, including: obtaining a weight value of each edge node; and obtaining the first target parameter based on at least one weight value and at least one sub-model parameter.

[0011] In this implementation, since the weight value of each edge node is used to measure the contribution or credibility of each edge node participating in the training, the first target parameter obtained by weighted averaging at least one sub-model parameter based on the weight value of each edge node can take into account the contribution of different edge nodes and is more accurate.

[0012] In an optional implementation, obtaining a first target parameter according to at least one weight value and at least one sub-model parameter includes: obtaining the first target parameter according to at least one weight value and at least one sub-model parameter based on a first algorithm; the first algorithm is: Among them, w global is the first target parameter, w i is the sub-model parameter of the ith edge node, q i is the weight value of the i-th edge node.

[0013] In this implementation, the first target parameter can be accurately calculated to obtain the best target model, thereby improving the accuracy of the target model in detecting abnormal traffic data and reducing the false alarm rate.

[0014] In an optional embodiment, the first preset condition is that the accuracy rate of the model detecting whether N traffic data are abnormal is greater than or equal to a first threshold, and the false alarm rate of the model detecting that abnormal traffic data in N traffic data is normal traffic data is less than or equal to a second threshold.

[0015] In this implementation, the trained target model can detect abnormal traffic data with higher accuracy and lower false alarm rate.

[0016] In an optional embodiment, the method also includes: obtaining L newly added traffic data in a second time period, the L traffic data including a P-type abnormal traffic data group, and the second time period is a time period after the first time period; if the L traffic data are greater than or equal to a preset threshold, and P is 1, then training the target model based on the L traffic data to obtain a first target model; if the L traffic data are greater than or equal to a preset threshold, and P is greater than 1, then calculating a second ratio between the number of abnormal traffic data in the class with the least number of abnormal traffic data in the P-type abnormal traffic data group and L, and when the second ratio meets a second preset condition, training the target model based on the L traffic data to obtain a first target model.

[0017] In this embodiment, the target model can be automatically incrementally learned so that the target model can perform incremental learning while performing abnormal traffic data detection in real time, so that the target model can automatically adapt to the updated data, and achieve better real-time and generalization in the scenario where the traffic data is continuously updated by at least one network device, thereby ensuring the real-time and high availability of the target model.

[0018] In the second aspect, the present invention provides a model training device, a transceiver module, used to obtain N traffic data transmitted by at least one network device in a first time period, N ≥ 1 and is a positive integer; the transceiver module is also used to send at least part of the N traffic data to at least one edge node managed by a central node, each edge node is deployed with a replica model, and the replica model is a replica of the global model; the transceiver module is also used to receive at least one first training result output by at least one edge node, each first training result is generated by each edge node training at least part of the received traffic data in the replica model, and each first training result includes a sub-model parameter.

[0019] The processing module is used to perform an aggregation operation on at least one sub-model parameter in at least one first training result to generate a first target parameter, and use the first target parameter to update the global model to obtain a first model; the processing module is also used to iteratively train the first model between a central node and at least one edge node if the first model does not meet the first preset condition, until the trained model meets the first preset condition, thereby obtaining a target model; the processing module is also used to use the target model to detect target traffic data to obtain a detection result, and the detection result is used to indicate whether the target traffic data is abnormal or normal.

[0020] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the model training method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the model training method of the first aspect or any corresponding embodiment thereof.

[0022] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the model training method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1is a topological diagram of a model training system according to an embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of the structure of a global model according to an embodiment of the present invention;

[0026] Figure 3 is a flow chart of a model training method according to an embodiment of the present invention;

[0027] Figure 4 is a flowchart of another model training method according to an embodiment of the present invention;

[0028] Figure 5 is a structural block diagram of a model training device according to an embodiment of the present invention;

[0029] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] The embodiments of the present invention are applied to perform anomaly detection on network traffic in multi-node scenarios such as cloud computing, industrial Internet and Internet of Things. Traffic anomaly detection aims to identify abnormal patterns, abnormal behaviors or possible network threats in network traffic or data streams. Traffic anomaly detection is commonly used in network security, system monitoring and network management to help detect and respond to various network problems, including network attacks, failures, performance issues, etc.

[0032] As described in the background art, the related network traffic data anomaly detection methods may result in low detection accuracy, high false alarm rate and poor efficiency for abnormal traffic data, resulting in false alarms or omissions for certain attack behaviors.

[0033] In order to solve the above technical problems, an embodiment of the present invention provides a model training method, which is based on edge computing and federated learning to train an accurate target model, and can effectively process high-dimensional complex data. Therefore, it has strong adaptability to various types of traffic data, thereby achieving the effect of improving the accuracy of detecting abnormal traffic data and reducing the false alarm rate.

[0034] Among them, edge computing is a distributed computing paradigm that aims to push computing and data processing capabilities from traditional central nodes to edge nodes at or close to data generation sources, which can reduce the burden on central nodes and is often used in applications that require low latency and real-time performance.

[0035] Federated learning is a distributed machine learning (deep learning) method that aims to enable multiple participants (such as edge devices, clients, servers, etc.) to collaborate to train machine learning or deep learning models.

[0036] Below Figure 1 Taking the model training system 100 shown as an example, the method provided in the embodiment of the present application is described. Figure 1 It is only a schematic diagram and does not constitute a limitation on the applicable scenarios of the technical solution provided in this application.

[0037] like Figure 1 As shown, Figure 1 is a topological diagram of a model training system according to an embodiment of the present invention. Figure 1 In the figure, the model training system 100 may include a central node 101, a first edge node 102, a second edge node 103 and a network device 104.

[0038] The central node 101 in the embodiment of the present invention may be any device with computing and communication capabilities, such as a server, a server or a virtual machine, etc. The central node 101 manages the first edge node 102 and the second edge node 103 .

[0039] The central node 101 is deployed with a global model. Figure 2 As shown, Figure 2 is a schematic diagram of the structure of a global model according to an embodiment of the present invention. The global model may be a one-dimensional DenseNet model.

[0040] The optimizer of the global model may be an Adam optimizer. The central node 101 may update the parameters of the global model by using a stochastic gradient descent method.

[0041] The core advantage of the DenseNet model lies in its densely connected structure. The middle layer can receive all the feature information from the previous layer. Compared with other neural networks, it can better reuse the underlying features and reduce feature forgetting during the learning process. This structure is not only applicable to two-dimensional data such as images, but also to one-dimensional network traffic data.

[0042] The first edge node 102 or the second edge node 103 in the embodiment of the present application may be any device with computing and communication capabilities, such as a server, a server or a virtual machine, etc. The first edge node 102 or the second edge node 103 is deployed with a replica model corresponding to the global model.

[0043] The network device 104 in the embodiment of the present application may be any device with computing and communication capabilities, such as a server, a server or a virtual machine, etc. It is understandable that the network device 104 may also be any node.

[0044] Optionally, the central node 101, the first edge node 102, and the second edge node 103 all have a graphics processing unit (GPU).

[0045] Figure 1 The model training system 100 shown is only used as an example and is not used to limit the technical solution of the present application. Those skilled in the art should understand that in the specific implementation process, the model training system 100 may also include other devices, and the number of edge nodes and network devices may also be determined according to specific needs without limitation.

[0046] According to an embodiment of the present invention, a model training method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0047] In this embodiment, a model training method is provided, which can be used for the above-mentioned central node. Figure 3 is a flow chart of a model training method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0048] S301: Acquire N traffic data transmitted by at least one network device within a first time period.

[0049] Wherein, N ≥ 1 and is a positive integer.

[0050] In the embodiment of the present invention, the N flow data include normal flow data and abnormal flow data. The abnormal flow data include M types of abnormal flow data groups. N≥M≥1 and is a positive integer.

[0051] It is understandable that each type of abnormal traffic data corresponds to a different attack type. For example, the attack type may be a cross-site scripting attack, a database injection attack, or a distributed denial of service attack.

[0052] In the embodiment of the present invention, the first time period may be any historical time period without limitation. For example, the first time period may be one month, one quarter, or seven days.

[0053] Exemplarily, the central node obtains real-time traffic data including normal traffic data and abnormal traffic data transmitted by at least one network device for 7 consecutive days and stores it in the traffic feature library, and labels the traffic data into multiple categories according to normal traffic data and abnormal traffic data to obtain the original data set.

[0054] S302: Send at least part of the N traffic data to at least one edge node managed by the central node.

[0055] Among them, each edge node is deployed with a replica model, and the replica model is a replica of the global model.

[0056] In some optional embodiments, the central node obtains the computing power of each edge node; calculates a first ratio between the computing power of each edge node and all computing powers of at least one edge node; sends normal traffic data of the first ratio to each edge node, and sends abnormal traffic data in each type of abnormal traffic data group of the first ratio to each edge node.

[0057] The computing power of each edge node is proportional to the number of cores of the central processing unit of each edge node. For example, if the number of cores of the central processing unit of the first edge node is 1 and the number of cores of the central processing unit of the second edge node is 2, the computing power of the second edge node is twice that of the first edge node.

[0058] It is understandable that if the computing power of each edge node is similar, the central node evenly distributes N traffic data according to the number of edge nodes. If the computing power of each edge node is quite different, the central node sends more traffic data to the edge node with stronger computing power and sends less traffic data to the edge node with weaker computing power. For example, if the computing power of the second edge node is twice that of the first edge node, the traffic data sent by the central node to the second edge node is twice that of the traffic data sent to the first edge node.

[0059] Optionally, before sending the N traffic data to at least one edge node managed by the central node, the central node may also pre-process the N traffic data included in the original data set. For example, the pre-processing may be a normalization process.

[0060] It can be understood that the normalization process can avoid affecting the convergence of the model. The N traffic data after preprocessing are divided into a training set and a test set according to a preset ratio. For example, the preset ratio can be 9:1.

[0061] S303: Receive at least one first training result output by at least one edge node.

[0062] Among them, part of the traffic data is the traffic data sent by the central node to each edge node. The part of the traffic data trained by each edge node in the replica model is proportional to the computing power of each edge node.

[0063] In an embodiment of the present invention, each first training result is generated by training at least part of the traffic data received by each edge node in the replica model. The first training result includes sub-model parameters obtained after the replica model is trained, the accuracy rate of the replica model in detecting whether part of the traffic data is abnormal, and the false alarm rate corresponding to the abnormal traffic data in the part of the traffic data being normal traffic data.

[0064] Exemplarily, each edge node receives part of the traffic data sent from the central node, inputs part of the traffic data into the replica model for training, and outputs a first training result.

[0065] S304: Perform an aggregation operation on at least one sub-model parameter in at least one first training result to generate a first target parameter, and use the first target parameter to update the global model to obtain a first model.

[0066] In some optional implementations, the central node obtains a weight value of each edge node; and obtains a first target parameter based on at least one weight value and at least one sub-model parameter.

[0067] Specifically, the central node obtains the first target parameter based on the first algorithm according to at least one weight value and at least one sub-model parameter.

[0068] Among them, the first algorithm is:

[0069] w global is the first target parameter, w i is the sub-model parameter of the ith edge node, q i is the weight value of the i-th edge node.

[0070] The weight value can be set according to the amount of traffic data received by each edge node. For example, if the amount of traffic data received by the first edge node is greater than that received by the second edge node, the weight value of the first edge node is greater than that of the second edge node.

[0071] It can be understood that the first target parameter of the first model is obtained by weighted averaging the sub-model parameters of each edge node, which enables the federated learning system to take into account the contributions of different edge nodes to obtain the best first model.

[0072] S305: If the first model does not meet the first preset condition, iteratively train the first model between the central node and at least one edge node until the trained model meets the first preset condition, thereby obtaining a target model.

[0073] The first preset condition is that the accuracy rate corresponding to the model detecting whether N traffic data are abnormal is greater than or equal to the first threshold, and the false alarm rate corresponding to the model detecting that abnormal traffic data in the N traffic data is normal traffic data is less than or equal to the second threshold.

[0074] In a possible design, the first threshold or the second threshold can be set according to actual needs without limitation. For example, the first threshold can be 98%; the second threshold can be 5%.

[0075] Exemplarily, after the central node updates the model parameters of the global model to the first target parameters and obtains the first model, if the first model does not meet the first preset conditions, the first target parameters are sent to each edge node to update the model parameters of the copy model of each edge node to obtain the first sub-model, and training continues based on the first sub-model until the second training result output after training meets the first preset conditions, and the target model is obtained.

[0076] It can be understood that during the iterative training process, the central node continuously updates the model parameters of the global model until both the accuracy and the false alarm rate meet the first preset condition.

[0077] S306: Use the target model to detect the target flow data and obtain a detection result.

[0078] The detection result is used to indicate whether the target flow data is abnormal or normal.

[0079] In the embodiment of the present invention, the target flow data may be any flow data.

[0080] In one example, the central node obtains target traffic data, inputs the target traffic data into a target model, and outputs a detection result.

[0081] Based on the above Figure 3 According to the method shown, the central node can send the N acquired traffic data to at least one edge node managed by the central node, train the replica model of each edge node, output the sub-model parameters, and perform an aggregation operation on at least one sub-model parameter to obtain an update of the global model deployed by the central node to obtain a first model; and iteratively train the first model between the central node and at least one edge node until a target model that meets the first preset condition is obtained through training, so that the target model can detect whether the target traffic data is abnormal.

[0082] Since the training of the target model in the embodiment of the present invention is performed by the central node distributing data to each edge node, compared with the method of directly training the target model at the central node, the excessive workload caused by large-scale traffic data training at the central node can be avoided; in addition, since the node performance and adaptation scenarios of each edge node are different, N traffic data can be trained in a diversified manner to obtain sub-model parameters obtained by training different edge nodes, thereby improving the accuracy of the target model in detecting abnormal traffic data and reducing the false alarm rate.

[0083] Furthermore, after completing the model training and obtaining the target model, the central node distributes the target model to each edge node, and at the same time continues to obtain the traffic data transmitted by at least one network device in real time.

[0084] like Figure 4 As shown, Figure 4 is a flow chart of another model training method according to an embodiment of the present invention. In order to ensure the real-time and availability of traffic data detection, after a period of time, the central node may also perform the following steps:

[0085] S401: Obtain L new traffic data in the second time period.

[0086] The L traffic data include P types of abnormal traffic data groups.

[0087] In the embodiment of the present invention, the second time period is a time period after the first time period. The second time period can be any time period, for example, the second time period can be three months.

[0088] S402: Determine whether L flow data are greater than or equal to a preset threshold.

[0089] The preset threshold value can be set according to actual needs without limitation. For example, the preset threshold value can be 300.

[0090] S403: If the L flow data are less than the preset threshold, continue to obtain new flow data until the new flow data is greater than or equal to the preset threshold.

[0091] S404: If the L traffic data are greater than or equal to a preset threshold, determine whether the L traffic data include a P-type abnormal traffic data group equal to 1;

[0092] S405: If P is 1, the target model is trained based on L traffic data to obtain a first target model.

[0093] S406: If P is greater than 1, calculate the second ratio between the number of abnormal traffic data in the class with the least number of abnormal traffic data in the P-class abnormal traffic data group and L, and when the second ratio meets the second preset condition, train the target model based on L traffic data to obtain the first target model.

[0094] The second preset condition is

[0095] r i is the second ratio of the i-th type of abnormal traffic. i is the number of abnormal traffic of type i.

[0096] It can be understood that when the second ratio meets the second preset condition, it can avoid the problem that the small amount of data in a certain type of abnormal traffic data group interferes with the training of the model, thereby affecting the detection effect of other types of data.

[0097] In the present embodiment, a model training device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0098] This embodiment provides a model training device, which is applied to a central node, and a global model is deployed in the central node; Figure 5 As shown, Figure 5 : is a structural block diagram of a model training device according to an embodiment of the present invention; the device comprises:

[0099] The transceiver module 501 is used to obtain N flow data transmitted by at least one network device in a first time period, where N≥1 and is a positive integer.

[0100] The transceiver module 501 is also used to send at least part of the N traffic data to at least one edge node managed by the central node. Each edge node is deployed with a replica model, which is a replica of the global model.

[0101] The transceiver module 501 is also used to receive at least one first training result output by at least one edge node, each first training result is generated by training at least part of the traffic data received by each edge node in the replica model, and each first training result includes sub-model parameters.

[0102] The processing module 502 is used to perform an aggregation operation on at least one sub-model parameter in at least one first training result to generate a first target parameter, and use the first target parameter to update the global model to obtain a first model.

[0103] The processing module 502 is also used to iteratively train the first model between the central node and at least one edge node if the first model does not meet the first preset condition, until the trained model meets the first preset condition, thereby obtaining a target model.

[0104] The processing module 502 is further used to detect the target flow data using the target model to obtain a detection result, and the detection result is used to indicate whether the target flow data is abnormal or normal.

[0105] In some optional embodiments, the N traffic data include normal traffic data and abnormal traffic data; the abnormal traffic data include M types of abnormal traffic data groups, N≥M≥1 and is a positive integer; the transceiver module 501 is specifically used to obtain the computing power of each edge node; the transceiver module 501 is also specifically used to calculate a first ratio between the computing power of each edge node and all the computing powers of at least one edge node; the transceiver module 501 is also specifically used to send the normal traffic data of the first ratio to each edge node, and send the abnormal traffic data in each type of abnormal traffic data group of the first ratio to each edge node.

[0106] In some optional implementations, the processing module 502 is specifically used to obtain a weight value of each edge node; the processing module 502 is also specifically used to obtain a first target parameter based on at least one weight value and at least one sub-model parameter.

[0107] In some optional implementations, the processing module 502 is further specifically configured to obtain a first target parameter based on a first algorithm according to at least one weight value and at least one sub-model parameter; the first algorithm is: Among them, w global is the first target parameter, w i is the sub-model parameter of the ith edge node, q i is the weight value of the i-th edge node.

[0108] In some optional embodiments, the first preset condition is that the accuracy of the model detecting whether N traffic data are abnormal is greater than or equal to a first threshold, and the false alarm rate corresponding to the model detecting that abnormal traffic data in the N traffic data is normal traffic data is less than or equal to a second threshold.

[0109] In some optional embodiments, the transceiver module 501 is also used to obtain L newly added traffic data in a second time period, the L traffic data include a P-type abnormal traffic data group, and the second time period is a time period after the first time period; the processing module 502 is also used to, if the L traffic data are greater than or equal to a preset threshold, and P is 1, then train the target model based on the L traffic data to obtain a first target model; the processing module 502 is also used to, if the L traffic data are greater than or equal to a preset threshold, and P is greater than 1, calculate the second ratio between the number of abnormal traffic data in the class with the least number of abnormal traffic data in the P-type abnormal traffic data group and L, and when the second ratio meets the second preset condition, train the target model based on the L traffic data to obtain the first target model.

[0110] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0111] The model training device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0112] The embodiment of the present invention also provides a computer device having the above Figure 5 The model training apparatus shown.

[0113] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0114] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0115] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0116] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0118] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0119] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0120] Part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes but is not limited to source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0121] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A model training method, characterized in that: Applied to a central node, the central node is deployed with a global model; the method comprises: Obtain N flow data transmitted by at least one network device in a first time period, where N ≥ 1 and is a positive integer; Sending at least part of the N traffic data to at least one edge node managed by the central node, each edge node being deployed with a replica model, the replica model being a replica of the global model; Receiving at least one first training result output by the at least one edge node, each of the first training results is generated by training the at least part of the traffic data received by each edge node in the replica model, and each of the first training results includes a sub-model parameter; Performing an aggregation operation on at least one of the sub-model parameters in the at least one first training result to generate a first target parameter, and using the first target parameter to update the global model to obtain a first model; If the first model does not meet the first preset condition, iteratively train the first model between the central node and at least one of the edge nodes until the trained model meets the first preset condition, thereby obtaining a target model; The target flow data is detected using the target model to obtain a detection result, where the detection result is used to indicate whether the target flow data is abnormal or normal.

2. The method according to claim 1, characterized in that The N flow data include normal flow data and abnormal flow data; the abnormal flow data include M types of abnormal flow data groups, N≥M≥1 and is a positive integer; The sending at least part of the N traffic data to at least one edge node managed by the central node includes: Obtaining the computing power of each edge node; Calculating a first ratio between the computing power of each of the edge nodes and all computing power of the at least one edge node; The normal traffic data of the first ratio is sent to each of the edge nodes, and the abnormal traffic data in each type of abnormal traffic data group of the first ratio is sent to each of the edge nodes.

3. The method according to claim 1, characterized in that The step of performing an aggregation operation on at least one of the sub-model parameters in the at least one first training result to generate a first target parameter includes: Obtaining a weight value of each edge node; The first target parameter is obtained according to at least one of the weight values ​​and at least one of the sub-model parameters.

4. The method according to claim 3, characterized in that The obtaining the first target parameter according to at least one of the weight values ​​and at least one of the sub-model parameters comprises: Based on a first algorithm, the first target parameter is obtained according to at least one of the weight values ​​and at least one of the sub-model parameters; the first algorithm is: Among them, w global is the first target parameter, w i is the sub-model parameter of the ith edge node, q i is the weight value of the i-th edge node.

5. The method according to claim 4, characterized in that The first preset condition is that the accuracy rate corresponding to the model detecting whether the N traffic data are abnormal is greater than or equal to a first threshold, and the false alarm rate corresponding to the model detecting that the abnormal traffic data in the N traffic data is normal traffic data is less than or equal to a second threshold.

6. The method according to claim 1, characterized in that The method further comprises: Acquire L newly added flow data in a second time period, wherein the L flow data include a P-type abnormal flow data group, and the second time period is a time period after the first time period; If the L flow data are greater than or equal to a preset threshold, and P is 1, the target model is trained based on the L flow data to obtain a first target model; If the L traffic data are greater than or equal to the preset threshold and P is greater than 1, then the second ratio between the number of abnormal traffic data in the class with the least number of abnormal traffic data in the P class abnormal traffic data group and L is calculated, and when the second ratio meets the second preset condition, the target model is trained based on the L traffic data to obtain the first target model.

7. A model training device, characterized in that: Applied to a central node, the central node is deployed with a global model; the device comprises: A transceiver module, used to obtain N flow data transmitted by at least one network device in a first time period, where N ≥ 1 and is a positive integer; The transceiver module is further used to send at least part of the N traffic data to at least one edge node managed by the central node, each edge node is deployed with a replica model, and the replica model is a replica of the global model; The transceiver module is further used to receive at least one first training result output by the at least one edge node, each of the first training results is generated by training the at least part of the traffic data received by each edge node in the replica model, and each of the first training results includes a sub-model parameter; a processing module, configured to perform an aggregation operation on at least one of the sub-model parameters in the at least one first training result to generate a first target parameter, and update the global model using the first target parameter to obtain a first model; The processing module is further configured to iteratively train the first model between the central node and at least one edge node if the first model does not meet the first preset condition, until the trained model meets the first preset condition, thereby obtaining a target model; The processing module is further used to detect the target flow data using the target model to obtain a detection result, and the detection result is used to indicate whether the target flow data is abnormal or normal.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the model training method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the model training method described in any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to make a computer execute the model training method described in any one of claims 1 to 6.

Images