Security inspection model upgrade method and device based on data distribution clustering federated learning

Abstract

The present application relates to a security inspection model upgrade method and device based on data distribution clustering federated learning. In this method, after the server sends the initial global model to each client node, each client node divides the corresponding local security inspection image into an ordinary first image and a high-value second image. Each client node builds a feature descriptor of the local security inspection image composed of a descriptor of ordinary data and a descriptor of high-value data. The feature descriptor ensures that the corresponding client contributes important information representing the data distribution characteristics of its local security inspection image. Then, the server clusters the client nodes through the feature descriptors, which can balance the contribution of each client to the global model, so that a class of client nodes with similar high-value data are more easily divided into the same cluster, so that the distribution difference of high-value data of client nodes in the same cluster is relatively small, thereby improving the effect of model aggregation.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of federated learning, and in particular to a method and device for upgrading a security inspection model based on data distribution clustering federated learning. Background Art

[0002] Since the image datasets stored in the security terminal nodes are specific, that is, they are not independent and identically distributed, and the data distribution of the datasets between the security terminal nodes is quite different, if the traditional federated learning framework is used, each security terminal node will contribute unequally to the global model, which may cause the global model update to be biased towards certain specific types of data. This deviation will accumulate and may lead to problems such as long global model training time and low inference accuracy. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The main purpose of the embodiments of the present disclosure is to propose a security inspection model upgrade method and device based on data distribution clustering federated learning, which can reduce the impact of data non-independent and identically distributed on model accuracy and achieve efficient and safe completion of model self-evolution upgrade.

[0005] A first aspect of an embodiment of the present application proposes a security inspection model upgrade method based on data distribution clustering federated learning, which is used for a server. The method includes:

[0006] Sending an initial global model to multiple client nodes;

[0007] Obtain corresponding feature descriptors of the plurality of client nodes, wherein the first Feature descriptors of client nodes Obtained through the following methods:

[0008] ;

[0009] ;

[0010] ;

[0011] ;

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] in, represents the rectified linear unit function; represents the hyperbolic tangent function; represents the weight coefficient; Represents element-wise dot product; Indicates the The client node extracts the feature vector of the corresponding local security inspection image; Represents the index of the local security inspection image; represents the data amount of the first image; represents the amount of data of the second image; Indicates the A client node extracts the data volume of the corresponding local security inspection image, wherein the first image refers to the local security inspection image with the detection result of the initial global model; and the second image refers to the local security inspection image without the detection result of the initial global model;

[0017] Clustering the multiple client nodes according to the feature descriptors to obtain a clustering result;

[0018] The initial global model of the plurality of client nodes is trained according to the clustering result until each client node obtains a security inspection model that completes the training of the initial global model.

[0019] In some embodiments, Obtained through the following methods:

[0020] Use any one of the residual network, convolutional neural network of the visual geometry group, EfficientNet, and twin network to extract the The local security image of the client node .

[0021] In some implementations, the initial global model of the plurality of client nodes is trained according to the clustering result, and each client node obtains a security inspection model trained with the initial global model, including:

[0022] Obtaining model parameters corresponding to the multiple client nodes; wherein the model parameters refer to model parameters obtained after the client node uses the initial global model to perform multiple rounds of training on the local security inspection image;

[0023] Perform intra-cluster model aggregation according to the model parameters of all client nodes in each cluster to obtain a cluster model corresponding to each cluster;

[0024] Performing inter-cluster model aggregation on the cluster models corresponding to each cluster to obtain a global model for the next number of iterations;

[0025] The global model of the next iteration number is sent to the multiple client nodes, so that the multiple client nodes start the training of the next iteration number based on the global model of the next iteration number, and iterate in sequence until each client node obtains the security inspection model that completes the training of the initial global model.

[0026] In some embodiments, The model parameters of a client node are obtained as follows:

[0027] The said Client node pair Perform data enhancement to obtain ;

[0028] The said The client nodes are based on the initial global model conduct Round iterative training;

[0029] The said The client nodes are based on the initial global model and conduct After rounds of iterative training, the The model parameters of the client nodes.

[0030] In some implementations, the performing intra-cluster model aggregation according to the model parameters of all client nodes in each cluster includes:

[0031] Selecting a corresponding client weight according to the model parameters of all client nodes in each cluster; wherein the size of the client weight is proportional to the amount of training data of all client nodes in the cluster, and the amount of training data of the client node is the sum of the amount of data of the first image and the amount of data of the second image of the client node;

[0032] The intra-cluster model aggregation is completed through the following formula:

[0033] ;

[0034] in, Indicates the The client node belongs to The cluster model of clusters, Indicates the the initial global model of client nodes, Indicates the The client weight of the client nodes, For the Clusters, Indicates the The model parameters of the client nodes.

[0035] In some implementations, performing inter-cluster model aggregation on the cluster models corresponding to each cluster to obtain a global model for the next iteration number includes:

[0036] Selecting a corresponding cluster weight according to the amount of training data corresponding to each cluster; wherein the size of the cluster weight is proportional to the amount of training data of the cluster, and the amount of training data of the cluster includes the sum of the amount of training data of all client nodes in the cluster;

[0037] The cluster model aggregation is completed through the following formula to obtain the global model for the next iteration:

[0038] ;

[0039] in, represents the global model for the next iteration number, represents the total number of clusters, Indicates The cluster weight of each cluster.

[0040] In some implementations, clustering the plurality of client nodes according to the feature descriptors includes:

[0041] identifying client nodes as core nodes and client nodes as edge nodes according to the feature descriptors using a density clustering algorithm, clustering the core nodes according to a hierarchical clustering algorithm, and grouping client nodes with similar data distribution into the same cluster;

[0042] or,

[0043] The plurality of client nodes are clustered according to the feature descriptors using a K-means clustering algorithm.

[0044] A second aspect of the embodiment of the present application proposes a security inspection model upgrade device based on data distribution clustering federated learning, the device comprising:

[0045] A model distribution unit, used for sending an initial global model to multiple client nodes;

[0046] A feature description acquisition unit is used to acquire the corresponding feature descriptors of the plurality of client nodes, wherein the first Feature descriptors of client nodes Obtained through the following methods:

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] in, represents the rectified linear unit function; represents the hyperbolic tangent function; represents the weight coefficient; Represents element-wise dot product; Indicates the The client node extracts the feature vector of the corresponding local security inspection image; Represents the index of the local security inspection image; represents the data amount of the first image; represents the amount of data of the second image; Indicates the A client node extracts the data volume of the corresponding local security inspection image, wherein the first image refers to the local security inspection image with the detection result of the initial global model; and the second image refers to the local security inspection image without the detection result of the initial global model;

[0056] A terminal node clustering unit, configured to cluster the plurality of client nodes according to the feature descriptors to obtain a clustering result;

[0057] A global model training unit is used to train the initial global model of the multiple client nodes according to the clustering result until each client node obtains a security inspection model that completes the training of the initial global model.

[0058] A third aspect of an embodiment of the present application proposes an electronic device, comprising at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by at least one of the controllers to enable at least one of the controllers to execute the security inspection model upgrade method based on data distribution clustering federated learning as described above.

[0059] A fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the security inspection model upgrade method based on data distribution clustering federated learning as described above is implemented.

[0060] The method provided in this embodiment has the following advantages:

[0061] After the server sends the initial global model to each client node, each client node divides the corresponding local security inspection image into an ordinary first image and a high-value second image, because the second image is a type of image data with complex features that is difficult to be detected by the global model. These data represent the weak link of the global model and are particularly important for the optimization of the security inspection model. Each client node mines the local high-value data and builds a feature descriptor of the local security inspection image composed of a descriptor of ordinary data and a descriptor of high-value data. The feature descriptor ensures that the corresponding client contributes important information representing the data distribution characteristics of its local security inspection image and can accurately reflect the data distribution of the client; then the server clusters the client nodes through the feature descriptors, which can balance the contribution of each client to the global model, so that a type of client nodes with similar high-value data are more easily divided into the same cluster, so that the distribution difference of high-value data of client nodes in the same cluster is relatively small, thereby improving the effect of model aggregation.

[0062] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0064] Figure 1 It is a flowchart of a security inspection model upgrade method based on data distribution clustering federated learning provided in an embodiment of the present application;

[0065] Figure 2 It is a structural schematic diagram of a security inspection model upgrading device based on data distribution clustering federated learning provided in an embodiment of the present application;

[0066] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0068] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0069] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0070] Reference Figure 1 Some embodiments of the present application provide a security inspection model upgrade method based on data distribution clustering federated learning, and the upgrade method includes the following steps S110 to S140, specifically:

[0071] Step S110: The server sends an initial global model to multiple client nodes.

[0072] Step S120: The server obtains corresponding feature descriptors of multiple client nodes.

[0073] Step S130: The server clusters the multiple client nodes according to the feature descriptors to obtain a clustering result.

[0074] Step S140: The server trains the initial global models of multiple client nodes according to the clustering results until each client node obtains a security inspection model that completes the training of the initial global model.

[0075] The client can be various security inspection terminals, such as security inspection machines deployed in aviation, high-speed rail stations and major conference areas. The server refers to a device connected to each security inspection terminal and used to upgrade the security inspection model in the security inspection terminal. The security inspection model is a model carried by the client for detecting the security inspection image scanned by the security inspection machine, for example, detecting a package.

[0076] In step S110, the initial global model is a model set in advance by the server, for example, it can be the security inspection model used before iteration. One of the purposes of this embodiment is to update the initial global model so that each client node can carry the latest security inspection model.

[0077] In step S120, the client needs to count the corresponding feature descriptors. The specific process includes:

[0078] Assume that it is client I, client I has J local security inspection images, and will continue to extract the feature vector of each local security inspection image ,in Indicates the data sample index of the local security inspection image.

[0079] Then high-value data and ordinary data are selected, that is, the first image (referring to the local security inspection image that can be detected by the initial global model) and the second image (referring to the local security inspection image that cannot be detected by the initial global model) are selected; it should be noted that in the field of deep learning, data that is difficult for the global model to recognize is called hard data, which has a greater impact on the loss of local model training. Compared with data that can be easily recognized by the local model, it is more valuable and can also be called high-value data.

[0080] Then construct the feature descriptor of the normal data (i.e. the first image) :

[0081] ;

[0082] in, represents the data amount of the first image; Indicates The feature vector of the local security inspection image extracted by the client node;

[0083] Then, build a feature descriptor for the high-value data (i.e., the second image) :

[0084] ;

[0085] in, represents the amount of data of the second image;

[0086] Then calculate the ratio of the first image to the second image in the local security inspection image:

[0087] ;

[0088] Indicates the data volume of local security inspection images.

[0089] Then calculate the ratio of the second image in the local security inspection image:

[0090] ;

[0091] Finally, nonlinear transformation of feature descriptors is used to improve the effect of feature descriptors.

[0092] ;

[0093] ;

[0094] in, The corrected linear unit function effectively handles common data feature identifiers by returning negative values ​​to zero and retaining positive values The sparsity of . is the hyperbolic tangent function. The feature descriptor of high-value data Map to to avoid the influence of extreme values ​​on subsequent feature clustering and cluster classification.

[0095] Then the feature interaction term is constructed to describe the correlation between the feature descriptors of the first image and the second image.

[0096] ;

[0097] in, is the element-wise dot product.

[0098] Finally, the overall feature descriptor is obtained by combining , the final overall feature descriptor It can be expressed as:

[0099] ;

[0100] in, is the weight coefficient for adjusting the interaction term.

[0101] The subsequent steps are to use feature descriptors for clustering and finally train and update the model.

[0102] The advantages of the method provided in this embodiment include:

[0103] After the server sends the initial global model to each client node, each client node divides the corresponding local security inspection image into an ordinary first image and a high-value second image, because the second image is a type of image data with complex features that is difficult to be detected by the global model. These data represent the weak link of the global model and are particularly important for the optimization of the security inspection model. Each client node mines the local high-value data and builds a feature descriptor of the local security inspection image composed of a descriptor of ordinary data and a descriptor of high-value data. The feature descriptor ensures that the corresponding client contributes important information representing the data distribution characteristics of its local security inspection image and can accurately reflect the data distribution of the client; then the server clusters the client nodes through the feature descriptors, which can balance the contribution of each client to the global model, so that a type of client nodes with similar high-value data are more easily divided into the same cluster, so that the distribution difference of high-value data of client nodes in the same cluster is relatively small, thereby improving the effect of model aggregation.

[0104] In some embodiments of the present application, Obtained through the following methods:

[0105] Use a pre-trained convolutional neural network to extract The local security image of the client node ; Among them, the convolutional neural network includes any one of the residual network, the convolutional neural network of the visual geometry group, EfficientNet and Siamese Network.

[0106] Among them, the convolutional neural network of the Visual Geometry Group, also known as VGG or VGGNet, is a deep convolutional neural network architecture developed by a research team at the University of Oxford, designed to solve computer vision tasks, especially image recognition tasks. EfficientNet is a network proposed by Google for target detection. Siamese Network is a type of neural network structure that consists of two or more identical networks.

[0107] In the client node, a convolutional neural network can be set up in advance, through which the local security inspection image can be extracted. , and then the feature vector can be used to construct the feature descriptor of high-value data.

[0108] In some embodiments of the present application, the above step S140 trains the initial global model of multiple client nodes according to the clustering result, and each client node obtains a security inspection model that completes the training of the initial global model, including the following steps S141 to S144:

[0109] Step S141, obtaining model parameters corresponding to multiple client nodes; wherein the model parameters refer to model parameters obtained after the client node uses the initial global model to perform multiple rounds of training on the local security inspection image.

[0110] In some embodiments of the present application, Model parameters for client nodes Obtained through the following methods:

[0111] Step S1411, Client node pair Perform data enhancement to obtain .

[0112] Step S1412, The client nodes are based on the initial global model conduct Rounds of iterative training.

[0113] Step S1413, The client nodes are based on the initial global model and conduct After a round of iterative training, we get Model parameters for client nodes .

[0114] In the training process of the security inspection model, this example fully mines high-value data (i.e., the second image), because high-value data often has complex features and is difficult to be detected by the model. These data represent the weak links of the model and are particularly important for the optimization of the security inspection model. By mining high-value data for targeted training, the performance of the model in complex scenarios can be significantly improved; and high-value data mining can ensure that the global model is exposed to more difficult examples during the training process, improving the generalization ability of the final security inspection model for different scenarios, thereby reducing missed detections and false detections.

[0115] Step S142: Perform intra-cluster model aggregation according to the model parameters of all client nodes in each cluster to obtain a cluster model corresponding to each cluster.

[0116] Step S142 performs intra-cluster model aggregation according to the model parameters of all client nodes in each cluster, including the following steps S1421 and S1422:

[0117] Step S1421, select the corresponding client weight according to the model parameters of all client nodes in each cluster; wherein the size of the client weight is proportional to the amount of training data of all client nodes in the cluster, and the amount of training data of the client node is the sum of the data of the first image and the data of the second image of the client node.

[0118] Step S1422: Complete the intra-cluster model aggregation through the following formula:

[0119] ;

[0120] in, Indicates The client node belongs to The cluster model of clusters, Indicates The initial global model of client nodes, Indicates The client weight of the client nodes, For the Clusters.

[0121] This embodiment sets a client weight for the corresponding client that is proportional to the amount of training data of all client nodes in the cluster. This can adjust the proportion of model parameters of the corresponding client nodes with richer training data when aggregating within the cluster, extract more useful features, and thus improve the cluster aggregation model effect.

[0122] Step S143: perform inter-cluster model aggregation on the cluster models corresponding to each cluster to obtain a global model for the next number of iterations.

[0123] In some embodiments of the present application, step S143 performs inter-cluster model aggregation on the cluster model corresponding to each cluster to obtain a global model for the next iteration number, including the following steps S1431 and S1432:

[0124] Step S1431, selecting a corresponding cluster weight according to the amount of training data corresponding to each cluster; wherein the cluster weight is proportional to the amount of training data of the cluster, and the amount of training data of the cluster includes the sum of the training data of all client nodes in the cluster.

[0125] Step S1432: complete the cluster model aggregation through the following formula to obtain the global model of the next iteration number:

[0126] ;

[0127] in, represents the global model for the next iteration number, represents the total number of clusters, Indicates The cluster weight of each cluster.

[0128] It should be noted that this embodiment selects a cluster weight for each cluster that is proportional to the amount of training data of the cluster. This enables the global model to be aggregated by adjusting the proportion of the corresponding cluster models with richer training data, thereby extracting more useful features and improving the effect of the aggregated global model.

[0129] Step S144: Send the global model of the next iteration number to multiple client nodes, so that the multiple client nodes start the training of the next iteration number based on the global model of the next iteration number, and iterate in sequence until each client node obtains the security inspection model that completes the training of the initial global model.

[0130] In this embodiment, the first training process of the initial global model is provided. It should be noted that the initial global model generally needs to be trained multiple times before the completed security inspection model can be obtained (i.e., multiple iterations including step S144). The condition for the end of the iteration may be reaching the specified number of iterations or reaching the accuracy preset by the global model. The training in the above steps first includes the clustering process (i.e., steps S141 to S143), and then includes the parameter aggregation model issuance and training process (i.e., step S144). In this embodiment, the model upgrade process only needs to perform a clustering process on each client node once. After clustering is performed, each subsequent iteration process does not need to cluster again. This embodiment can cluster multiple client nodes based on feature descriptors, which can make it easier for similar data to be classified into the same cluster, thereby improving the effect of model aggregation.

[0131] In some embodiments of the present application, clustering multiple client nodes according to feature descriptors in step S130 to obtain clustering results includes the following steps S1310 and S1320:

[0132] Step S1310: using a density clustering algorithm to identify client nodes that are core nodes and client nodes that are edge nodes according to feature descriptors;

[0133] Step S1320: further cluster the core nodes using a hierarchical clustering algorithm, and classify client nodes with similar data distribution into the same cluster.

[0134] Or:

[0135] Step S1330 : clustering the multiple client nodes according to the feature descriptors using a K-means clustering algorithm.

[0136] For ease of understanding, the following embodiment is provided, namely, a security inspection model upgrade method based on data distribution clustering federated learning, the method comprising steps S910 to S990:

[0137] Step S910: data statistics and client selection.

[0138] Shared Security terminal nodes , count the data volume of local security inspection images of each security inspection terminal node , if any The amount of data from security inspection terminal nodes Larger than the preset data volume , then these security inspection terminal nodes are selected as the client nodes participating in this training, that is, ,in .

[0139] Step S920: The server sends the initial global model.

[0140] The server sends the global model To client node As the initial global model .

[0141] Step S930: mining high-value data of client nodes.

[0142] Among them, the client node Using the initial global model Inference is performed on local security inspection images to mine high-value data. The data volume of the first image with recognition results (local security inspection images detected by the model) is , the data volume of the second image without recognition result (local security inspection image missed by the model) is , .

[0143] Step S940: extracting client node features and constructing feature descriptors.

[0144] The feature descriptor representing the local data distribution is obtained through feature extraction , in some embodiments, specifically includes:

[0145] Step S941: feature extraction.

[0146] Use pre-trained convolutional neural networks (such as ResNet, convolutional neural networks from the visual geometry group, EfficientNet, etc.) to extract the feature vector of each local security inspection image ,in Represents the index of the data sample.

[0147] Step S942: Feature descriptor of the first image Calculate the feature descriptor of the common data of the client node :

[0148] ;

[0149] Step S943: Feature descriptor of the second image Calculate the feature descriptor of the high-value data of the client node :

[0150] ;

[0151] Step S944: Calculate the ratio of the first image to the second image in the local security inspection image.

[0152] Calculate the ratio of the first image in the local security inspection image:

[0153] ;

[0154] Calculate the ratio of the second image in the local security inspection image:

[0155] ;

[0156] Step S945: Non-linear transformation of feature descriptors.

[0157] Through the nonlinear transformation function and Transform feature descriptors:

[0158] ;

[0159] ;

[0160] in, The corrected linear unit function effectively handles common data feature identifiers by returning negative values ​​to zero and retaining positive values The sparsity of is the hyperbolic tangent function, which transforms the feature descriptor of high-value data into Map it to the range between positive 1 and negative 1 to avoid the influence of extreme values ​​on subsequent feature clustering and cluster classification.

[0161] Step S946: construct feature interaction items.

[0162] Considering the correlation between the feature descriptors of the first image and the second image, the formula for constructing the feature interaction term is:

[0163] ;

[0164] in, is the element-wise dot product.

[0165] Step S947: Overall feature descriptor combination.

[0166] Final overall feature descriptor It can be expressed as:

[0167] ;

[0168] in, is the weight coefficient for adjusting the interaction term.

[0169] Step S950: The server performs feature clustering and cluster division.

[0170] The client node sends the local security image data volume and feature descriptors Upload to the server, the server uses a hybrid clustering algorithm to transform the feature descriptors of the client nodes Divided into The specific steps are as follows:

[0171] Step S951: First, use a density clustering algorithm (such as DBSCAN) to use feature descriptors Identify core nodes and edge nodes where data is densely distributed.

[0172] Density clustering algorithms are used to identify core nodes and edge nodes where data is densely distributed.

[0173] Set two key hyperparameters: (indicates the radius of the neighborhood) and MinPts (indicates the minimum number of points in the neighborhood);

[0174] Feature Descriptor As input, for each node, the feature descriptor Calculate its The number of points in the neighborhood, if the number of points is greater than or equal to MinPts, then the node is marked as a core node, otherwise, it is marked as a non-core node;

[0175] For each non-core node, if it is located in a core node If a point is neither a core node nor an edge node, it is marked as a noise point.

[0176] Core node: a point whose label is not -1;

[0177] Edge node: a point with label -1 but in the neighborhood of a core node;

[0178] Noise points: points with label -1 that are not in the neighborhood of any core node.

[0179] Step S952: Use a hierarchical clustering algorithm to further cluster the core nodes, and group the security terminal nodes with similar data distribution into the same cluster. .

[0180] Steps to implement hierarchical clustering:

[0181] Calculate the distance matrix: Use the feature descriptors of every two core nodes to calculate the distance between each two, and then form a distance matrix, usually using Euclidean distance or cosine distance.

[0182] Construct a clustering tree: Use a bottom-up approach to merge the two most similar nodes into one cluster, and repeat this process until all nodes are in one cluster or the set number of clusters is reached.

[0183] Pruning cluster tree: Prune the cluster tree according to the preset distance threshold or number of clusters to obtain the final clustering result.

[0184] Through the above steps, it is possible to identify core nodes and edge nodes with dense data distribution, and further cluster core nodes with similar data distribution. This method can effectively organize and manage the data distribution of security terminal nodes in practical applications.

[0185] Step S953: The server calculates the data volume of each cluster's local security inspection image And the total amount of all data for this training :

[0186] ;

[0187] ;

[0188] Step S960: The client node trains the local model and updates the parameters.

[0189] Each client node Using local security images and the initial global model Perform multiple iterations of training to update local model parameters , and returns it to the server. The specific steps are as follows:

[0190] Step S961: Model initialization.

[0191] The initial global model As Initialization parameters.

[0192] Step S962: data enhancement.

[0193] For high value data (i.e. second image) Perform data enhancement to generate enhanced high-value data .

[0194] Step S963: hierarchical training.

[0195] Use First Image conduct rounds of iterative training, using the learning rate .

[0196] Introducing high-value data (i.e., second images) and enhanced high-value data conduct Rounds of iterative training, using a higher learning rate .

[0197] Step S964, weighted loss function;

[0198] The weighted loss function used is as follows:

[0199] ;

[0200] in, .

[0201] Step S965, optimizing algorithm;

[0202] Use an optimization algorithm such as Adam or RMSProp to minimize the local loss function.

[0203] Step S966: local training is completed;

[0204] Each client node The updated model parameters Upload to the server.

[0205] Step S970: The server aggregates the models within the cluster.

[0206] The server's local model parameters for each client node in the same cluster Use the optimization method to aggregate the models within the cluster, and each cluster can get a cluster model , the specific steps are as follows:

[0207] Step S971: Set the corresponding client weight according to the amount of training data of each local model parameter , the client weight size and the amount of training data of each client node in the cluster Proportional to:

[0208] ;

[0209] Step S972: The specific formula for cluster model aggregation is:

[0210] ;

[0211] Step S980: The server performs inter-cluster model aggregation.

[0212] Server Node Pair Cluster Model Perform inter-cluster model aggregation to update the global model , the specific steps are as follows:

[0213] Step S981: Set the corresponding cluster weight according to the amount of training data of each cluster , the cluster weight size and the amount of cluster data Proportional to:

[0214] ;

[0215] Step S982: The specific formula for cluster model aggregation is:

[0216] ;

[0217] Step S990: The server evaluates and updates the model.

[0218] The updated global model To conduct an assessment, the specific steps are as follows:

[0219] Step S991: If the model performance does not meet the preset requirements, the server will Cluster Model Sent to the client nodes in the corresponding cluster ( ), as the initial global model for the next training cycle, and repeat the above training steps.

[0220] Step S992: If the model performance reaches the preset requirements, the training ends and the server sends the global model Sent to all client nodes participating in the training , and then complete the update and upgrade of the corresponding security inspection model of each security inspection terminal.

[0221] This method does not need to integrate the security terminal node data. Through the privacy protection mechanism of "data does not change but the model changes, data is available but not visible", combined with the terminal node data distribution clustering method, the federated learning process is optimized, which effectively reduces the impact of data non-independent and identically distributed on model accuracy, and realizes efficient and safe completion of model self-evolution upgrade.

[0222] After the server sends the initial global model to each client node, each client node divides the corresponding local security inspection image into an ordinary first image and a high-value second image, because the second image is a type of image data with complex features that is difficult to be detected by the global model. These data represent the weak link of the global model and are particularly important for the optimization of the security inspection model. Each client node mines the local high-value data and builds a feature descriptor of the local security inspection image composed of a descriptor of ordinary data and a descriptor of high-value data. The feature descriptor ensures that the corresponding client contributes important information representing the data distribution characteristics of its local security inspection image and can accurately reflect the data distribution of the client; then the server clusters the client nodes through the feature descriptors, which can balance the contribution of each client to the global model, so that a type of client nodes with similar high-value data are more easily divided into the same cluster, so that the distribution difference of high-value data of client nodes in the same cluster is relatively small, thereby improving the effect of model aggregation.

[0223] This method fully mines the high-value data in the local security inspection images of each client, because high-value data is often data with complex features that are difficult to be detected by the model. These data represent the weak links of the model and are particularly important for the optimization of the security inspection model. By mining high-value data for targeted training (such as steps S961 to S965), the performance of the global model in complex scenarios can be significantly improved; and high-value data mining can ensure that the global model is exposed to more difficult examples during the training process, thereby improving the generalization ability of the final security inspection model for different scenarios, thereby reducing missed detections and false detections.

[0224] An embodiment of the present application provides a security inspection model upgrade device based on data distribution clustering federated learning, the device comprising a model distribution unit 1100, a feature description acquisition unit 1200, a terminal node clustering unit 1300 and a global model training unit 1400, specifically including:

[0225] The model distribution unit 1100 is used to send the initial global model to multiple client nodes.

[0226] The feature description acquisition unit 1200 is used to acquire corresponding feature descriptors of multiple client nodes, wherein the first Feature descriptors of client nodes Obtained through the following methods:

[0227] ;

[0228] ;

[0229] ;

[0230] ;

[0231] ;

[0232] ;

[0233] ;

[0234] ;

[0235] in, represents the rectified linear unit function; represents the hyperbolic tangent function; represents the weight coefficient; Represents element-wise dot product; Indicates The feature vector of the local security inspection image extracted by the client node; Indicates the index of the local security inspection image; represents the data amount of the first image; Indicates the data volume of the second image; the local security inspection image indicates the The security inspection images obtained by the client nodes, represents the data volume of the local security inspection image, the first image refers to the local security inspection image with the detection result of the initial global model; the second image refers to the local security inspection image without the detection result of the initial global model.

[0236] The terminal node clustering unit 1300 is used to cluster multiple client nodes according to the feature descriptors to obtain clustering results.

[0237] The global model training unit 1400 is used to train the initial global models of multiple client nodes according to the clustering results, until each client node obtains a security inspection model that completes the training of the initial global model.

[0238] It should be noted that the present device embodiment and the above-mentioned method embodiment are based on the same inventive concept, so the relevant contents of the above-mentioned method embodiment are also applicable to the present device embodiment and will not be repeated here.

[0239] After the server sends the initial global model to each client node, each client node fully mines the local high-value data (i.e., the second image), because high-value data is often data with complex features and difficult to be detected by the model. These data represent the weak links of the model and are particularly important for the optimization of the security inspection model. The feature descriptor of the entire local security inspection image is obtained through the descriptor of ordinary data (i.e., the feature vector of the first image and the data volume of the first image) and the high-value data descriptor (i.e., the feature vector of the second image and the data volume of the second image). The feature descriptor ensures that the corresponding client contributes important information representing the characteristics of its local data distribution and can accurately reflect the data distribution of the client. The server clusters the client nodes through the feature descriptors, which can balance the contribution of each client to the global model, making it easier for high-value data of the same type to be divided into the same cluster, so that the distribution difference of high-value data in the same cluster is relatively small, thereby improving the effect of model aggregation.

[0240] Reference Figure 3 The embodiment of the present application further provides an electronic device, the electronic device comprising:

[0241] at least one memory;

[0242] at least one processor;

[0243] at least one program;

[0244] The program is stored in the memory, and the processor executes at least one program to implement the security inspection model upgrade method based on data distribution clustering federated learning implemented in the present disclosure.

[0245] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0246] The electronic device according to the embodiment of the present application is described in detail below.

[0247] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0248] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the security inspection model upgrade method based on data distribution clustering federated learning in the embodiment of this application.

[0249] Input / output interface 1800, used to implement information input and output;

[0250] Communication interface 1900, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0251] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0252] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0253] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned security inspection model upgrade method based on data distribution clustering federated learning.

[0254] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0255] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0256] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0257] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0258] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0259] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0260] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0261] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0262] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0263] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0264] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0265] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A security inspection model upgrade method based on data distribution clustering federated learning, characterized in that: For a server, the method comprises: Sending an initial global model to multiple client nodes; Obtain corresponding feature descriptors of the plurality of client nodes, wherein the first Feature descriptors of client nodes Obtained through the following methods: ； ； ； ； ； ； ； ； in, represents the rectified linear unit function; represents the hyperbolic tangent function; represents the weight coefficient; Represents element-wise dot product; Indicates the The client node extracts the feature vector of the corresponding local security inspection image; Represents the index of the local security inspection image; represents the data amount of the first image; represents the data amount of the second image; Indicates the A client node extracts the data volume of the corresponding local security inspection image, wherein the first image refers to the local security inspection image with the detection result of the initial global model; and the second image refers to the local security inspection image without the detection result of the initial global model; Clustering the multiple client nodes according to the feature descriptors to obtain a clustering result; The initial global models of the multiple client nodes are trained according to the clustering results until each client node obtains a security inspection model trained with the initial global model, wherein the initial global models of the multiple client nodes are trained according to the clustering results until each client node obtains a security inspection model trained with the initial global model, including: Obtaining model parameters corresponding to the multiple client nodes; wherein the model parameters refer to model parameters obtained after the client node uses the initial global model to perform multiple rounds of training on the local security inspection image; Perform intra-cluster model aggregation according to the model parameters of all client nodes in each cluster to obtain a cluster model corresponding to each cluster; Performing inter-cluster model aggregation on the cluster models corresponding to each cluster to obtain a global model for the next number of iterations; The global model of the next iteration number is sent to the multiple client nodes, so that the multiple client nodes start the training of the next iteration number based on the global model of the next iteration number, and iterate in sequence until each client node obtains the security inspection model that completes the training of the initial global model.

2. The security inspection model upgrade method based on data distribution clustering federated learning according to claim 1 is characterized in that: Obtained through the following methods: Use any one of the residual network, convolutional neural network of the visual geometry group, EfficientNet, and twin network to extract the The local security image of the client node , the convolutional neural network of the visual geometry group is a VGG network.

3. The security inspection model upgrade method based on data distribution clustering federated learning according to claim 1 is characterized in that: No. The model parameters of a client node are obtained as follows: The said Client node pair Perform data enhancement to obtain ; The said The client nodes are based on the initial global model conduct Round iterative training; The said The client nodes are based on the initial global model and conduct After rounds of iterative training, the The model parameters of each client node.

4. The security inspection model upgrade method based on data distribution clustering federated learning according to claim 3 is characterized in that: The performing intra-cluster model aggregation according to the model parameters of all client nodes in each cluster includes: Selecting a corresponding client weight according to the model parameters of all client nodes in each cluster; wherein the size of the client weight is proportional to the amount of training data of all client nodes in the cluster, and the amount of training data of the client node is the sum of the amount of data of the first image and the amount of data of the second image of the client node; The intra-cluster model aggregation is completed through the following formula: ； in, Indicates the The client node belongs to The cluster model of clusters, Indicates the the initial global model of client nodes, Indicates the The client weight of the client nodes, For the Clusters, Indicates the The model parameters of each client node.

5. The security inspection model upgrade method based on data distribution clustering federated learning according to claim 4 is characterized in that: The clustering of cluster models corresponding to each cluster to obtain a global model for the next number of iterations includes: Selecting a corresponding cluster weight according to the amount of training data corresponding to each cluster; wherein the size of the cluster weight is proportional to the amount of training data of the cluster, and the amount of training data of the cluster includes the sum of the amount of training data of all client nodes in the cluster; The cluster model aggregation is completed through the following formula to obtain the global model for the next iteration: ； in, represents the global model for the next iteration number, represents the total number of clusters, Indicates The cluster weight of each cluster.

6. The security inspection model upgrade method based on data distribution clustering federated learning according to claim 1 is characterized in that: The clustering of the plurality of client nodes according to the feature descriptors comprises: identifying client nodes as core nodes and client nodes as edge nodes according to the feature descriptors using a density clustering algorithm, clustering the core nodes according to a hierarchical clustering algorithm, and grouping client nodes with similar data distribution into the same cluster; or, The plurality of client nodes are clustered according to the feature descriptors using a K-means clustering algorithm.

7. A security inspection model upgrade device based on data distribution clustering federated learning, characterized in that: The device comprises: A model distribution unit, used for sending an initial global model to multiple client nodes; A feature description acquisition unit is used to acquire the corresponding feature descriptors of the plurality of client nodes, wherein the first Feature descriptors of client nodes Obtained through the following methods: ； ； ； ； ； ； ； ； in, represents the rectified linear unit function; represents the hyperbolic tangent function; represents the weight coefficient; Represents element-wise dot product; Indicates the The client node extracts the feature vector of the corresponding local security inspection image; Represents the index of the local security inspection image; represents the data amount of the first image; represents the data amount of the second image; Indicates the A client node extracts the data volume of the corresponding local security inspection image, wherein the first image refers to the local security inspection image with the detection result of the initial global model; and the second image refers to the local security inspection image without the detection result of the initial global model; A terminal node clustering unit, configured to cluster the plurality of client nodes according to the feature descriptors to obtain a clustering result; A global model training unit, used for training the initial global model of the multiple client nodes according to the clustering result, until each client node obtains a security inspection model trained with the initial global model, wherein the initial global model of the multiple client nodes is trained according to the clustering result, and each client node obtains a security inspection model trained with the initial global model, including: Obtaining model parameters corresponding to the multiple client nodes; wherein the model parameters refer to model parameters obtained after the client node uses the initial global model to perform multiple rounds of training on the local security inspection image; Perform intra-cluster model aggregation according to the model parameters of all client nodes in each cluster to obtain a cluster model corresponding to each cluster; Performing inter-cluster model aggregation on the cluster models corresponding to each cluster to obtain a global model for the next number of iterations; The global model of the next iteration number is sent to the multiple client nodes, so that the multiple client nodes start the training of the next iteration number based on the global model of the next iteration number, and iterate in sequence until each client node obtains the security inspection model that completes the training of the initial global model.

8. An electronic device, characterized in that: It includes at least one controller and a memory for communicating with the controller; the memory stores instructions that can be executed by the at least one controller, and the instructions are executed by at least one of the controllers to enable at least one of the controllers to execute the security inspection model upgrade method based on data distribution clustered federated learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the security inspection model upgrading method based on data distribution clustering federated learning as described in any one of claims 1 to 6.

Images