Neural network-oriented unsupervised clustering algorithm and device and electronic device thereof

Abstract

The application discloses a neural network-oriented unsupervised clustering algorithm and device and electronic equipment thereof, and relates to the technical field of networks, and comprises the following steps: obtaining a plurality of neural networks, and selecting a reference network from the neural networks; obtaining the hidden element similarity of the reference network and the neural networks other than the reference network; performing maximum bipartite weight matching layer by layer based on the hidden element similarity of the reference network and the hidden element similarity of the neural networks other than the reference network, to obtain a matching result; and adjusting the weight connection order of each layer of the neural networks other than the reference network according to the matching result. The application greatly improves the clustering accuracy and the performance of the aggregated model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network, and particularly relates to a neural network-oriented unsupervised clustering algorithm and a device thereof. BACKGROUND

[0002] With the continuous development of the Internet of Things, mobile phones, wearable devices and autonomous vehicles all use distributed networks; among them, the devices in the distributed network will generate a large amount of data and generate a variety of task processing needs, which challenges the computing power of the devices.

[0003] In the prior art, collaborative learning participated by multiple clients has emerged, which continuously improves the computing power of the devices, making it possible for the devices to store data locally and complete calculations; among them, collaborative learning, as a learning framework that can train on a large scale on devices that generate data, is widely used; but the devices in the distributed network will generate a large amount of data, and timely processing of the large amount of local data through neural networks is a problem that needs to be solved at present, and using collaborative learning cannot effectively utilize the connection between each client and cannot obtain high clustering accuracy. SUMMARY

[0004] In order to solve the above problems existing in the prior art, the application provides a neural network-oriented unsupervised clustering algorithm and a device thereof. The technical problem to be solved by the application is solved by the following technical scheme:

[0005] In the first aspect, the application provides a neural network-oriented unsupervised clustering algorithm, comprising:

[0006] obtaining a plurality of neural networks and selecting a reference network therefrom;

[0007] obtaining the similarity of hidden elements of the reference network and the neural networks other than the reference network;

[0008] based on the similarity of hidden elements of the reference network and the similarity of hidden elements of the neural networks other than the reference network, performing maximum bipartite weight matching layer by layer to obtain a matching result;

[0009] adjusting the weight connection order of each layer of the neural networks other than the reference network according to the matching result.

[0010] Optionally, obtaining a plurality of neural networks and selecting a reference network therefrom includes two ways, which are: selection by model performance and selection by discriminability;

[0011] The process of selection by model performance includes: respectively adopting different neural networks to classify the same test data, and taking the neural network with the highest classification accuracy as the reference network;

[0012] The process of selecting by the discrimination degree comprises: obtaining variances of the multiple neural networks respectively, and selecting the reference network according to the variances of the neural networks.

[0013] Optionally, the process of performing the maximum bipartite weight matching layer by layer based on the hidden element similarity of the reference network and the hidden element similarity of the neural network other than the reference network to obtain the matching result comprises:

[0014] selecting corresponding layers of the reference network and the neural network other than the reference network, and constructing multiple bipartite graphs;

[0015] obtaining the maximum matching of all the constructed bipartite graphs by using the Hungarian algorithm;

[0016] obtaining the maximum weight matching of all the constructed bipartite graphs by using the KM algorithm based on the maximum matching of all the constructed bipartite graphs;

[0017] obtaining corresponding layers of the reference network and the neural network other than the reference network layer by layer, constructing multiple bipartite graphs for each layer, and obtaining the maximum weight matching of all the constructed bipartite graphs, that is, obtaining the matching result.

[0018] Optionally, the method for obtaining the hidden element similarity of the reference network comprises the Euclidean distance, the Hamming distance or the cosine similarity.

[0019] In a second aspect, the present application further provides a neural network-oriented unsupervised clustering device, comprising:

[0020] a selection module configured to obtain multiple neural networks and select a reference network therefrom;

[0021] an obtaining module configured to obtain hidden element similarities of the reference network and the neural network other than the reference network;

[0022] a matching module configured to perform the maximum bipartite weight matching layer by layer based on the hidden element similarity of the reference network and the hidden element similarity of the neural network other than the reference network to obtain a matching result;

[0023] a weight connection module configured to adjust the weight connection order of each layer of the neural network other than the reference network according to the matching result.

[0024] In a third aspect, the present application further provides an electronic device, comprising: a processor and a machine readable storage medium;

[0025] The machine readable storage medium stores machine executable instructions capable of being executed by the processor;

[0026] The processor is configured to execute the machine executable instructions to implement the algorithm steps in the above embodiments.

[0027] The present application has the following beneficial effects:

[0028] The application provides a neural network-oriented unsupervised clustering algorithm and device and electronic equipment thereof, obtains a reference network from multiple neural networks, obtains the similarity of hidden elements of each layer of the neural network, and performs maximum bipartite weight matching layer by layer to eliminate the influence of the permutation invariance of the hidden elements in the neural network; in addition, for different networks in distributed learning, the matching of the hidden elements of each layer is realized, which greatly improves the clustering accuracy and the performance of the aggregated model.

[0029] The application will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of the neural network-oriented unsupervised clustering algorithm provided by the embodiment of the application;

[0031] Figure 2 is a bipartite graph provided by the embodiment of the application;

[0032] Figure 3 is a structural schematic diagram of the neural network-oriented unsupervised clustering device provided by the embodiment of the application;

[0033] Figure 4 is a structural schematic diagram of the electronic equipment provided by the embodiment of the application. DETAILED DESCRIPTION

[0034] The application will be further described in detail below with reference to the drawings and embodiments, but the embodiments of the application are not limited thereto.

[0035] In the prior art, clustering is a process of dividing a set of physical or abstract objects into multiple classes composed of similar objects, and the cluster generated by clustering is a set of data objects, which are similar to each other in the same cluster and different from the objects in other clusters. Since the client uploads a neural network instead of original training data, how to cluster the super-high-dimensional data with millions or tens of millions of parameters becomes a difficulty. In addition, the relationship between the hidden elements in the neural network (such as the convolution kernel in the convolutional neural network and the hidden state in the long short-term memory network) is parallel, that is, the extraction of the same feature may be responsible for the elements in different positions on different networks, which also hinders the implementation of clustering or aggregation methods.

[0036] Therefore, the application provides a neural network-oriented unsupervised clustering algorithm and device, which designs an unsupervised clustering algorithm for neural networks in distributed learning to better mine the internal relationship between the networks.

[0037] Please refer to Figure 1 , Figure 1A flow chart of the neural network-oriented unsupervised clustering algorithm provided by the embodiment of the present application, the neural network-oriented unsupervised clustering algorithm provided by the present application comprises:

[0038] S101, a plurality of neural networks are acquired, and a reference network is selected from the plurality of neural networks;

[0039] S102, a hidden element similarity of the reference network and a neural network other than the reference network is acquired;

[0040] S103, based on the hidden element similarity of the reference network and the hidden element similarity of the neural network other than the reference network, maximum bipartite weight matching is performed layer by layer to obtain a matching result;

[0041] S104, according to the matching result, a weight connection order of each layer of the neural network other than the reference network is adjusted.

[0042] Specifically, the neural network-oriented unsupervised clustering algorithm provided in the embodiment is designed to solve the defects that the neural network parameters uploaded by the clients in the distributed network are large in quantity and high in dimension, and the hidden elements in different neural networks may have an adverse effect on the extraction of the same feature. The unsupervised clustering algorithm can better mine the internal relationship between networks, eliminate the permutation invariance of the neural network hidden elements, and improve the clustering accuracy and model performance.

[0043] In an alternative embodiment of the present application, acquiring a plurality of neural networks and selecting a reference network from the plurality of neural networks includes two ways, which are model performance selection and degree of differentiation selection;

[0044] The model performance selection process includes: different neural networks are respectively used to classify the same test data, and the neural network with the highest classification accuracy is taken as the reference network;

[0045] The degree of differentiation selection process includes: the variances of the plurality of neural networks are respectively acquired, and the neural network with a suitable variance is selected as the reference network.

[0046] In an alternative embodiment of the present application, please refer to Figure 2 , Figure 2 The process of performing maximum bipartite weight matching layer by layer based on the hidden element similarity of the reference network and the hidden element similarity of the neural network other than the reference network to obtain a matching result includes:

[0047] Selecting corresponding layers of the reference network and the neural network other than the reference network to construct a plurality of bipartite graphs;

[0048] Obtaining the maximum matching of all the constructed bipartite graphs by using the Hungarian algorithm;

[0049] Based on the maximum matching of all bipartite graphs constructed, the maximum weight matching of all bipartite graphs constructed is obtained by using the KM algorithm.

[0050] The corresponding layers of the reference network and the neural network other than the reference network are obtained layer by layer, a plurality of bipartite graphs are constructed for each layer, and the maximum weight matching of all bipartite graphs constructed is obtained, that is, the matching result is obtained.

[0051] It should be noted that, Figure 2 As shown in the figure, the other network is the neural network other than the reference network, and A, B and C are the number of layers of the reference network, and a, b and c are the other neural networks.

[0052] In an optional embodiment of the present application, the method for obtaining the similarity of the hidden elements of the reference network includes the Euclidean distance, the Hamming distance or the cosine similarity.

[0053] Please continue to refer to Figure 1 and Figure 2 As shown in the figure, in an optional embodiment of the present application, the unsupervised clustering algorithm for neural networks provided in the embodiment includes:

[0054] S101, build a selection framework of the reference network, and select the reference network from the multiple neural networks uploaded by the client through the selection framework of the reference network.

[0055] Optionally, the neural network uploaded by the client is generally a convolutional neural network (CNN) or a long short-term memory network (LSTM); wherein the CNN is composed of convolutional layers, pooling layers and fully connected layers, etc., to extract features of a certain thing according to a certain model, and then classify, identify, decide or predict the thing according to the extracted features; LSTM is a class of feedback neural networks (RNN), mainly used for processing time series data, and has a wide range of applications in stock trend prediction, relationship classification in NLP and speech recognition.

[0056] In the embodiment, the reference network is selected by the need of the distributed learning task. According to different needs, the selection criteria are divided into two categories, which are selected by model performance and selected by discrimination.

[0057] Among them, the reference network is selected by the model performance, and the level of the model performance is reflected by the accuracy of the neural network in completing the distributed learning task; for example, multiple clients use CNN for image classification tasks, and the reference network is selected by comparing the accuracy of each client model in classifying the same test data, and the neural network with the highest accuracy is selected as the reference network of the unsupervised neural network clustering algorithm.

[0058] The reference network is selected by the degree of distinction, and the degree of distinction is reflected by the variance of the neural network. In a certain range, the higher the variance, the higher the degree of distinction of the network. For example, a complex classifier with a very deep depth and many hidden nodes is used to fit the data set. The large and deep neural network can fully learn the characteristics of the sample, but if it is not set properly, it will result in over-high variance, leading to misclassification. A suitable variance can be obtained by using regularization or increasing the number of samples. It can be understood that when there is a relatively strict requirement for the neural network classification standard, a network with a suitable variance needs to be selected as the reference network.

[0059] The embodiment selects a reference network from a plurality of neural networks uploaded by a client to eliminate the permutation invariance of the hidden elements of the neural network by feature matching. The selection method of the reference network is not limited in the embodiment, and a person skilled in the art can select it according to the requirements of the distributed learning task.

[0060] S102, the similarity of the hidden elements of the reference network and the neural networks other than the reference network is calculated.

[0061] Optionally, the hidden elements are different according to different user-uploaded networks, for example, the convolution kernel in the convolutional neural network and the hidden state in the long short-term memory network. The relationship between the hidden elements in the neural network is parallel, that is, the extraction of the same feature may be responsible for the elements in different positions on different networks. Therefore, it will hinder the implementation of the clustering or aggregation method.

[0062] The embodiment calculates the similarity of the hidden elements of the reference network and the neural networks other than the reference network. The higher the similarity of the hidden elements, the higher the probability of being responsible for the same feature extraction. The calculated similarity is used to provide a weight value reference for the maximum bipartite matching. Optionally, the method for calculating the similarity includes but is not limited to the Euclidean distance, the Hamming distance or the cosine similarity.

[0063] S103, based on the similarity of the hidden elements of the reference network and the similarity of the hidden elements of the neural networks other than the reference network, the maximum bipartite matching is performed layer by layer to obtain a matching result.

[0064] S1031, the corresponding layers of the reference network and the neural networks other than the reference network are selected to construct a plurality of bipartite graphs.

[0065] The benchmark network is one of the n neural networks uploaded by the client selected through the benchmark network selection framework. According to the correspondence between the hidden elements of the corresponding layers of the benchmark network and the neural networks other than the benchmark network and the extracted features, n-1 bipartite graphs are constructed. The hidden elements of the corresponding layers of the benchmark network and the neural networks other than the benchmark network are selected to construct a bipartite graph. In the embodiment, the threshold of the similarity is set to 0.6. If the similarity between two hidden elements is greater than or equal to 0.6, the extracted features of the two hidden elements are consistent or related. If the similarity between two hidden elements is less than 0.6, the extracted features of the two hidden elements are inconsistent, and the two hidden elements cannot be connected. The threshold of the connection between the bipartite graphs is set by a person skilled in the art according to business needs, which is not limited in the embodiment.

[0066] S1032, obtaining the maximum matching of all the constructed bipartite graphs by using the Hungarian algorithm.

[0067] The matching is a set of edges (i.e. the connection in the above step), wherein any two edges do not have a common vertex. The maximum matching refers to the matching with the maximum number of matching edges in all matchings, wherein the maximum matching involves two concepts, i.e. the alternating path and the augmenting path. The path formed by successively passing through the non-matching edge, the matching edge and the non-matching edge from a matching point is the alternating path. The alternating path is the augmenting path if another non-matching point (not the starting point) is passed through.

[0068] In the embodiment, the Hungarian algorithm is used to find the augmenting path of the constructed bipartite graph, and after multiple iterations, the matching with the maximum number of matching edges is finally found. The maximum matching result is the premise of obtaining the maximum bipartite weight matching by using the KM algorithm.

[0069] S1033, obtaining the maximum weight matching of all the constructed bipartite graphs by using the KM algorithm based on the maximum matching of all the constructed bipartite graphs.

[0070] To facilitate the use of the KM algorithm for processing, the similarity value of the hidden elements obtained in the above step S102 is expanded. The expansion multiple is determined according to the actual situation, which is not limited in the application.

[0071] The embodiment utilizes the KM algorithm to increase the edge weight on the bipartite graph on the basis of the maximum matching obtained in the step S1032, and obtains the maximum weight matching by calculating the maximum value of the sum of the matching edge weights; the point weight of the left point is set as the maximum edge weight, and the right point edge weight is set as 0, when the sum of the left and right point edge weights is equal to the edge weight of the edge connecting the two points, the matching can be performed; each time according to the Hungarian matching, when a conflict occurs, the left point edge weight is-1 and the right point edge weight is+1 when the last point has no matching edge, and the matching is performed again, if the matching cannot be performed, the left point edge weight is-1 and the right point edge weight is+1 until the matching is successful.

[0072] S1034, repeating the steps S1031-S1033 layer by layer until the matching of the last corresponding layer of the reference network and the neural network other than the reference network is completed, and obtaining the matching result.

[0073] The maximum weight matching is found by layer-by-layer KM algorithm, the overall structure of the neural network other than the reference network is adjusted, and the overall matching of the neural network other than the reference network and the reference network is realized.

[0074] In the embodiment, the similarity is calculated by calculating the hidden elements in the neural network, and the layer-by-layer maximum bipartite weight matching is performed by using the KM algorithm, the neural network structure other than the reference network is adjusted to match the reference network, that is, other structures are adjusted to better meet the distributed learning task requirements, so that the clustering accuracy and the model performance after aggregation can be greatly improved.

[0075] S104, according to the matching result, adjusting the weight connection order of each layer of the neural network other than the reference network.

[0076] According to the result of the maximum weight matching, the weight connection order of the neural network other than the reference network is adjusted, the influence of the hidden element permutation invariance in the neural network is eliminated, and the clustering accuracy and the model performance after aggregation can be greatly improved.

[0077] Through the above steps, the embodiment can obtain each neural network uploaded by the client by establishing a complete set of unsupervised neural network clustering algorithm rules in the distributed learning scenario, select the reference network according to the needs by the reference network selection framework, calculate the similarity of the hidden elements of the neural network, and perform layer-by-layer maximum bipartite weight matching by using the KM algorithm, so as to eliminate the influence of the hidden element permutation invariance in the neural network, thereby greatly improving the clustering accuracy and the model performance after aggregation.

[0078] Based on the same inventive concept, please refer to Figure 3 , Figure 3is a structural schematic diagram of a neural network-oriented unsupervised clustering device provided by an embodiment of the present application, and the present application further provides a neural network-oriented unsupervised clustering device, which is applied to a neural network-oriented unsupervised clustering algorithm provided by the above-mentioned embodiment of the present application, and the algorithm can be referred to the above-mentioned embodiment, which will not be repeated here; the device comprises:

[0079] The selection module 201 is configured to acquire a plurality of neural networks and select a reference network therefrom.

[0080] The acquisition module 202 is configured to acquire the similarity of the hidden elements of the reference network and the neural networks other than the reference network.

[0081] The matching module 203 is configured to perform maximum bipartite weight matching layer by layer based on the similarity of the hidden elements of the reference network and the similarity of the hidden elements of the neural networks other than the reference network, to obtain a matching result.

[0082] The weight connection module 204 is configured to adjust the weight connection order of each layer of the neural networks other than the reference network according to the matching result.

[0083] Specifically, the neural network-oriented unsupervised clustering device provided by the embodiment can face the scenario of increasing distributed learning, select a reference network according to task requirements, calculate the similarity of the hidden elements of the model, and perform layer-by-layer maximum bipartite weight distribution by using the KM algorithm, so that the influence of the invariance of the replacement of the hidden elements in the neural network is greatly eliminated, and the clustering accuracy and the performance of the aggregated model can be greatly improved by using the unsupervised neural network clustering algorithm.

[0084] Based on the same inventive concept, please refer to Figure 4 , Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application, and the present application further provides an electronic device, which comprises a processor 301 and a machine readable storage medium 303.

[0085] The machine readable storage medium 303 stores machine executable instructions capable of being executed by the processor 301.

[0086] The processor 301 is configured to execute the machine executable instructions to implement the neural network-oriented unsupervised clustering algorithm provided by the above-mentioned embodiment.

[0087] Specifically, please continue to refer to Figure 4As shown, the electronic device provided in this embodiment further includes a communication interface 302 and a communication bus 304, wherein the processor 301, the communication interface 302, and the machine-readable storage medium 303 realize communication with each other through the communication bus 304. The processor 301 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 301 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. Optionally, the processor 301 can be integrated with a graphics processor (GPU) that is responsible for rendering and drawing content to be displayed on the display screen.

[0088] The machine-readable storage medium 303 can include one or more computer-readable storage media, which can be non-transitory. The machine-readable storage medium 303 can further include a high-speed random access memory and a non-volatile memory such as one or more disk storage devices, flash storage devices. Optionally, the machine-readable storage medium 303 is at least used to store a computer program, wherein the computer program, after being loaded and executed by the processor, can implement the unsupervised neural network clustering algorithm provided in the above-described embodiments. In addition, the resources stored by the machine-readable storage medium 303 can further include an operating system and data, etc. The storage mode can be temporary storage or permanent storage. The operating system can include Windows, Unix, Linux, etc. The data can include but is not limited to data involved in the network behavior analysis and situation awareness method for dynamic data, etc.

[0089] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flow diagrams and / or block diagrams, and combinations of flows and / or blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0090] Furthermore, these computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks. Figure 1 one or more flows and / or blocks.

[0092] The present application provides a neural network-oriented unsupervised clustering algorithm and device, and an electronic device thereof. The neural network-oriented unsupervised clustering algorithm and device obtain a reference network from a plurality of neural networks, and obtain similarity of hidden elements of each layer of the neural network, and perform maximum bipartite weight matching layer by layer, thereby eliminating the influence of permutation invariance of the hidden elements in the neural network. In addition, for different networks in distributed learning, the matching of the hidden elements of each layer is implemented, thereby greatly improving the clustering accuracy and the performance of the aggregated model.

[0093] The above content is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those of ordinary skill in the art to which the present application belongs, without departing from the concept of the present application, a number of simple derivations or substitutions can be made, and all of them should be regarded as falling within the protection scope of the present application.

Claims

1. An unsupervised clustering apparatus, characterized by, The application is applied to an Internet of Things distributed collaborative learning scene, and comprises the following: A selection module is configured to build a selection framework of a benchmark network, and select the benchmark network from a plurality of neural networks uploaded by clients through the selection framework of the benchmark network; the plurality of clients perform image classification tasks using CNNs, and select the benchmark network by comparing the accuracy of the classification of the same test data by the models of the clients, and select the neural network with the highest accuracy as the benchmark network of an unsupervised neural network clustering algorithm; A obtaining module is configured to obtain the similarity of hidden elements of the benchmark network and neural networks other than the benchmark network; the similarity of the hidden elements of the benchmark network and the neural networks other than the benchmark network is calculated, the higher the similarity of the hidden elements, the higher the probability of being responsible for the same feature extraction, and the calculated similarity is used to provide a weight value reference for bipartite maximum weight matching; the method for calculating the similarity comprises Euclidean distance, Hamming distance or cosine similarity; A matching module is configured to perform maximum bipartite weight matching layer by layer based on the similarity of the hidden elements of the benchmark network and the similarity of the hidden elements of the neural networks other than the benchmark network, and obtain a matching result; the matching module comprises the following steps: selecting corresponding layers of the benchmark network and the neural networks other than the benchmark network, constructing a plurality of bipartite graphs, obtaining the maximum matching of all the constructed bipartite graphs by using the Hungarian algorithm, obtaining the maximum weight matching of all the constructed bipartite graphs by using the KM algorithm based on the maximum matching of all the constructed bipartite graphs, and obtaining the corresponding layers of the benchmark network and the neural networks other than the benchmark network layer by layer, constructing a plurality of bipartite graphs for each layer, and obtaining the maximum weight matching of all the constructed bipartite graphs, thereby obtaining the matching result; A weight connection module is configured to adjust the weight connection order of each layer of the neural networks other than the benchmark network according to the matching result.

2. An unsupervised clustering algorithm applied to an Internet of Things distributed collaborative learning scenario, characterized in that, The application comprises the following: A plurality of neural networks are obtained, and a benchmark network is selected from the plurality of neural networks; The benchmark network is selected from a plurality of neural networks uploaded by clients through a selection framework of the benchmark network; The plurality of clients perform image classification tasks using CNNs, and select the benchmark network by comparing the accuracy of the classification of the same test data by the models of the clients, and select the neural network with the highest accuracy as the benchmark network of an unsupervised neural network clustering algorithm; The similarity of hidden elements of the benchmark network and neural networks other than the benchmark network is obtained; the similarity of the hidden elements of the benchmark network and the neural networks other than the benchmark network is calculated, the higher the similarity of the hidden elements, the higher the probability of being responsible for the same feature extraction, and the calculated similarity is used to provide a weight value reference for bipartite maximum weight matching; the method for calculating the similarity comprises Euclidean distance, Hamming distance or cosine similarity; The maximum bipartite weight matching is obtained layer by layer based on the implicit element similarity of the reference network and the implicit element similarity of the neural network other than the reference network, including: selecting the corresponding layers of the reference network and the neural network other than the reference network, constructing multiple bipartite graphs; obtaining the maximum matching of all the constructed bipartite graphs by using the Hungarian algorithm; obtaining the maximum weight matching of all the constructed bipartite graphs by using the KM algorithm based on the maximum matching of all the constructed bipartite graphs; obtaining the corresponding layers of the reference network and the neural network other than the reference network layer by layer, constructing multiple bipartite graphs at each layer, and obtaining the maximum weight matching of all the constructed bipartite graphs, that is, obtaining the matching result; According to the matching result, adjusting the weight connection order of each layer of the neural network other than the reference network.

3. The unsupervised clustering algorithm of claim 2, wherein, The obtaining of the multiple neural networks and the selection of the reference network therefrom includes two modes, which are: selection by model performance and selection by discriminability. The process of selection by model performance includes: respectively adopting different neural networks to classify the same test data, and taking the neural network with the highest classification accuracy as the reference network. The process of selection by discriminability includes: respectively obtaining the variances of the multiple neural networks, and selecting the reference network according to the variances of the neural networks.

4. An electronic device, comprising: It includes: a processor and a machine readable storage medium; The machine readable storage medium stores machine executable instructions that can be executed by the processor; The processor is configured to execute the machine executable instructions to implement the algorithm steps of any one of claims 2-3.

Images