A management and control method and device for deep learning model

Abstract

The present invention discloses a management and control method and device for a deep learning model, wherein the method comprises: receiving task information and a configuration file input by a user, training a corresponding deep learning model according to the task information and the configuration file, and generating a model file; integrating the model file and its related model information to generate a model package; optimizing the model package based on pruning and knowledge distillation, and generating a version iteration graph of the model package based on a snapshot technology; extracting key features and version information from the optimized model package, generating a label list according to the key features and the version information, associating and storing the label list and the model package, and establishing a multi-label index mechanism; receiving a query condition sent by a user, searching based on the multi-label index mechanism according to the query condition, finding a corresponding model package for deployment, and recording application information of the corresponding model package. The method can effectively manage the deep learning model and facilitate retrieval.

Description

Technical Field

[0001] The present invention relates to the field of network management technology, and in particular to a management and control method and device for a deep learning model. Background Art

[0002] In the context of the development of modern power grid systems, the continuous improvement of the level of intelligence and the widespread integration of renewable energy have made the complexity of power grid flow management increasingly complex. Traditional power grid flow management methods mainly rely on preset rules and manual intervention, which face many challenges when processing large-scale, multi-dimensional data. In order to meet these challenges, the introduction of deep learning technology has become a key turning point in improving the level of intelligence in power grid flow management. As an important branch of artificial intelligence, deep learning can learn and identify complex patterns and features from massive data by simulating the mechanism of human brain processing information. This ability is particularly important for power grid flow analysis because it can help predict and optimize the operating status of the power grid and improve the stability and efficiency of the power grid. However, the training and deployment of deep learning models is a complex process involving large amounts of data processing, network structure selection, and hyperparameter adjustment. As the power grid environment continues to change, the model needs to be updated regularly to maintain its accuracy and effectiveness.

[0003] The natural requirements of the power grid, such as real-time, large data processing, high complexity, security, and scalability, require deep learning models to be able to adapt to rapidly changing environments and be updated frequently. However, since the update of deep learning models usually requires a lot of manual knowledge input, frequent updates are often unrealistic in actual operations.

[0004] Patent document CN110554995A discloses a deep learning model management method and system, which selects a deep learning model framework from a server, modifies the hyperparameters of the deep learning network according to actual needs; creates a data set and inputs it as a training sample into the modified deep learning network for training; renames the trained deep learning model and training data set according to the established naming rules and uploads them to the server for storage; selects a specified deep learning model from the server to evaluate various indicators of the predicted samples. However, this method only renames the trained deep learning model according to the established naming rules and then stores it, and cannot effectively manage the deep learning model. Summary of the invention

[0005] The present invention provides a deep learning model management and control method and device, which can effectively manage the deep learning model and facilitate retrieval.

[0006] A management and control method for a deep learning model, comprising:

[0007] Receive task information and configuration files input by the user, train the corresponding deep learning model according to the task information and configuration files, and generate a model file;

[0008] Integrate the model files and related model information to generate a model package;

[0009] Optimizing the model package based on pruning and knowledge distillation, and generating a version iteration graph of the model package based on snapshot technology;

[0010] Extracting key features and version information from the optimized model package, generating a tag list based on the key features and version information, associating and storing the tag list with the model package, and establishing a multi-tag indexing mechanism;

[0011] Receive the query conditions sent by the user, perform retrieval based on the multi-label index mechanism according to the query conditions, find the corresponding model package for deployment, and record the application information of the corresponding model package.

[0012] Furthermore, the task information includes task description, task type, associated target level and traffic level; the configuration file includes model settings, version selection, model structure and model hyperparameters.

[0013] Further, the model package is optimized based on pruning, including:

[0014] Taking the filter of the model file in the model package as a point in the Euclidean space;

[0015] Calculate the Euclidean distance of filters in the same layer and find the geometric center of all filters in the same layer;

[0016] Calculate the distance between each filter and the geometric center, take the filter whose distance is less than a preset distance as the pruning target filter, and set the pruning target filter to 0;

[0017] The model package is optimized based on knowledge distillation, including:

[0018] Use the pruned model file as the teacher network and set up the student network;

[0019] Inputting the training samples into the teacher network to obtain the teacher network output result;

[0020] The output result of the teacher network is used as a soft label, the true label of the training sample is used as a hard label, the soft label and the hard label are input into the student network for training, the loss function is calculated and the parameters of the student network are updated during the training process until the student network converges, and the converged student network is used as the optimized model file.

[0021] Furthermore, the version iteration graph includes snapshots of various versions of the model file, and the snapshots include checkpoints during the iteration process of the model file.

[0022] Furthermore, the tag list includes a plurality of tags, and the tags include a separator flag, a tag type flag, a tag length flag, and a type identification value flag;

[0023] Establish a multi-label indexing mechanism, including:

[0024] Combining multiple labels of the model file to obtain a label data vector;

[0025] Dividing the label data vector according to the separator to obtain a plurality of inverted list vectors;

[0026] Clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters;

[0027] Utilizing a multi-constraint balanced clustering algorithm to optimize the obtained multiple clusters, and obtain multiple lists;

[0028] Create an index list based on the partitioned list.

[0029] Furthermore, a hierarchical clustering algorithm is used to cluster the multiple inverted list vectors to obtain multiple clusters, including:

[0030] Randomly select a preset number of inverted list vectors as initial cluster centers;

[0031] Calculate the Euclidean distance between other inverted list vectors and the initial cluster center, and search for the inverted list vector closest to the initial cluster center according to the Euclidean distance to merge them to form a cluster;

[0032] Iteratively perform the following steps until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster;

[0033] If the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is allocated to the corresponding multiple clusters.

[0034] Furthermore, the obtained multiple clusters are optimized using a multi-constraint balanced clustering algorithm to obtain multiple lists, including:

[0035] Based on the obtained clusters, an objective function on the variance of the length of the inverted list vector is established;

[0036] Optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list;

[0037] The objective function is as follows:

[0038] ;

[0039] Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, Represents the inverted list vector length variance.

[0040] Further, searching based on the multi-label index mechanism according to the query condition includes:

[0041] Generate a query condition vector according to the query condition;

[0042] Calculating the distance between the query condition vector and the centroid of each list in the index list, and if the distance meets a preset condition, using the corresponding list as the search list;

[0043] Matching the query condition vector with the search list to obtain a corresponding model package;

[0044] The preset conditions are:

[0045] ;

[0046] q represents the query condition vector, X ij Represents the list of distances to be calculated, c ij Represents list X ij The center of mass, represents the hyperparameter, Dist(q,c ij ) represents the query condition vector q and the centroid c ij The distance Represents the list X to be calculated by the query condition vector q ij Perform a search query.

[0047] A management and control device for a deep learning model, comprising:

[0048] A training module is used to receive task information and configuration files input by a user, train a corresponding deep learning model according to the task information and configuration files, and generate a model file;

[0049] An integration module, used to integrate the model files and related model information to generate a model package;

[0050] An optimization module optimizes the model package based on pruning and knowledge distillation, and generates a version iteration graph of the model package based on snapshot technology;

[0051] An indexing module, used to extract key features and version information from the optimized model package, generate a tag list based on the key features and version information, associate and store the tag list with the model package, and establish a multi-tag indexing mechanism;

[0052] The deployment module is used to receive the query conditions sent by the user, perform retrieval based on the multi-label index mechanism according to the query conditions, find the corresponding model package for deployment, and record the application information of the corresponding model package.

[0053] Furthermore, the task information includes task description, task type, associated target level and traffic level; the configuration file includes model settings, version selection, model structure and model hyperparameters.

[0054] Furthermore, the optimization module optimizes the model package based on pruning, including:

[0055] Taking the filter of the model file in the model package as a point in the Euclidean space;

[0056] Calculate the Euclidean distance of filters in the same layer and find the geometric center of all filters in the same layer;

[0057] Calculate the distance between each filter and the geometric center, take the filter whose distance is less than a preset distance as the pruning target filter, and set the pruning target filter to 0;

[0058] The optimization module optimizes the model package based on knowledge distillation, including:

[0059] Use the pruned model file as the teacher network and set up the student network;

[0060] Inputting the training samples into the teacher network to obtain the teacher network output result;

[0061] The output result of the teacher network is used as a soft label, the true label of the training sample is used as a hard label, the soft label and the hard label are input into the student network for training, the loss function is calculated and the parameters of the student network are updated during the training process until the student network converges, and the converged student network is used as the optimized model file.

[0062] Furthermore, the version iteration graph includes snapshots of various versions of the model file, and the snapshots include checkpoints during the iteration process of the model file.

[0063] Furthermore, the tag list includes a plurality of tags, and the tags include a separator flag, a tag type flag, a tag length flag, and a type identification value flag;

[0064] The index module establishes a multi-label index mechanism, including:

[0065] Combining multiple labels of the model file to obtain a label data vector;

[0066] Dividing the label data vector according to the separator to obtain a plurality of inverted list vectors;

[0067] Clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters;

[0068] Utilizing a multi-constraint balanced clustering algorithm to optimize the obtained multiple clusters, and obtain multiple lists;

[0069] Create an index list based on the partitioned list.

[0070] Furthermore, the index module clusters the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters, including:

[0071] Randomly select a preset number of inverted list vectors as initial cluster centers;

[0072] Calculate the Euclidean distance between other inverted list vectors and the initial cluster center, and search for the inverted list vector closest to the initial cluster center according to the Euclidean distance to merge them to form a cluster;

[0073] Iteratively perform the following steps until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster;

[0074] If the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is allocated to the corresponding multiple clusters.

[0075] Furthermore, the index module optimizes the obtained multiple clusters using a multi-constraint balanced clustering algorithm to obtain multiple lists, including:

[0076] Based on the obtained clusters, an objective function on the variance of the length of the inverted list vector is established;

[0077] Optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list;

[0078] The objective function is as follows:

[0079] ;

[0080] Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, Represents the inverted list vector length variance.

[0081] Furthermore, the deployment module performs retrieval based on the multi-label index mechanism according to the query condition, including:

[0082] Generate a query condition vector according to the query condition;

[0083] Calculating the distance between the query condition vector and the centroid of each list in the index list, and if the distance meets a preset condition, using the corresponding list as the search list;

[0084] Matching the query condition vector with the search list to obtain a corresponding model package;

[0085] The preset conditions are:

[0086] ;

[0087] q represents the query condition vector, X ij Represents the list of distances to be calculated, c ij Represents list X ij The center of mass, represents the hyperparameter, Dist(q,c ij ) represents the query condition vector q and the centroid c ij The distance Represents the list X to be calculated by the query condition vector q ij Perform a search query.

[0088] A computer storage medium stores computer instructions, which are used to execute the above method when called.

[0089] The management and control method and device of the deep learning model provided by the present invention have at least the following beneficial effects:

[0090] (1) The efficiency of power grid operation and maintenance has been significantly improved through the automated task distribution and configuration process. Operation and maintenance personnel can quickly set task parameters and select the most suitable model for deployment based on real-time network conditions and analysis requirements. This flexibility not only reduces the time cost of manual operation, but also allows the system to respond quickly to emergencies, train in real time or call pre-trained models to ensure the stable operation of the power grid.

[0091] (2) The model training phase uses snapshot technology to maintain the historical version of the model and perform iterative updates, which not only ensures the integrity and consistency of the model data, but also provides a basis for the continuous optimization of the model. With the accumulation of power grid data and the advancement of technology, the model can be continuously updated and optimized, thereby improving the accuracy and efficiency of the analysis;

[0092] (3) The multi-label retrieval mechanism greatly improves the intelligence and accuracy of model retrieval. Users can quickly locate the specific model version they need through multi-dimensional label combinations. This method not only improves retrieval efficiency but also reduces the risk of incorrect deployment. Accurate model deployment is crucial for traffic analysis of power grid equipment, which directly affects the success rate of analysis tasks and the safe operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 A flowchart of an embodiment of the management and control method of the deep learning model provided by the present invention.

[0094] Figure 2 A schematic diagram of an embodiment of the management and control method of the deep learning model provided by the present invention.

[0095] Figure 3 A flowchart of an embodiment of model optimization in the management and control method of the deep learning model provided by the present invention.

[0096] Figure 4 A flowchart of an embodiment of a multi-label indexing mechanism in the management and control method of the deep learning model provided by the present invention.

[0097] Figure 5 A flowchart of an embodiment of searching according to query conditions in the management and control method of the deep learning model provided by the present invention.

[0098] Figure 6 A schematic structural diagram of an embodiment of a management and control device for a deep learning model provided by the present invention. DETAILED DESCRIPTION

[0099] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0100] refer to Figure 1 and Figure 2 In some embodiments, a deep learning model management and control method is provided, including:

[0101] S1. Receive task information and configuration files input by a user, train a corresponding deep learning model according to the task information and configuration files, and generate a model file;

[0102] S2, integrating the model file and its related model information to generate a model package;

[0103] S3, optimizing the model package based on pruning and knowledge distillation, and generating a version iteration graph of the model package based on snapshot technology;

[0104] S4, extracting key features and version information from the optimized model package, generating a tag list according to the key features and version information, associating and storing the tag list with the model package, and establishing a multi-tag indexing mechanism;

[0105] S5. Receive the query conditions sent by the user, perform a search based on the multi-label indexing mechanism according to the query conditions, find the corresponding model package for deployment, and record the application information of the corresponding model package.

[0106] Further, in step S1, the power grid operation and maintenance personnel log in to the system client and select the network traffic analysis task to be performed. Determine the specific requirements of the task, such as traffic type, time window, target device, etc. Enter the task name and description so that other operation and maintenance personnel can understand the purpose of the task and specify the power grid equipment or area to be analyzed. Then configure the task parameters, such as data sampling rate, feature extraction method, model selection, etc., and generate a configuration file. According to the nature of the task, decide whether to train a new model in real time or select a trained model from the model warehouse for deployment.

[0107] In the model training phase of the system server, the model parameters, such as model architecture, hyperparameters, optimizer, etc., are accurately set according to the provided task information and configuration files, and the model training process is started according to the predetermined training settings.

[0108] In addition, the system server is responsible for the following four core functions:

[0109] Model initialization (INIT): When the system server starts, the model is initialized, including loading the initial model file or creating a blank model. After initialization, the system can start other operations, such as training, deployment, or version control.

[0110] Model Addition (ADD): When you need to add a new model version, perform the model addition operation. When you need to use new training data or need to try different hyperparameters, select the model architecture to add. Train or fine-tune the new data and update the model parameters.

[0111] Model submission (CMT): After model training or updating is completed, perform the model submission operation to save the current version of the model to the model warehouse. The model warehouse will generate a model file that records the model's version information, training parameters, and performance indicators.

[0112] Model Copy (COPY): When you need to create a copy that is similar to an existing model, perform a model copy operation. Copy the model file prototype, update the copy's version information to distinguish different model instances, and store the copy in the model hosting warehouse for backup, parallel deployment, or different experiments.

[0113] In step S2, after the training is completed, the system integrates the automatically generated model file with the relevant model information to form a complete model package model_p, which can be stored in the hosting warehouse for future use or directly used in the deployment of the current system to deal with various situations occurring in the power grid equipment flow.

[0114] The task information includes task description, task type, associated target level and traffic level; the configuration file includes model settings, version selection, model structure and model hyperparameters.

[0115] Furthermore, in step S3, the generated model package model_p is parsed to extract the model structure, weight parameters, version information, etc., and analyze the model size, computational complexity, memory usage and other indicators for optimization. The warehouse system will perform pruning optimization and knowledge distillation on the model package model_p. The warehouse system selects appropriate pruning strategies, such as channel pruning and weight pruning, based on the performance requirements of the model package, to make model_p smaller and less computationally intensive, while still maintaining certain performance. The warehouse system uses the knowledge distillation method to use the already trained large model as the knowledge source, and uses the prediction results to train the pruned small model, so that it has a smaller size and computational complexity. model_p will be The format stores and manages the model package. Each model has a name (Name) and a unique version number (Id). N is the model structure level, which describes the model's hierarchy and connection method. W is the model parameter, including the model's weight and bias. The metadata source M contains device information related to the model, etc. The model configuration F records the model's hyperparameters and other configuration options. The key weight parameters and different versions of the model will be updated, iterated and stored through snapshot technology to ensure the integrity and consistency of the model data.

[0116] Further, refer to Figure 3 In step S3, the model package is optimized based on pruning, including:

[0117] S31, taking the filter of the model file in the model package as a point in the Euclidean space;

[0118] S32, calculating the Euclidean distance of filters in the same layer, and finding the geometric center of all filters in the same layer;

[0119] S33, calculating the distance between each filter and the geometric center, taking the filter whose distance is less than the preset distance as the pruning target filter, and setting the pruning target filter to 0;

[0120] The model package is optimized based on knowledge distillation, including:

[0121] S34, using the pruned model file as the teacher network and setting the student network;

[0122] S35, inputting the training sample into the teacher network to obtain the teacher network output result;

[0123] S36. Use the output result of the teacher network as a soft label, use the true label of the training sample as a hard label, input the soft label and the hard label into the student network for training, calculate the loss function and update the parameters of the student network during the training process until the student network converges, and use the converged student network as the optimized model file.

[0124] Specifically, in step S31, the filters of the model files in the model package are taken as points in the Euclidean space, assuming that these points are ,and .

[0125] In step S32, the geometric center of the filter is calculated. The geometric center is an estimate of the center of a point in Euclidean space. Make the objective function The sum of the Euclidean distances is minimal:

[0126] ; (1)

[0127] Where x* represents the geometric center, R d represents the set of all filter points, x represents the point to be calculated, a (i) Represents the points of the filter.

[0128] Further, in step S33, the geometric center is used as a common attribute of the filter. If a filter is close to the geometric center, that is, the distance between the filter and the geometric center is less than a preset distance, it is considered that the filter overlaps with other filter information, and its function can be replaced by other filters. The pruning target filter satisfies the following equation:

[0129] ; (2)

[0130] in, is the pruning target filter, x represents the weight vector, It is composed of overlapping filters, F i,j Represents the weights of other filters associated with filter Fi.

[0131] Filter pruning methods based on geometric centers do not rely on canonical size assumptions for filters, but instead identify potentially overlapping or redundant information by computing the centers of filters.

[0132] Furthermore, in step S34, the pruned model file is used as the teacher network, and the lightweight model selected according to the task is used as the student network.

[0133] In step S35, the training sample is input into the teacher network to obtain the teacher network output result, and the output of the teacher network is set to q.

[0134] In step S36, the output result q of the teacher network is used as a soft label, and the real label of the training sample is used as a hard label. The soft label and the hard label are input into the student network for training. During the training process, the loss function is calculated and the parameters of the student network are updated. The output result p of the student network is obtained by learning the difference between the output p of the teacher network and its output result q, so that its output result p is close to the output result q of the teacher network until the student network converges. The converged student network is used as the optimized model file.

[0135] Among them, the loss function is:

[0136] ; (3)

[0137] Among them, CE represents cross entropy, y is the unique hot encoding of the true label, and is a hyperparameter used to control the weight of cross entropy.

[0138] In some embodiments, the distillation temperature T is used in knowledge distillation to adjust the smoothness of the soft label to make it closer to the true label. In order for the student network to better learn similar information in the teacher network, the softmax function of the distillation process is improved:

[0139] In some embodiments, the distillation temperature T is used in knowledge distillation to adjust the smoothness of the soft label to make it closer to the true label. In order for the student network to better learn similar information in the teacher network, the softmax function of the distillation process is improved:

[0140] ; (4)

[0141] in, It is the object of students' online learning. It is the output logits before the softmax layer of the teacher network. When the distillation temperature T is 1, the formula is the original softmax function. When T approaches 0, the function is similar to one-hot encoding. When the distillation temperature T is large, the output result distribution will be smoother, retaining the similar information between the data.

[0142] The loss function in knowledge distillation includes distillation loss and student loss. Distillation loss measures the difference between the soft labels of the student network and the soft labels of the teacher network. Student loss measures the difference between the hard labels of the student network and the true labels. During the training process of the student network, the weight of the student network is updated according to the feedback of the loss function. Through continuous iteration, the student network gradually learns the knowledge of the teacher network and achieves similar performance to the teacher network.

[0143] In the process of knowledge distillation, it is necessary to assign weights to the teacher network and the real sample labels. By introducing a third network C, the weights of the teacher network T and the student network S are considered. The knowledge distillation of the third network C is performed with the teacher network T and the student network S as the new teacher network, and the improvement values ​​of the teacher network T and the student network S are obtained respectively. and , we can get the weights assigned to the teacher network T and the student network S and :

[0144] ; (5)

[0145] In the process of knowledge distillation, the hyperparameters and Applied to the loss function to balance the impact of the teacher network output and the true label on the student network training.

[0146] Furthermore, in step S3, the version iteration graph includes snapshots of each version of the model file, and the snapshots include checkpoints during the iteration process of the model file. Specifically, each model file has its predecessor version and successor version, which together constitute the version iteration graph. The user is responsible for defining each version of the model, where Represents the model file For users in the model file The updated version of the model is written later, and the model file A series of snapshots These snapshots represent checkpoints during the training process. Treat as Version The latest snapshot of express.

[0147] For example, v i Represents model file A, and model file A has three parameters a, b, and c. After some training operations, it becomes model file B, and this model file B is the latest version of the model file v j When training a model, you may design multiple parameter changes. When parameter a in model file A is changed, you may get an intermediate product or an intermediate model. Since the model training iteration time is sometimes very long, the system will take a snapshot here and assume it is snapshot s1. When snapshot s1 continues to iterate, parameters b and c change and then checkpoint records are made, and snapshots s2 and s3 are obtained. Therefore, snapshots s1, s2, and s3 are model file v. i , where snapshot s3 is the model file v i The latest snapshot of .

[0148] Further, in step S4, the tag list includes multiple tags, and the tags include a separator flag, a tag type flag, a tag length flag, and a type identification value flag.

[0149] Different models have multiple labels and custom features according to different task requirements. In order to unify the description of labels of different models, a unified description of labels driven by model tasks is proposed.

[0150] The tag is divided into four types of flags: D character (separator): The D character is used to separate different contents in the tag and occupies 1 byte; T character (tag type): The T character is user-defined and indicates the type of the tag and occupies 1 byte; L character (tag length): The L character indicates the length of the tag and occupies 1 byte; V character (type identification value): The V character indicates the type identification value under the current tag type, and its memory usage is dynamic.

[0151] Each label that the user adds to the model can be converted into the above-mentioned form of label, and the collection of multiple labels represents the label list of the model file.

[0152] Specifically, refer to Figure 4 In step S4, a multi-label indexing mechanism is established, including:

[0153] S41, combining multiple labels of the model file to obtain a label data vector;

[0154] S42, dividing the label data vector according to the separator to obtain a plurality of inverted list vectors;

[0155] S43, clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters;

[0156] S44, optimizing the obtained multiple clusters using a multi-constraint balanced clustering algorithm to obtain multiple lists;

[0157] S45. Create an index list based on the divided list.

[0158] In step S41, the model file contains a large amount of label information when it is set, which is uniformly described by the above-mentioned description method, and the label of each model is combined to obtain the label data vector X.

[0159] Furthermore, in step S42, the label data vector X is divided according to the separator D to obtain a plurality of inverted list vectors.

[0160] Furthermore, in step S43, a hierarchical clustering algorithm is used to cluster the multiple inverted list vectors to obtain multiple clusters, including:

[0161] S431, randomly selecting a preset number of inverted list vectors as initial cluster centers;

[0162] S432, calculating the Euclidean distances between other inverted list vectors and the initial cluster center, searching for the inverted list vector closest to the initial cluster center according to the Euclidean distance, merging them, and forming a cluster;

[0163] S433, iteratively execute step S433 until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster.

[0164] Among them, if the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is assigned to the corresponding multiple clusters. Specifically, if the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a very small preset range (almost equal), then such inverted list vectors are boundary vectors. In order to solve the data boundary problem, a closure multi-cluster allocation strategy is adopted to improve the visibility of vectors at the boundary of the list. If a certain inverted list vector has almost the same distance from some clusters, it is allowed to be assigned to multiple nearest clusters instead of just the nearest one:

[0165] ; (6)

[0166] Among them, Dist is the distance calculation function, represents the centroid of the cluster, is a hyperparameter used to expand the list, and x represents the boundary vector.

[0167] With multi-cluster assignment, boundary vectors are assigned to multiple clusters, increasing the probability that they will be called back during the search. These boundary vectors will be called back if any recent posting list is searched. This approach helps improve the efficiency of approximate nearest neighbor search because it reduces the risk of missing true neighbors due to boundary issues.

[0168] Furthermore, in step S44, the obtained multiple clusters are optimized using a multi-constraint balanced clustering algorithm to obtain multiple lists, including:

[0169] S441, based on the obtained clusters, establishing an objective function on the variance of the length of the inverted list vector;

[0170] S442, optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list;

[0171] The objective function is as follows:

[0172] ;

[0173] Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, Represents the inverted list vector length variance.

[0174] The multi-constrained balanced clustering algorithm reduces the number of disk accesses by limiting the length of each list, thereby quickly loading in memory. In order to balance the list length, it is necessary to minimize the variance of the list length. , the algorithm achieves uniform distribution of the vector, and finally divides the label data vector X into N lists .

[0175] In addition, multi-constrained balanced clustering can be optimized by adjusting the hyperparameters , balancing the trade-off between clustering quality and clustering balance.

[0176] Further, refer to Figure 5In step S5, searching is performed based on the multi-label index mechanism according to the query condition, including:

[0177] S51, generating a query condition vector according to the query condition;

[0178] S52, calculating the distance between the query condition vector and the centroid of each list in the index list, and if the distance meets a preset condition, taking the corresponding list as the search list;

[0179] S53, matching the query condition vector with the search list to obtain a corresponding model package;

[0180] The preset conditions are:

[0181] ;

[0182] q represents the query condition vector, X ij Represents the list of distances to be calculated, c ij Represents list X ij The center of mass, represents the hyperparameter, Dist(q,c ij ) represents the query condition vector q and the centroid c ij The distance Represents the list X to be calculated by the query condition vector q ij Perform a search query.

[0183] The above dynamic search determines whether to search a list. If the query condition vector q is consistent with the list The centroid The distance between the query and the nearest centroid is less than or equal to Distance times, the list will be used for search. This method reduces query latency by reducing unnecessary list accesses while maintaining high recall.

[0184] refer to Figure 6 In some embodiments, a management and control device for a deep learning model is provided, comprising:

[0185] The training module 201 is used to receive task information and configuration files input by the user, train the corresponding deep learning model according to the task information and configuration files, and generate a model file;

[0186] An integration module 202 is used to integrate the model files and related model information to generate a model package;

[0187] An optimization module 203 optimizes the model package based on pruning and knowledge distillation, and generates a version iteration graph of the model package based on a snapshot technology;

[0188] An indexing module 204 is used to extract key features and version information from the optimized model package, generate a tag list according to the key features and version information, associate and store the tag list with the model package, and establish a multi-tag indexing mechanism;

[0189] The deployment module 205 is used to receive the query conditions sent by the user, perform retrieval based on the multi-label index mechanism according to the query conditions, find the corresponding model package for deployment, and record the application information of the corresponding model package.

[0190] Furthermore, the task information includes task description, task type, associated target level and traffic level; the configuration file includes model settings, version selection, model structure and model hyperparameters.

[0191] Furthermore, the optimization module 203 optimizes the model package based on pruning, including:

[0192] Taking the filter of the model file in the model package as a point in the Euclidean space;

[0193] Calculate the Euclidean distance of filters in the same layer and find the geometric center of all filters in the same layer;

[0194] Calculate the distance between each filter and the geometric center, take the filter whose distance is less than a preset distance as the pruning target filter, and set the pruning target filter to 0;

[0195] The optimization module 203 optimizes the model package based on knowledge distillation, including:

[0196] Use the pruned model file as the teacher network and set up the student network;

[0197] Inputting the training samples into the teacher network to obtain the teacher network output result;

[0198] The output result of the teacher network is used as a soft label, the true label of the training sample is used as a hard label, the soft label and the hard label are input into the student network for training, the loss function is calculated and the parameters of the student network are updated during the training process until the student network converges, and the converged student network is used as the optimized model file.

[0199] Furthermore, the version iteration graph includes snapshots of various versions of the model file, and the snapshots include checkpoints during the iteration process of the model file.

[0200] Furthermore, the tag list includes a plurality of tags, and the tags include a separator flag, a tag type flag, a tag length flag, and a type identification value flag;

[0201] The index module 204 establishes a multi-label index mechanism, including:

[0202] Combining multiple labels of the model file to obtain a label data vector;

[0203] Dividing the label data vector according to the separator to obtain a plurality of inverted list vectors;

[0204] Clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters;

[0205] Utilizing a multi-constraint balanced clustering algorithm to optimize the obtained multiple clusters, and obtain multiple lists;

[0206] Create an index list based on the partitioned list.

[0207] Furthermore, the indexing module 204 clusters the plurality of inverted list vectors using a hierarchical clustering algorithm to obtain a plurality of clusters, including:

[0208] Randomly select a preset number of inverted list vectors as initial cluster centers;

[0209] Calculate the Euclidean distance between other inverted list vectors and the initial cluster center, and search for the inverted list vector closest to the initial cluster center according to the Euclidean distance to merge them to form a cluster;

[0210] Iteratively perform the following steps until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster;

[0211] If the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is allocated to the corresponding multiple clusters.

[0212] Furthermore, the indexing module 204 optimizes the obtained multiple clusters using a multi-constraint balanced clustering algorithm to obtain multiple lists, including:

[0213] Based on the obtained clusters, an objective function on the variance of the length of the inverted list vector is established;

[0214] Optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list;

[0215] The objective function is as follows:

[0216] ;

[0217] Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, Represents the inverted list vector length variance.

[0218] Furthermore, the deployment module 205 performs retrieval based on the multi-label index mechanism according to the query condition, including:

[0219] Generate a query condition vector according to the query condition;

[0220] Calculating the distance between the query condition vector and the centroid of each list in the index list, and if the distance meets a preset condition, using the corresponding list as the search list;

[0221] Matching the query condition vector with the search list to obtain a corresponding model package;

[0222] The preset conditions are:

[0223] ;

[0224] q represents the query condition vector, X ij Represents the list of distances to be calculated, c ij Represents list X ij The center of mass, represents the hyperparameter, Dist(q,c ij ) represents the query condition vector q and the centroid c ij The distance Represents the list X to be calculated by the query condition vector q ij Perform a search query.

[0225] In some embodiments, a computer storage medium is also provided, wherein the computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the above method.

[0226] The deep learning model management and control method and device provided in the above embodiments have at least the following beneficial effects:

[0227] (1) The efficiency of power grid operation and maintenance has been significantly improved through the automated task distribution and configuration process. Operation and maintenance personnel can quickly set task parameters and select the most suitable model for deployment based on real-time network conditions and analysis requirements. This flexibility not only reduces the time cost of manual operation, but also allows the system to respond quickly to emergencies, train in real time or call pre-trained models to ensure the stable operation of the power grid.

[0228] (2) The model training phase uses snapshot technology to maintain the historical version of the model and perform iterative updates, which not only ensures the integrity and consistency of the model data, but also provides a basis for the continuous optimization of the model. With the accumulation of power grid data and the advancement of technology, the model can be continuously updated and optimized, thereby improving the accuracy and efficiency of the analysis;

[0229] (3) The multi-label retrieval mechanism greatly improves the intelligence and accuracy of model retrieval. Users can quickly locate the specific model version they need through multi-dimensional label combinations. This method not only improves retrieval efficiency but also reduces the risk of incorrect deployment. Accurate model deployment is crucial for traffic analysis of power grid equipment, which directly affects the success rate of analysis tasks and the safe operation of the power grid.

[0230] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A management and control method for a deep learning model, characterized in that: include: Receive task information and configuration files input by the user, train the corresponding deep learning model according to the task information and configuration files, and generate a model file; Integrate the model files and related model information to generate a model package; Optimizing the model package based on pruning and knowledge distillation, and generating a version iteration graph of the model package based on snapshot technology; Extracting key features and version information from the optimized model package, generating a tag list based on the key features and version information, associating and storing the tag list with the model package, and establishing a multi-tag indexing mechanism; Receive the query conditions sent by the user, perform retrieval based on the multi-label index mechanism according to the query conditions, find the corresponding model package for deployment, and record the application information of the corresponding model package; The tag list includes a plurality of tags, each of which includes a separator flag, a tag type flag, a tag length flag, and a type identification value flag; The multi-label indexing mechanism is established, including: Combining multiple labels of the model file to obtain a label data vector; Dividing the label data vector according to the separator to obtain a plurality of inverted list vectors; Clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters; Utilizing a multi-constraint balanced clustering algorithm to optimize the obtained multiple clusters, and obtain multiple lists; Create an index list based on the partitioned list; The step of clustering the plurality of inverted list vectors using a hierarchical clustering algorithm to obtain a plurality of clusters includes: Randomly select a preset number of inverted list vectors as initial cluster centers; Calculate the Euclidean distance between other inverted list vectors and the initial cluster center, and search for the inverted list vector closest to the initial cluster center according to the Euclidean distance to merge them to form a cluster; Iteratively perform the following steps until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster; If the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is allocated to the corresponding multiple clusters.

2. The method according to claim 1, characterized in that The task information includes task description, task type, associated target level and traffic level; the configuration file includes model settings, version selection, model structure and model hyperparameters.

3. The method according to claim 1, characterized in that The model package is optimized based on pruning, including: Taking the filter of the model file in the model package as a point in the Euclidean space; Calculate the Euclidean distance of filters in the same layer and find the geometric center of all filters in the same layer; Calculate the distance between each filter and the geometric center, take the filter whose distance is less than a preset distance as the pruning target filter, and set the pruning target filter to 0; The model package is optimized based on knowledge distillation, including: Use the pruned model file as the teacher network and set up the student network; Inputting the training samples into the teacher network to obtain the teacher network output result; The output result of the teacher network is used as a soft label, the true label of the training sample is used as a hard label, the soft label and the hard label are input into the student network for training, the loss function is calculated and the parameters of the student network are updated during the training process until the student network converges, and the converged student network is used as the optimized model file.

4. The method according to claim 1, characterized in that: The version iteration graph includes snapshots of various versions of the model file, and the snapshots include checkpoints during the iteration process of the model file.

5. The method according to claim 1, characterized in that The multiple clusters obtained are optimized by using a multi-constraint balanced clustering algorithm to obtain multiple lists, including: Based on the obtained clusters, an objective function on the variance of the length of the inverted list vector is established; Optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list; The objective function is as follows: Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, It represents the variance of the length of the inverted list vector, and the subscript F is the symbol for finding the norm.

6. The method according to claim 1, characterized in that The retrieving based on the multi-label index mechanism according to the query condition includes: Generate a query condition vector according to the query condition; Calculating the distance between the query condition vector and the centroid of each list in the index list, and if the distance meets a preset condition, using the corresponding list as the search list; Matching the query condition vector with the search list to obtain a corresponding model package; The preset conditions are: ； q represents the query condition vector, X ij Represents the list of distances to be calculated, c ij Represents list X ij The center of mass, represents the hyperparameter, Dist(q,c ij ) represents the query condition vector q and the centroid c ij The distance Represents the list X to be calculated by the query condition vector q ij Perform a search query.

7. A management and control device for a deep learning model, characterized in that: include: A training module is used to receive task information and configuration files input by a user, train a corresponding deep learning model according to the task information and configuration files, and generate a model file; An integration module, used to integrate the model files and related model information to generate a model package; An optimization module optimizes the model package based on pruning and knowledge distillation, and generates a version iteration graph of the model package based on snapshot technology; An indexing module, used to extract key features and version information from the optimized model package, generate a tag list based on the key features and version information, associate and store the tag list with the model package, and establish a multi-tag indexing mechanism; A deployment module, used to receive query conditions sent by users, perform retrieval based on the multi-label index mechanism according to the query conditions, find the corresponding model package for deployment, and record application information of the corresponding model package; The tag list includes a plurality of tags, each of which includes a separator flag, a tag type flag, a tag length flag, and a type identification value flag; The index module establishes a multi-label index mechanism, including: Combining multiple labels of the model file to obtain a label data vector; Dividing the label data vector according to the separator to obtain a plurality of inverted list vectors; Clustering the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters; Utilizing a multi-constraint balanced clustering algorithm to optimize the obtained multiple clusters, and obtain multiple lists; Create an index list based on the partitioned list; The index module clusters the multiple inverted list vectors using a hierarchical clustering algorithm to obtain multiple clusters, including: Randomly select a preset number of inverted list vectors as initial cluster centers; Calculate the Euclidean distance between other inverted list vectors and the initial cluster center, and search for the inverted list vector closest to the initial cluster center according to the Euclidean distance to merge them to form a cluster; Iteratively perform the following steps until each cluster contains a set number of inverted list vectors: reselect the cluster center, calculate the Euclidean distance between other inverted list vectors and the cluster center, merge the inverted list vectors closest to the cluster center, and update the cluster; If the Euclidean distances between the inverted list vector and multiple cluster centers are equal or the difference is within a preset range, the inverted list vector is allocated to the corresponding multiple clusters.

8. The device according to claim 7, characterized in that The optimization module optimizes the model package based on pruning, including: Taking the filter of the model file in the model package as a point in the Euclidean space; Calculate the Euclidean distance of filters in the same layer and find the geometric center of all filters in the same layer; Calculate the distance between each filter and the geometric center, take the filter whose distance is less than a preset distance as the pruning target filter, and set the pruning target filter to 0; The optimization module optimizes the model package based on knowledge distillation, including: Use the pruned model file as the teacher network and set up the student network; Inputting the training samples into the teacher network to obtain the teacher network output result; The output result of the teacher network is used as a soft label, the true label of the training sample is used as a hard label, the soft label and the hard label are input into the student network for training, the loss function is calculated and the parameters of the student network are updated during the training process until the student network converges, and the converged student network is used as the optimized model file.

9. The device according to claim 7, characterized in that The index module optimizes the obtained multiple clusters using a multi-constraint balanced clustering algorithm to obtain multiple lists, including: Based on the obtained clusters, an objective function on the variance of the length of the inverted list vector is established; Optimizing the allocation of the inverted list vectors in the cluster and solving the objective function so that the inverted list length variance is minimized, and the optimized cluster obtained is the list; The objective function is as follows: ； Among them, X represents the label data vector, C represents the centroid of the cluster, H represents the cluster obtained by clustering, λ is a hyperparameter, N represents the number of clusters, and H li Indicates l Whether the inverted list vector is assigned to the i-th cluster, Represents the inverted list vector length variance.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the method according to any one of claims 1 to 6.

Images