Model classification method and device, electronic equipment and storage medium

Abstract

The invention provides a model classification method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: extracting at least one initial classification feature of a to-be-classified model; according to the at least one initial classification feature, an initial classification result and initial classification precision are determined, and the initial classification precision is used for indicating the classification accuracy degree of the initial classification result; when the initial classification precision does not meet a preset end condition, iteratively extracting at least one target classification feature of the to-be-classified model, and determining a target classification result and target classification precision according to the at least one target classification feature; and when the target classification precision meets a preset ending condition, taking the target classification result as a final classification result. According to the method and the device, the classification features can be iteratively extracted for multiple times, and the accuracy of the classification features can be improved, so that the accuracy and the reliability of model classification are ensured while the efficiency of model classification is ensured, the efficiency of network operation management is improved, and the user experience is ensured.

Description

Background Art

[0002] With the development of computer technology, AI models with various AI (Artificial Intelligence) capabilities have been developed and applied in large quantities. Therefore, an AI platform or AI system is needed to manage AI models to improve the efficiency of network operation and management and ensure user experience.

[0003] In related technologies, due to the development of a large number of AI platforms and AI systems, AI capabilities are repeatedly built and fragmented. Therefore, there is an urgent need for a method to classify AI models, which can ensure the efficiency of AI model classification while ensuring the accuracy and reliability of model classification, thereby improving the efficiency of network operation and management and ensuring user experience.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a model classification method, device, electronic device and storage medium, which at least to a certain extent overcome the problem in related technologies that it is impossible to accurately and reliably classify models while ensuring the efficiency of AI model classification, thereby reducing the efficiency of network operation and management and user experience.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a model classification method is provided, comprising: extracting at least one initial classification feature of a model to be classified; determining an initial classification result and an initial classification accuracy based on the at least one initial classification feature, wherein the initial classification accuracy is used to indicate the classification accuracy of the initial classification result; when the initial classification accuracy does not meet a preset end condition, iteratively extracting at least one target classification feature of the model to be classified, and determining a target classification result and a target classification accuracy based on the at least one target classification feature; when the target classification accuracy meets the preset end condition, taking the target classification result as the final classification result.

[0008] In some exemplary embodiments, extracting at least one initial classification feature of the model to be classified includes: inputting the model to be classified into a feature extraction model to obtain at least one initial classification feature;

[0009] The method of determining an initial classification result and an initial classification accuracy based on at least one initial classification feature includes: inputting at least one initial classification feature into a classification model to obtain the initial classification result and the initial classification accuracy.

[0010] In some exemplary embodiments, the model classification method provided by the embodiments of the present disclosure further includes: obtaining the model to be classified;

[0011] The step of extracting at least one initial classification feature of the model to be classified includes: when the user has not classified the model to be classified according to the current classification label, extracting at least one initial classification feature of the model to be classified.

[0012] In some exemplary embodiments, after taking the target classification result as the final classification result, the method further includes: clustering the model to be classified to obtain at least one model clustering cluster; determining the classification label corresponding to at least one model clustering cluster; and updating the current classification label according to the classification label corresponding to at least one model clustering cluster to obtain an updated classification label.

[0013] In some exemplary embodiments, after the current classification label is updated according to the classification label corresponding to at least one model clustering cluster to obtain the updated classification label, it also includes: clustering the updated classification label to obtain multiple label clustering clusters; screening at least one group of label clustering clusters that meet preset merging conditions from the multiple label clustering clusters, wherein any group of label clustering clusters includes at least two of the label clustering clusters; and merging the updated classification labels corresponding to at least one group of label clustering clusters to obtain merged classification labels.

[0014] In some exemplary embodiments, the model to be classified is an AI model, and the type of the initial classification feature includes at least one of AI capability application scenario features, AI capability solution features, algorithm features corresponding to the AI ​​model, and label data features.

[0015] In some exemplary embodiments, the model classification method provided by the embodiments of the present disclosure further includes: constructing the feature extraction model based on the BPSO (Discrete Binary Particle Swarm Optimization) algorithm; and constructing the classification model based on the CNN (Convolutional Neural Networks) algorithm.

[0016] According to another aspect of the present disclosure, there is also provided a model classification device, comprising:

[0017] An initial classification feature extraction module, used to extract at least one initial classification feature of the model to be classified;

[0018] An initial classification result determination module, used to determine an initial classification result and an initial classification accuracy according to at least one initial classification feature, where the initial classification accuracy is used to indicate the classification accuracy of the initial classification result;

[0019] A target classification feature extraction module, used for iteratively extracting at least one target classification feature of the model to be classified when the initial classification accuracy does not meet the preset end condition, and determining the target classification result and the target classification accuracy according to the at least one target classification feature;

[0020] The final classification result determination model is used to take the target classification result as the final classification result when the target classification accuracy meets the preset end condition.

[0021] In some exemplary embodiments, the initial classification feature extraction module is used to input the model to be classified into a feature extraction model to obtain at least one initial classification feature;

[0022] The initial classification result determination module is used to input at least one initial classification feature into the classification model to obtain the initial classification result and the initial classification accuracy.

[0023] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0024] A model to be classified obtaining module, used to obtain the model to be classified;

[0025] The initial classification feature extraction module is used to extract at least one initial classification feature of the model to be classified when the user has not classified the model to be classified according to the current classification label.

[0026] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0027] A model clustering module for classification, used for clustering the model to be classified to obtain at least one model clustering cluster;

[0028] A classification label determination module, used to determine a classification label corresponding to at least one model cluster;

[0029] The current classification label updating module is used to update the current classification label according to the classification label corresponding to at least one model clustering cluster to obtain an updated classification label.

[0030] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0031] A classification label clustering module is used to cluster the updated classification labels to obtain multiple label clusters;

[0032] A label cluster screening module, used for screening at least one group of label clusters satisfying a preset merging condition from a plurality of label clusters, wherein any group of label clusters includes at least two of the label clusters;

[0033] The classification label merging module is used to merge the updated classification labels corresponding to at least one group of label clusters to obtain merged classification labels.

[0034] In some exemplary embodiments, the model to be classified is an AI model, and the type of the initial classification feature includes at least one of an AI capability application scenario feature, an AI capability solution feature, an algorithm feature corresponding to the AI ​​model, and a label data feature.

[0035] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0036] The model building module is used to build the feature extraction model based on the BPSO algorithm and build the classification model based on the CNN algorithm.

[0037] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned model classification methods by executing the executable instructions.

[0038] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the model classification method described in any one of the above is implemented.

[0039] According to another aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the model classification method provided in various optional ways in any embodiment of the present disclosure.

[0040] The technical solution provided in the embodiments of the present disclosure can perform multiple iterative extractions on classification features, which can improve the accuracy of classification features, thereby ensuring the accuracy and reliability of model classification while ensuring the efficiency of model classification, thereby improving the efficiency of network operation management and ensuring user experience.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0043] Figure 1 A schematic diagram of an exemplary system architecture in an embodiment of the present disclosure is shown;

[0044] Figure 2 A flow chart of a model classification method in an embodiment of the present disclosure is shown;

[0045] Figure 3 A schematic diagram showing an AI system in an embodiment of the present disclosure is shown;

[0046] Figure 4 A schematic diagram of a classification method of an AI model in an embodiment of the present disclosure is shown;

[0047] Figure 5 A schematic diagram of a model classification device in an embodiment of the present disclosure is shown;

[0048] Figure 6 A schematic diagram showing an electronic device in an embodiment of the present disclosure;

[0049] Figure 7 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0051] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0052] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0053] Figure 1 FIG. 1 shows an exemplary system architecture diagram to which the model classification method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0054] Among them, the terminal device 101 can extract at least one initial classification feature of the model to be classified. And according to the at least one initial classification feature, the initial classification result and the initial classification accuracy can be determined, and the initial classification accuracy is used to indicate the classification accuracy of the initial classification result. When the initial classification accuracy does not meet the preset end condition, the terminal device 101 can iteratively extract at least one target classification feature of the model to be classified, and determine the target classification result and the target classification accuracy according to the at least one target classification feature. When the target classification accuracy meets the preset end condition, the terminal device 101 can use the target classification result as the final classification result and send it to the server 103.

[0055] Alternatively, the server 103 may extract at least one initial classification feature of the model to be classified. And based on the at least one initial classification feature, the initial classification result and the initial classification accuracy may be determined, and the initial classification accuracy is used to indicate the classification accuracy of the initial classification result. When the initial classification accuracy does not meet the preset end condition, the server 103 may iteratively extract at least one target classification feature of the model to be classified, and determine the target classification result and the target classification accuracy based on the at least one target classification feature. When the target classification accuracy meets the preset end condition, the server 103 may use the target classification result as the final classification result and send it to the terminal device 101.

[0056] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0057] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0058] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0059] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0060] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0061] Those skilled in the art will know that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0062] Under the above system architecture, a model classification method is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0063] In some embodiments, the model classification method provided in the embodiments of the present disclosure can be executed by a terminal device of the above-mentioned system architecture; in other embodiments, the model classification method provided in the embodiments of the present disclosure can be executed by a server in the above-mentioned system architecture; in other embodiments, the model classification method provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above-mentioned system architecture through interaction.

[0064] Figure 2 A flow chart of a model classification method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the model classification method provided in the embodiment of the present disclosure includes the following steps S202 to S208.

[0065] S202: extract at least one initial classification feature of the model to be classified.

[0066] In some embodiments, the model to be classified may be an AI model, and the type of the initial classification feature includes at least one of AI capability application scenario features, AI capability solution features, algorithm features corresponding to the AI ​​model, and label data features.

[0067] The embodiments of the present disclosure do not limit the application scenario characteristics of the AI ​​capability. For example, any AI model can correspond to any AI capability. In some embodiments, the AI ​​capability can be a traffic prediction capability, a speech recognition capability, an encryption and decryption capability, etc. For example, taking an AI model for traffic prediction as an example, the AI ​​capability application scenario characteristics corresponding to the AI ​​model can be an application scenario for traffic prediction.

[0068] The embodiments of the present disclosure do not limit the AI ​​capability solution feature. For example, taking an AI model for speech recognition as an example, the AI ​​capability solution feature corresponding to the AI ​​model can be a solution for performing speech recognition on speech data.

[0069] The embodiments of the present disclosure do not limit the algorithm features corresponding to the AI ​​model. For example, taking the AI ​​model used for encryption and decryption as an example, the algorithm features corresponding to the AI ​​model can be the algorithm used to encrypt and decrypt data.

[0070] The disclosed embodiments do not limit the label data features. For example, taking an AI model for traffic prediction as an example, the label data features corresponding to the AI ​​model can be used for traffic prediction. In one possible implementation, the label data features can be used to describe the approximate classification of each AI model. For example, the label data features corresponding to the traffic prediction capability can be used to describe the AI ​​model for traffic prediction, but may not describe the type of traffic predicted by the AI ​​model.

[0071] In some embodiments, the model classification method provided by the embodiments of the present disclosure may further include: obtaining the model to be classified.

[0072] In this case, extracting at least one initial classification feature of the model to be classified includes: when the user does not classify the model to be classified according to the current classification label, extracting at least one initial classification feature of the model to be classified.

[0073] Exemplarily, each AI model can be stored in an AI system or AI platform. Taking the AI ​​system as an example, when registering, the user can classify each AI model according to the current classification label pre-set in the current system. And, when the user cannot classify a certain AI model by the pre-set current classification label, the AI ​​model can be used as a model to be classified for model classification.

[0074] In some embodiments, extracting at least one initial classification feature of the model to be classified may include: inputting the model to be classified into a feature extraction model to obtain at least one initial classification feature.

[0075] In this case, determining the initial classification result and the initial classification accuracy based on at least one initial classification feature may include: inputting at least one initial classification feature into a classification model to obtain the initial classification result and the initial classification accuracy.

[0076] In some exemplary embodiments, after the model to be classified is obtained, the model to be classified may be input into a feature extraction model to obtain initial classification features.

[0077] The disclosed embodiments do not limit the type of feature extraction model, and the type of the feature extraction model can be determined based on experience or application scenarios. For example, the feature extraction model can be a model built based on the BPSO algorithm, or a model built based on the LSTM (Long Short-Term Memory) algorithm, etc.

[0078] For example, the feature extraction model is constructed based on the BPSO algorithm. The BPSO algorithm is a discrete binary particle swarm algorithm that can optimize discrete space constraint problems. For discrete problems such as feature selection, a binary version of the BPSO algorithm can also be used for optimization. Therefore, the embodiment of the present disclosure can improve the accuracy and reliability of feature extraction by constructing a feature extraction model based on the BPSO algorithm.

[0079] In some exemplary embodiments, after the initial classification features are obtained, the initial classification features may be input into a classification model to classify the model to be classified.

[0080] The disclosed embodiments do not limit the type of classification model, and the type of the classification model can be determined based on experience or application scenarios. For example, the classification model can be constructed based on a CNN algorithm. Alternatively, the classification model can be constructed based on a logistic regression algorithm.

[0081] For example, the classification model is constructed based on the CNN algorithm. The CNN algorithm is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of DL (Deep Learning). The combination of the CNN algorithm and the BPSO algorithm can better optimize the optimal feature set, thereby improving the accuracy and reliability of the model classification.

[0082] S204: Determine an initial classification result and an initial classification accuracy according to at least one initial classification feature, where the initial classification accuracy is used to indicate the classification accuracy of the initial classification result.

[0083] In an exemplary embodiment, the initial classification result may be a classification result obtained by classifying the model to be classified using the initial classification features.

[0084] In addition, the initial classification accuracy can be expressed as the accuracy of the classification result, that is, the percentage of correct classification results. Exemplarily, when the model to be classified is classified, it can be obtained by predicting.

[0085] In some embodiments, the feature extraction model and the classification model may be trained first, and the trained feature extraction model and the classification model may be used to perform model classification for models to be classified that cannot be classified by the user through the current classification labels preset in advance.

[0086] Exemplarily, when training the feature extraction model, an AI model with known labels can be obtained first and input into the feature extraction model to obtain the corresponding initial classification results and initial classification accuracy.

[0087] The initial classification accuracy can be described by the degree of matching between the initial classification result and the known label corresponding to the AI ​​model to describe the classification accuracy of the initial classification result. When the initial classification accuracy meets the preset end condition, the training can be terminated, and the model to be classified is input into the feature extraction model and the classification model.

[0088] S206, when the initial classification accuracy does not meet the preset end condition, iteratively extract at least one target classification feature of the model to be classified, and determine the target classification result and the target classification accuracy according to the at least one target classification feature.

[0089] The embodiments of the present disclosure do not limit the preset end condition, and the preset end condition can be limited based on experience or application scenarios. For example, the preset end condition can be that the initial classification accuracy is not less than a preset threshold, and when the initial classification accuracy is the accuracy of the classification result, the preset threshold can be set to 90%, 85% or 70%, etc.

[0090] It should be noted that different types of AI models may correspond to different preset thresholds.

[0091] In some embodiments, the method of iteratively extracting at least one target classification feature of the model to be classified and determining the target classification result and the target classification accuracy based on the at least one target classification feature may be the same as the steps in S202 to S204 and will not be repeated here.

[0092] For example, the preset end condition is that the initial classification accuracy is not less than 90%. When the end condition is not met, the first target classification feature can be determined based on the BPSO algorithm, and the first target classification feature is the feature after an iterative update of the initial classification feature. After that, the first target classification feature can be input into the CNN model again for model classification, and the first target classification result and the first target classification accuracy can be obtained.

[0093] If the first target classification accuracy still does not meet the preset end condition, the second target classification feature can be determined based on the BPSO algorithm. The second target classification feature is the feature after the initial classification feature is iteratively updated twice. After that, the second target classification feature is input into the CNN model again for model classification to obtain the third target classification result and the third target classification accuracy. Repeat this process until the target classification accuracy meets the preset end condition.

[0094] S208: When the target classification accuracy meets the preset end condition, the target classification result is used as the final classification result.

[0095] In some embodiments, after the target classification result is used as the final classification result, the model classification method provided by the embodiments of the present disclosure may also include: clustering the model to be classified to obtain at least one model clustering cluster; determining the classification label corresponding to at least one model clustering cluster; and updating the current classification label according to the classification label corresponding to at least one model clustering cluster to obtain an updated classification label.

[0096] The disclosed embodiments do not limit the type of the clustering algorithm. Exemplarily, the clustering algorithm may be a supervised clustering algorithm, an unsupervised clustering algorithm, etc. Taking the unsupervised clustering algorithm as an example, the K-means unsupervised clustering algorithm may be used to cluster the models to be classified.

[0097] Exemplarily, after obtaining at least one model clustering cluster, the classification label corresponding to the at least one model clustering cluster can be determined. At this time, the classification label corresponding to the at least one model clustering cluster can be determined by at least one of the AI ​​capability application scenario characteristics, AI capability solution characteristics, algorithm characteristics corresponding to the AI ​​model, and label data characteristics. Alternatively, the classification label corresponding to the at least one model clustering cluster can be manually set.

[0098] In an exemplary embodiment, after determining the classification label corresponding to at least one model clustering cluster, it is determined whether the preset current classification label includes the classification label corresponding to each model clustering cluster. If there is a classification label corresponding to at least one model clustering cluster that is not included in the preset current classification label, the original AI capability classification label library can be updated to achieve the update of the current classification label.

[0099] In some embodiments, after the current classification label is updated according to the classification label corresponding to at least one model clustering cluster to obtain the updated classification label, the model classification method provided by the embodiment of the present disclosure may also include: clustering the updated classification label to obtain multiple label clustering clusters; screening at least one group of label clustering clusters that meet preset merging conditions from the multiple label clustering clusters, wherein any group of label clustering clusters includes at least two of the label clustering clusters; and merging the updated classification labels corresponding to at least one group of label clustering clusters to obtain merged classification labels.

[0100] In an exemplary embodiment, the updated classification labels can be clustered by the K-means unsupervised clustering algorithm. In some embodiments, clustering can be performed according to the functional type corresponding to the label. It should be noted that after obtaining the merged classification labels, the above-mentioned current classification labels can be updated according to the merged classification labels.

[0101] It should be noted that the acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0102] The improved method of the disclosed embodiment can perform multiple iterative extractions on classification features, which can improve the accuracy of classification features, thereby ensuring the efficiency of model classification while ensuring the accuracy and reliability of model classification, thereby improving the efficiency of network operation management and ensuring user experience.

[0103] In some embodiments, a schematic diagram of an AI system that can perform model classification provided by an embodiment of the present disclosure can be as follows: Figure 3 shown.

[0104] exist Figure 3 In the example, the user can send the user intention to the AI ​​capability management system through the external system. For example, the user intention can be to classify the AI ​​model. At this time, the user can register the AI ​​model in the AI ​​capability management system through the AI ​​capability registration module included in the AI ​​capability management system by indicator access, mirror access or model access.

[0105] Among them, the indicator access is used to register existing AI models and can periodically provide information such as intelligent capability status and value indicators. The image access can be used to encapsulate the AI ​​model and its operating environment as an inference or training image for registration. The model access can be used to manage the AI ​​model code model (including model parameter files and data specification requirements).

[0106] After successful registration, users can upload AI models that meet the sharing or retraining conditions to the AI ​​capability management system for registration through the AI ​​capability registration module. The uploaded data includes at least one of the reasoning or training image, related training data, verification data, and output data.

[0107] After successful registration, the AI ​​model effect evaluation module can evaluate the effect of the AI ​​model based on value evaluation indicators (such as accuracy, variance, etc.). In addition, the AI ​​capability registration module can update the classification label library accordingly based on the labels corresponding to the registered AI model.

[0108] In addition, the AI ​​model retraining module is used to compare the effect evaluation result of the AI ​​model with a preset evaluation threshold, and to retrain the AI ​​model that is lower than the preset evaluation threshold until the effect evaluation result is greater than or equal to the threshold.

[0109] After the effect evaluation modules of each AI model meet the preset evaluation threshold, the AI ​​model can be classified through the AI ​​capability classification module. For example, the user can first classify each AI model based on the current classification label stored in the classification label library management module. When the user cannot classify a certain AI model through the preset current classification label, the AI ​​model can be used as a model to be classified and the model classification can be performed through the AI ​​capability classification module.

[0110] In addition, the unsupervised clustering model can be used to cluster AI models that cannot correspond to the current classification labels, construct new labels based on experience, and update the original classification label library accordingly.

[0111] It should be noted that for AI models that cannot be classified, the construction of new legal entity classification labels and the updating of label libraries based on unsupervised clustering methods combined with application scenarios and experience can better map user intentions and enhance user perception.

[0112] The AI ​​capability library can be used to store AI models after each classification. The AI ​​capability sharing module can be used to share corresponding AI models based on the AI ​​capabilities contained in the user's intent.

[0113] Exemplarily, the sharing method may include: at least one of standardized sharing, retraining sharing, inference calling, model embedded reference, and image downloading.

[0114] Among them, the standardized sharing method can be a standardized AI capability, and standardized sharing can be used for direct sharing through intelligent asset management. The retraining sharing method can be used to retrain the managed AI capabilities so that they can be reused in different regions, professions, and scenarios. The reasoning call method can be used to call the corresponding AI model through the API (Application Program Interface) interface, so as to perform model reasoning in the cloud. The model embedded reference method can be used to package and pack AI models and embed them into other systems. The image download method can be used to package and download AI models and their operating environment to other systems, and other systems provide computing resources.

[0115] In an exemplary embodiment, the AI ​​capability management system may also return result feedback corresponding to the user's intention to the external system. For example, if the user intends to classify a certain AI model, the result feedback may be the classification result of the AI ​​model. For another example, if the user intends to obtain a certain AI capability, the result feedback may be the return of the AI ​​model corresponding to the AI ​​capability.

[0116] In some embodiments, a schematic diagram of a classification method of an AI model can be as follows: Figure 4 shown.

[0117] exist Figure 4 In the process, the model can be constructed first. Exemplarily, the constructed model can be composed of a BPSO algorithm and a CNN algorithm. Among them, the AI ​​model classification feature set can be obtained first. The AI ​​model classification feature set includes multiple current classification labels. Afterwards, the initial classification features can be obtained from the AI ​​model classification feature set through the BPSO algorithm and substituted into the CNN algorithm for model classification to obtain the output initial classification accuracy. Afterwards, it can be confirmed whether the initial classification accuracy meets the preset end condition. If not, the target classification features can be iteratively extracted through the BPSO algorithm until the preset end condition is met. When the budget end condition is met, the AI ​​model classification can be performed according to the currently determined target classification features.

[0118] For example, at this time, the target classification features can be substituted into the CNN algorithm to obtain the final classification result, and the classification label can be updated accordingly.

[0119] Afterwards, the target classification features and the current classification labels included in the AI ​​capability label library can be clustered by an unsupervised clustering algorithm to obtain a clustering result. Exemplarily, at this time, the AI ​​model can be qualified according to the clustering result to obtain a new classification label. The new classification label and the current classification label included in the AI ​​capability label library are clustered to merge the classification labels of the functional type to update the AI ​​capability label library.

[0120] Based on the same inventive concept, the present disclosure also provides a model classification device in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0121] Figure 5 A schematic diagram of a model classification device in an embodiment of the present disclosure is shown. Figure 5 As shown, the device comprises:

[0122] An initial classification feature extraction module 501 is used to extract at least one initial classification feature of a model to be classified;

[0123] An initial classification result determination module 502 is used to determine an initial classification result and an initial classification accuracy according to at least one initial classification feature, where the initial classification accuracy is used to indicate the classification accuracy of the initial classification result;

[0124] The target classification feature extraction module 503 is used to iteratively extract at least one target classification feature of the model to be classified when the initial classification accuracy does not meet the preset end condition, and determine the target classification result and the target classification accuracy according to the at least one target classification feature;

[0125] The final classification result determination model 504 is used to take the target classification result as the final classification result when the target classification accuracy meets the preset end condition.

[0126] In some exemplary embodiments, the initial classification feature extraction module 501 is used to input the model to be classified into the feature extraction model to obtain at least one initial classification feature;

[0127] The initial classification result determination module 502 is used to input at least one initial classification feature into the classification model to obtain an initial classification result and an initial classification accuracy.

[0128] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0129] A model acquisition module to be classified is used to obtain the model to be classified;

[0130] The initial classification feature extraction module 501 is used to extract at least one initial classification feature of the model to be classified when the user does not classify the model to be classified according to the current classification label.

[0131] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0132] The model to be classified clustering module is used to cluster the model to be classified to obtain at least one model clustering cluster;

[0133] A classification label determination module, used to determine a classification label corresponding to at least one model cluster;

[0134] The current classification label updating module is used to update the current classification label according to the classification label corresponding to at least one model clustering cluster to obtain an updated classification label.

[0135] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0136] A classification label clustering module is used to cluster the updated classification labels to obtain multiple label clusters;

[0137] A label cluster screening module, used to screen at least one group of label clusters satisfying a preset merging condition from a plurality of label clusters, wherein any group of label clusters includes at least two label clusters;

[0138] The classification label merging module is used to merge the updated classification labels corresponding to at least one group of label clusters to obtain merged classification labels.

[0139] In some exemplary embodiments, the model to be classified is an AI model, and the types of initial classification features include at least one of AI capability application scenario features, AI capability solution features, algorithm features corresponding to the AI ​​model, and label data features.

[0140] In some exemplary embodiments, the model classification device provided by the embodiments of the present disclosure further includes:

[0141] The model building module is used to build a feature extraction model based on the BPSO algorithm and a classification model based on the CNN algorithm.

[0142] The improved device of the embodiment of the present disclosure can perform multiple iterative extractions on classification features, which can improve the accuracy of classification features, thereby ensuring the accuracy and reliability of model classification while ensuring the efficiency of model classification, thereby improving the efficiency of network operation management and ensuring user experience.

[0143] It should be noted that the above-mentioned initial classification feature extraction module 501, initial classification result determination module 502, target classification feature extraction module 503, and final classification result determination model 504 correspond to S202 to S208 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0144] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0145] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present disclosure is described. Figure 6 The electronic device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0146] like Figure 6As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).

[0147] The storage unit stores a program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 610 can execute the following steps of the above method embodiment to extract at least one initial classification feature of the model to be classified; based on at least one initial classification feature, determine the initial classification result and the initial classification accuracy, and the initial classification accuracy is used to indicate the classification accuracy of the initial classification result; when the initial classification accuracy does not meet the preset end condition, iteratively extract at least one target classification feature of the model to be classified, and determine the target classification result and the target classification accuracy based on at least one target classification feature; when the target classification accuracy meets the preset end condition, the target classification result is used as the final classification result.

[0148] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0149] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0150] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0151] The electronic device 600 may also communicate with one or more external devices 640 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0152] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0153] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above model classification method when executed by a processor.

[0154] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided, which may be a readable signal medium or a readable storage medium. Figure 7 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. Figure 7 As shown, the computer-readable storage medium stores a program product capable of implementing the above method of the present disclosure. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0155] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0157] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0158] In a specific implementation, the program code for performing the operation of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0159] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0160] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0161] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope of the present disclosure is indicated by the appended claims.

Claims

1. A model classification method, characterized in that: include: Extracting at least one initial classification feature of the model to be classified; Determine an initial classification result and an initial classification accuracy according to at least one initial classification feature, wherein the initial classification accuracy is used to indicate the classification accuracy of the initial classification result; When the initial classification accuracy does not meet the preset end condition, iteratively extracting at least one target classification feature of the model to be classified, and determining a target classification result and a target classification accuracy according to the at least one target classification feature; When the target classification accuracy meets the preset end condition, the target classification result is used as the final classification result.

2. The model classification method according to claim 1, characterized in that: The step of extracting at least one initial classification feature of the model to be classified includes: Inputting the model to be classified into a feature extraction model to obtain at least one initial classification feature; The step of determining the initial classification result and the initial classification accuracy according to at least one initial classification feature includes: At least one initial classification feature is input into the classification model to obtain the initial classification result and the initial classification accuracy.

3. The model classification method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining the model to be classified; The step of extracting at least one initial classification feature of the model to be classified includes: When the user does not classify the model to be classified according to the current classification label, at least one initial classification feature of the model to be classified is extracted.

4. The model classification method according to claim 3, characterized in that: After taking the target classification result as the final classification result, the method further includes: Clustering the models to be classified to obtain at least one model cluster; Determine a classification label corresponding to at least one model clustering cluster; The current classification label is updated according to the classification label corresponding to at least one model cluster to obtain an updated classification label.

5. The model classification method according to claim 4, characterized in that: After the current classification label is updated according to the classification label corresponding to at least one model cluster to obtain the updated classification label, the method further includes: Clustering the updated classification labels to obtain multiple label clusters; Screening at least one group of label clusters satisfying a preset merging condition from a plurality of label clusters, wherein any group of label clusters includes at least two of the label clusters; The updated classification labels corresponding to at least one group of label clusters are merged respectively to obtain merged classification labels.

6. The model classification method according to claim 1 or 2, characterized in that: The model to be classified is an artificial intelligence AI model, and the type of the initial classification feature includes at least one of AI capability application scenario features, AI capability solution features, algorithm features corresponding to the AI ​​model, and label data features.

7. The model classification method according to claim 2, characterized in that: The method further comprises: Constructing the feature extraction model based on discrete binary particle swarm BPSO algorithm; The classification model is constructed based on the convolutional neural network (CNN) algorithm.

8. A model classification device, characterized in that: include: An initial classification feature extraction module, used to extract at least one initial classification feature of the model to be classified; An initial classification result determination module, used to determine an initial classification result and an initial classification accuracy according to at least one initial classification feature, wherein the initial classification accuracy is used to indicate the classification accuracy of the initial classification result; A target classification feature extraction module, used for iteratively extracting at least one target classification feature of the model to be classified when the initial classification accuracy does not meet the preset end condition, and determining the target classification result and the target classification accuracy according to the at least one target classification feature; The final classification result determination model is used to take the target classification result as the final classification result when the target classification accuracy meets the preset end condition.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the model classification method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model classification method according to any one of claims 1 to 7 is implemented.

Images