A knowledge graph-based deep learning model recommendation method

Abstract

The present application belongs to the technical field of software engineering, and specifically relates to a deep learning model recommendation method based on a knowledge graph. The method comprises: a model knowledge graph construction method, taking open source code repositories and component-related text corpus as input, and outputting a model knowledge graph integrating AI repositories, models, components, implementations and other knowledge; a model recommendation method, taking existing and modified model implementation code as input, and outputting multiple most similar model reference implementations and corresponding explanation information; the explanation information comprises components used by the model, relationships between the components, descriptive knowledge related to the components and the like. The present application is based on a model knowledge graph integrating multiple sources of knowledge, and according to the code of the model input by a user, a model architecture is matched in a high-level semantic manner, so that a similar model is obtained. The present application fully utilizes background knowledge in the field of artificial intelligence, promotes the reuse of deep learning model implementations, and improves the development efficiency of AI application developers.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of software engineering, and particularly relates to a deep learning model recommendation method based on a knowledge graph. BACKGROUND

[0002] With the development of Artificial Intelligence (AI) technology, more and more software applications integrate deep learning models, thereby realizing various intelligent capabilities, such as intelligent human-computer interaction, intelligent recommendation, intelligent decision-making, etc. Generally speaking, application developers first need to find suitable deep learning models, and then write codes or reuse existing models. Since many application developers are not AI technology experts, finding similar models and then customizing them become a common development method. For this purpose, developers need to refer to some similar models to learn how to adjust some model components and their usage.

[0003] However, for AI application developers, it is not easy to find similar models and learn about their components and usage. AI has always been a very popular research field, and every year a large number of researchers publish a large number of AI papers, propose new AI models and components, open source their AI models, and release them to GitHub code repository hosting platforms. For example, Google Research released the code of the BERT model to the GitHub repository. In 2019, more than 120,000 peer-reviewed AI papers were published in various conferences and journals, and the number is still growing. In 2018, the number of repositories on GitHub exceeded 100 million. Although there are a large number of open source code repositories on GitHub, how to filter out AI-related repositories containing AI models and identify models in them is still a great challenge for AI application developers. At present, in the artificial intelligence community, researchers have established a free and open platform PapersWithCode containing AI papers, AI repositories, AI models, components, and other AI-related resources through crowdsourcing. PapersWithCode supports searching for models, components, and other AI resources. However, AI application developers still cannot find similar models from PapersWithCode. Because PapersWithCode does not support input model search and only supports input AI paper title, AI model, component, etc. And PapersWithCode cannot parse the model, so it cannot know the information about the components in the model.

[0004] GitHub, as a famous open source code hosting platform, can support users to input keywords to search repositories or codes, thus supporting model search to some extent. However, it is still difficult to search similar models from GitHub, because GitHub only supports inputting keywords with limited length, while the length of model implementation is often very long, which is difficult to directly input into GitHub. Moreover, the search results of GitHub are obtained based on simple keyword matching, and there are many noises in the results, such as repositories or codes irrelevant to AI.

[0005] In academia, there are many studies on deep learning models, but most of them focus on the model architecture itself or its security, explainability, fairness, etc., and there are few studies on model recommendation, especially similar model recommendation. Some general techniques such as code clone detection and sample code search can be applied to similar model recommendation. Code clone detection technology can detect syntax identical or highly similar codes from code repositories, and sample code search can also search similar code snippets by inputting codes. However, these methods are designed for general sample code search purposes and are not suitable for specific model recommendation tasks. This is because the implementation of the model may not be similar in code, but similar in the high-level semantics of the architecture or components of the model. Therefore, model recommendation needs to match the architecture of the model and consider the semantic relationship of the components with relevant background knowledge. In addition, customized modification of the model often needs to integrate multiple models, and needs to learn different components from different models. The final modified model may use multiple components from different models. In addition, the selection of the model also needs many AI-related background knowledge, but AI application developers may not be experts in the field of AI, so it is also important to provide necessary explanation information for the model, such as the components used in the model, the relationship between the components, the characteristics of the components, etc. SUMMARY

[0006] The purpose of the present application is to provide a deep learning model recommendation method based on a knowledge graph to promote the reuse of deep learning model implementation and improve the development efficiency of AI application developers.

[0007] The deep learning model recommendation method based on the knowledge graph provided by the present application includes an offline model knowledge graph construction method and an online model recommendation method, which realizes deep learning model recommendation, and the overall process is as shown in Figure 1 As for the model knowledge graph construction method, the present application takes open source code repositories and component-related text corpus as input, and outputs a model knowledge graph that integrates multiple sources of knowledge (including AI repositories, models, components, implementation methods, and other related knowledge). The high-level model of the knowledge graph constructed by the present application is as shown in Figure 2The model recommendation method is described. As to the model recommendation method, the present application takes the existing and modified model implementation code as input, and outputs a plurality of most similar model references and corresponding explanation information. The explanation information includes components used by the model, relationships between the components, descriptive knowledge related to the components, and the like.

[0008] The present application is based on a model knowledge graph that fuses multi-source knowledge. The model knowledge graph can match model architectures based on user input model implementation code, and obtain similar model references in high-level semantics. The present application can construct a model knowledge graph by fusing multi-source knowledge, fully utilize background knowledge in the field of artificial intelligence, promote the reuse of deep learning model implementation, and improve the development efficiency of AI application developers.

[0009] (I) Offline model knowledge graph construction method

[0010] First, the trained classifier is used to identify AI-related open source repositories. Then, the model implementation class is extracted from the repository code, and the components and the dependency relationships between the components are extracted from the model implementation code through abstract syntax tree analysis and heuristic rules. Then, based on the prefix-based pattern mining method, high-level semantic concepts are mined from the model and the components, and the high-level concepts of the components are linked to the component types provided in PapersWithCode. In addition, the characteristics of the components are extracted from the text corpus of the components using dependency relation analysis and part-of-speech tagging, the descriptions of the components are extracted using heuristic rules, and the open relationships between the components are extracted based on open information extraction technology. The specific steps are as follows.

[0011] (1) AI repository identification.

[0012] A text classifier is trained using manually collected ReadMe datasets to classify the ReadMe text in open source code repositories and identify AI-related repositories. Each open source code repository often contains a ReadMe file, which may record the background, main content, dependency configuration, reference documents and other important information of the open source project. These information is very useful for understanding and using open source code repositories. The specific operation is as follows:

[0013] First, the data on PapersWithCode (https: / / github.com / paperswithcode / paperswithcode-data) is used to manually construct a ReadMe label dataset. Then, a text classifier model is trained on the dataset, which can predict whether the ReadMe text of an open source code repository is an AI repository based on the ReadMe text. Finally, the text classifier model is used to identify AI repositories from a large number of open source repositories.

[0014] (2) Train a text classifier to extract models.

[0015] A text classifier (also a model's class code classifier) is trained on a dataset constructed by humans to extract models from open-source code. The AI repository contains a large amount of code, only a small part of which is about implementing models. In code files, models are usually implemented by a complete class code block, and the content of the class code block (such as class name, method name contained in the class) has obvious characteristics that distinguish it from non-model implementation code, so a text classification method is used to extract model implementation code. The specific operation is as follows:

[0016] First, use heuristic rules (such as files ending in ".py") to identify code files from open-source code repositories, and further use regular expressions (such as ) to cut class code blocks from the identified code files; then, use the data on PapersWithCode to manually construct a label dataset for models, which contains randomly sampled class code blocks and their annotations of whether they are model implementation classes; then, use a supervised learning method to train a model classifier that can predict whether a class code block is a model implementation class based on the class code block; finally, use the trained text classifier to classify the class code blocks extracted from large-scale open-source code, and extract models based on the classification results.

[0017] (3) Component and dependency extraction.

[0018] Use abstract syntax tree analysis and heuristic rules to extract component instances, components, and dependencies between components from model implementation class code. Components often perform input-output transmission, which is represented in code as parameter transmission, and can be extracted by analyzing the abstract syntax tree of the code. In addition, the code where the component appears is also considered as the usage code of the component, and the implementation code of the component is searched from the code repository where the model is implemented. The specific operation is as follows:

[0019] First, use a code static analysis tool to convert the model implementation class code into an abstract syntax tree, traverse the abstract syntax tree, and identify class objects involved in assignment statements in the constructor as component instances and class names as components. Then, based on the abstract syntax tree analysis, obtain the parameter transmission relationship between local variables in the code, and use it as the dependency relationship of the component corresponding to the local variable. Finally, since the same code repository often implements components in the form of classes, use string matching to match the class names of all class code blocks extracted from the code repository with the component names identified, and if the class name and the component name are the same, it is considered as the implementation code of the component.

[0020] (4) High-level concept mining.

[0021] In order to make the model knowledge graph have higher level of concept understanding ability and deeper semantic reasoning ability, it is necessary to extract high-level semantic concepts from the model and components. The invention adopts a prefix and suffix based pattern mining method to mine high-level semantic concepts from the extracted models and components. The specific operation is as follows:

[0022] Firstly, all model or component nodes are "hump type" disassembled to obtain a word segmentation list; then, from the word segmentation list, a set of N-gram prefix and suffix words is obtained; then, the prefix and suffix word set is matched with the model or component set, and the matched prefix and suffix word is added to the high-level concept set as a high-level concept; finally, the class relationship is added between the model or component containing the high-level concept and the corresponding high-level concept.

[0023] (5) Component type linking.

[0024] The component type information in PapersWithCode contains component type, component type description, and component information. The invention links the extracted component high-level concept and the component type in PapersWithCode. First, after preprocessing, 350 component types and 1,802 components are obtained from the component data set provided by PapersWithCode, and they are converted into component-component type mapping as the input of the algorithm; then, each component in the component high-level concept and the component type information is traversed in turn, and keyword matching is performed; if the matching is successful, the high-level concept is directly linked to the corresponding component type; otherwise, the pre-trained Wikipedia word vector provided by Google (https: / / code.google.com / archive / p / word2vec / ) is used to vectorize the high-level concept and the component, and at the same time, the word-based Jaccard algorithm is used to measure the similarity between the high-level concept and the component by calculating the cosine similarity and Jaccard text similarity between the vectors. If the sum of the two similarities is greater than the self-defined similarity threshold, the high-level concept is linked to the corresponding component type.

[0025] (6) Component descriptive knowledge extraction.

[0026] In order to enrich the model knowledge graph, and to enhance the explainability of the model, the application extracts component descriptive knowledge. First, component-related text corpus is screened out from PapersWithCode, Wikipedia, Wikidata and AI-related paper abstracts; then, three different ways are used to analyze the text corpus respectively to obtain three types of descriptive knowledge to add more descriptive knowledge to the component, including component feature extraction, component description extraction and component open relationship extraction. The component feature extraction step uses natural language processing tools to perform dependency relation analysis and part-of-speech tagging on the sentences in the component corpus, and the nominative parses the subject in the sentence. If the subject is a component, the adjective or adverb or noun phrase in the sentence is extracted as the feature of the component. The component description extraction step extracts the sentences starting with the component name, or the sentences containing both the component name and the component feature as the description of the component. The component open relationship extraction step uses an open relationship extraction tool to extract the scored (head entity, relationship, tail entity) triple open relationship from the component corpus, and filters out the triple whose score is greater than a certain threshold and whose head entity and tail entity are both components to supplement the relationship between the components in the knowledge graph.

[0027] (II) Online model reference implementation recommendation method

[0028] First, the components used by the model and the dependency relationship between the components are identified from the input model implementation code that needs to be modified; then, the identified components are mapped to the high-level concepts in the model knowledge graph, and the model implementations with the same high-level concepts are searched as candidate model implementations; finally, through the kernel method of the graph, the similarity between each candidate model and the corresponding high-level concept relationship graph of the input model is calculated, and the similarity is sorted and clustered to select the top k most similar model implementations as the reference implementation of the model and the corresponding explanation. The specific steps are as follows:

[0029] (1) Component and dependency relationship extraction.

[0030] From the user input model implementation code, the components used in the user input model implementation and the dependency relationship between the components are extracted by code static analysis to generate a component dependency relationship graph. The overall method is similar to the component and dependency relationship extraction method in the model knowledge graph construction method. The specific operation is as follows:

[0031] First, the code static analysis tool is used to convert the user input model implementation class code into an abstract syntax tree, and the class objects involved in the assignment statements in the constructor are identified as component instances, and the class names are identified as components. Then, based on the abstract syntax tree analysis, the parameter passing relationship between local variables in the code is obtained, which is used as the dependency relationship of the component corresponding to the local variable.

[0032] (2) Candidate result generation.

[0033] In order to obtain the results similar in high-level semantics, the components are abstracted in high level. First, the component list used in the model implementation code input by the user is obtained. Then, the components are conceptually mapped to the concept nodes in the model knowledge graph, and the concept nodes are expanded upward to obtain the most abstract concept nodes that each component can be mapped to. Finally, according to the abstract concept node set of the component, the model implementation with intersection with the abstract concept is screened, and the candidate results similar in high-level semantics are obtained. The specific operation is as follows:

[0034] First, the components extracted in the component and dependency extraction step are directly mapped to a specific component node in the model knowledge graph according to the name of the component. If the direct text matching does not map to the component node, the component is fuzzy matched with the high-level concept node in the model knowledge graph. If the component name contains a component concept, the component is mapped to the component concept node. If it still cannot be mapped successfully, it is considered that the component may be extracted incorrectly or has very low usage, which has little effect on the result, and can be directly discarded.

[0035] Then, after the component is conceptually mapped to the corresponding node in the model knowledge graph, the node is taken as the starting point to expand upward in the knowledge graph, and the nodes having the superior-inferior relationship with the starting point are taken as new starting points. When the expansion reaches the component type node or the topmost concept node that can be expanded, the expansion is stopped, and the node is taken as the abstract concept node of the component. For example, the node "nn.LSTM" obtained after conceptual mapping is conceptually expanded to find its high-level concept "LSTM" node, and further expanded to find its component type "Recurrent Neural Networks" node. Finally, the most abstract concept of the component "nn.LSTM" is "Recurrent Neural Networks". After conceptual mapping and conceptual expansion, the abstract concept list corresponding to the component list in the model implementation input by the user can be obtained.

[0036] In order to further screen the similar model implementations as candidate results, the components used in each model implementation in the model knowledge graph are processed by conceptual expansion to obtain the "model implementation-component abstract concept list" mapping, which is matched with the abstract concept list corresponding to the components in the model implementation input by the user to screen the model implementations with intersection as the candidate results.

[0037] (3) Candidate result sorting.

[0038] The model can be considered as a subgraph composed of components and dependency relationships between the components. The present application calculates subgraph similarity of the subgraph of the user input model and the subgraph of all candidate models on the basis of screening out the candidate model, and sorts the candidate results according to the subgraph similarity, so as to return similar models with high-level semantic similarity to the user input model. The specific operation is as follows:

[0039] Firstly, the model implementation-component abstract concept list obtained in the last step is used to convert the user input model and the candidate model into a subgraph composed of component abstract concepts. Each node in the subgraph is a component abstract concept corresponding to the component in the model implementation, and each edge corresponds to the component dependency relationship in the model implementation. Then, the Weisfeiler-Leman Graph Kernel algorithm [1] is used to calculate the graph similarity between the input user input model subgraph and the candidate model subgraph. The Weisfeiler-Leman Graph Kernel algorithm can map graph data to fixed-length vector representations in the same high-dimensional space, and the vector representations of graph data with similar structured information are very close, i.e. the cosine similarity is low. Next, the candidate models with similarity greater than the self-defined threshold are screened out, and the candidate models are SinglePass clustered [2] according to the graph similarity. Highly similar candidate models will be clustered into a cluster, and each cluster only retains the candidate model of the cluster center. Finally, the most similar top k clustered model sample codes and their explanations are returned as the result, and the explanations include the components used by the model, the relationships between the components, the descriptive knowledge related to the components, and the like obtained based on the model knowledge graph. The present application supports an interactive model recommendation user interface, as shown in Figure 3 Firstly, the model implementation code input by the user is parsed to analyze the components used in the model and the dependency relationships between the components. Then, the component high-level concepts and the component dependency relationships with high similarity are searched out as references through the concept understanding and semantic reasoning ability of the model knowledge graph. Finally, the model sample code is supplemented with various explainability information such as components, component dependency relationships, component descriptive knowledge, and correspondence with the components in the input. In addition, the user can also screen the search results to screen out models using specific components.

[0040] The method of the present application has the following characteristics:

[0041] (1) The model knowledge graph construction method is designed, taking the open source code repository and the component related text corpus as the input, and outputting a model knowledge graph which fuses the knowledge of AI repository, model, component, implementation code and the like, and specifically includes the key steps of AI repository identification, model and implementation extraction, component and dependency relationship extraction, high-level concept mining, component type linking, component descriptive knowledge extraction and the like;

[0042] (2) The model recommendation method is designed, taking the implementation code of the existing and modified model as the input, and based on the model knowledge graph which fuses multiple source knowledge, the similar model reference is obtained by matching the model architecture at the high-level semantics. In the meantime, the explanation information including the components used by the model, the relationship between the components, the descriptive knowledge related to the components and the like is provided. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The overall flowchart of the present application.

[0044] Figure 2 The high-level concept model graph of the model knowledge graph involved in the present application.

[0045] Figure 3 The model reference implementation recommendation user interface involved in the present application. DETAILED DESCRIPTION

[0046] The present application will be further described below by way of examples in conjunction with the drawings.

[0047] By using the method of the present application, based on nearly 70,000 AI repositories on GitHub, the model knowledge graph is constructed and the deep learning model recommendation based on the model knowledge graph is implemented, and the specific steps are as follows:

[0048] (1) The classifier of AI repository.

[0049] Using the data on PapersWithCode, a ReadMe label dataset is artificially constructed, and various text classification models are compared, and finally a convolutional neural network (CNN) is selected as the text classification model for AI repository classification. The CNN text classification model is composed of an Embedding layer (vector layer), a convolutional layer, a pooling layer, a fully connected layer, and a Softmax layer (normalization layer). First, the Embedding layer is used to encode the input open source code repository ReadMe text into a vector representation. Then, the convolutional layer extracts different features from the vector into the pooling layer. Then, the pooling layer selects important features from the extracted features and inputs them into the fully connected layer. The fully connected layer integrates the extracted features and inputs them into the Softmax layer. Finally, the Softmax layer outputs the result, i.e., whether the input ReadMe text is AI-related, thereby determining whether the corresponding code repository is an AI repository.

[0050] To train the CNN text classification model, a supervised learning method is used. Using the repository links provided by PapersWithCode, the ReadMe text of the code repository is crawled, and 1,000 ReadMe are randomly selected to construct positive sample data. Another 1,000 ReadMe are randomly selected from large-scale open source repositories to construct negative sample data. The specific model implementation uses the Kashgari framework (https: / / kashgari-zh.bmio.net / ). Kashgari is an NLP transfer learning framework commonly used for text tagging and text classification.

[0051] (2) Model class code classifier.

[0052] According to the Kashgari framework, various text classification models are built and compared, and finally a two-layer bidirectional long short-term memory network (Bi-directional Long Short-Term Memory, BiLSTM for short) is selected as the classifier of the model class code to identify the model implementation class code. The model is composed of an Embedding layer, a two-layer BiLSTM layer, a Dropout layer (random inactivation), a fully connected layer and a Softmax layer. First, the Embedding layer is used to encode the input model implementation class to be identified into a vector representation; then, the context information is extracted through the two-layer BiLSTM layer; then, in order to prevent overfitting, the Dropout layer randomly sets some hidden layer node weights to be invalid, and enters the fully connected layer; the fully connected layer integrates the information extracted before and inputs it to the Softmax layer; finally, the Softmax layer outputs the result, that is, whether the input class code is a model implementation class.

[0053] In order to train the model implementation class code classifier, according to the models and corresponding code repositories in the PapersWithCode dataset, 18,033 class codes with class name equal to model name are selected from the class code blocks in the corresponding code repository as positive sample data. Then, the same number of non-model implementation class codes are manually selected from the remaining class codes as negative sample data, and the class code label data set is divided into training set, validation set and test set according to the ratio of 6:2:2.

[0054] (3) Component descriptive knowledge extraction.

[0055] Component descriptive knowledge extraction includes component property extraction, component description extraction, and component open relationship extraction. The present application uses the natural language processing tool spaCy (https: / / spacy.io / ) to perform dependency relationship analysis and part-of-speech tagging on the sentences in the component corpus. The subject in the sentence is parsed, and if the subject is a component, the adjective or adverb or noun phrase is extracted from the sentence as the property of the component. At the same time, using the component property extraction step, the extracted component properties, and according to the heuristic rules, sentences starting with the component name or containing both the component name and the component properties are extracted from the component corpus as the description of the component. Finally, the OpenIE tool of StanfordNLP (https: / / nlp.stanford.edu / software / openie.html) is used to extract the triple open relationship (head entity, relationship, tail entity) from the component corpus, and then the open relationship between the components with high scores is selected to supplement the relationship between the components in the knowledge graph.

[0056] (4) Model knowledge graph construction.

[0057] First, 65,279 Python open source code repositories are cloned from GitHub according to the dataset containing 93,303 open source repository links provided by PapersWithCode. At the same time, 81 open source code repositories of different versions of common Python deep learning frameworks such as TensorFlow, PyTorch, Keras, MXNet, Chainer and Theano are also cloned. In addition, 4,710 open source code repositories are identified from the AI repository identification method of the text classification model, and the Python third-party library provided by the Libraries.io dataset (https: / / libraries.io / ). The above three parts of the open source code repository are used as the knowledge graph construction open source code repository dataset. Further, 1,647 component text description fragments are obtained from the component dataset provided by PapersWithCode; 4,847 component concept descriptions are obtained from the Wikipedia dataset and Wikidata dataset; 680,475 components contained in the abstracts of 65,084 AI-related papers are screened out. The above four parts of component-related text data are used as the component corpus used for model knowledge graph construction. Through the above knowledge graph construction steps, a model knowledge graph with 651,328 entity nodes and 1,810,335 relationships is generated. Among them, the entity nodes include 21,359 open source repository nodes, 42,577 model implementation nodes, 29,019 component implementation nodes, 24,063 model nodes, 20,189 model high-level concept nodes, 102,969 component nodes, 222,381 component instance nodes, 198,80 component high-level concept nodes, 161,970 component usage code nodes, 350 component type nodes, 4,087 component property nodes and 2,484 component description nodes. The relationships include 119,654 subclass of relationships, 3838 belong to relationships, 664423 use relationships, 121363 provide relationships, 126463 implement relationships, 222381 instance of relationships, 270976 has usagecode relationships, 118073 follow relationships and 163,164 open relationships.

[0058] The present application evaluates the key steps of model knowledge graph construction through multiple experiments, wherein the accuracy rate of AI warehouse recognition reaches 98.94%, and the accuracy rate of model implementation class extraction reaches 98.00%. Further, 384 tuples in the model knowledge graph are sampled for manual annotation, and the final accuracy rate reaches 90.63%. Thus, it is proved that each step of model knowledge graph construction is very effective in general, and the overall quality of the model knowledge graph is high. We invite 10 developers to complete the model modification task experiment to verify the effectiveness of the present application in model recommendation. In the experiment, a group of experiment participants are required to use the present application to find similar models for the given AI model and complete the modification task. Another group of experiment participants use the GitHub method as a comparison to complete the same task. The experiment proves that the present application is effective in recommending models. Using the present application, the participants' satisfaction with the modification results of the model is about 36.4% higher than using GitHub, the task completion time is shortened by about 41.2%, the search times are reduced by about 58.5%, and the search result ranking is improved by about 79.5%. The experiment participants think that the present application is more useful and easier to use than GitHub.

[0059] REFERENCES

[0060] [1] Shervashidze N, Schweitzer P, Leeuwen EJ, Mehlhorn K, Borgwardt KM. Weisfeiler-Lehman Graph Kernels. Journal of Machine Learning Research, 2011, 12.

[0061] [2] Papka R, Allan J. On-line new event detection using single-pass clustering[J]. University of Massachusetts, Amherst, 1998, 10(290941.290954).

Claims

1.A knowledge graph-based deep learning model recommendation method, characterized in that, The method is divided into two parts: offline model knowledge graph construction method and online model recommendation method, and the deep learning model recommendation is realized. (1) The offline model knowledge graph construction method takes the open source code repository and the component related text corpus as input, and outputs a model knowledge graph that integrates multiple sources of knowledge, including AI repository, model, component, and implementation related knowledge; including: Firstly, the trained classifier is used to identify the AI related open source repository; Then, the implementation class of the model is extracted from the repository code, and the dependency relationship between the components is extracted from the model implementation code through abstract syntax tree analysis and heuristic rules; Then, based on the prefix and suffix pattern mining method, high-level semantic concepts are mined from the model and component, and the high-level concepts of the component are linked with the component types provided in PapersWithCode; In addition, the characteristics of the component are extracted from the text corpus of the component using dependency relation analysis and part-of-speech tagging, the description of the component is extracted using heuristic rules, and the open relationship between components is extracted based on open information extraction technology; (2) The online model recommendation method takes the existing and modified model implementation code as input, and outputs multiple most similar model references and corresponding explanation information; the explanation information includes the components used by the model, the relationship between the components, and the descriptive knowledge related to the components; including: Firstly, the components used by the model and the dependency relationship between the components are identified from the input model implementation code that needs to be modified; Then, the identified components are mapped to the high-level concepts in the model knowledge graph, and the model implementations with the same high-level concepts are searched as candidate model implementations; Finally, through the kernel method of the graph, the similarity between each candidate model and the corresponding high-level concept relationship graph of the input model is calculated, and the similarity is sorted and clustered to select the top k most similar model implementations as the reference implementation of the model and the corresponding explanation; Based on the model knowledge graph that integrates multiple sources of knowledge, the model architecture is matched according to the user input model implementation code, and the similar model reference is obtained; by constructing the model knowledge graph by integrating multiple sources of knowledge, the background knowledge in the field of artificial intelligence is fully utilized, the reuse of deep learning model implementation is promoted, and the development efficiency of AI application developers is improved. 2.The knowledge graph based deep learning model recommendation method of claim 1, wherein, The specific steps of the offline model knowledge graph construction method are as follows: (1) AI repository identification; A text classifier is trained by manually collecting ReadMe dataset, which is used to classify the ReadMe text in open source code repository and identify AI related repositories; the specific process is as follows: Firstly, the ReadMe label dataset is constructed manually based on the data on PapersWithCode; Then, a text classifier model is trained on the dataset, which can predict whether the ReadMe text of the open source code repository is an AI repository; Finally, the text classifier model is used to identify the AI repository from a large number of open source repositories; (2) Train the text classifier and extract the model; A text classifier, also called a class code classifier of a model, is trained from a manually constructed dataset to extract models from open source code; the specific process is as follows: First, heuristic rules are used to identify code files from open source code repositories, and regular expressions are used to further cut class code blocks from the identified code files; Then, using the data on PapersWithCode, a manually constructed model label dataset is constructed, which contains randomly sampled class code blocks and their annotations of whether they are model implementation classes; Next, a supervised learning method is used to train a model classifier that can predict whether a class code block is a model implementation class based on the class code block; Finally, the trained text classifier is used to classify class code blocks extracted from large-scale open source code, and models are extracted based on the classification results; (3) Component and dependency relationship extraction; Abstract syntax tree analysis and heuristic rules are used to extract component instances, components, and dependency relationships between components from model implementation class code; the specific process is as follows: First, use a code static analysis tool to convert the model implementation class code into an abstract syntax tree, traverse the abstract syntax tree, and identify class objects involved in assignment statements in the constructor as component instances and class names as components; Then, based on the abstract syntax tree analysis, the parameter passing relationship between local variables in the code is obtained, which is used as the dependency relationship of the component to which the local variable belongs; Finally, since the same code repository implements components in the form of classes, the class names of all class code blocks extracted from the code repository are matched with the component names identified, and if the class name and component name are the same, they are considered as the implementation code of the component; (4) High-level concept mining; In order to make the model knowledge graph have higher-level concept understanding ability and deeper semantic reasoning ability, high-level semantic concepts are extracted from models and components; the specific method is to use a prefix and suffix-based pattern mining method to extract high-level semantic concepts from the extracted models and components; the specific process is as follows: First, the "CamelCase" is broken down to obtain a list of tokens; Next, from the token list, a set of N-gram prefixes and suffixes is obtained; Then, the prefix and suffix word set is matched with the model or component set, and the successfully matched prefix and suffix word is added to the high-level concept set as a high-level concept; Finally, the model or component containing the high-level concept and the corresponding high-level concept are added with a generic relationship; (5) Component type linking; The component type information in PapersWithCode includes component type, component type description, and component information; the extracted component high-level concept is linked to the component type in PapersWithCode; the specific process is as follows: First, after preprocessing, 350 component types and 1,802 components are obtained from the component dataset provided by PapersWithCode, and the component-component type mapping is converted into an algorithm input; Then, each component in the component high-level concept and component type information is traversed in turn to perform keyword matching; if matching is successful, the high-level concept is directly linked to the corresponding component type; otherwise, the high-level concept and the component are vectorized by using the pre-trained Wikipedia vector provided by Google, and the similarity between the high-level concept and the component is measured by calculating the cosine similarity and the Jaccard text similarity between the vectors by using the word-based Jaccard algorithm; if the sum of the two similarities is greater than a self-defined similarity threshold, the high-level concept is linked to the corresponding component type; (6) Component descriptive knowledge extraction; First, component-related text corpus is screened out from PapersWithCode, Wikipedia, Wikidata and AI-related paper abstracts; Then, three different ways are used to analyze the text corpus to obtain three types of descriptive knowledge to add more descriptive knowledge to the component, including component feature extraction, component description extraction and component open relationship extraction; wherein: Component feature extraction is to parse the dependency relationship and part-of-speech tagging of the sentence in the component corpus by using a natural language processing tool, and the subject in the sentence is parsed by using the genitive case; if the subject is a component, the adjective or adverb or noun phrase in the sentence is extracted as the feature of the component; Component description extraction is to extract the sentence starting with the component name, or the sentence containing both the component name and the component feature as the description of the component; Component open relationship extraction is to extract the head entity, relationship and tail entity triple open relationship with a score from the component corpus by using an open relationship extraction tool, and to filter out the triple whose score is greater than a certain threshold and whose head entity and tail entity are both components to supplement the relationship between the components in the knowledge graph. 3.The knowledge graph based deep learning model recommendation method of claim 2, wherein, The online model reference implementation recommendation method comprises the following steps: (1) Component and dependency relationship extraction From the user input model implementation code, the components used in the user input model implementation and the dependency relationships between the components are extracted by using code static analysis to generate a component dependency relationship graph; the specific process is as follows: First, the user input model implementation class code is converted into an abstract syntax tree by using a code static analysis tool, the class objects involved in the assignment statements in the constructor function are recognized as component instances, and the class names are recognized as components by traversing the abstract syntax tree; Then, the parameter passing relationship between local variables in the code is analyzed based on the abstract syntax tree to obtain the dependency relationship of the local variable with respect to the component; (2) Candidate result generation; The specific operation is as follows: First, the components extracted in the component and dependency relationship extraction step are directly mapped to a specific component node in the model knowledge graph according to the name of the component; if the text matching is not directly mapped to the component node, the component is fuzzy matched with the high-level concept node in the model knowledge graph; if the component name contains a certain component concept, the component is mapped to the component concept node; if the mapping is still unsuccessful, it is considered that the component may be extracted incorrectly or has a very low usage rate, which has little effect on the result, and the component can be directly discarded; Then, the component is mapped to the corresponding node in the model knowledge graph, and the node is used as the starting point to expand upwards in the knowledge graph to obtain nodes with hierarchical relationships as new starting points. When the expansion reaches the component type node or the topmost concept node, it stops and takes the node as the abstract concept node of the component. After concept mapping and expansion, an abstract concept list corresponding to the component list in the user-input model implementation is obtained. To further filter similar model implementations as candidate results, the components used in each model implementation in the model knowledge graph are processed through concept expansion to obtain a "model implementation-component abstract concept list" mapping, which is matched with the abstract concept list corresponding to the components in the user-input model implementation to filter out model implementations with intersections as candidate results. (3) Candidate result sorting A model is regarded as a subgraph composed of components and their dependencies. Based on the selected candidate models, the subgraph similarity of the user-input model and all candidate models is calculated, and the candidate results are sorted according to the subgraph similarity to return similar models with high-level semantic similarity to the user-input model. The specific process is as follows: First, use the "model implementation-component abstract concept list" obtained in the previous step to convert the user-input model and candidate models into subgraphs composed of component abstract concepts. Each node in the subgraph is a component abstract concept corresponding to a component in the model implementation, and each edge corresponds to a component dependency in the model implementation. Then, use the Weisfeiler-Leman Graph Kernel algorithm to calculate the graph similarity between the user-input model subgraph and the candidate model subgraph. The Weisfeiler-Leman Graph Kernel algorithm maps graph data to fixed-length vector representations in the same high-dimensional space, and graph data with similar structured information has very close vector representations, i.e., low cosine similarity. Next, filter out candidate models with a similarity greater than a custom threshold, and perform Single Pass clustering on the candidate models based on the graph similarity. Highly similar candidate models will be clustered into a cluster, and each cluster will only retain the candidate model as the cluster center. Finally, return the top k most similar clustered model examples and their explanations as the result. The explanations include the components used in the model, the relationships between the components, and the descriptive knowledge related to the components based on the model knowledge graph. 4.The knowledge graph based deep learning model recommendation method of claim 3, characterized in that, User interface supporting interactive model recommendation First, analyze the components used in the model and their dependencies by parsing the user-input model implementation code. Then, search for model example codes with high similarity in component high-level concepts and component dependency relationships as references through the concept understanding and semantic reasoning capabilities of the model knowledge graph. Finally, the model example code is supplemented with components, component dependencies, component descriptive knowledge, and correspondence with components in the input. In addition, users can also filter search results to find models that use specific components. 5.The knowledge graph based deep learning model recommendation method according to any one of claims 1-4, characterized in that, The text classifier for AI warehouse identification adopts a convolutional neural network (CNN) model composed of a vector layer, a convolution layer, a pooling layer, a full connection layer, and a normalization layer. First, the vector layer is used to encode the input open source code repository ReadMe text into a vector representation. Then, the convolution layer extracts different features from the vector into the pooling layer. The pooling layer then selects important features from the extracted features and inputs them into the full connection layer. The full connection layer integrates the previously extracted features and inputs them into the normalization layer. Finally, the normalization layer outputs the result, i.e., whether the input ReadMe text is AI-related, thereby determining whether the corresponding code repository is an AI warehouse. The model's class code classifier adopts a two-layer bidirectional long short-term memory network (BiLSTM) composed of a vector layer, a two-layer BiLSTM layer, a Dropout layer, a full connection layer, and a Softmax layer. First, the vector layer is used to encode the input model implementation class to be identified into a vector representation. Then, the two-layer BiLSTM layer extracts context information. To prevent overfitting, the Dropout layer randomly sets some hidden layer node weights to zero and enters the full connection layer. The full connection layer integrates the previously extracted information and inputs it into the Softmax layer. Finally, the Softmax layer outputs the result, i.e., whether the input class code is a model implementation class. 6.The knowledge graph based deep learning model recommendation method according to any one of claims 1-4, characterized in that, ​

Images