Multi-dimensional label automatic classification and display system and method in AI model community

Abstract

The invention relates to the technical field of artificial intelligence model management, in particular to a multi-dimensional label automatic classification and display system and method in an AI model community, and the system comprises a file analysis and preprocessing module, a dimension information analysis layer and a label generation and classification module. The file analysis and preprocessing module responds to an analysis request of a model warehouse, generates a file path list of the model warehouse, and analyzes metadata information from a document file; the dimension information analysis layer comprises a structure analysis module, a purpose identification module, a framework analysis module and a scale calculation module which work in parallel; according to the method, the multi-source model file can be automatically analyzed and preprocessed, multiple mainstream model formats are supported, and the structure information of the model is automatically extracted; the purpose and the framework of the model can be automatically identified according to the model file, so that the tag generation accuracy is improved; the scale of the model can be automatically calculated according to the model file, the total parameter number of the model and the total size of the model are output, and manual calculation is not needed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence model management technology, and in particular to a system and method for automatic classification and display of multi-dimensional tags in AI model communities. Background Technology

[0002] With the development of deep learning technology, models are increasingly widely used in various artificial intelligence tasks. A large number of pre-trained models are released and shared in file formats (such as .pt, .bin, .nnx, .safetensrs, etc.), and these models are widely distributed in public model repositories (such as HuggingFace, MdelScpe) and enterprise internal MLps platforms. Model files not only contain weight parameter information, but also implicitly contain key attributes such as the model's structural design, intended use, framework type, and parameter size.

[0003] Existing model management platforms generally rely on developers manually uploading structural metadata or filling in configuration files (such as cnfig.jsn) to label the model's purpose, architecture type, number of parameters, and other tags. This method depends on manual input, is prone to errors and inconsistencies, and struggles to cover all the detailed features of the file itself. Existing technologies also face the following challenges: 1. Model file structures are complex and diverse, lacking a unified standard and difficult to parse universally.

[0004] 2. There is a lack of mechanisms for automatically identifying model structure and hierarchical information.

[0005] 3. The purpose and task type of the model are difficult to infer directly from the file.

[0006] 4. Parameter size, accuracy and other indicators require complex calculations to obtain.

[0007] 5. Multidimensional tags lack a unified classification system and organizational logic. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by proposing a multi-dimensional label automatic classification and display system and method for AI model communities.

[0009] Firstly, in order to achieve the above objectives, the present invention adopts the following technical solution: a multi-dimensional label automatic classification and display system in an AI model community, including a file parsing and preprocessing module, a dimension information parsing layer, and a label generation and classification module; The file parsing and preprocessing module responds to the analysis request of the model repository, generates a list of file paths of the model repository, and parses metadata information from the document files; The dimensional information parsing layer includes a structure parsing module, a purpose identification module, a framework parsing module, and a scale calculation module that work in parallel. The structure parsing module extracts a set of structural features representing the model architecture design from the configuration file and code file based on the file path list and metadata information. The purpose identification module infers the application task type of the model based on the metadata information, document description content and code usage examples, and generates a set of purpose features. The framework parsing module identifies the deep learning framework that the model depends on based on the metadata information, file name characteristics, and configuration file content, and generates a framework feature set. The scale calculation module calculates the number of parameters and storage size of the model based on the weight file and configuration file, and generates a scale feature set. The tag generation and classification module receives the set of structural features, the set of usage features, the set of framework features, and the set of scale features. It uses a preset mapping table to perform keyword matching and maps the matching results to a standard tag library to generate standardized multidimensional tags for classification and display.

[0010] The system operates in a modular fashion. When the underlying service issues an analysis request, the system first accesses the model repository through the file parsing module to obtain the file structure. Subsequently, four parallel parsing modules process the data from different dimensions: the structure parsing module focuses on analyzing the model's hierarchy and architecture type; the usage identification module focuses on uncovering the model's application scenarios; the framework parsing module focuses on identifying underlying dependency libraries; and the scale calculation module focuses on quantifying the model size. Finally, the tag generation module aggregates all the above information and uses a built-in knowledge base to transform the messy metadata into standard tags.

[0011] For example, suppose a user uploads a model repository named Bert-Base-Chinese. The system will automatically initiate the process, identifying that the model is based on the BERT architecture, used for text classification, based on the PyTorch framework, and has 110 million parameters, ultimately displaying these four standard tags on the page.

[0012] As a further aspect of the present invention, the file parsing and preprocessing module is specifically configured as follows: By listening to message queues and consuming analysis requests, Git commands are used to pull remote model repositories to the local machine; Iterate through the repository to generate a list of file paths containing all subfiles; Parse the readme document in the root directory of the repository, extract the metadata information in YAML format and convert it into a JSON format object. The metadata information includes at least the tags field, the pipeline_tag field or the library_name field.

[0013] The file parsing and preprocessing module triggers tasks by listening to consumers in a message queue (such as Kafka). Once a task is received, the system calls the Git command-line tool to synchronize files from the remote repository to a local temporary directory via cloning or pulling operations. Next, the system uses a file traversal algorithm (such as TreeWalk) to generate a list containing all file paths. Simultaneously, the system reads the README.md file in the root directory, parses the YAML-formatted metadata block in the file header, converts it into a JSON object, and extracts key fields such as tags and pipeline tags.

[0014] For example: The system detects a new task and executes `git clone` to pull the repository. After traversing the repository, a list is generated: [config.json, pytorch_model.bin, README.md]. The `---` section at the beginning of the README file is read, and the library_name: transformers and tags: [text-classification] are extracted and temporarily stored as metadata objects.

[0015] As a further aspect of the present invention, the method by which the structure parsing module extracts the set of structural features includes: Examine the tags field in the metadata information and extract the structure-related tags; Read the configuration file and extract the model type field, architecture field, and structural parameter field. The structural parameter field includes the number of attention heads, the number of layers, or the size of the hidden layers. When the configuration file contains specific architecture feature fields, the architecture type is determined based on the combination of these fields; the specific architecture feature fields include attend_heads, encoder / decoder, kernel_size, or hidden_states.

[0016] First, the system checks the `tags` field in the metadata to directly obtain the user-annotated structural information. More importantly, it reads the `config.json` file from the model repository. The system parses this JSON file, looking for the `model_type` and `architecture` fields. Additionally, the system scans specific parameter fields, such as `num_attention_heads` (number of attention heads) and `hidden_size` (hidden layer size). The system also establishes feature field matching logic: if the configuration file contains `attend_heads` and an `encoder` or `decoder` field, the system determines it's a Transformer architecture; if it contains `kernel_size` (kernel size), it determines it's a CNN architecture.

[0017] For example, when parsing a model's config.json, if the system reads model_type: gpt2, it directly generates GPT2 structure labels. If the type is not read, but the file contains num_attention_heads: 12 and n_layer: 12, the system infers that it belongs to the Transformer class architecture based on rules.

[0018] As a further aspect of the present invention, the structure parsing module is also configured to analyze code files: Locate the class definitions that inherit from a deep learning framework base class, including nn.Module or tf.keras.Model; The class name pattern of the class definition is analyzed, the model architecture is inferred through the keywords in the class name, and the inference results are added to the structural feature set.

[0019] When configuration files lack structural information, the system performs in-depth analysis of Python code files in the repository (such as modeling_xxx.py). The system searches for class definitions using regular expressions or abstract syntax trees (such as ASTs), focusing on classes that inherit from the base classes of deep learning frameworks (e.g., torch.nn.Module or tf.keras.Model). Once found, the system analyzes the class name string, extracting uppercase letter combinations or specific keyword patterns to infer the model architecture.

[0020] For example, when the system scans a code file, it finds the line `classBertModel(nn.Module)`. The system recognizes that this class inherits from `nn.Module` and that the class name contains `Bert`, therefore it infers that the model is based on the BERT architecture and generates the corresponding label.

[0021] As a further aspect of the present invention, the method by which the purpose identification module generates the purpose feature set includes: Prioritize extracting the pipeline tag field from metadata information as task tags; Perform semantic analysis on the self-described document and its content, identify task description words and application scenario descriptions through large models or keyword matching, and format the recognition results into task descriptions. The analysis model uses code examples to extract model class names from import statements, infer the model's purpose based on the class name suffix, and filter out common prefixes.

[0022] The system prioritizes reading the `pipeline_tag` field from the metadata, which is the most direct task identifier. If missing, the system will perform semantic analysis on the README document. This can be achieved by calling the large language model interface or using a keyword matching algorithm. The system pre-defines a set of task description terms (such as `classification` and `detection`) and searches for these terms and their context (such as application scenario descriptions) in the document. Identified non-standard descriptions are formatted into a unified task ID.

[0023] For example: The metadata lacks a task tag. The system scans the README document and finds the sentence "This model is designed for recognizing cats in images." The system matches the keywords "recognizing" and "images," and through semantic analysis, categorizes it as an Image Classification task.

[0024] As a further aspect of the present invention, the purpose identification module, when analyzing the code examples used in the model, further includes: Identify the type of input preprocessing object; if a tokenizer object exists, it is identified as a text task; if a processor object exists, it is identified as an image task. Identify the calling method of the model object, and determine the task type based on the method name, which includes generate, classify, predict, or encode.

[0025] The system analyzes import statements and object instantiation patterns in the code. It checks whether the code imports specific preprocessor classes, such as Tokenizer (typically for NLP tasks) and ImageProcessor (typically for visual tasks). Furthermore, the system examines the calling methods of model objects. If `model.generate` is called in the code, the system infers a generative task; if `model.classify` or `predict` is called, it infers a discriminative task.

[0026] For example, if the code contains `from transformers import AutoModelForTokenClassification`, the system will parse the suffix `TokenClassification` and directly determine that the purpose is text classification or named entity recognition. Alternatively, if the code contains `inputs=tokenizer`, the system will confirm that this is a natural language processing-related model.

[0027] As a further aspect of the present invention, the frame parsing module generates the frame feature set in the following ways: Extract the library name field from the metadata information; Traverse the list of file paths, extract the file extensions and filenames of all files, and match them with the file characteristics of known frames; Read the configuration file, parse the auto-mapping fields, extract the framework prefix information indicating the loaded classes, and filter and clean the values.

[0028] The system first checks the `library_name` field in the metadata. Next, it iterates through file extensions: `.pt` or `.bin` typically correspond to PyTorch, `.ckpt` to TensorFlow or JAX, and `.safetensors` to SafeTensors. Furthermore, the system reads `config.json` again and parses the `auto_map` field. This field usually contains the classpath required for model loading (e.g., `transformers.models.x`), and the system extracts the prefix of this path to determine the framework.

[0029] For example, if the file list contains `model.tf`, the system initially identifies it as a TensorFlow application. Additionally, the `auto_map` field in `config.json` has a value of `{AutoModel: modeling_tf_bert.TFBertModel}`. The system extracts the `TF` prefix, confirming it as a TensorFlow framework.

[0030] As a further aspect of the present invention, the scale calculation module is specifically configured as follows: Load the model index file and configuration file; If an index file exists, the weight mapping table and tensor mapping information are extracted directly from it. If the index file is missing, a list of tensor names will be dynamically generated based on the model structure parameters in the configuration file, and a virtual weight mapping table will be constructed. Determine the default data type. If the data type is missing in the tensor information, infer it based on the tensor name pattern and standardize the data type names of various frameworks into a unified format. The scaling module calculates the model parameters in the following way: For each tensor in the weight mapping table, calculate the number of parameters for that tensor based on its shape tuple; Determine the number of bytes occupied by a single parameter based on the standardized data type, and calculate the storage size of the tensor in bytes; It supports specifying a particular fragment file via a file path parameter, and only tensor information belonging to that fragment file is counted. Summarize the calculation results of all tensors and output the formatted total number of model parameters and the total size of the model file.

[0031] The system attempts to load `model.safetensors.index.json` (the index file). If it exists, it directly reads the `weight_map` to obtain the mapping relationship between tensor names and file slices. If the index file does not exist, the system loads `config.json` and uses its structure parameters (such as number of layers and dimensions) to dynamically generate the expected list of tensor names, constructing a virtual weight mapping table. Simultaneously, the system determines the default data precision type (e.g., `float32`) and uniformly converts the type names of different frameworks (e.g., `fp16`, `bfloat16`) to the standard format (e.g., `F16`, `BF16`).

[0032] For example: The repository has no index file, but config.json shows 12 layers. The system generates a list of names such as layer.0.weight, layer.1.weight, etc., based on the logic. Simultaneously, it reads that torch_dtype is float16 in the configuration, normalizes it, and records it as type F16, which serves as the benchmark for subsequent calculations.

[0033] The system iterates through each tensor in the weight mapping table. For each tensor, the system obtains its shape information, typically a list containing multiple dimensions. The system multiplies all dimension values ​​in the list sequentially; the product is the number of parameters for that tensor. Next, the system looks up the corresponding byte size based on the tensor's data type (e.g., 4 bytes for F32, 2 bytes for F16). The system multiplies the number of parameters by the byte size of a single parameter to obtain the tensor's storage footprint. The system supports calculating only tensors contained in specific fragment files. Finally, the system sums the parameter counts of all tensors to obtain the total number of model parameters and sums the storage footprints of all tensors to obtain the total model file size.

[0034] For example, a tensor has the shape [768, 3072] and the data type is float32 (4 bytes).

[0035] The number of parameters calculated is 768 multiplied by 3072, resulting in 2,359,296 parameters.

[0036] Storage size calculation: 2,359,296 multiplied by 4 bytes, resulting in 9,437,184 bytes (approximately 9 MB). The system performs this operation on all layers in the model, and the final summary output is: total parameters 110M, total size 420MB.

[0037] As a further embodiment of the present invention, the tag generation and classification module includes a knowledge base layer, which contains a mapping table and a standard tag library; The label generation and classification module is configured as follows: Load the mapping table, traverse the structural feature set, purpose feature set, framework feature set and scale feature set, and if an element in the feature set matches a keyword in the mapping table, obtain the corresponding intermediate label. The intermediate tag is retrieved from the standard tag library. If a mapping relationship exists, the intermediate tag is replaced with a standard tag defined in the standard tag library, and the standard tag is assigned to the corresponding dimension category.

[0038] The system loads a pre-defined mapping table. This table defines the mapping relationship from non-standard keywords to intermediate tags. The system receives the feature set extracted in the previous steps (such as bert-base, text-gen), traverses the mapping table, and performs fuzzy or exact matching. If a match is successful, the intermediate tag is obtained. Subsequently, the system retrieves the intermediate tag from the standard tag library, replaces it with a standardized tag for front-end display (such as normalizing text-gen to Text Generation), and categorizes it according to specific dimensions such as structure, purpose, framework, or scale based on tag attributes.

[0039] For example: The system parses the feature keyword `img-cls`. The mapping table defines `img-cls->ImageClassification`. The system confirms that `Image Classification` belongs to the usage dimension under the computer vision category in the standard library. The final generated label data structure is: {Category: Usage, Label: ImageClassification}.

[0040] The system also includes a persistence layer: The persistence layer is configured to call the AI ​​Hub service interface to associate and store the generated and categorized standard tag set with the corresponding model project ID, so as to support the display and filtering of tags in the model community interface.

[0041] After generating the final set of standard tags, the system calls a persistence layer interface (such as AIHubService) via an API. The system serializes the standard tag set into JSON format and binds it to the ProjectID of the currently analyzed model. The data is written to a relational table in the database or to document storage, so that the front-end interface can quickly retrieve the corresponding model list directly by tag ID when the user filters or browses models.

[0042] For example, if the model ID is 1001 and the generated labels are [BERT, PyTorch], the system calls the `save_labels(model_id=1001, labels=[...])` interface to add a new record to the database, so that when a user clicks the PyTorch filter button on the webpage, model 1001 can be found.

[0043] Secondly, in order to achieve the above objectives, the present invention adopts the following technical solution: a multi-dimensional label automatic classification and display system in the AI ​​model community, including: a resource acquisition and preprocessing module, a multi-dimensional feature extraction subsystem, a scale computing power evaluation module, and a label standardization mapping module; The resource acquisition and preprocessing module, based on the analysis request command of the base service, triggers the version control tool to pull the target model repository through the message queue listening mechanism, traverses the file tree to generate a file path index list, and parses the description document to extract metadata key-value pairs to obtain the original model feature dataset; The multidimensional feature extraction subsystem includes a structural topology analysis unit, a semantic purpose reasoning unit, and a frame compatibility verification unit. The structural topology analysis unit performs feature matching on the configuration file and code class definition based on the original model feature dataset, identifies the network layer features and hyperparameter fields of the deep learning model, and obtains the model structural dimension feature vector. The semantic purpose inference unit, based on the original model feature dataset, performs keyword semantic analysis and context matching on the description document and code call examples to infer the downstream task type of the model and obtain the model purpose dimension feature vector. The framework compatibility verification unit analyzes the dependency library declaration, file extension and automatic mapping configuration fields based on the original model feature dataset to determine the underlying dependency environment of the model and obtain the model framework dimension feature vector. The scale computing power assessment module loads the model index file or dynamically generates a tensor name list based on the file path index list, parses the tensor shape tuples and data precision types in each segment file, calculates the total number of parameters of the whole model and the storage space occupied, and obtains the model scale dimension feature index. The label standardization mapping module loads a pre-set multidimensional knowledge base mapping table based on the model structure dimension feature vector, model purpose dimension feature vector, model framework dimension feature vector, and model scale dimension feature index, performs matching operations between feature values ​​and standard labels, and generates standardized multidimensional classification display labels.

[0044] As a further aspect of the present invention, the specific logic for data cleaning performed by the resource acquisition and preprocessing module is as follows: The Git tool is used to pull the repository and generate a TreeWalk structure. The relative paths of all sub-files are then extracted and stored in a path list. At the same time, locate the readme document in the model repository and convert it into a JSON object; Inspect the unstructured tag fields in the object and perform a cleaning operation, which includes removing null values, removing leading and trailing spaces, filtering empty strings, and converting them to lowercase characters. Store the cleaned results in the metadata collection.

[0045] As a further aspect of the present invention, the specific logic for the feature matching performed by the structural topology analysis unit is as follows: Extract the model type field and architecture field from the model configuration file; Execution architecture feature keyword matching: If the fields attend_heads and encoder / decoder are detected, it is marked as a Transformer architecture feature; If the kernel_size, stride, and channels fields are detected, it is marked as a CNN architecture feature; If the fields hidden_states and sequence_length are detected, it is marked as an RNN architecture feature; Analyze the class definitions in the code file, identify the class names that inherit from the deep learning base class (nn.Module or tf.keras.Model), infer the architecture variants through class name patterns, and add the identified GPT or BERT class name features to the structural feature set.

[0046] As a further aspect of the present invention, the specific logic for the semantic purpose reasoning unit to perform semantic analysis includes: Prioritize extracting pipeline tags from metadata; If the tag is missing, keyword matching is performed on the description document content to identify task description words including classification, generation, and detection. The code parsing uses the import statements in the example to infer the task type by extracting the string after AutoModelFor; at the same time, it detects the method call patterns in the code. If a call to the generate method is detected, it is marked as a generation task; a call to the classify method is marked as a classification task; a call to the predict method is marked as a prediction task; and a call to the encode method is marked as a feature extraction task.

[0047] As a further aspect of the present invention, the framework compatibility verification unit determines the specific underlying environment dependent on the model as follows: Extract the library name field from the metadata; Read the auto-mapping fields in the configuration file, extract the mapping values ​​using regular expressions, and filter out AutoModel and Auto key names that have no specific meaning. By combining the file path index list, the distribution of file extensions for a specific frame is statistically analyzed, and the above information is aggregated and standardized into a set of frame tags in lowercase format.

[0048] As a further aspect of the present invention, the scale computing power assessment module adopts a dual data source processing mechanism: Establish an index file existence detection logic, prioritize loading the model index file and extracting the weight mapping table and tensor information table from it; When a missing index file is detected, dynamic generation logic is triggered to parse the model configuration file, dynamically generate a list of tensor names based on the model hierarchy configuration, and construct a virtual weight mapping table, which maps all tensors to standard fragment file names by default.

[0049] As a further aspect of the present invention, the scale computing power assessment module performs a priority inference strategy when parsing tensor properties: First, try to read the precise shape and data type directly from the tensor information table of the index file; If information is missing, an inference algorithm is invoked based on the tensor name pattern and model configuration to complete it; By using a type standardization method, the data type names of different frameworks are uniformly converted into a standard format, which includes mapping float32 to F32, bfloat16 to BF16, and fp8_e4m3 to F8_E4M3.

[0050] As a further aspect of the present invention, the model scale dimension feature index output by the scale computing power evaluation module is a four-tuple structure, including: Fragment-level tensor statistics mapping is used to record the list of tensors and their independent parameter counts contained in each fragment file; The total number of parameters in the entire model is the sum of the products of all tensor shape tuples; The total number of bytes stored in the entire model is the sum of the products of the number of tensor parameters and the byte width of the corresponding data type; Data type distribution statistics are used to record the proportion of different precision types in the model.

[0051] As a further aspect of the present invention, the label standardization mapping module performs label classification as follows: Aggregate the original tag set output by the four modules: structure, purpose, framework, and scale; Load the predefined system mapping table and iterate through the original tag set to perform keyword fuzzy matching; The matched tags are mapped to the standard tag library. If a mapping relationship exists, the original tags are replaced with the corresponding normalized tags in the standard tag library, and the classification results are stored in the database through a persistent interface.

[0052] As a further embodiment of the present invention, the system is deployed based on a microservice architecture, and the resource acquisition and preprocessing module consumes analysis requests by listening to the Kafka message queue; the multi-dimensional feature extraction subsystem adopts a parallel processing mechanism to synchronously execute parsing tasks of structure, purpose and framework dimensions; the system also includes a persistence layer to store the finally generated tag data through the AIHub service interface.

[0053] Thirdly: The method for automatic classification and display of multi-dimensional labels in the AI ​​model community is based on the aforementioned automatic classification and display system for multi-dimensional labels in the AI ​​model community, and includes the following steps: S1: In response to the analysis request of the model repository, generate a list of file paths of the model repository and extract metadata information from the document files; S2: Based on the file path list and metadata information, extract a set of structural features representing the model architecture design from the configuration file and code file; S3: Based on the metadata information, documentation, and code usage examples, infer the application task type of the model and generate a set of usage features; S4: Based on the metadata information, file name characteristics and configuration file content, identify the deep learning framework that the model depends on and generate a framework feature set; S5: Based on the weight file and configuration file, calculate the number of model parameters and storage size, and generate a set of scale features; S6: Receive the set of structural features, the set of usage features, the set of framework features, and the set of scale features, perform keyword matching using a preset mapping table, and map the matching results to a standard tag library to generate standardized multidimensional tags for classification and display.

[0054] Compared with the prior art, the advantages and positive effects of the present invention are as follows: 1. This invention can automatically parse and preprocess multi-source model files, supports multiple mainstream model formats, and automatically extracts the structural information of the model; 2. This invention can automatically identify the purpose and framework of a model based on the model file, thereby improving the accuracy of label generation; 3. This invention can automatically calculate the scale of the model based on the model file, and output the total number of parameters and the total size of the model, without the need for manual calculation; 4. This invention can generate standardized tags based on the extracted dimensional information and automatically classify them according to a preset dictionary. Attached Figure Description

[0055] Figure 1 This is a system flowchart of Embodiment 1 of the present invention; Figure 2 This is a structural diagram of the dimensional information parsing layer in this invention; Figure 3 This is a system flowchart of Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the structure of the multidimensional feature extraction subsystem in this invention; Figure 5 This is a flowchart of the method for automatic classification and display of multidimensional tags in the AI ​​model community in this invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] Example 1: Please see Figures 1-2The present invention provides a technical solution: an automatic classification and display system for multidimensional tags in an AI model community, characterized in that the system includes a file parsing and preprocessing module, a dimension information parsing layer, and a tag generation and classification module; The file parsing and preprocessing module responds to the analysis request of the model repository, generates a list of file paths of the model repository, and parses metadata information from the document files; The dimensional information parsing layer includes a structure parsing module, a purpose identification module, a framework parsing module, and a scale calculation module that work in parallel. The structure parsing module extracts a set of structural features representing the model architecture design from the configuration file and code file based on the file path list and metadata information. The purpose identification module infers the application task type of the model based on the metadata information, document description content and code usage examples, and generates a set of purpose features. The framework parsing module identifies the deep learning framework that the model depends on based on the metadata information, file name characteristics, and configuration file content, and generates a framework feature set. The scale calculation module calculates the number of parameters and storage size of the model based on the weight file and configuration file, and generates a scale feature set. The tag generation and classification module receives the set of structural features, the set of usage features, the set of framework features, and the set of scale features. It uses a preset mapping table to perform keyword matching and maps the matching results to a standard tag library to generate standardized multidimensional tags for classification and display.

[0058] For example: When the system receives an analysis request for the "DeepSeek-Coder" model repository, it pulls the repository metadata, generates a list of file paths, and parses them in parallel. The structure module parses the LlamaForCausalLM field in the configuration file and identifies it as the Llama architecture.

[0059] The purpose module parses the keyword "codegeneration" in the README document and identifies it as a code generation task.

[0060] The framework module recognizes files with the .safetensors extension and identifies them as SafeTensors format.

[0061] The scale module uses statistical weight file data to calculate a parameter quantity of 1.3 Billion.

[0062] Output: Generate and store a standardized tag set [Llama, CodeGeneration, SafeTensors, 1.3B].

[0063] The file parsing and preprocessing module is specifically configured as follows: By listening to message queues and consuming analysis requests, Git commands are used to pull remote model repositories to the local machine; Iterate through the repository to generate a list of file paths containing all subfiles; Parse the readme document in the root directory of the repository, extract the metadata information in YAML format and convert it into a JSON format object. The metadata information includes at least the tags field, the pipeline_tag field or the library_name field.

[0064] The system listens to and analyzes the request queue through a Kafka consumer. After parsing the repository address in the request, it calls a standard Git command-line tool (such as / usr / bin / git) through an operating system subprocess to execute the `git clone [repository address]` or `git fetch` command, synchronizing the remote model repository to the local temporary storage area. During this process, the 'file metadata' mainly includes version control information obtained through commands such as `git log` and `git ls-tree`, such as the file's commit hash (SHA-1), last modified time, author information, and attributes like file size and permissions obtained through file system interfaces (such as Python's `os.path` module). This metadata, along with the entity files, is cached locally for subsequent module analysis.

[0065] Request queue definition: Using the Apache Kafka message middleware, create a Topic named model-analysis-request; The message body is in JSON format: {"repo_url":"https: / / github.com / xxx","project_id":"1001","trigger_source":"manual"}.

[0066] Repository address parsing logic: Use regular expressions to validate the URL format: ^https?: / / (github|gitlab).+\.git$; After extracting the repository path, use the command `git clone {url} / tmp / {project_id}`.

[0067] Git operation implementation details: Timeout mechanism: Set a 30-minute timeout, and automatically retry 3 times on failure; Temporary directory management: Automatically clean up / tmp / {project_id} after the task is completed.

[0068] The system executes a directory traversal algorithm (such as TreeWalk) to scan all levels of directories in the local storage area and build a list object containing the relative or absolute paths of all files, which serves as the file index source for subsequent modules.

[0069] The specific implementation of the algorithm is shown in the following example: def generate_file_list(root_dir): file_list = []; for root, dirs, files in os.walk(root_dir): For file in files: rel_path = os.path.relpath(os.path.join(root, file), root_dir); file_list.append(rel_path); return sorted(file_list) # Sort by letter to ensure consistency; in: The [config.json, model.safetensors, README.md] files in the example are representative files; The actual output includes all files: ['config.json', 'pytorch_model.bin', 'README.md', 'vocab.txt', 'tokenizer_config.json'].

[0070] The system locates and reads the README document (e.g., README.md) in the root directory of the repository. The system identifies the YAML-formatted metadata block (e.g., MetadataBlock) in the document header and converts it into a JSON object using a YAML parser. From this object, the system extracts key fields such as tags, pipeline_tag, and library_name, and performs null removal, noise reduction, and lowercase normalization.

[0071] The specific steps of YAML parsing are as follows: The system uses Python's PyYAML library to process the readme document. Specifically, it first locates the YAML-formatted metadata block enclosed by the --- delimiter in the README.md file through file reading operations; then, it calls the yaml.safe_load() function to parse this YAML string into a Python dictionary object; finally, it calls the json.dumps() function of the Python standard library json to serialize the dictionary object into a JSON-formatted string. This process realizes the conversion of metadata from a document-embedded format to a standard data exchange format that the program can process.

[0072] Field extraction fault tolerance logic: If the pipeline_tag field is missing, it will fall back to parsing the task keywords (such as "text-classification") in the tags field.

[0073] Example correction: Complete JSON object example: {"library_name": "transformers", "tags": ["text-classification", "pytorch"], "pipeline_tag": null}.

[0074] For example: After the system pulls the repository, it iterates through and generates a list of files [config.json, model.safetensors, README.md]. It reads the header metadata block of README.md and parses it to obtain a JSON object {library_name: transformers, tags: [text-classification, pytorch]}.

[0075] The structural parsing module extracts the set of structural features in the following ways: Examine the tags field in the metadata information and extract the structure-related tags; Read the configuration file and extract the model type field, architecture field, and structural parameter field. The structural parameter field includes the number of attention heads, the number of layers, or the size of the hidden layers. When the configuration file contains specific architecture feature fields, the architecture type is determined based on the combination of these fields; the specific architecture feature fields include attend_heads, encoder / decoder, kernel_size, or hidden_states.

[0076] The determination rule table is defined as follows:

[0077] For example, if both "num_attention_heads": 12 and "encoder": true exist in the configuration file, it is marked as a Transformer architecture.

[0078] The structure parsing module is also configured to analyze code files: Locate the class definitions that inherit from a deep learning framework base class, including nn.Module or tf.keras.Model; The class name pattern of the class definition is analyzed, the model architecture is inferred through the keywords in the class name, and the inference results are added to the structural feature set.

[0079] When the configuration file information is insufficient, the structure parsing module performs static analysis on the source code: The system iterates through Python code files (such as modeling_.py) in the repository. It uses abstract syntax trees (such as ASTs) or regular expressions to find class definition statements.

[0080] The system filters class definitions using static code analysis. Specifically, it uses Python's `ast` (Abstract Syntax Tree) module to parse code files: First, it constructs the Abstract Syntax Tree (AST) of the code, traversing and identifying all class definition nodes (`ClassDef`). Then, it examines the base class list (`bases`) of each class node to determine if it contains base class identifiers of known deep learning frameworks such as `torch.nn.Module`, `tensorflow.keras.Model`, or `flax.linen.Module`. In this way, the system can accurately filter model classes that inherit from these base classes, and then analyze their architecture.

[0081] Specific base class identifiers include nn.Module in PyTorch or tf.keras.Model in TensorFlow / Keras.

[0082] For the selected model classes, the system analyzes their class name strings. It extracts core architectural terms from the class names using camelCase segmentation or substring matching. The system maintains a table of architectural keywords (e.g., Bert, GPT, ResNet); if a class name contains a term from this table, that term is used as an architectural feature.

[0083] AST parsing example: Using Python's ast module to traverse the syntax tree and identify class definition nodes: tree = ast.parse(source_code)for node in ast.walk(tree): if isinstance(node, ast.ClassDef): class_name = node.name# Such as "BertModel" Inheritance relationship detection: Check whether the base class of the class node is nn.Module (e.g., base_id = node.bases[0].id).

[0084] Architecture keyword list: Pre-defined list ["Bert", "GPT", "ResNet"]; Keywords can be matched using camelCase segmentation (e.g., re.findall('[AZ][az]+', class_name)).

[0085] For example: The system scans the code file and finds the definition of classBertModel (e.g., nn.Module). The system identifies the base class as nn.Module and extracts the string "Bert" from the class name, inferring that the model architecture is BERT.

[0086] The purpose identification module generates the purpose feature set in the following ways: Prioritize extracting the pipeline tag field from metadata information as task tags; Perform semantic analysis on the self-described document and its content, identify task description words and application scenario descriptions through large models or keyword matching, and format the recognition results into task descriptions. The analysis model uses code examples to extract model class names from import statements, infer the model's purpose based on the class name suffix, and filter out common prefixes.

[0087] The formatting rules for task descriptions are as follows:

[0088] The logic for extracting class name suffixes is as follows: # Extract task type from AutoModelForXXX: def extract_task_from_classname(class_name): if 'AutoModelFor' in class_name: suffix = class_name.split('AutoModelFor')[-1]; return suffix.lower().replace('_', '-'); return None.

[0089] Instructions for the function of production line labels: Direct mapping: pipeline_tag: from text-classification to text classification tasks; Avoid NLP analysis and improve efficiency by over 90%.

[0090] The purpose identification module infers the model task type through document semantic analysis: The system prioritizes reading the pipeline_tag field value from the metadata, which typically directly defines the standard task type (such as text-classification).

[0091] If the above fields are missing, the system reads the main text of the README document. The system has a pre-built task description keyword library (including classification, detection, segmentation, etc.). The system performs full-text search or calls the large language model interface to identify task description terms and application scenario descriptions (such as chatbot, question answering) appearing in the document. The identified terms are formatted into uniform task identifiers.

[0092] The system scans for import statements in the code. For example, from `from transformers import AutoModelForImageClassification`, it uses regular expressions to extract the suffix string "Image Classification" after "Auto Model For" and converts it into a standard task name.

[0093] For example, the system retrieves the sentence "This model is designed for recognizing cats in images" from the README document. By matching keywords "recognizing" and "images," the system maps this to features for the Image Classification task.

[0094] The purpose identification module, when analyzing code examples used in the model, also includes: Identify the type of input preprocessing object; if a tokenizer object exists, it is identified as a text task; if a processor object exists, it is identified as an image task. Identify the calling method of the model object, and determine the task type based on the method name, which includes generate, classify, predict, or encode.

[0095] Specific rules for object recognition: Tokenizer object recognition: Class names include: Tokenizer, BertTokenizer, GPT2Tokenizer; Instantiation pattern: tokenizer = AutoTokenizer.from_pretrained(...); Processor object identification: Class names include: Processor, ImageProcessor, FeatureExtractor; Instantiation mode: processor = ViTImageProcessor(...); The method call analysis is as follows: # Detect method call pattern if 'generat' in method_name.lower(): task_type = 'text-generation'elif 'classif' in method_name.lower(): task_type = 'classification'.

[0096] The purpose identification module assists in identifying purposes by analyzing object instantiation and method call patterns in the code: Preprocessing object recognition: Preprocessing classes instantiated in the system analysis code. If the code contains an instantiation or object call of the Tokenizer class, the system determines that the input modality is text (such as NLP tasks); if it contains ImageProcessor or FeatureExtractor, the system determines that the input modality is an image (such as CV tasks).

[0097] The system detects the method call signature of the model instance: If a call to the generate() method is detected, it is determined to be a generative task (such as Generation).

[0098] If the classify() method or the output layer contains classification logic, it is determined to be a classification task (such as Classification).

[0099] If the predict() or detect() method is detected, it is determined to be a prediction or detection task.

[0100] The `encode()` method was detected, indicating a feature extraction or embedding task.

[0101] For example, the code contains `inputs=tokenizer` (e.g., `text`) and `model.generate` (e.g., `inputs`). Based on this, the system identifies the text input preprocessing and generation method calls, and determines it as a Text Generation task.

[0102] The framework parsing module generates the framework feature set in the following ways: Extract the library name field from the metadata information; Traverse the list of file paths, extract the file extensions and filenames of all files, and match them with the file characteristics of known frames; Read the configuration file, parse the auto-mapping fields, extract the framework prefix information indicating the loaded classes, and filter and clean the values.

[0103] The framework parsing module determines the deep learning framework that the model depends on by fusing multi-source features: Extract the value of the library_name field directly from the metadata.

[0104] Iterate through the list of file paths and check if a file with a specific file extension exists: .pt or .bin correspond to the PyTorch framework.

[0105] .ckpt or .h5 correspond to the TensorFlow framework.

[0106] .safetensors corresponds to the SafeTensors format (usually associated with PyTorch or TensorFlow).

[0107] .onnx corresponds to ONNXRuntime.

[0108] .nnx or .msgpack may correspond to the JAX / Flax framework.

[0109] The system reads the `auto_map` field from `config.json`. This field defines the loading path for model classes (e.g., `transformers.models.x`). The system parses this path string and extracts the framework prefix identifier (e.g., the `TF` prefix points to TensorFlow, and the `Flax` prefix points to JAX).

[0110] For example, if the file list contains `model.tf` and the `auto_map` field in `config.json` has a value of `{AutoModel: modeling_tf_bert.TFBertModel}`, the system extracts the TF prefix and filename characteristics to determine that the framework is TensorFlow.

[0111] The scale calculation module is specifically configured as follows: Load the model index file and configuration file; If an index file exists, the weight mapping table and tensor mapping information are extracted directly from it. If the index file is missing, a list of tensor names will be dynamically generated based on the model structure parameters in the configuration file, and a virtual weight mapping table will be constructed. Determine the default data type. If the data type is missing in the tensor information, infer it based on the tensor name pattern and standardize the data type names of various frameworks into a unified format. The scale calculation module is responsible for constructing the data model required for parameter calculation: The system checks if `model.safetensors.index.json` or `pytorch_model.bin.index.json` exists in the file list. If it does, it reads the file and extracts the `weight_map` (weight mapping table) and `tensors` (tensor metadata) fields.

[0112] If the index file is missing (common in single-file models), the system reads config.json to obtain structural parameters such as the number of layers, hidden layer dimensions, and vocabulary size. The system then calls its internal generation logic to dynamically generate the expected list of tensor names based on these parameters, constructing a virtual weight mapping table containing all tensor names.

[0113] The system reads the torch_dtype field from config.json as the default precision. The system has a built-in type mapping table that converts type strings from different frameworks (such as fp16, bfloat16, float32) into standard identifiers (such as F16, BF16, F32) to determine the byte width later.

[0114] For example: The system loads config.json and reads that torch_dtype is float16 (normalized to F16). Since there is no index file, the system constructs a list of tensors in memory containing names such as layer.0.weight, layer.1.weight, etc., based on n_layers=12 in the configuration.

[0115] The scaling module calculates the model parameters in the following way: For each tensor in the weight mapping table, calculate the number of parameters for that tensor based on its shape tuple; Determine the number of bytes occupied by a single parameter based on the standardized data type, and calculate the storage size of the tensor in bytes; It supports specifying a particular fragment file via a file path parameter, and only tensor information belonging to that fragment file is counted. Summarize the calculation results of all tensors and output the formatted total number of model parameters and the total size of the model file.

[0116] The scale calculation module traverses each tensor entry in the weight mapping table.

[0117] In deep learning frameworks, the shape of a tensor describes its size in each dimension. For example, a tensor with shape [Batch_size, Sequence_length, Hidden_size] has its shape attribute represented as a list of integers [b, s, h] containing the values ​​of these dimensions. The total number of parameters in a tensor (i.e., the number of scalar values ​​that can be trained or stored) is mathematically equal to the product of the sizes of all its dimensions. Therefore, for this tensor, the total number of parameters is b × s × h. The system uses this mathematical principle to precisely calculate the number of parameters for each tensor by traversing the list of shapes and performing cumulative multiplication.

[0118] The system queries the byte width table based on the standardized data type (e.g., F32=4 bytes, F16=2 bytes, INT8=1 byte). The system multiplies the total number of parameters calculated above with the byte width, and the result is the storage size of the tensor in bytes.

[0119] If the analysis request specifies a particular file segment, the system only counts the tensors mapped to that file. Otherwise, the system sums the number of parameters and the storage size in bytes for all tensors to obtain the total number of parameters and the total file size of the model.

[0120] The system converts the calculated total number of parameters and storage bytes into a human-readable format. The conversion rules are as follows: For parameter values, when the value is greater than or equal to 10^9, divide by 10^9 and use 'B' (Billion) as the unit (e.g., 7e9 -> 7B); when the value is greater than or equal to 10^6, divide by 10^6 and use 'M' (Million) as the unit (e.g., 110e6 -> 110M). For storage size, binary units are used, carrying over every 1024 (2^10) bits, and are converted to KB, MB, GB, TB, etc. (e.g., 15032385536 bytes / (1024^3) ≈ 14 GB). The system generates a formatted string according to the above rules and adds it to the scale feature set.

[0121] For example: A tensor has the shape [4096, 4096] and the data type is F16 (2 bytes).

[0122] Number of parameters = 4096 × 4096 = 16777216.

[0123] Storage size = 16777216 × 2 = 33554432 bytes. The system adds this value to the total, and finally outputs a total parameter size of 110M.

[0124] The tag generation and classification module includes a knowledge base layer, which contains a mapping table and a standard tag library. The label generation and classification module is configured as follows: Load the mapping table, traverse the structural feature set, usage feature set, framework feature set, and scale feature set, and if the elements in the structural feature set, usage feature set, framework feature set, and scale feature set match the keywords in the mapping table, then obtain the corresponding intermediate tags; The intermediate tag is retrieved from the standard tag library. If a mapping relationship exists, the intermediate tag is replaced with a standard tag defined in the standard tag library, and the standard tag is assigned to the corresponding dimension category.

[0125] The label generation and classification module is responsible for the standardized mapping of feature data: The system loads a predefined mapping table (such as a Mapping Table) and a standard tag library (such as StandardTaxonomy). The mapping table defines the correspondence between non-standard keywords (such as img-cls) and intermediate tags; the standard tag library defines the final display tags and their respective dimensions (such as Computer Vision -> Image Classification).

[0126] The system receives four feature sets output by the aforementioned modules. It iterates through each element in the set, performing a key-value lookup in the mapping table. It supports exact matching and fuzzy matching based on string inclusion relationships. If a match is successful, the corresponding intermediate label is retrieved.

[0127] The system retrieves intermediate tags and searches them in a standard tag library. If a mapping path exists, the intermediate tag is replaced with a standard name defined in the standard tag library, and the tag is assigned to a specific dimension category (such as structure, purpose, framework, scale) based on the hierarchical relationship defined in the library.

[0128] For example: The input feature set contains chatglm.

[0129] Retrieve mapping table: chatglm is mapped to the intermediate label chatglm.

[0130] Searching the standard library: chatglm belongs to the Architecture dimension, and its standard display name is Chat GLM.

[0131] Generate the final label object: {Category: Architecture, Label: Chat GLM}.

[0132] The system also includes a persistence layer: The persistence layer is configured to call the AI ​​Hub service interface to associate and store the generated and categorized standard tag set with the corresponding model project ID, so as to support the display and filtering of tags in the model community interface.

[0133] The persistence layer is responsible for storing and associating the analysis results: The system serializes the categorized list of standard label objects into a JSON string or a binary data object.

[0134] The system constructs a network request (such as HTTP POST or RPC) to call the persistence layer service interface (such as AIHubService). The request parameters include the unique identifier of the current model project (Project ID) and the serialized tag data.

[0135] Upon receiving the request, the persistence service performs a database update operation. The system writes the tag data to the record associated with the ProjectID. If a relational database is used, the corresponding column fields are updated; if a search engine (such as Elasticsearch) is used, the indexed documents are updated to support tag-based inverted index queries.

[0136] For example, if model ID is 1001 and the generated label data is [{dim: Usage, val: NLP}, {dim: Size, val: 7B}], the system calls the storage interface, and the record for this model in the database is updated. When the user filters the NLP category on the front-end interface, the system can retrieve and return model 1001.

[0137] Example 2: Please see Figures 3-4 The present invention provides another technical solution: an automatic classification and display system for multidimensional labels in an AI model community, including: a resource acquisition and preprocessing module, a multidimensional feature extraction subsystem, a scale computing power evaluation module, and a label standardization mapping module; The resource acquisition and preprocessing module, based on the analysis request command of the base service, triggers the version control tool to pull the target model repository through the message queue listening mechanism, traverses the file tree to generate a file path index list, and parses the description document to extract metadata key-value pairs to obtain the original model feature dataset; The multidimensional feature extraction subsystem includes a structural topology analysis unit, a semantic purpose reasoning unit, and a frame compatibility verification unit. The structural topology analysis unit performs feature matching on the configuration file and code class definition based on the original model feature dataset, identifies the network layer features and hyperparameter fields of the deep learning model, and obtains the model structural dimension feature vector. The semantic purpose inference unit, based on the original model feature dataset, performs keyword semantic analysis and context matching on the description document and code call examples to infer the downstream task type of the model and obtain the model purpose dimension feature vector. The framework compatibility verification unit analyzes the dependency library declaration, file extension and automatic mapping configuration fields based on the original model feature dataset to determine the underlying dependency environment of the model and obtain the model framework dimension feature vector. The scale computing power assessment module loads the model index file or dynamically generates a tensor name list based on the file path index list, parses the tensor shape tuples and data precision types in each segment file, calculates the total number of parameters of the whole model and the storage space occupied, and obtains the model scale dimension feature index. The label standardization mapping module loads a pre-set multidimensional knowledge base mapping table based on the model structure dimension feature vector, model purpose dimension feature vector, model framework dimension feature vector, and model scale dimension feature index, performs matching operations between feature values ​​and standard labels, and generates standardized multidimensional classification display labels.

[0138] The specific logic for data cleaning performed by the resource acquisition and preprocessing module is as follows: The Git tool is used to pull the repository and generate a TreeWalk structure. The relative paths of all sub-files are then extracted and stored in a path list. At the same time, locate the readme document in the model repository and convert it into a JSON object; Inspect the unstructured tag fields in the object and perform a cleaning operation, which includes removing null values, removing leading and trailing spaces, filtering empty strings, and converting them to lowercase characters. Store the cleaned results in the metadata collection.

[0139] The specific logic for feature matching performed by the structural topology analysis unit is as follows: Extract the model type field and architecture field from the model configuration file; Execution architecture feature keyword matching: If the fields attend_heads and encoder / decoder are detected, it is marked as a Transformer architecture feature; If the kernel_size, stride, and channels fields are detected, it is marked as a CNN architecture feature; If the fields hidden_states and sequence_length are detected, it is marked as an RNN architecture feature; Analyze the class definitions in the code file, identify the class names that inherit from the deep learning base class (nn.Module or tf.keras.Model), infer the architecture variants through class name patterns, and add the identified GPT or BERT class name features to the structural feature set.

[0140] The specific logic for semantic analysis performed by the semantic purpose reasoning unit includes: Prioritize extracting pipeline tags from metadata; If the tag is missing, keyword matching is performed on the description document content to identify task description words including classification, generation, and detection. The code parsing uses the import statements in the example to infer the task type by extracting the string after AutoModelFor; at the same time, it detects the method call patterns in the code. If a call to the generate method is detected, it is marked as a generation task; a call to the classify method is marked as a classification task; a call to the predict method is marked as a prediction task; and a call to the encode method is marked as a feature extraction task.

[0141] The framework compatibility verification unit determines the specific underlying environment that the model depends on: Extract the library name field from the metadata; Read the auto-mapping fields in the configuration file, extract the mapping values ​​using regular expressions, and filter out AutoModel and Auto key names that have no specific meaning. By combining the file path index list, the distribution of file extensions for a specific frame is statistically analyzed, and the above information is aggregated and standardized into a set of frame tags in lowercase format.

[0142] The scale computing power assessment module adopts a dual data source processing mechanism: Establish an index file existence detection logic, prioritize loading the model index file and extracting the weight mapping table and tensor information table from it; When a missing index file is detected, dynamic generation logic is triggered to parse the model configuration file, dynamically generate a list of tensor names based on the model hierarchy configuration, and construct a virtual weight mapping table, which maps all tensors to standard fragment file names by default.

[0143] When parsing tensor properties, the scale computing power assessment module executes a priority inference strategy: First, try to read the precise shape and data type directly from the tensor information table of the index file; If information is missing, an inference algorithm is invoked based on the tensor name pattern and model configuration to complete it; By using a type standardization method, the data type names of different frameworks are uniformly converted into a standard format, which includes mapping float32 to F32, bfloat16 to BF16, and fp8_e4m3 to F8_E4M3.

[0144] The model scale dimension feature index output by the scale computing power assessment module is a four-tuple structure, including: Fragment-level tensor statistics mapping is used to record the list of tensors and their independent parameter counts contained in each fragment file; The total number of parameters in the entire model is the sum of the products of all tensor shape tuples; The total number of bytes stored in the entire model is the sum of the products of the number of tensor parameters and the byte width of the corresponding data type; Data type distribution statistics are used to record the proportion of different precision types in the model.

[0145] The specific steps of the label standardization mapping module in performing label classification are as follows: Aggregate the original tag set output by the four modules: structure, purpose, framework, and scale; Load the predefined system mapping table and iterate through the original tag set to perform keyword fuzzy matching; The matched tags are mapped to the standard tag library. If a mapping relationship exists, the original tags are replaced with the corresponding normalized tags in the standard tag library, and the classification results are stored in the database through a persistent interface.

[0146] The system is deployed based on a microservice architecture. The resource acquisition and preprocessing module listens to the Kafka message queue to consume analysis requests. The multi-dimensional feature extraction subsystem adopts a parallel processing mechanism to synchronously execute parsing tasks of structure, purpose, and framework dimensions. The system also includes a persistence layer to store the finally generated tag data through the AIHub service interface.

[0147] Implementation Three: Please see Figure 5 Based on Embodiment 1, this invention provides a technical solution: a method for automatic classification and display of multi-dimensional tags in an AI model community, comprising the following steps: S1: In response to the analysis request of the model repository, generate a list of file paths of the model repository and extract metadata information from the document files; S2: Based on the file path list and metadata information, extract a set of structural features representing the model architecture design from the configuration file and code file; S3: Based on the metadata information, documentation, and code usage examples, infer the application task type of the model and generate a set of usage features; In this embodiment, the specific set of usage features is as follows: usage_features = [ {"source": "pipeline_tag", "value": "text-classification", "confidence": 0.95}, {"source": "code_analysis", "value": "token-classification", "confidence": 0.80}; ]; S4: Based on the metadata information, file name characteristics and configuration file content, identify the deep learning framework that the model depends on and generate a framework feature set; In this embodiment, the specific set of frame features is as follows: framework_features = [ {"source": "file_extension", "value": ".safetensors", "confidence": 0.90}, {"source": "config_analysis", "value": "pytorch", "confidence": 0.85}; ]; S5: Based on the weight file and configuration file, calculate the number of model parameters and storage size, and generate a set of scale features; In this embodiment, the specific set of scale features is as follows: scale_features = { "total_params": 110000000, "size_mb": 420.5 "precision": "float16" }

[0148] S6: Receive the set of structural features, the set of usage features, the set of framework features, and the set of scale features, perform keyword matching using a preset mapping table, and map the matching results to a standard tag library to generate standardized multidimensional tags for classification and display.

[0149] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multi-dimensional label automatic categorization and display system in an AI model community, characterized in that, The system comprises a file parsing and preprocessing module, a dimension information parsing layer, and a label generation and classification module; The file parsing and preprocessing module generates a file path list of the model repository and parses metadata information from the document file in response to an analysis request of the model repository; The dimension information parsing layer comprises a structure parsing module, a use identification module, a framework parsing module, and a scale calculation module working in parallel; The structure parsing module extracts a structure feature set representing the model architecture design from the configuration file and the code file based on the file path list and the metadata information; The use identification module infers the application task type of the model based on the metadata information, the document description content, and the code usage example, and generates a use feature set; The framework parsing module identifies the deep learning framework on which the model depends based on the metadata information, the file name feature, and the configuration file content, and generates a framework feature set; The scale calculation module calculates the parameter quantity and storage size of the model based on the weight file and the configuration file, and generates a scale feature set; The label generation and classification module receives the structure feature set, the use feature set, the framework feature set, and the scale feature set, performs keyword matching using a preset mapping table, maps the matching result to a standard label library, generates standardized multi-dimensional labels for classification and display. 2.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 1, wherein, The file parsing and preprocessing module is specifically configured to: Pull the remote model repository to the local by using Git instructions through listening to the message queue to consume the analysis request; Generate a file path list containing all subfiles by traversing the repository; Parse the readme document under the root directory of the repository, extract the YAML format metadata information and convert it to a JSON format object, and the metadata information at least contains tags field, pipeline_tag field or library_name field. 3.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 2, wherein, The structure parsing module extracts the structure feature set in the following way: Check the tags field in the metadata information to extract structure-related labels; Read the configuration file to extract the model type field, the architecture field, and the structure parameter field, which includes the number of attention heads, the number of layers, or the size of the hidden layer; When the configuration file contains specific architecture feature fields, determine the architecture type according to the field combination features; the specific architecture feature fields include attend_heads, encoder / decoder, kernel_size, or hidden_states. 4.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 3, wherein, The structure parsing module is also configured to analyze the code file: Find the class definition inherited from the deep learning framework base class, which includes nn.Module or tf.keras.Model; Analyze the class name pattern of the class definition, infer the model architecture through the keywords in the class name, and add the inference result to the structure feature set. 5.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 1, wherein, The use identification module generates the use feature set in the following way: Preferentially extract the pipeline label field in the metadata information as the task label; Perform semantic analysis on the self-description document and the document content, identify task description words and application scenario descriptions through large models or keyword matching, and format the identification results as task descriptions; The analysis model uses code examples to extract model class names from import statements, and infers the model purpose based on the class name suffix, and filters out general prefixes. 6.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 5, wherein, The use recognition module, when analyzing model usage code examples, further includes: Identify the type of input preprocessing object, if there is a tokenizer object, identify it as a text task, if there is a processor object, identify it as an image task; Identify the calling method of the model object, and determine the task type according to the method name, the method name including generate, classify, predict or encode. 7.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 1, wherein, The framework parsing module generates the framework feature set in the following manner: Extract the library name field in the metadata information; Traverse the file path list, extract the suffix and file name of all files, and match them with the file features of known frameworks; Read the configuration file, parse the automatic mapping field, extract the framework prefix information indicating the loaded class, and perform value filtering and cleaning. 8.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 1, wherein, The scale calculation module is specifically configured to: Load the model index file and the configuration file; If there is an index file, directly extract the weight mapping table and tensor mapping information therein; If the index file is missing, dynamically generate a tensor name list based on the model structure parameters in the configuration file, and construct a virtual weight mapping table; Determine the default data type, if the data type is missing in the tensor information, infer it based on the tensor name pattern, and standardize the data type names of various frameworks into a unified format; The scale calculation module calculates the model parameter quantity in the following manner: For each tensor in the weight mapping table, calculate the number of parameters for that tensor based on its shape tuple; Determine the number of bytes occupied by a single parameter based on the standardized data type, and calculate the storage byte size of the tensor; Support specifying a specific shard file through a file path parameter, and only count the tensor information belonging to that shard file; Summarize the calculation results of all tensors, and output the formatted total model parameter quantity and total model file size. 9.The multi-dimensional label auto-categorization and presentation system in an AI model community of claim 1, wherein, The label generation and classification module includes a knowledge base layer, which contains a mapping table and a standard label library; The label generation and classification module is configured to: Load the mapping table, traverse the structure feature set, usage feature set, framework feature set, and scale feature set, and if the elements in the structure feature set, usage feature set, framework feature set, and scale feature set match the keywords in the mapping table, obtain the corresponding intermediate label; Search the intermediate label in the standard label library, if there is a mapping relationship, replace the intermediate label with the standard label defined in the standard label library, and classify the standard label into the corresponding dimension category; The system further includes a persistence layer: The persistence layer is configured to call the AI Hub service interface, associate and store the generated and classified standard label set with the corresponding model project ID, to support label display and filtering in the model community interface.

10. A method for multi-dimensional label automatic categorization and display in an AI model community, characterized in that, The application discloses a multi-dimensional label automatic classification and display system in an AI model community. S1: In response to an analysis request of a model warehouse, a file path list of the model warehouse is generated, and metadata information parsed from a document file is obtained; S2: Based on the file path list and the metadata information, a structure feature set representing model architecture design is extracted from a configuration file and a code file; S3: Based on the metadata information, document description content and code usage examples, an application task type of the model is inferred, and a use feature set is generated; S4: Based on the metadata information, a file name feature and a configuration file content, a deep learning framework relied on by the model is identified, and a framework feature set is generated; S5: Based on a weight file and a configuration file, a parameter quantity and a storage occupation size of the model are calculated, and a size feature set is generated; S6: The structure feature set, the use feature set, the framework feature set and the size feature set are received, keyword matching is performed by using a preset mapping table, and a matching result is mapped to a standard label library, so that a standardized multi-dimensional label is generated to be classified and displayed.

Images