Industrial software component classification method and system based on proxy labels and knowledge distillation

Abstract

The invention discloses an industrial software component classification method and system based on proxy labels and knowledge distillation. Ontology-guided text preprocessing is realized by constructing an industrial field term library, and manual annotation dependence is remarkably reduced in combination with a double-source proxy tag generation mechanism (teacher model weak tags and large model supervision data); a knowledge distillation framework of hierarchical semantic constraints is designed, knowledge of a complex teacher model is migrated to a lightweight student model, and real-time reasoning requirements are met while high classification precision is kept; a multi-task graph structure constraint and dynamic weight adjustment mechanism is creatively fused, and the adaptability of the model to industrial component short texts, multi-language mixed description and newly-added component types is effectively improved. According to the technical scheme, the low-code platform component management efficiency is remarkably optimized, domain knowledge fusion is enhanced, and core support is provided for industrial software intelligence.

Description

Technical Field

[0001] This invention relates to the field of intelligent industrial software technology, and in particular to a method and system for classifying industrial software components based on agent tagging and knowledge distillation. Background Technology

[0002] In today's era of accelerated digital transformation, low-code platforms in industrial software have become crucial tools for enterprises to build various applications due to their rapid development and ease of use. Low-code platforms offer a wide variety of software components, covering multiple functional areas such as data processing, user interface display, and business logic implementation. Accurately categorizing these components helps developers quickly find the components they need, improving development efficiency; facilitates resource management and maintenance for platform administrators; and promotes the optimization of the platform ecosystem, encouraging component reuse and sharing.

[0003] However, with the development of low-code platforms, the number of components is constantly increasing, and their functions are becoming increasingly complex and their descriptions more diverse. Traditional manual classification methods are not only inefficient and prone to errors, but also struggle to adapt to the rapid updates and changes in components. Existing automated classification technologies also face numerous challenges when applied to low-code platform software components. On the one hand, acquiring large amounts of labeled data is costly, and the labeling process requires specialized knowledge and is time-consuming and labor-intensive. On the other hand, software component descriptions often suffer from problems such as short texts, multilingual mixing, and semantic ambiguity, making it difficult for traditional classification models to accurately understand and classify them. Furthermore, new component types are constantly emerging, and the adaptability and scalability of models urgently need improvement. Therefore, there is an urgent need for an efficient and flexible classification method that can achieve high-precision classification with limited labeled data and quickly adapt to new component types and description methods, thereby promoting the efficient development of low-code platforms.

[0004] The prior art related to this invention suffers from the following key bottlenecks:

[0005] 1. Limitations of traditional machine learning methods

[0006] Technical solution: The solution employs algorithms such as Support Vector Machine (SVM) and Naive Bayes (NB), and relies on manual feature engineering (such as bag-of-words model and TF-IDF) to quantify the component text descriptions.

[0007] Inherent defects:

[0008] Feature extraction requires extensive domain knowledge, making it difficult to adapt to complex scenarios in industrial components, such as short text (e.g., "robotic arm trajectory calibration V2.1") and mixed multilingual scenarios (e.g., "data acquisition module + filtering algorithm").

[0009] The demand for labeled data is large, the labeling process requires the participation of professional engineers, and the cost is high.

[0010] Classification accuracy is significantly affected by text noise and class imbalance. For example, misclassification of "safety-critical components" (such as emergency stop control modules) may lead to industrial accidents.

[0011] 2. Bottlenecks in Deep Learning Models

[0012] Technical solution: Fine-tuning based on pre-trained models such as BERT and RoBERTa, and capturing semantic features using self-supervised learning.

[0013] Inherent defects:

[0014] The large number of parameters (RoBERTa model has over 100 million parameters) results in high deployment costs and significant inference latency in industrial edge devices or resource-constrained environments (single component classification time > 500ms).

[0015] The fine-tuning phase still requires large-scale labeled data, which cannot solve the core problem of scarce labeled samples in industrial scenarios.

[0016] It is difficult to handle semantically ambiguous descriptions specific to industrial components (such as "multi-axis linkage controller" which may belong to "motion control" or "mechanical drive").

[0017] 3. Common Challenges in the Industry

[0018] Labeling costs and data scarcity: Industrial component classification requires labeling by domain experts, and the cost of obtaining samples for a single category exceeds 1,000 yuan, while new component types continue to emerge (such as industrial metaverse interface components);

[0019] Real-time requirements: Developers need to obtain classification results in a short time to maintain the smoothness of development, which existing deep learning models cannot meet;

[0020] Poor domain adaptability: General models have difficulty recognizing the semantic relationships of industry-specific terms such as "PLC signal processing" and "OPC UA protocol adaptation".

[0021] In summary, the field of industrial software component classification urgently needs to overcome three major technical challenges:

[0022] 1. The contradiction between the demand for high precision and the shortage of labeled data;

[0023] 2. The contradiction between complex semantic understanding and lightweight deployment;

[0024] 3. The contradiction between dynamically adding new component types and the model's expansion capabilities. Summary of the Invention

[0025] This invention addresses the shortcomings of existing technologies by providing a method and system for classifying industrial software components based on proxy tagging and knowledge distillation. Through an innovative architecture that integrates proxy tag generation, knowledge distillation, and large-scale model supervision, it systematically solves the pain points in industrial scenarios found in existing technologies.

[0026] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0027] A classification method for industrial software components based on agent tagging and knowledge distillation, characterized by the following steps:

[0028] S1: Construct a terminology database TermBank for the industrial software component domain. This database contains functional categories of terms (FuncClass(t)). i ) and Hierarchical path HierPath(t i );

[0029] S2: Preprocess the input text description of the software components, including:

[0030] Text cleaning: Remove HTML tags, special characters, and punctuation marks, and unify lowercase;

[0031] Ontology-guided word segmentation: Based on TermBank, the longest matching algorithm is used to identify domain terms and generate a word segmentation sequence SegText;

[0032] Type-aware filtering: Based on predicted component type Filter out stop words and hardware parameter noise;

[0033] S3: Generate proxy label training data:

[0034] Using labeled dataset D labeled Training teacher models;

[0035] Generate high-confidence predictions from unlabeled data using large models and the reasons for the prediction j This forms a large model supervised dataset D. big-selected ;

[0036] Using the teacher model, a high-confidence weakly labeled dataset D is generated from the remaining unlabeled data. weak ;

[0037] Merge datasets: D augmented =D augmented1 ∪D weak ;

[0038] S4: Build a lightweight student model and train it using knowledge distillation:

[0039] Embedding constraint technique is adopted: by minimizing the cosine distance and mapping matrix constraints, the hidden representation of the student model is made semantically consistent with that of the teacher model;

[0040] Design a multi-task loss function:

[0041] Where α(t) represents the dynamic weight coefficient of hard label loss, β(t) represents the dynamic weight coefficient of soft label loss, γ(t) represents the dynamic weight coefficient of embedding constraint loss, δ(t) represents the dynamic weight coefficient of hard label supervision loss of large model, and ε(t) represents the dynamic weight coefficient of rational alignment loss of large model. Represents the hard-label loss function; Represents the soft-label loss function; This represents the embedding constraint loss function;

[0042] Introducing a large model supervision signal: using the cross-entropy loss L from the prediction results. big - Hard and predictive rationale feature alignment loss Strengthen supervision;

[0043] S5: Use the trained student model to classify industrial software components in real time and output the classification results.

[0044] According to the industrial software component classification method of claim 1, the construction of the domain terminology database in step S1 satisfies:

[0045] Term t i Associated triples {t i FuncClass(t i ),HierPath(t i )};

[0046] The hierarchical path description format is a multi-level structure of "basic functions → data processing → acquisition".

[0047] According to the method described in claim 1, the ontology-guided word segmentation algorithm in step S2 includes:

[0048] A sliding window mechanism is used to match terms in the TermBank;

[0049] When the continuous subsequence ω i …ω j Matching term t k At that time, t k As a whole word segmentation unit;

[0050] Unmatched segments retain the basic word segmentation results.

[0051] According to the industrial software component classification method of claim 1, the large model supervision data generation in step S3 needs to satisfy:

[0052] Set the confidence threshold τ big ,filter The prediction results;

[0053] Apply a thought chain prompting mechanism to the output of the large model to guide the generation of prediction rationale related to industrial scenarios.

[0054] According to the method described in claim 1, the embedding constraint technique in step (4) specifically comprises:

[0055] Construct a domain mapping matrix M to transform the general embedding into an industry-specific representation:

[0056] Minimize the loss function:

[0057] Among them, h teacher The hidden representation of the teacher model; h student β represents the hidden representation of the student model; β represents the weight coefficients of the mapping matrix constraints; M represents the domain mapping matrix; The word w i The general embedding vector; Represents the L2 norm;

[0058] The industrial software component classification method according to claim 1 is characterized in that the multi-task loss function includes a dynamic weight adjustment mechanism:

[0059] α(t)=α0·σ(λ term • Terminology coverage);

[0060] Where α0 represents the basic weight coefficient; σ represents the sigmoid function; λ term A scaling factor indicating term coverage;

[0061]

[0062] Where β0 represents the basic weight coefficient; KL divergence represents the KL divergence between the output distributions of the teacher model and the student model;

[0063]

[0064] Where γ0 represents the basic weight coefficient; τ big This represents the confidence threshold for large model predictions.

[0065] The industrial software component classification method according to claim 1 is characterized in that step S4 further introduces a multi-task learning framework:

[0066] The primary task is component classification, while auxiliary tasks include named entity recognition or sentiment analysis.

[0067] Loss due to graph structure constraints:

[0068] in, The graph-constrained loss function for the k-th auxiliary task is represented by y; M represents the number of samples in the training batch; i represents the sample index, 1≤i≤M; i,k x represents the true label of the i-th sample in the k-th auxiliary task; i θ represents the textual description of the software component for the i-th sample; aux,k This represents the model parameters of the output layer of the k-th auxiliary task; (y i,k |x i ;θ aux,k ) represents the standard negative log-likelihood loss of the auxiliary task; β represents the balance coefficient of the graph embedding constraint; GraphEmb(y i,k ) represents the embedding vector of the auxiliary task label in the graph structure; GraphEmb(y i ) represents the embedding vector of the main task label in the graph structure; y i This represents the true class label of the i-th sample in the main task (component classification).

[0069] The industrial software component classification method according to claim 9, characterized in that the multi-task knowledge distillation includes:

[0070] Main task distillation loss

[0071] Among them, P teacher(c|x) P represents the teacher model's predicted probability for category c; student represents the predicted probability distribution of the student model; μ represents the weighting coefficient of the cosine similarity loss;

[0072] Task-related attention mechanism: Dynamically weighted auxiliary task loss.

[0073] Where Q represents the query vector of the hidden representation of the student model; K k Let d represent the feature vector of the k-th auxiliary task; d represents the dimension of the vector.

[0074] This invention also discloses an industrial software component classification system that implements the industrial software component classification method according to any one of claims 1-6, characterized in that it comprises:

[0075] Terminology Management Module: Builds and maintains the domain terminology database TermBank;

[0076] Preprocessing engine: performs text cleaning, ontology-guided word segmentation, and type-aware filtering;

[0077] Proxy label generator: coordinates the teacher model and the large model to generate weakly labeled data;

[0078] Knowledge distillation training module: Implements embedding constraints and multi-task loss to train a lightweight student model;

[0079] Real-time classification interface: Deploy student models to provide low-latency classification services.

[0080] The system according to claim 7, wherein the knowledge distillation training module comprises:

[0081] The hierarchical semantic dictionary Hier-Syn(w) stores synonyms of the term w at the component level;

[0082] Graph Structure Alignment Unit: Parses the prediction rationale of the large model into a directed graph and aligns it with the student model output through graph embedding.

[0083] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-mentioned industrial software component classification method.

[0084] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described industrial software component classification method.

[0085] Compared with the prior art, the advantages of the present invention are as follows:

[0086] 1. Significantly reduces annotation dependency

[0087] The innovative approach integrates dual-source proxy label generation (weak labels from teacher models and supervision signals from large models), significantly reducing the need for manual annotation and effectively solving the problem of scarce labeled data in industrial scenarios.

[0088] By generating prediction reasons through large-scale model reasoning, the training set can be efficiently expanded using a small amount of seed data, breaking through the dependence of traditional models on large-scale annotation.

[0089] 2. Achieving a balance between high precision and high efficiency

[0090] Knowledge distillation techniques based on hierarchical constraints enable lightweight models to maintain high classification accuracy while significantly reducing the number of parameters.

[0091] By combining industrial semantic embedding mapping, the semantic understanding of short texts and multilingual mixed descriptions is improved, meeting the real-time classification needs of low-code platforms.

[0092] 3. Enhance robustness in complex scenarios

[0093] By using ontology-guided word segmentation and type-aware filtering, the problems of terminological ambiguity and noise interference in the description of industrial components are effectively solved.

[0094] By adopting a multi-task graph structure constraint, the model's adaptability to new component types is improved, avoiding the performance degradation of traditional models in dynamically expanded scenarios.

[0095] 4. Optimize platform resource management efficiency

[0096] The classification results drive the intelligent organization of the component library, greatly improving the retrieval efficiency of developers and the reusability of components;

[0097] Hierarchical classification output provides the platform with an interpretable component organization structure, reducing the complexity of system operation and maintenance.

[0098] 5. Deep integration of industrial knowledge

[0099] By using an industrial terminology database and graph embedding semantic constraints, professional knowledge such as mechanical control and protocol interfaces is encoded into the model, thereby enhancing the classification reliability of safety-critical components.

[0100] The dynamic weight adjustment mechanism enables the model to continuously adapt to changes in the distribution of industrial data, ensuring long-term deployment stability. Attached Figure Description

[0101] Figure 1 This is a flowchart of the industrial software component classification method according to an embodiment of the present invention;

[0102] Figure 2 This is a framework diagram of the industrial software component classification method according to an embodiment of the present invention. Detailed Implementation

[0103] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0104] like Figure 1 and 2 As shown, this invention provides a method for classifying industrial software components based on proxy tagging and knowledge distillation, comprising the following steps:

[0105] (1) Construction of the domain terminology database. Establish an industrial component terminology dictionary TermBank = {t1, t2, ..., t...} m}, where each term t i Associate its functional category FuncClass(t) i ) and hierarchical relationship HierPath(t i ). For example, t i = "Data Acquisition Module", corresponding to FuncClass(t i) = "Data Processing", HierPath(t i = "Basic functions → Data processing → Acquisition".

[0106] (2) Clean the input text description of the software components, removing irrelevant characters and noise. Specific operations include:

[0107] a. Remove HTML tags: If the component description text contains HTML tags, remove them using regular expressions.

[0108] b. Remove special characters and punctuation marks: Keep letters, numbers, and common separators (such as spaces and newlines), and replace the rest with spaces.

[0109] c. Unified lowercase: Convert all characters to lowercase to reduce the size of the vocabulary and improve consistency.

[0110] (3) Segment the cleaned text into words or sub-words. For English text, spaces and punctuation marks are usually used as delimiters; for Chinese text, word segmentation tools (such as jieba) can be used for segmentation. The segmented text is represented as: Text={ω1,ω2,…,ω n}

[0111] Where, ω i This represents the i-th word or sub-word unit, where n is the length of the text.

[0112] (4) Ontology-guided word segmentation algorithm. For the input text Text={ω1,ω2,…,ω… n}, using the longest matching method to prioritize the identification of domain terms, if the continuous subsequence ω i …ω j The term t constitutes the term in TermBank k :

[0113] The word segmentation result is represented as SegText={s1,s2,…,s p}, where s i For terms or basic words. Example: If the input text is "robotic arm data acquisition module", and both "robotic arm" and "data acquisition" are domain terms, the word segmentation result will be {robotic arm, data acquisition, module} instead of being split into "robotic", "arm", "data", "acquisition", etc.

[0114] (5) Define a component type set CompType = {c1, c2, ..., c q}, each type corresponds to a dedicated stop word list StopWords = (c i For example: StopWords control component. (c控制){Execute, Start, Stop}. Hardware parameter patterns are defined using regular expressions: HardwarePatterns = {V\d+\.\d+,\d+V,IP\d+}, used to match meaningless noise such as "220V" and "V1.3".

[0115] (6) Filtering. Predicted values ​​for a given component type. (Preliminary predictions can be made using a teacher model), the filtering rules are as follows:

[0116]

[0117]

[0118] (7) Construct a hierarchical semantic dictionary Hier-Syn(w) to store synonyms of word w at different component levels (e.g., "filtering" corresponds to "filtering" at the "signal processing" level). Based on the component hierarchical path HierPath predicted by the teacher model, select the most semantically relevant synonyms:

[0119]

[0120]

[0121]

[0122] Where φ(·) is the embedding function of the teacher model, and P is the hierarchical path.

[0123] (8) Use industrial-grade custom BPE to segment multilingual text into subwords, and generate initial embeddings using a large model (such as GPT-4):

[0124] (9) Introduce a domain mapping matrix M to transform the general embedding into an industry domain representation, and ensure semantic consistency between the student model and the teacher model through embedding constraint loss:

[0125]

[0126] (10) The final embedding vector is:

[0127] X=[ν1′,ν2′,…ν m ′,ν [CLS] ,ν [SEP] ]

[0128] (11) When selecting a subset of labeled data as seed data from the industrial software component text description dataset, the category distribution of the industrial software components should be fully considered. For example, the number and importance of data processing, interface display, and business logic implementation components may differ within the industrial software component dataset. When selecting seed data, stratified sampling should be performed according to the proportion of each category in actual industrial applications to ensure that each category has reasonable representativeness in the seed data. Assume the labeled dataset is as follows:

[0129]

[0130] Where, x i It is a text description, y i is the corresponding category label, and N is the number of labeled data.

[0131] (12) Using labeled data D labeled Train a high-performance "teacher" model, such as RoBERTa. During training, in addition to minimizing the cross-entropy loss on the labeled data, prior knowledge of the industrial software component domain can be introduced. For example, when training the teacher model, knowledge of component relationships in specific industrial domains can be encoded. For instance, in a robotic arm grasping an object, there is a specific collaborative relationship between the camera control component and the robotic arm's movement control component. Integrating this relationship knowledge into the model training allows the teacher model to better learn the intrinsic connections between industrial software components. The goal of the teacher model is to minimize the cross-entropy loss on the labeled data.

[0132]

[0133] Wherein, P(y i |x i ;θ teacher ) is the teacher model in the input x i Predict label y i The probability, θ teacher These are the parameters of the teacher model.

[0134] (13) Select a large number of unlabeled text descriptions from the dataset as the unlabeled dataset. Where M represents the number of unlabeled data.

[0135] Large models are used to process unlabeled data. When using large models to process unlabeled data, specific thought processes and prompts are designed to address the specialization of industrial software components. For example, in a task where a robotic arm grasps an object, the large model can guide the user to think from professional perspectives such as camera recognition, object localization, and the movement of the robotic arm when generating predictions and justifications. The large model proceeds step-by-step through the thought process prompts to obtain predictions and generate justifications for those predictions. For each unlabeled sample x... j Large model output predicted category and the reasons for the prediction j .

[0136] (14) To ensure high quality of the generated information, a large model prediction confidence threshold τ is set. big (For example, 90%), select samples that meet the criteria. Regarding the prediction results, when a large model predicts the probability of a certain class... If the sample is not found, retain it. During the screening process, consider the specific requirements of industrial software components. For example, in industrial software components for medical device control, for components involving safety-critical functions, increase the screening threshold to ensure classification accuracy. Form a large-scale model-supervised screening dataset:

[0137]

[0138] in, The teacher model is x j Predicted category labels.

[0139] (15) The filtered large model supervision data D big-selected Compared with the original labeled dataset D labeled The dataset is merged to form the first expanded dataset:

[0140] D augmented1 =D labeled ∪D big-selected

[0141] During the merging process, check the balance of data for each category of industrial software components. If the proportion of data for a certain component becomes unbalanced after expansion, oversampling or undersampling methods can be used for adjustment.

[0142] (16) Use the trained teacher model to generate predicted labels for the unlabeled data (excluding data already filtered by the large model). For each remaining unlabeled sample x k The teacher model outputs its predicted class distribution:

[0143] P k =P(y|x) k ;θ teacher )

[0144] When generating weak labels, the functional similarity and hierarchical relationship of industrial software components should be considered. For example, in the software components of an industrial internet platform, although components with similar data processing functions belong to different subcategories, the confidence requirements can be appropriately relaxed when generating weak labels to increase the diversity of training data. At the same time, the hierarchical relationship between components, such as the relationship between basic data acquisition components and upper-level data analysis components, can be utilized to improve the accuracy of weak labels.

[0145] (17) To ensure the quality of the generated weak labels is high, a confidence threshold τ for the teacher model prediction is set (e.g., 95%), and samples that meet the criteria are selected:

[0146]

[0147] For mission-critical industrial software components, increase the threshold (e.g., 98%); for general components, use a relatively low threshold (e.g., 92%).

[0148] The selected high-confidence weakly labeled samples D weak Added to the dataset D after the first expansion augmented1 In this process, a second expanded training set is formed.

[0149] D augmented =D augmented1 ∪D weak

[0150] (18) Select a lightweight model as the "student," such as ALBERT, which has far fewer parameters than the teacher model (such as RoBERTa). When selecting a lightweight student model (such as ALBERT), evaluate the model's ability to handle long sequences and complex terms commonly found in industrial software component text. Assume the teacher model has θ parameters. teacher The parameters of the student model are θ student .

[0151] (19) Define a new loss function. In addition to the loss based on hard labels, soft labels and embedding constraints, add a large model supervision loss.

[0152] Hard label loss: The predictions from the teacher model are used as hard labels for the augmented training set D. augmented ={(x i ,y i The student model aims to minimize the cross-entropy loss with hard labels. In addition, a semantic matching penalty term from a domain terminology corpus is introduced into the traditional cross-entropy loss to enhance the model's ability to recognize industry-specific vocabulary.

[0153]

[0154] Where T i For sample x i The corresponding domain terminology set (e.g., "data acquisition module," "robotic arm control"); γ is the term matching penalty coefficient, dynamically adjusted through the domain terminology database TermBank. In addition to hard labels, the output probability distribution of the teacher model can be used as soft labels, allowing the student model to learn the uncertainty information of the teacher model. For each sample x... i The output probability distribution of the teacher model is as follows:

[0155] Ρ teacher,i =Ρ(y|x i ;θ teacher )

[0156] Soft label loss: The output probability distribution of the teacher model is used as a soft label. The goal of the student model is to minimize the KL divergence loss between its output distribution and the teacher model's output distribution.

[0157]

[0158] Where depth(c) represents the depth of category c in the hierarchical structure; Importance(c) is predefined by domain experts (e.g., weight of safety-critical components = 1.0, weight of general components = 0.3).

[0159] Embedding constraint loss: Calculate the similarity between the hidden representations of the teacher and student models in the last layer to ensure that the semantic representation of the student model is consistent with that of the teacher model, and introduce a joint constraint of the domain mapping matrix M and the hierarchical semantic dictionary Hier-Syn:

[0160]

[0161] Among them, the Frobenius norm regularization term of the increasing matrix Control domain mapping matrix complexity. and These are the teacher model and the student model for sample x. i The hidden representation is cosine similarity (·,·).

[0162] Large Model Supervision Loss: For samples in a large model supervision dataset, the student model learns the prediction results and rationale behind the large model's predictions. First, the feature vector transformation of the prediction rationale is explicitly defined as follows:

[0163] f reason (reason j = BigModel-Embedding(reason) j )

[0164] Simultaneously, the predicted reasons are transformed into feature vectors (e.g., through text embedding methods), and the prediction results are used as additional hard-label supervision to construct a cross-entropy loss. The cosine similarity between the student model's output features and the feature vectors of the predicted reasons from the large model is calculated.

[0165]

[0166] (20) By combining the above three loss functions and introducing dynamic weight adjustment based on data characteristics to replace the fixed hyperparameter combination, the adaptability of the model to the distribution of industrial data is improved, forming the final optimization objective of the student model:

[0167]

[0168] Where α(t), β(t), γ(t), δ(t), and ∈(t) are dynamic weighting coefficients.

[0169]

[0170]

[0171] Where B is the batch size, σ(·) is the sigmoid function, and KL is the KL divergence.

[0172] (Average confidence level of the large model in batch b);

[0173] For components related to core industrial sectors or key business operations, increase the weight of the corresponding loss terms to make the model focus more on the accurate classification of these components.

[0174] (21) Using the expanded training set D augmented and comprehensive loss function A lightweight student model is trained until convergence. In this way, the student model not only learns the output distribution of the teacher model but also retains the semantic representation of the teacher model, thus maintaining high performance while reducing the number of parameters. During training, the model's performance on an industrial software component test set needs to be evaluated periodically.

[0175] (22) In the software component classification task, the main task can be combined with the auxiliary task of generating reasoning for classification to enhance the model's understanding of text semantics and improve the interpretability of the classification. We further expanded the training set by introducing data from the auxiliary task through multi-task learning. The main task and auxiliary task are defined as follows:

[0176] a. Main task: Low-code platform software component classification task, the goal of which is to classify the text descriptions of software components into predefined categories.

[0177] b. Auxiliary tasks: Select natural language processing tasks related to the main task, such as named entity recognition (NER), sentiment analysis, and semantic role labeling. These tasks can help the model learn richer linguistic features.

[0178] (23) Construct a multi-task model that shares the underlying pre-trained language model (such as BERT or SciBERT) and adds a specific output layer for each task. Let the shared underlying model be M, and the output layer of the main task be f. main The output layer of the auxiliary task is f aux,k (k represents the kth auxiliary task).

[0179] (24) For each task (including the main task and auxiliary tasks), an independent "teacher" model is trained. When training the teacher model related to industrial software components, expert-annotated data from the industrial field is used for fine-tuning to improve the accuracy of the model in classifying industrial software components and auxiliary tasks. The goal of each teacher model is to minimize the loss function of the corresponding task. Based on the original cross-entropy loss, an alignment penalty term for industrial terms and task-related attention weights are introduced. The loss process for the main task is defined as follows:

[0180]

[0181] Where I(·) is the indicator function; I(t∈SegText i =1 when term t is recognized by the word segmenter.

[0182] Bind auxiliary tasks (such as named entity recognition) to the hierarchical structure of the main task, and introduce graph structure constraints:

[0183]

[0184] Among them, y i,k It is the label of the kth auxiliary task; θ aux,k These are the parameters corresponding to the teacher model for auxiliary tasks;

[0185] GraphEmb(·) maps task labels to embedding vectors in an industrial component relationship graph, capturing the functional relationships between components such as “data acquisition module” and “sensor control”.

[0186] (25) In the multi-task student model, knowledge distillation is introduced, enabling the student model to learn not only the real labels but also the output distribution of the teacher model. Specifically, the main task distillation method is as follows:

[0187]

[0188] PathSim(·) is based on the semantic similarity between the hierarchical path HierPath(y) predicted by the teacher model and the implicit path StudentPath(x) of the student model, and calculates the path matching degree through dynamic time warping (DTW).

[0189] (26) To ensure that the semantic representation of the student model is consistent with that of the teacher model, embedding constraints are introduced. When calculating the embedding constraint loss of the industrial software component-related models, a semantic similarity measurement method in the industrial domain is combined to calculate the similarity between the hidden representations of the teacher model and the student model in the last layer, thus obtaining the embedding constraint loss. The industrial domain mapping matrix and the task association matrix are then integrated:

[0190]

[0191] Where w k,i =softmax(TaskRelevance(k,SegText i The weighting is based on the relevance between word segmentation results and the task; TaskEmb k It is a semantic embedding vector for the auxiliary task, which forces the student model representation to align with the task features.

[0192] (27) Convert the prediction reasoning generated by the large model into a directed graph and align it structurally with the student model output.

[0193] Reason text embedding:

[0194]

[0195] Structure embedding:

[0196] GraphAlign(r j =GCN(ParseToGraph(r) j ))

[0197]

[0198] GraphAlign(r j The reason for the prediction r j It is parsed as a structure embedding of a "cause-effect" directed graph.

[0199] (28) Combining the large model supervision loss, for the large model supervision data in the multi-task scenario, we also construct cross-entropy loss and prediction reason difference loss.

[0200] Supervision loss for the main task large model:

[0201]

[0202] (here θ)student These are the parameters of the multi-task student model on the main task. For auxiliary tasks, assuming the main model also has corresponding prediction results and justifications for the auxiliary tasks, taking the k-th auxiliary task as an example, we construct the cross-entropy loss.

[0203]

[0204] And the loss due to differences in the reasons for prediction

[0205]

[0206] Among them, D big-selected-aux,k It is the supervised dataset selected by the large model for the k-th auxiliary task, θ student-aux,k These are the parameters of the multi-task student model on the k-th auxiliary task, f student-aux,k (x j ) is the student model's response to x on the k-th auxiliary task. j The output characteristics.

[0207] (29) Integrating the supervision signals of the large model for both main and auxiliary tasks:

[0208]

[0209]

[0210] The auxiliary task supervision weights are dynamically adjusted based on the confidence level of the large model and the difficulty of the task.

[0211] (30) Combining multi-task loss, distillation loss, embedding constraint loss, and large model supervision loss to form the final optimization objective of the multi-task student model:

[0212]

[0213] in,

[0214] Terminology coverage index weighting;

[0215] Task-related soft weights;

[0216] γ(t), δ(t), ∈(t) are related to the prediction confidence P of the large model. big Adaptive decay coefficient with training rounds.

[0217] Based on the importance and application requirements of industrial software components in different industrial scenarios, the weights of various losses in the final optimization objective of the multi-task student model are finely adjusted.

[0218] (31) Use the training set that has been augmented multiple times (containing the original labeled data, weakly labeled data generated by the teacher model, and supervised data from the large model) and the comprehensive loss function. Train the multi-task lightweight student model until it converges.

[0219] (32) Select a suitable optimization algorithm to update the parameters of the student model. In the industrial software component classification task, based on the gradient change characteristics during model training and the scale of industrial data, the parameter update rule when using the Adam optimizer is as follows:

[0220]

[0221] Where η is the learning rate. This is the gradient of the loss function with respect to the model parameters. During training, hyperparameters can be adjusted using the validation set, and early stopping can be used to avoid overfitting. When selecting the validation set, priority should be given to selecting software component datasets that are representative of the industry to improve the stability and reliability of the model.

[0222] In another embodiment of the present invention, an industrial software component classification system is provided. This system can be used to implement the above-described industrial software component classification method, specifically including:

[0223] Terminology Management Module: Builds and maintains the domain terminology database TermBank;

[0224] Preprocessing engine: performs text cleaning, ontology-guided word segmentation, and type-aware filtering;

[0225] Proxy label generator: coordinates the teacher model and the large model to generate weakly labeled data;

[0226] Knowledge distillation training module: Implements embedding constraints and multi-task loss to train a lightweight student model;

[0227] Real-time classification interface: Deploy student models to provide low-latency classification services.

[0228] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of an industrial software component classification method.

[0229] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0230] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the industrial software component classification method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by a processor.

[0231] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0232] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0233] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0234] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0235] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. A classification method for industrial software components based on agent tagging and knowledge distillation, characterized in that, Includes the following steps: S1: Construct a terminology database TermBank for the industrial software component domain. This database contains functional categories of terms (FuncClass(t)). i ) and Hierarchical path HierPath(t i ); S2: Preprocess the input text description of the software components, including: Text cleaning: Remove HTML tags, special characters, and punctuation marks, and unify lowercase; Ontology-guided word segmentation: Based on TermBank, the longest matching algorithm is used to identify domain terms and generate a word segmentation sequence SegText; Type-aware filtering: Based on predicted component type Filter out stop words and hardware parameter noise; S3: Generate proxy label training data: Using labeled dataset D labeled Training teacher models; Generate high-confidence predictions from unlabeled data using large models and the reasons for the prediction j This forms a large model supervised dataset D. big-selected ; Using the teacher model, a high-confidence weakly labeled dataset D is generated from the remaining unlabeled data. weak ; Merge datasets: D augmented =D augmented1 ∪D weak ; S4: Build a lightweight student model and train it using knowledge distillation: Embedding constraint technique is adopted: by minimizing the cosine distance and mapping matrix constraints, the hidden representation of the student model is made semantically consistent with that of the teacher model; Design a multi-task loss function: Where α(t) represents the dynamic weight coefficient of hard label loss, β(t) represents the dynamic weight coefficient of soft label loss, γ(t) represents the dynamic weight coefficient of embedding constraint loss, δ(t) represents the dynamic weight coefficient of hard label supervision loss of large model, and ε(t) represents the dynamic weight coefficient of rational alignment loss of large model. Represents the hard-label loss function; Represents the soft-label loss function; This represents the embedding constraint loss function; Introducing a large model supervision signal: using the cross-entropy loss L from the prediction results. big -hard and prediction rationale feature alignment loss L big -Reason strengthens supervision; S5: Use the trained student model to classify industrial software components in real time and output the classification results.

2. The industrial software component classification method according to claim 1, characterized in that, The construction of the domain terminology database in step S1 satisfies: Term t i Associated triples {t i FuncClass(t i ),HierPath(t i )}; The hierarchical path description format is a multi-level structure of "basic functions → data processing → acquisition".

3. The method according to claim 1, characterized in that, The ontology-guided word segmentation algorithm in step S2 includes: A sliding window mechanism is used to match terms in the TermBank; When the continuous subsequence ω i …ω j Matching term t k At that time, t k As a whole word segmentation unit; Unmatched segments retain the basic word segmentation results.

4. The industrial software component classification method according to claim 1, characterized in that, The generation of large model supervision data in step S3 must meet the following requirements: Set the confidence threshold τ big ,filter The prediction results; Apply a thought chain prompting mechanism to the output of the large model to guide the generation of prediction rationale related to industrial scenarios.

5. The method according to claim 1, characterized in that, The embedding constraint technique in step (4) is specifically as follows: Construct a domain mapping matrix M to transform the general embedding into an industry-specific representation: Minimize the loss function: Among them, h teacher The hidden representation of the teacher model; h student β represents the hidden representation of the student model; β represents the weight coefficients of the mapping matrix constraints; M represents the domain mapping matrix; The word w i The general embedding vector; This represents the L2 norm.

6. The industrial software component classification method according to claim 1, characterized in that, The multi-task loss function includes a dynamic weight adjustment mechanism: Where α0 represents the basic weight coefficient; σ represents the sigmoid function; λ term A scaling factor indicating term coverage; Where β0 represents the basic weight coefficient; KL divergence represents the KL divergence between the output distributions of the teacher model and the student model; Where γ0 represents the basic weight coefficient; τ big This represents the confidence threshold for large model predictions.

7. The industrial software component classification method according to claim 1, characterized in that, Step S4 further introduces a multi-task learning framework: The primary task is component classification, while auxiliary tasks include named entity recognition or sentiment analysis. Loss due to graph structure constraints: in, The graph-constrained loss function for the k-th auxiliary task is represented by y; M represents the number of samples in the training batch; i represents the sample index, 1≤i≤M; i,k x represents the true label of the i-th sample in the k-th auxiliary task; i θ represents the textual description of the software component for the i-th sample; aux,k This represents the model parameters of the output layer of the k-th auxiliary task; (y i,k |x i ;θ aux,k ) represents the standard negative log-likelihood loss of the auxiliary task; β represents the balance coefficient of the graph embedding constraint; GraphEmb(y i,k ) represents the embedding vector of the auxiliary task label in the graph structure; GraphEmb(y i ) represents the embedding vector of the main task label in the graph structure; y i This represents the true class label of the i-th sample in the main task.

8. The industrial software component classification method according to claim 1, characterized in that, Multi-task knowledge distillation includes: Main task distillation loss Among them, P teacher(c|x) P represents the teacher model's predicted probability for category c; student represents the predicted probability distribution of the student model; μ represents the weighting coefficient of the cosine similarity loss; Task-related attention mechanism: Dynamically weighted auxiliary task loss; Where Q represents the query vector of the hidden representation of the student model; K k Let d represent the feature vector of the k-th auxiliary task; d represents the dimension of the vector.

9. An industrial software component classification system implementing the industrial software component classification method according to any one of claims 1-8, characterized in that, include: Terminology Management Module: Builds and maintains the domain terminology database TermBank; Preprocessing engine: performs text cleaning, ontology-guided word segmentation, and type-aware filtering; Proxy label generator: coordinates the teacher model and the large model to generate weakly labeled data; Knowledge distillation training module: Implements embedding constraints and multi-task loss to train a lightweight student model; Real-time classification interface: Deploy student models to provide low-latency classification services.

10. The system according to claim 9, characterized in that, The knowledge distillation training module includes: The hierarchical semantic dictionary Hier-Syn(w) stores synonyms of the term w at the component level; Graph Structure Alignment Unit: Parses the prediction rationale of the large model into a directed graph and aligns it with the student model output through graph embedding.

Images