An ontology and RoBERTa model-based aviation field meta-model recommendation method

Abstract

The application provides an aviation field meta-model recommendation method based on an ontology and a RoBERTa model, which comprises the following steps: obtaining an aviation field ontology file and an aviation field meta-model resource file; converting an XML format meta-model file into a text sequence; dividing the text sequence into a training set and a test set; performing text normalization on the training set, and enhancing a word segmenter by using an aviation field ontology dictionary to perform ontology-enhanced word segmentation; constructing a RoBERTa model, expanding a concept set of the aviation field ontology to a model vocabulary, defining and minimizing a total loss function; receiving a meta-model context sequence to be processed, inserting a mask mark at a specified position, inputting the trained RoBERTa model, obtaining a mask position probability distribution output by the model, and generating a candidate recommendation list arranged in descending order of confidence. In the above technical solution, the technical problems of low recommendation accuracy, dependence on artificial rules and inability to adapt to complex dynamic semantic environments in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a meta-model recommendation method for the aviation domain based on ontology and the RoBERTa model. Background Technology

[0002] Ontology is a branch of Western philosophy that studies the essence of existence, as opposed to epistemology. In information science, the term refers to a standardized system for describing domain knowledge, constructing a structured knowledge framework by defining concepts, attributes, and relationships. In artificial intelligence, it is viewed as a formalized system for describing shared conceptual models, used to symbolize human knowledge so that computers can understand it. The core functions of ontology include unifying terminology, achieving semantic network connections, and promoting the structured expression of knowledge. Its construction requires two steps: establishing conceptual relationships and applying formal tools. In the field of standard digitization, ontology is used to clarify standard concept definitions, attribute content, and rules of conceptual relationships, constructing multi-dimensional semantic networks.

[0003] RoBERTa is an optimization proposed by Facebook AI Research based on BERT. Its core idea is to fully unlock the potential of BERT through a more thorough and optimized training strategy. The RoBERTa model utilizes a larger training dataset and longer training time to ensure more comprehensive model learning. This model employs dynamic masking technology, randomly generating masks for the input data to enhance its generalization ability, and gains language understanding capabilities through self-supervised learning on a large amount of unlabeled raw text.

[0004] A domain metamodel can be simply understood as a "model of models." It doesn't directly describe the business logic itself, but rather a set of rules and a language for building models for a specific domain. The domain metamodel provides a unified language, ensuring that all experts, developers, and product managers within the domain have a consistent understanding of core concepts, eliminating communication ambiguities. During the modeling process, it provides specifications for creating models within that domain, ensuring that all models follow unified semantics and rules, thereby improving model quality and consistency. In model-driven development, defining a domain metamodel automatically generates code, configuration files, or development tools, significantly improving software development efficiency.

[0005] As modern software systems become increasingly complex, model-driven engineering (MDE) offers significant advantages in improving software quality and reducing maintenance costs during software development. However, the successful implementation of MAE requires reliable auxiliary modeling tools. During modeling, modelers must strictly adhere to unified professional domain standards, accurately defining all domain concepts. The cross-disciplinary and interdisciplinary nature of this process makes modeling extremely challenging for technical personnel. Furthermore, the significant manpower required to manually complete model rules further increases costs and complexity for enterprises. Therefore, there is an urgent need to design an efficient and accurate auxiliary modeling method to quickly facilitate the implementation of MAE.

[0006] Currently, there are technical solutions for automatically completing scenario concepts in the aviation domain metamodel to aid modeling. The first approach involves building a logic-driven rule engine and utilizing a model transformation mechanism to output recommendation results for local models. The second approach extracts domain knowledge from aviation-related text data and embeds it into the model using pre-trained words to achieve automatic recommendation functionality. Both solutions have limitations. The first approach requires significant manpower to customize model transformation logic and relies on a logic inference engine, making it complex and difficult to implement. The second approach requires pre-stocking aviation-related knowledge resources and precise targeting, but the timeliness, accuracy, and version control of these resources are uncertain during acquisition, and the extraction process is extremely cumbersome. Summary of the Invention

[0007] This application provides a meta-model recommendation method for the aviation domain based on ontology and the RoBERTa (Robustly Optimized BERT Approach) model, in order to solve the technical problems of low recommendation accuracy, reliance on manual rules, and inability to adapt to complex dynamic semantic environments in existing technologies.

[0008] Firstly, this paper presents a meta-model recommendation method for the aerospace domain based on ontology and the RoBERTa model, including the following steps:

[0009] Obtain the aerospace ontology file and aerospace metamodel resource file;

[0010] The XML-formatted metamodel file is converted into a text sequence using a tree-like text serialization algorithm, which maps the metamodel's hierarchical structure, element types, and dependencies into a natural language-like sequence containing special tags.

[0011] The text sequence is divided into a training set and a test set;

[0012] The training set is text normalized, and the word segmenter is enhanced using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation.

[0013] Construct a RoBERTa model, extending the concept set of the aerospace ontology to the model vocabulary; define and minimize the total loss function;

[0014] Receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence.

[0015] In the above technical solution, the following steps are taken: First, acquire aviation domain ontology files and aviation domain metamodel resource files. Then, convert the XML-formatted metamodel files into text sequences using a tree-like text serialization algorithm. This algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags. Next, divide the text sequences into training and testing sets. Normalize the training set and enhance the word segmenter using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation. Construct a RoBERTa model, expanding the concept set of the aviation domain ontology to the model vocabulary. Define and minimize the total loss function. Receive the metamodel context sequence to be processed, insert mask tags at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence. This solves the technical problems of low recommendation accuracy, reliance on manual rules, and inability to adapt to complex dynamic semantic environments in existing technologies.

[0016] In one specific implementation scheme, it also includes:

[0017] The accuracy of the recommendation list is evaluated using the test set, and the Top-K hit rate and average reciprocal ranking metrics are calculated.

[0018] In one specific implementation, the total loss function is a weighted average of the dynamic masking language model loss LMLM and the ontology semantic consistency constraint loss LONTO, wherein,

[0019] , where λ is the weighting coefficient.

[0020] In one specific implementation, the tree-structured text serialization algorithm follows these rules:

[0021] Special tags including [CLASS], [PROPERTY], [RELATION], [SOURCE], and [TARGET] are used to define the different constituent units of the metamodel;

[0022] The class's property list is encapsulated in [PROPERTIES] and Between the markers;

[0023] The definition of a relationship includes explicit declarations of the source class and the target class.

[0024] In one specific implementation scheme, a semantic obfuscation-based masking sampling strategy is adopted. This strategy determines the mask position based on the importance weight W(c) of concepts in the aerospace domain ontology; the formula for calculating the importance weight W(c) is as follows:

[0025] ;

[0026] Where freq(c) is the frequency of occurrence of concept c in the ontology, depth(c) is the depth of concept in the classification tree, deg(c) is the connectivity of concept with other nodes, and α, β, and γ are preset adjustment coefficients.

[0027] In one specific implementation, the ontology semantic consistency constraint loss LONto employs a contrastive learning framework, and its calculation process includes:

[0028] Extract concept pairs with direct relationships from the aerospace ontology as positive sample pairs (ci, cj);

[0029] Randomly sampled, unrelated concept pairs are used as negative sample pairs;

[0030] The [CLS] Token vector output by the RoBERTa model is mapped to the semantic space through a semantic projection head to obtain the vector z;

[0031] The similarity loss between positive sample pairs is calculated using the cross-entropy loss calibrated by normalized temperature, which encourages positive sample pairs to move closer together in the vector space and negative sample pairs to move further apart.

[0032] In one specific implementation, the association relationship of positive sample pairs includes the ontology's subClassOf, hasPart, and The relationship between axiom definitions.

[0033] In a specific feasible implementation, the model input representation consists of the sum of three vector components: token embedding, segment embedding, and position embedding.

[0034] The segment embedding is used to distinguish whether the input sequence comes from metamodel text or ontology axiom text, so as to achieve the fusion of multi-source information.

[0035] In one feasible implementation, the recommendation reasoning also includes a post-filtering step, which uses the axiom set of the aviation domain ontology to perform logical verification on the candidate recommendation list generated by the model, and removes candidate words that do not conform to the ontology constraints, so as to improve the domain compliance of the recommendation.

[0036] Secondly, a meta-model recommendation system for the aviation domain based on ontology and the RoBERTa model is provided, including:

[0037] The data preparation module is used to acquire aerospace ontology files and aerospace metamodel resource files;

[0038] The format conversion module is used to convert XML format metamodel files into text sequences using a tree-like text serialization algorithm. The tree-like serialization algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags.

[0039] The data classification module is used to divide text sequences into training and test sets.

[0040] The data processing module is used to normalize the text of the training set and enhance the word segmenter with an aerospace domain ontology dictionary, performing ontology-enhanced word segmentation.

[0041] The model training module is used to build the RoBERTa model, expanding the concept set of the aerospace ontology to the model vocabulary; defining and minimizing the total loss function;

[0042] The recommendation inference module is used to receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the probability distribution of the mask positions output by the model, and generate a candidate recommendation list sorted in descending order of confidence.

[0043] The validity verification module is used to evaluate the accuracy of the recommendation list using the test set, and to calculate the Top-K hit rate and average reciprocal ranking metrics.

[0044] In the above technical solution, the following steps are taken: First, acquire aviation domain ontology files and aviation domain metamodel resource files. Then, convert the XML-formatted metamodel files into text sequences using a tree-like text serialization algorithm. This algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags. Next, divide the text sequences into training and testing sets. Normalize the training set and enhance the word segmenter using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation. Construct a RoBERTa model, expanding the concept set of the aviation domain ontology to the model vocabulary. Define and minimize the total loss function. Receive the metamodel context sequence to be processed, insert mask tags at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence. This solves the technical problems of low recommendation accuracy, reliance on manual rules, and inability to adapt to complex dynamic semantic environments in existing technologies. Attached Figure Description

[0045] Figure 1 A flowchart illustrating the aviation domain metamodel recommendation method based on ontology and RoBERTa model provided in this application embodiment;

[0046] Figure 2 The diagram shows the structure of the aerospace metamodel recommendation method based on ontology and the RoBERTa model provided in this application embodiment. Detailed Implementation

[0047] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. Through these descriptions, the features and advantages of the present application will become clearer and more apparent.

[0048] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0049] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other. The following detailed description, in conjunction with specific accompanying drawings, illustrates the embodiments.

[0050] exist Figure 1 This application provides a method for recommending meta-models in the aerospace domain based on ontology and the RoBERTa model, including the following steps:

[0051] Obtain the aerospace ontology file and aerospace metamodel resource file;

[0052] The XML-formatted metamodel file is converted into a text sequence using a tree-like text serialization algorithm, which maps the metamodel's hierarchical structure, element types, and dependencies into a natural language-like sequence containing special tags.

[0053] The text sequence is divided into a training set and a test set;

[0054] The training set is text normalized, and the word segmenter is enhanced using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation.

[0055] Construct a RoBERTa model, extending the concept set of the aerospace ontology to the model vocabulary; define and minimize the total loss function;

[0056] Receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence.

[0057] In the above technical solution, the following steps are taken: First, acquire aviation domain ontology files and aviation domain metamodel resource files. Then, convert the XML-formatted metamodel files into text sequences using a tree-like text serialization algorithm. This algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags. Next, divide the text sequences into training and testing sets. Normalize the training set and enhance the word segmenter using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation. Construct a RoBERTa model, expanding the concept set of the aviation domain ontology to the model vocabulary. Define and minimize the total loss function. Receive the metamodel context sequence to be processed, insert mask tags at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence. This solves the technical problems of low recommendation accuracy, reliance on manual rules, and inability to adapt to complex dynamic semantic environments in existing technologies.

[0058] Specifically, the beneficial effects include:

[0059] Recommendation accuracy and intelligence are significantly improved: By introducing the ontology semantic consistency constraint loss LONTO, this invention transforms the RoBERTa model from a mere statistical language model into an expert system with "common sense" in the aviation field. It can understand the hierarchy (e.g., "Boeing 737" is a type of "passenger aircraft") and relationships (e.g., "wing" includes "flaps") between concepts, thus avoiding the "semantic drift" produced by traditional word embedding methods. Experiments show that the Hit@5 metric of this invention is improved by more than 40% compared to the Word2Vec-based solution.

[0060] Completely eliminating reliance on manual rules: This invention employs a purely data-driven end-to-end training model. Modelers do not need to write and maintain complex logical rule bases; the system can automatically learn implicit patterns from massive meta-models and ontology data. This greatly reduces the implementation difficulty and maintenance cost of the system, enabling non-domain experts to build high-quality meta-models with the system's assistance.

[0061] Possessing strong contextual understanding and generalization capabilities: The RoBERTa model's deep Transformer structure and dynamic masking mechanism enable it to capture long-range semantic dependencies. Even when faced with unfamiliar modeling scenarios, the model can infer reasonable concepts based on context, solving the problem of poor generalization ability in traditional solutions. Simultaneously, ontology-enhanced word segmentation effectively handles aviation-related terminology, resolving the OOV (Out-of-Vocabulary) problem.

[0062] Supporting modeling assistance throughout its entire lifecycle: This invention not only provides creation suggestions in the early stages of modeling, but also recommends more scientific and standardized naming and relationships by analyzing the context of existing models during model iteration and reconstruction, continuously improving the maturity of the model. Furthermore, the generated recommendation list includes confidence scores, providing quantitative evidence for modelers' decision-making.

[0063] High scalability: The methodology of this invention has good universality. The proposed "ontology injection pre-training" framework can be easily transferred to other professional fields (such as automobile manufacturing and healthcare), simply by replacing the corresponding domain ontology and meta-model dataset, which has extremely high promotional value.

[0064] In one specific implementation scheme, it also includes:

[0065] The accuracy of the recommendation list is evaluated using the test set, and the Top-K hit rate and average reciprocal ranking metrics are calculated.

[0066] In one specific implementation, the total loss function is a weighted average of the dynamic masking language model loss LMLM and the ontology semantic consistency constraint loss LONTO, wherein,

[0067] , where λ is the weighting coefficient.

[0068] In one specific implementation, the tree-structured text serialization algorithm follows these rules:

[0069] Special tags including [CLASS], [PROPERTY], [RELATION], [SOURCE], and [TARGET] are used to define the different constituent units of the metamodel;

[0070] The class's property list is encapsulated in [PROPERTIES] and Between the markers;

[0071] The definition of a relationship includes explicit declarations of the source class and the target class.

[0072] In one specific implementation scheme, a semantic obfuscation-based masking sampling strategy is adopted. This strategy determines the mask position based on the importance weight W(c) of concepts in the aerospace domain ontology; the formula for calculating the importance weight W(c) is as follows:

[0073] ;

[0074] Where freq(c) is the frequency of occurrence of concept c in the ontology, depth(c) is the depth of concept in the classification tree, deg(c) is the connectivity of concept with other nodes, and α, β, and γ are preset adjustment coefficients.

[0075] In one specific implementation, the ontology semantic consistency constraint loss LONto employs a contrastive learning framework, and its calculation process includes:

[0076] Extract concept pairs with direct relationships from the aerospace ontology as positive sample pairs (ci, cj);

[0077] Randomly sampled, unrelated concept pairs are used as negative sample pairs;

[0078] The [CLS] Token vector output by the RoBERTa model is mapped to the semantic space through a semantic projection head to obtain the vector z;

[0079] The similarity loss between positive sample pairs is calculated using the cross-entropy loss calibrated by normalized temperature, which encourages positive sample pairs to move closer together in the vector space and negative sample pairs to move further apart.

[0080] In one specific implementation, the association relationship of positive sample pairs includes the ontology's subClassOf, hasPart, and The relationship between axiom definitions.

[0081] In a specific feasible implementation, the model input representation consists of the sum of three vector components: token embedding, segment embedding, and position embedding.

[0082] The segment embedding is used to distinguish whether the input sequence comes from metamodel text or ontology axiom text, so as to achieve the fusion of multi-source information.

[0083] In one feasible implementation, the recommendation reasoning also includes a post-filtering step, which uses the axiom set of the aviation domain ontology to perform logical verification on the candidate recommendation list generated by the model, and removes candidate words that do not conform to the ontology constraints, so as to improve the domain compliance of the recommendation.

[0084] In a specific feasible implementation, the aviation domain metamodel recommendation method based on ontology and the RoBERTa model constructs a data-driven intelligent recommendation engine. By introducing an aviation domain ontology as a structured knowledge source and combining it with a large-scale aviation metamodel dataset, the RoBERTa model is pre-trained and fine-tuned in a domain-adaptive manner. This model can deeply understand the semantic connotations of the aviation domain, the hierarchy and relationships between concepts, and thus, during the metamodel construction process, automatically and accurately recommend the most relevant classes, attributes, or relationships for the current modeling context, and sort them by confidence, significantly improving modeling efficiency and model quality. The method includes the following steps:

[0085] Step A: Preparation and Normalization of Multi-Source Heterogeneous Data

[0086] Prepare a set of structured aerospace ontology files (OWL format) and no fewer than 10,000 aerospace metamodel resource files (XML format, such as Ecore or UML XMI format). The ontology files define the core concept set Conto, attribute set Ponto, relation set Ronto, and axiom set Aonto for the aerospace domain.

[0087] Step B: Metamodel XML to Tree-like Text Serialization

[0088] Since native XML format is not suitable as direct input to a language model, this step designs a metamodel tree-based text serialization algorithm. This algorithm traverses the XML DOM tree, converting the metamodel's hierarchical structure, element types, and dependencies into a linear sequence resembling natural language. The serialization function is defined as follows: , where T is the generated text sequence.

[0089] Specifically, for a metamodel fragment containing class ClassA, having attributes Attr1 and Attr2, and having an Association1 relationship with ClassB, its serialization process follows the following rules:

[0090] Class definition: [CLASS] ClassA [PROPERTIES].

[0091] Attribute definition: [PROPERTY] Attr1 : Type_A ; [PROPERTY] Attr2 : Type_B .

[0092] Relationship definition: [RELATION] Association1 [SOURCE] ClassA [TARGET] ClassB

[0093] The final generated text sequence Ti is: [CLASS] ClassA [PROPERTIES] [PROPERTY] Attr1: Type_A ; [PROPERTY] Attr2 : Type_B [END_PROPERTIES] [RELATION] Association1[SOURCE] ClassA [TARGET] ClassB [END_CLASS].

[0094] Through this step, the complex graph structure meta-model is transformed into sequence data that the model can understand.

[0095] Step C: Dataset Classification and Sampling Strategy Design

[0096] Serialized text dataset The dataset is divided into a training set Dtrain and a test set Dtest according to a predetermined ratio (e.g., 8:2).

[0097] Design a semantic obfuscation-based masking sampling strategy, Strategymask. This strategy does not simply randomly select tokens for masking; instead, it performs weighted sampling based on the conceptual importance of the aviation ontology. Define the importance weight function W(c), where... :

[0098] ;

[0099] Where freq(c) is the frequency of concept c in the ontology, depth(c) is the depth of the concept in the classification tree (the deeper the better), deg(c) is the connectivity of the concept with other nodes, and α, β, and γ are adjustment coefficients. During sampling, concepts with higher importance weights are more likely to be selected for masking.

[0100] Step D: Text preprocessing and word segmentation incorporating ontology knowledge

[0101] The training set Dtrain undergoes deep preprocessing. Unlike general NLP tasks, this step introduces an aeronautical ontology dictionary to enhance the traditional word segmenter.

[0102] Text normalization: unifying terminology, for example, mapping "airplane", "aircraft" to the standard concept "Aircraft" in the ontology.

[0103] Ontology-enhanced word segmentation: Construct an ontology vocabulary tree for the aviation domain and use the longest matching algorithm to segment the serialized text. For example, for the string "LandingGear", a general word segmenter may segment it into "Landing" and "Gear", while the ontology-enhanced word segmenter will recognize it as a complete domain concept [LandingGear].

[0104] Data augmentation: leveraging hierarchical relationships in ontology ( ) and synonyms ( Enhancement operations, such as back-translation and random insertion of synonyms, are performed on the training text to improve the robustness of the model.

[0105] Step E: RoBERTa model pre-training and fine-tuning with ontology injection

[0106] We construct an improved RoBERTa model architecture, whose input embedding layer, encoding layer, and output layer are all optimized for aeronautical metamodel recommendation tasks.

[0107] (1) Model input representation

[0108] The model's input consists of the sum of three vectors:

[0109] ;

[0110] in:

[0111] Etoken(ti) is a token embedding. To address the issue of rare concepts in the aviation field, we expanded the RoBERTa vocabulary by adding all concepts Conto from the ontology as special tokens to the vocabulary.

[0112] Esegment(sj) is a segment embedding used to distinguish whether the input sequence belongs to the metamodel text or the ontology axiom text.

[0113] Eposition(pk) is a position embedding function that uses RoBERTa's sine and cosine functions to encode the absolute position.

[0114] (2) Dynamic Masking Language Model

[0115] In the pre-training phase, a dynamic masking mechanism is employed. At the beginning of each training epoch, mask positions are dynamically generated according to the sampling strategy `Strategymask` defined in step C. Let the input sequence be... The sequence after masking is The training objective of the model is to maximize the masked area. Predicted probability:

[0116] ;

[0117] Where M is the set of mask positions, and θ is the model parameter.

[0118] (3) Ontology semantic consistency constraint loss (Novel Contribution)

[0119] To force the model to strictly adhere to the logical constraints of the aviation ontology while learning language patterns, this invention innovatively introduces the ontology semantic consistency constraint loss LONTO. This loss function uses the axiom set Aonto of the ontology to penalize illogical predictions.

[0120] For example, if the ontology axiom states that "an engine is a power plant", that is... When the model predicts "Engine" in the context, its output probability in the "PowerPlant" category should be boosted. We formalize this constraint as a contrastive learning objective.

[0121] We define positive sample pairs (ci, cj) as concepts in the ontology with direct association (such as subClassOf, hasPart), while negative sample pairs are random concepts without association. We construct a semantic projection head. The model output [CLS] Token vector h is mapped to the semantic space: z=g(h).

[0122] The loss function LONto uses normalized temperature-calibrated cross-entropy loss. ):

[0123] ;

[0124] in For the set of positive sample pairs, For the set of negative sample pairs, It represents the cosine similarity, and τ is the temperature parameter.

[0125] The final pre-training total loss function is:

[0126] ;

[0127] Here, λ is a balancing coefficient used to adjust the weights of the language modeling task and the ontology knowledge injection task. By minimizing LTotal, the model can not only learn to predict words, but also internalize the knowledge system of the aviation domain.

[0128] (4) Hyperparameter configuration and optimization

[0129] Model hyperparameters were configured as follows: Transformer layers were set to 12, hidden layer dimension to 768, and attention heads to 12. The optimizer used was AdamW, with an initial learning rate set to... And use linear preheating ( The training program uses a combination of batch size and decay strategies. The batch size is set to 64, and the number of training epochs is set to 50.

[0130] Step F: Context-Aware Recommendation Inference

[0131] After model training is complete, the inference phase begins. Given a meta-model fragment Mpartial being built, it is first serialized to obtain the context sequence Cseq.

[0132] Mask position determination: Based on the cursor position of the modeler or the semantic gap position automatically detected by the system (such as an unnamed attribute or an incomplete relationship), a special [MASK] token is inserted at the position.

[0133] Model inference: The Cseq data containing [MASK] is input into the trained RoBERTa model. The output of the last layer of the model corresponds to the latent vector hmask∈Rd at the [MASK] location.

[0134] Candidate set generation: Map hmask through a fully connected layer (Output Layer) to a vector with dimensions equal to the size of the vocabulary. Applying the Softmax function to this vector yields the probability distribution of each token in the vocabulary as a filler word at the [MASK] position:

[0135] ;

[0136] in and These are the weights and biases of the output layer.

[0137] Top-K filtering and confidence calculation: Selecting the K candidate words with the highest probabilities The corresponding probability value This is the confidence score, Scoresi, for the recommendation. To improve the professionalism of the recommendations, we use ontology to perform post-filtering on the candidate set, removing words that, although highly probable in the language model, do not conform to ontology axioms (e.g., recommending "color" as a subclass of "engine"). The final recommendation list, RecommendList, is returned to the user interface in descending order of Scoresi.

[0138] Step G: Closed-loop validity verification and iteration

[0139] The recommender system is objectively evaluated using the test set Dtest reserved in step C. Samples are selected from Dtest, and a recommender list is generated using the method in step F. A series of standardized metrics are then calculated.

[0140] Top-K hit rate ( ): The proportion of correct answers appearing in the first K positions of the recommended list. Among them,

[0141] ;

[0142] Where Q is the query set and I is the indicator function. It is the ranking of the correct word in the recommendation list.

[0143] Mean Reciprocal Rank (MRR): Measured by the average of the reciprocals of the first correct answer.

[0144] ;

[0145] Semantic similarity score: The cosine similarity between the recommended word vectors and the real word vectors is calculated using the trained model to evaluate the semantic accuracy of the recommendations.

[0146] Based on the evaluation results, the hyperparameter λ in step E can be dynamically adjusted, or the process can return to step D for further data augmentation, forming a closed loop of continuous optimization.

[0147] In a specific feasible implementation, the intelligent recommendation method for the aviation domain based on ontology and RoBERTa model includes the following steps:

[0148] Step A: Data Preparation

[0149] First, collect and organize knowledge resources in the aviation field.

[0150] Aviation ontology construction: Core concepts are extracted from publicly available aviation knowledge graphs, airworthiness standard documents (such as CCAR-25-R4), and internal enterprise knowledge bases. (Using...) Tools for building aviation ontology This ontology contains the following core elements:

[0151] Concepts include: Aircraft, Wing, Fuselage, Empennage, Landing Gear, Engine, Flight Control System, Hydraulic System, etc.

[0152] The Ponto property set includes properties such as hasPart, isPartOf, hasSubsystem, poweredBy, and controls.

[0153] Axiom set Aonto: For example, (Landing gear is a type of equipment) (The wing has at least one flap).

[0154] Metamodel resource collection: Exported over 10,000 metamodel files from existing aerospace design projects, primarily for the Eclipse Modeling Framework (EMF). Documents and UML These documents describe various models ranging from flight control logic and avionics architecture to manufacturing processes. All documents are stored in XML format.

[0155] Step B: Format Conversion (Serialization)

[0156] Develop a Java tool called MetaModelSerializer to convert XML to tree-like text.

[0157] Taking a simplified landing gear system meta-model, LandingGear.ecore, as an example, its XML fragment is as follows:

[0158] <ecore:epackage name="LandingGear">

[0159] <eclassifiers xsi:type="ecore:EClass" name="GearAssembly">

[0160] <estructuralfeatures xsi:type="ecore:EAttribute" name="gearID" eType="ecore:EDataType http: / / www.eclipse.org / emf / 2002 / XMLType# / / EString" / >

[0161] <estructuralfeatures xsi:type="ecore:EReference" name="hydraulicLines" eType="# / / HydraulicLine" / >

[0162] < / eclassifiers>

[0163] <eclassifiers xsi:type="ecore:EClass" name="HydraulicLine">

[0164] <estructuralfeatures xsi:type="ecore:EAttribute" name="pressure" eType="ecore:EDataType http: / / www.eclipse.org / emf / 2002 / XMLType# / / EFloat" / >

[0165] < / eclassifiers>

[0166] < / ecore:epackage>

[0167] MetaModelSerializer parses this XML and generates the following text sequence T1:

[0168] [PACKAGE] LandingGear [CLASS] GearAssembly [ATTRIBUTES] [ATTRIBUTE]gearID : EString [END_ATTRIBUTES] [REFERENCES] [REFERENCE] hydraulicLines ->HydraulicLine [END_REFERENCES] [END_CLASS] [CLASS] HydraulicLine [ATTRIBUTES][ATTRIBUTE] pressure : EFloat [END_ATTRIBUTES] [END_CLASS] [END_PACKAGE].

[0169] In this way, all 10,000+ XML files were converted Text files constitute the original corpus.

[0170] Step C: Data Classification and Sampling Strategy Design

[0171] Shuffle all serialized text and divide it into training and test sets in an 80:20 ratio. For example, obtain 8000 training samples and 2000 test samples.

[0172] Next, a masking sampling strategy based on ontology importance is designed. (Loading) This process analyzes all concepts and their relationships, and sets up formulas.

[0173] .

[0174] For the concept Engine: freq=0.8 (high frequency), depth=3 (depth from Thing to Engine), deg=10 (connecting 10 other concepts). The calculated W(Engine) = 0.4 × 0.8 + 0.4 × 3 + 0.2 × 10 = 4.72.

[0175] For the concept Bolt: freq=0.2, depth=5, deg=2. The calculated value is W(Bolt)=0.4×0.2+0.4×5+0.2×2=2.48.

[0176] When generating training samples, the system selects core concepts like Engine for masking with a higher probability (proportional to W(c)). This strategy forces the model to focus more on learning core knowledge in the aerospace field.

[0177] Step D: Data Processing

[0178] Constructing an aviation domain dictionary: Extracting all concept names (Aircraft, LandingGear, etc.) and attribute names (hasPart, gearID, etc.) from the ontology, and constructing... .

[0179] Ontology-enhanced tokenization: Modify Hugging Face's BertTokenizer to load it during initialization. In processing text sequences At that time, the word segmenter will identify it as Instead of splitting HydraulicLine into .

[0180] Data augmentation: Using the synonym attribute in the ontology, randomly replace "Main Wheel" in the text.

[0181] At the same time, back translation technology is used ( Increase language diversity.

[0182] Step E: Model pre-training and fine-tuning

[0183] This is the most crucial implementation step. We use the PyTorch and Transformers libraries to build the model.

[0184] 1. Model Initialization: Loading The pre-trained weights are used as a starting point. Then, the word embedding matrix of the model is expanded according to the aviation dictionary constructed in step D. The embedding vectors of newly added tokens (such as [PACKAGE], [CLASS], LandingGear) are initialized by averaging the existing word vectors.

[0185] 2. Construct the training data stream: Use DataCollatorForLanguageModeling, setting mlm=True and mlm_probability=0.15. Implement a custom Dataset class, which... In the method, the input text is dynamically masked according to the strategy in step C.

[0186] 3. Define the loss for ontology semantic consistency constraints:

[0187] import torch

[0188] import torch.nn as nn

[0189] import torch.nn.functional as F

[0190] class OntologyConsistencyLoss(nn.Module):

[0191] def __init__(self, temperature=0.07):

[0192] super().__init__()

[0193] self.temperature = temperature

[0194] def forward(self, features, ontology_pairs):

[0195] # features: [batch_size, hidden_dim]

[0196] # ontology_pairs: list of (positive_pair_indices, negative_pair_indices)

[0197] # L2 Normalize features

[0198] features = F.normalize(features, dim=1)

[0199] # Compute similarity matrix

[0200] sim_matrix = torch.matmul(features, features.T) / self.temperature

[0201] # Create labels based on ontology pairs

[0202] labels = torch.arange(features.size(0)).to(features.device)

[0203] loss = F.cross_entropy(sim_matrix, labels)

[0204] return loss

[0205] During the training loop, we sample a batch of positive sample pairs (e.g., (Aircraft, Airplane)) and negative sample pairs (e.g., (Aircraft, Bolt)) from the ontology in real time. The indices of these samples are passed to the loss function, which encourages the model to bring positive sample pairs closer together in the latent space and push negative sample pairs further apart.

[0206] 4. Training execution:

[0207] set up , ,

[0208] .

[0209] The optimizer uses AdamW, and works in conjunction with... Learning rate scheduler.

[0210] Total loss calculation: Here, λ = 0.5.

[0211] Initiate multi-GPU distributed training and train for approximately 72 hours on a server equipped with an NVIDIA A100 graphics card until the loss converges.

[0212] Step F: Recommendation Reasoning

[0213] Suppose a modeling engineer is designing a new metamodel for a "braking system". He creates a class `BrakeSystem` and adds a property `maxTemp`. Now he needs to define the type of this property. His current context sequence `Cseq` is:

[0214]

[0215] 1. Input encoding: Input the above sequence into the trained model.

[0216] 2. Model forward propagation: The model outputs the hidden state of each token. We extract the hidden vector hmask corresponding to the [MASK] position.

[0217] 3. Probability Calculation: Feed hmask into the LM Head (linear layer + Softmax) of the model to obtain the probability distribution of the entire vocabulary.

[0218] 4. Generate a recommendation list: Filter out the top 5 tokens with the highest probability:

[0219] EFloat ( ); EDouble ( EInt ( );

[0220] TemperatureUnit (confidence level: 0.01); PressureUnit (confidence level: 0.005).

[0221] 5. Post-processing: The system identifies EFloat as a standard EMF data type, consistent with the hasDataType axiom of aerospace ontology, and therefore recommends it as the first choice. Meanwhile, although TemperatureUnit ranks fourth in probability, its semantic relevance to "temperature" allows the system to provide it to engineers as a "semantically related but different type" suggestion. Finally, the recommended list is displayed on the interface: , , .

[0222] Step G: Validation

[0223] We used 2000 reserved test samples for verification. We selected a meta-model fragment about "flight control computer" from the test set and manually replaced the type EInt of the memorySize attribute with [MASK] to form query q1.

[0224] Run the recommendation system and get the recommendation list. EInt is found to be ranked first.

[0225] Repeat this process 2000 times and calculate each indicator:

[0226] (In 78% of cases, the correct answer is ranked first);

[0227] (In 95% of cases, the correct answer is among the top five).

[0228] MRR: 0.86.

[0229] These metrics are significantly higher than the baseline model based on Word2Vec ( (Approximately 0.45), demonstrating the effectiveness of the present invention.

[0230] In this embodiment, the beneficial effects include: by deeply integrating aviation domain ontology knowledge and an improved RoBERTa pre-trained language model, the challenge of intelligent recommendation in the construction of aviation domain meta-models is successfully solved. Through an innovative ontology semantic consistency constraint loss function, this invention endows the language model with domain-specific logical reasoning capabilities, significantly improving the accuracy and professionalism of recommendations. Simultaneously, the end-to-end training paradigm completely eliminates reliance on manual rules, providing powerful intelligent tools to support model-driven engineering in the aviation domain. This invention significantly lowers the modeling threshold, improves model quality, and accelerates the development cycle of aviation software.

[0231] In one specific implementation scheme, the application of the present invention in the scenario of model renaming and reconstruction includes:

[0232] During the iterative development of the aviation project, an engineer discovered that the class name Storage in the early FuelSys.ecore metamodel was too general and could not accurately express the meaning of "fuel tank". He hoped the system could recommend a more professional name.

[0233] 1. The engineer selected the Storage class.

[0234] 2. The system captures its context: ;

[0235] .

[0236] 3. Perform the recommendation to obtain the list:

[0237] FuelTank (confidence level: 0.92);

[0238] Reservoir (confidence level: 0.05);

[0239] Container (Confidence level: 0.02).

[0240] 4. The engineers adopted FuelTank's suggestion and completed the refinement of the model.

[0241] This embodiment demonstrates that the present invention can not only be used to fill gaps, but also to optimize existing models and improve their semantic accuracy.

[0242] In one specific implementation scheme, the process of handling long-range dependencies according to the present invention includes:

[0243] Consider a complex avionics system metamodel where a Sensor class is defined at the beginning of a file, and its associated DataBus class is defined at the end of the file. When an engineer adds a reference to [MASK] in the Sensor class, that [MASK] should actually point to the DataBus at the end of the file.

[0244] Thanks to RoBERTa's Transformer architecture's global attention mechanism, it can capture the implicit correlation between Sensors and DataBus throughout the sequence. During recommendation, DataBus can still be recommended with high confidence, while traditional... The model or RNN model typically fails in this scenario. This demonstrates the superiority of the present invention in handling large, complex meta-models.

[0245] exist Figure 2 In this application, embodiments provide an aviation domain metamodel recommendation system based on ontology and the RoBERTa model, including:

[0246] The data preparation module is used to acquire aerospace ontology files and aerospace metamodel resource files;

[0247] The format conversion module is used to convert XML format metamodel files into text sequences using a tree-like text serialization algorithm. The tree-like serialization algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags.

[0248] The data classification module is used to divide text sequences into training and test sets.

[0249] The data processing module is used to normalize the text of the training set and enhance the word segmenter with an aerospace domain ontology dictionary, performing ontology-enhanced word segmentation.

[0250] The model training module is used to build the RoBERTa model, expanding the concept set of the aerospace ontology to the model vocabulary; defining and minimizing the total loss function;

[0251] The recommendation inference module is used to receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the probability distribution of the mask positions output by the model, and generate a candidate recommendation list sorted in descending order of confidence.

[0252] The validity verification module is used to evaluate the accuracy of the recommendation list using the test set, and to calculate the Top-K hit rate and average reciprocal ranking metrics.

[0253] In the above technical solution, the following steps are taken: First, acquire aviation domain ontology files and aviation domain metamodel resource files. Then, convert the XML-formatted metamodel files into text sequences using a tree-like text serialization algorithm. This algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags. Next, divide the text sequences into training and testing sets. Normalize the training set and enhance the word segmenter using an aviation domain ontology dictionary, performing ontology-enhanced word segmentation. Construct a RoBERTa model, expanding the concept set of the aviation domain ontology to the model vocabulary. Define and minimize the total loss function. Receive the metamodel context sequence to be processed, insert mask tags at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence. This solves the technical problems of low recommendation accuracy, reliance on manual rules, and inability to adapt to complex dynamic semantic environments in existing technologies.

[0254] Those skilled in the art will know that this application can be implemented as a system, method, or computer program product.

[0255] Therefore, this disclosure can be implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this application can also be implemented as a computer program product in one or more computer-readable media, the computer-readable media containing computer-readable program code.

[0256] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0257] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application. Based on this, various substitutions and improvements can be made to this application, all of which fall within the protection scope of this application.

Claims

1. A method for recommending an aviation domain meta-model based on an ontology and a RoBERTa model, characterized in that, Includes the following steps: Obtain the aerospace ontology file and aerospace metamodel resource file; The XML-formatted metamodel file is converted into a text sequence using a tree-like text serialization algorithm, which maps the metamodel's hierarchical structure, element types, and dependencies into a natural language-like sequence containing special tags. The text sequence is divided into a training set and a test set; Text normalization is performed on the training set, and the word segmenter is enhanced using an aviation domain ontology dictionary to perform ontology-enhanced word segmentation. Construct the RoBERTa model and extend the concept set of aerospace ontology to the model vocabulary; Define and minimize the total loss function; Receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the mask position probability distribution of the model output, and generate a candidate recommendation list sorted in descending order of confidence.

2. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 1, characterized in that, Also includes: The accuracy of the recommendation list is evaluated using the test set, and the Top-K hit rate and average reciprocal ranking metrics are calculated.

3. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 2, characterized in that, The total loss function is a weighted sum of the dynamic masking language model loss LMLM and the ontology semantic consistency constraint loss LONTO, where, , where λ is the weighting coefficient.

4. The aerospace domain metamodel recommendation method based on ontology and the RoBERTa model according to claim 3, characterized in that, The tree-structured text serialization algorithm follows these rules: Special tags including [CLASS], [PROPERTY], [RELATION], [SOURCE], and [TARGET] are used to define the different constituent units of the metamodel; The class's property list is encapsulated in [PROPERTIES] and Between the markers; The definition of a relationship includes explicit declarations of the source class and the target class.

5. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 4, characterized in that, A semantic obfuscation-based masking sampling strategy is adopted, which determines the mask position based on the importance weight W(c) of concepts in the aerospace ontology; the formula for calculating the importance weight W(c) is as follows: ； Where freq(c) is the frequency of occurrence of concept c in the ontology, depth(c) is the depth of concept in the classification tree, deg(c) is the connectivity of concept with other nodes, and α, β, and γ are preset adjustment coefficients.

6. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 5, characterized in that, The ontology semantic consistency constraint loss LONTO employs a contrastive learning framework, and its calculation process includes: Extract concept pairs with direct relationships from the aerospace ontology as positive sample pairs (ci, cj); Randomly sampled, unrelated concept pairs are used as negative sample pairs; The [CLS] Token vector output by the RoBERTa model is mapped to the semantic space through a semantic projection head to obtain the vector z; The similarity loss between positive sample pairs is calculated using the cross-entropy loss calibrated by normalized temperature, which encourages positive sample pairs to move closer together in the vector space and negative sample pairs to move further apart.

7. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 6, characterized in that, The association relationships between positive sample pairs include the ontology's subClassOf, hasPart, and The relationship between axiom definitions.

8. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 7, characterized in that, The model input representation consists of the sum of three vector components: token embedding, segment embedding, and position embedding. The segment embedding is used to distinguish whether the input sequence comes from metamodel text or ontology axiom text, so as to achieve the fusion of multi-source information.

9. The aerospace domain metamodel recommendation method based on ontology and RoBERTa model according to claim 8, characterized in that, The recommendation reasoning also includes a post-filtering step, which uses the axiom set of the aviation domain ontology to perform logical verification on the candidate recommendation list generated by the model, and removes candidate words that do not conform to the ontology constraints, so as to improve the domain compliance of the recommendations.

10. A meta-model recommendation system for the aviation domain based on ontology and the RoBERTa model, characterized in that, include: The data preparation module is used to acquire aerospace ontology files and aerospace metamodel resource files; The format conversion module is used to convert XML format metamodel files into text sequences using a tree-like text serialization algorithm. The tree-like serialization algorithm maps the hierarchical structure, element types, and dependencies of the metamodel into natural language-like sequences containing special tags. The data classification module is used to divide text sequences into training and test sets. The data processing module is used to normalize the text of the training set and enhance the word segmenter with an aerospace domain ontology dictionary, performing ontology-enhanced word segmentation. The model training module is used to build the RoBERTa model, which expands the concept set of the aerospace ontology to the model vocabulary; Define and minimize the total loss function; The recommendation inference module is used to receive the meta-model context sequence to be processed, insert mask markers at specified positions, input the trained RoBERTa model, obtain the probability distribution of the mask positions output by the model, and generate a candidate recommendation list sorted in descending order of confidence. The validity verification module is used to evaluate the accuracy of the recommendation list using the test set, and to calculate the Top-K hit rate and average reciprocal ranking metrics.

Images