BERT-based space science task demand classification method, system and product

Abstract

The invention belongs to the technical field of natural language processing, and particularly relates to a BERT-based space science task demand classification method, system and product. The method comprises the following steps: inputting a space science task demand text into a pre-established and trained classification model to obtain estimation probabilities of the demand text in different categories, and selecting the category with the highest estimation probability to realize classification; the categories comprise a scientific target, a performance demand, a load demand and an engineering demand; the classification model adopts a BERT model, and a complex structure and non-sequence content of a language are learned by using an attention mechanism, a full-connection neural network layer and position coding. According to the method, the SysML demand model for natural language processing and space science task demonstration modeling is combined, a basis is provided for automatically constructing the SysML demand model, the MBSE modeling speed is increased, the system design efficiency is improved, and the design cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular, relates to a BERT-based space science mission requirement classification method, system and product. Background Art

[0002] Model Based System Engineering (MBSE) is an interdisciplinary approach that can provide comprehensive solutions to the needs of different stakeholders. In the field of aerospace, MBSE plays a key role in space science missions. Requirements analysis is the first step in designing a system. At present, the workflow of requirements analysis in MBSE usually involves soliciting customer opinions through a series of design studies and interviews, and forming a SysML requirements model. These requirements will be mapped into technical languages ​​such as design, function, and performance. Engineers can start more detailed design work by checking these technical languages. At the same time, in engineering, quantitative indicator parameters can be used as conditions for meeting requirements. The indicator parameters are recorded through the SysML requirement model and associated with the corresponding requirement content, so as to achieve unified management of indicators generated at different stages of the project, support the tracking of requirements, and perform consistency verification. SysML is a general system architecture modeling language for system engineering applications, and an enabling technology for model-based system engineering (MBSE).

[0003] However, the current mainstream MBSE modeling tools can only build demand models by manually drawing model diagrams or forming demand tables by importing Excel. Since manual modeling is costly, time-consuming, and requires highly specialized knowledge, it is necessary to study methods for automatically generating demand models based on demand text.

[0004] The first step to realize the automatic generation of demand models from demand text is demand classification. In recent years, natural language processing has made breakthrough progress in text classification tasks. Models such as recurrent neural networks, convolutional neural networks, and Transformer can learn the semantic information and contextual relationships of text and improve the accuracy of text classification. Although the field of natural language processing has shown good prospects in text classification, the application scenarios of text classification are mainly limited to fields such as sentiment analysis, news classification, and spam identification. It is rarely used in aerospace engineering requirements. Therefore, how to build a demand classification model for SysML requirements of space science missions is a problem worth studying. Summary of the invention

[0005] In view of the problems of high MBSE modeling learning cost, complicated modeling process and low efficiency in the prior art, the purpose of the present invention is to overcome the above-mentioned defects of the prior art and propose a space science mission requirement classification method based on BERT.

[0006] In order to achieve the above objectives, the present invention proposes a space science mission requirement classification method based on BERT, comprising:

[0007] Inputting the space science mission requirement text into a pre-established and trained classification model to obtain the estimated probability of the requirement text in different categories, and selecting the category with the highest estimated probability for classification; the categories include: scientific objectives, performance requirements, payload requirements, and engineering requirements;

[0008] The classification model adopts the BERT model, which uses attention mechanism, fully connected neural network layers and position encoding to learn the complex structure and non-sequential content of language.

[0009] Preferably, the scientific goal refers to a clear, measurable, specific and practical research direction formulated for a specific problem or hypothesis;

[0010] The performance requirements refer to the various technical and operational parameters that need to be met in order to achieve scientific goals and ensure mission success, including probes, payloads, communications, data volume analysis, mission management, lifespan, and orbital parameters;

[0011] The payload requirements refer to the requirements for scientific instruments or equipment in order to achieve specific scientific goals, including the functions, workflow and structural design of the payload;

[0012] The engineering requirements refer to the quantitative requirements of various indicators proposed during the engineering development phase, including thermal control, power consumption, communication, size, weight, posture and data transmission indicators.

[0013] Preferably, the classification model includes 12 encoder layers and a linear classification layer connected in sequence, and the processing process includes:

[0014] The space science mission requirement text is represented by tokens after passing through 12 encoder layers. The first token is input into the linear classification layer to obtain the estimated probability of the requirement text in different categories.

[0015] Preferably, the method further includes a step of training the classification model, including:

[0016] Step 1) Collect space science mission plan data and establish a data set;

[0017] Step 2) Preprocess the data set to achieve data enhancement;

[0018] Step 3) Divide the preprocessed data set into a training set, a test set, and a validation set. Divide the data in the training set by words and convert them into word vectors and input them into the model. Compare the predicted results with the true values ​​and update the model parameters. After each training iteration, use the validation set to verify the model training effect and export the model parameters with the highest current accuracy. Repeat this step until the training requirements are met and a trained classification model is obtained. Apply it to the test set to verify the model training effect.

[0019] Preferably, the step 1) comprises:

[0020] Collect space science mission plan documents, from which scientific goals, performance requirements, payload requirements, and engineering requirements are collected, and each type of requirement is labeled as 0, 1, 2, and 3 respectively to establish a data set.

[0021] Preferably, the step 2) comprises:

[0022] Text splitting: Split text into multiple parallel or independent data based on the key characters "and", semicolon, comma and / or period;

[0023] Noise addition: randomly add meaningless special symbols to part of the text instead of correct mathematical symbols;

[0024] Synonym Replacement: Randomly swap proper nouns and acronyms in your text.

[0025] Preferably, the learning rate in the training of step 3) is 2e-5, the optimizer is AdamW, and the loss function is CrossEntropyLoss.

[0026] On the other hand, the present invention proposes a space science mission requirement classification system based on BERT, comprising:

[0027] A classification output module is used to input the space science mission requirement text into a pre-established and trained classification model, obtain the estimated probability of the requirement text in different categories, and select the category with the highest estimated probability for classification; the categories include: scientific objectives, performance requirements, payload requirements, and engineering requirements;

[0028] The classification model adopts the BERT model, which uses attention mechanism, fully connected neural network layers and position encoding to learn the complex structure and non-sequential content of language.

[0029] In a third aspect, the present invention further provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] 1. The present invention combines natural language processing with the SysML requirement model for space science mission demonstration modeling, provides a basis for automatically building the SysML requirement model, accelerates the MBSE modeling speed, and improves the system design efficiency.

[0032] 2. Learning and understanding space science missions requires a certain amount of learning costs. Thanks to the space science mission requirement classification model, the requirements can be classified, which can speed up the understanding of the mission and reduce the learning cost.

[0033] 3. The space science mission requirement classification model trained by the present invention can be applied to other aerospace missions to provide a reference for requirement classification, thereby improving the efficiency of space science mission design and reducing design costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is the confusion matrix of the space science mission requirements classification model test results;

[0035] Figure 2 This is a flow chart of the BERT-based space science mission requirements classification method;

[0036] Figure 3 This is a schematic diagram of the BERT-based space science mission requirements classification model;

[0037] Figure 4 It is a text distribution map of the space science mission plan dataset. DETAILED DESCRIPTION

[0038] The present invention proposes a space science mission requirement classification method based on BERT, which uses space science mission related reports as data sources, screens the requirement text in the document to generate a data set, and increases the diversity of the data set by data enhancement. The requirement text in the data set is encoded into a text vector, and provided to the space science mission requirement classification model for training. The trained classification model is used to classify the space science mission requirements, and supports the subsequent automatic creation of SysML requirement models based on the classification results. BERT (Bidirectional Encoder Representations from Transformers) is a language representation model.

[0039] The steps of the BERT-based space science mission requirements classification method are as follows:

[0040] Step 1: Collect space science mission plan data. Because there is a lack of open source data sets related to space science mission requirements, it is necessary to collect space science mission plan data to generate a data set. Since labeling space science mission requirement data sets requires professional knowledge, and the relevant plan documents of space science missions have been classified and summarized according to different types of requirements, this article uses this type of document as the data source and selects the requirement text from it as the data set. The space science mission requirement data set includes four types of requirements, namely scientific goals, performance requirements, payload requirements, and engineering requirements.

[0041] a) Scientific objectives refer to clear, measurable, specific and practical research directions formulated for specific problems or hypotheses. Scientific objectives provide guidance for scientific research and are the starting point for MBSE modeling of space science missions.

[0042] b) Performance requirements refer to the various technical and operational parameters that need to be met in order to achieve scientific goals and ensure mission success. Performance requirements put forward requirements for all aspects of the mission, including detectors, payloads, communications, data volume analysis, mission management, lifespan, and orbit.

[0043] c) Payload requirements refer to the requirements for scientific instruments or equipment in order to achieve specific scientific goals, including the functions, workflow, structural design, etc. of the payload

[0044] d) Engineering requirements refer to the quantitative requirements of various indicators proposed during the engineering development phase, including thermal control, power consumption, communication, size, weight, posture, data transmission and other aspects.

[0045] The document structure of a space science mission plan usually includes chapters such as mission description, scientific objectives, scientific requirements, and payload. The mission description chapter summarizes the core content of the four types of requirements; the scientific objectives chapter summarizes all scientific objectives; the scientific requirements chapter analyzes and summarizes all performance requirements; the payload chapter introduces all payloads involved in the mission, including information such as functions, working principles, structures, and index parameters, from which payload requirements and engineering requirements can be screened out.

[0046] Some requirements in the document are described in a table, that is, the subject and content of the requirements are scattered in different fields, so the requirements lack a subject. The subject and content of the requirements need to be recombined to form a complete sentence; the other part of the requirements appear in the form of paragraphs, which need to be rewritten to summarize the paragraphs into a sentence of requirement text.

[0047] Step 2: Preprocess the data set. This is mainly divided into two steps: data enhancement and data set classification:

[0048] a) Data enhancement: Since public space science mission documents are limited, it is necessary to transform the collected required data through data enhancement to increase the diversity and volume of the data set and improve the generalization ability of the model. In this paper, text splitting, noise addition and synonym replacement are used:

[0049] (1) Text splitting: Split long text into multiple parallel or independent data based on key characters such as "and", semicolon, comma and period.

[0050] (2) Noise addition: Since some demand texts contain mathematical symbols, garbled characters may appear when using the dataset due to inconsistent character set encoding. In order to improve the model's tolerance to such problems, meaningless special symbols are randomly added to some texts instead of correct mathematical symbols.

[0051] (3) Synonym replacement: Since space science tasks contain a large number of proper nouns and their acronyms, in order to enhance the model’s understanding of different expressions, the proper nouns and acronyms in the text are randomly swapped.

[0052] b) Dataset classification: In one embodiment, the data set is split into a training set, a test set, and a validation set in a ratio of 8:1:1 to ensure that there is no duplication of data in the three data sets, thereby ensuring the authenticity and effectiveness of the testing process and evaluating the prediction accuracy of the model when facing unknown data.

[0053] Step 3: Training the space science mission requirements classification model: Since the dataset implemented in this paper is a small-scale dataset, in order to ensure good classification results with small samples, this paper uses a pre-trained language model - Bidirectional Encoder Representations from Transformers (BERT), and fine-tunes it using the space science mission requirements dataset to achieve the purpose of space science mission requirements classification. BERT is able to learn the complex structure and non-sequential content of language by using attention mechanisms, fully connected neural network layers, and position encoding. Text classification through BERT requires taking the first token position output by the last layer of BERT as the vector representation of the text, and inputting this vector into the fully connected layer for classification. The specific steps for training the space science mission requirements classification model are:

[0054] a) Input the training set into the BERT pre-training model for word vector representation

[0055] b) Input the word vector into the fully connected neural network layer for classification prediction

[0056] c) Compare the predicted results with the true values ​​and update the model parameters

[0057] d) Apply the model in training to the validation set and derive the model parameters with the highest accuracy

[0058] e) Repeat the above four steps until the training is completed

[0059] Apply the trained model to the test set to verify the model training effect.

[0060] The technical effect of the BERT-based space science mission requirement classification method is mainly reflected in the space science mission requirement classification model correctly classifying the requirement type. For example, using the LISA Yellow Book as a data source to create a data set, the test results after model training are: accuracy 96.97%, recall rate 96.97%, F1 value 96.93%, and the confusion matrix of the test results is as follows: Figure 1 As shown, it can be seen that the model correctly classifies scientific objectives, performance requirements and payload requirements, and only a small number of engineering requirements are misclassified as scientific objectives.

[0061] The BERT-based space science mission requirements classification method mainly consists of three steps: space science mission plan data collection, data preprocessing, and model training. The main process of the BERT-based space science mission requirements classification method is as follows: Figure 2 shown.

[0062] Figure 3 A schematic diagram of a BERT-based space science mission requirement classification model is provided to meet the classification of the requirement "Research the growth mechanism of MBHs before the epoch of reionization". The requirement text is input into the pre-trained BERT and tokenized after 12 encoder layers. The first token is input into the linear classification layer to obtain the estimated probability of the requirement in four categories (scientific objectives, performance requirements, payload requirements, and engineering requirements), and the requirement is classified into the category with the highest estimated probability, which is "scientific objectives" in this case.

[0063] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0064] Example 1

[0065] In order to illustrate in detail the various steps of the BERT-based space science mission requirement classification method, this embodiment uses the LISA space gravitational wave detection mission as an example for description.

[0066] Step 1: Collect space science mission plan data. The data source of this experiment is the public documents of the LISA space gravitational wave detection mission. "LISA—Unveiling a hidden Universe", "LISAMeasurementRequirements Flowdown Guide", "LISAScience Requirements Document" and "MRD.Mission Requirement Document" are used as data sources to collect scientific goals, performance requirements, payload requirements and engineering requirements, and each type of requirement is labeled 0, 1, 2, and 3 respectively.

[0067] Step 2: Preprocess the dataset. First, the dataset is enhanced by text splitting, noise addition, and synonym replacement. After data enhancement, the dataset has 300 data, of which 95% of the text lengths are within 194. Therefore, in this embodiment, all text lengths are unified to 200. For text vectors with a length of less than 200, they are padded with zeros, and for text vectors with a length of more than 200, they are truncated. The text length distribution of the dataset is as follows: Figure 4 As shown in the figure, the data is randomly split into 240 training sets, 30 test sets, and 30 validation sets in a ratio of 8:1:1.

[0068] Step 3: Train the space science mission requirement classification model. Use bert-base-uncased as the pre-trained model for fine-tuning. The hyperparameters are shown in Table 1. Divide the data of the training set by words and convert them into Embeddings to input into the model for training. After each training iteration, use the validation set to test the model training effect and export the model parameters with the highest current accuracy.

[0069] Table 1 Hyperparameters of space science mission requirements classification model

[0070] Hyperparameters describe Parameter Value class_num Number of classification types 4 Max_len Maximum text length 200 batch_size The size of each training batch 8 epochs Iteration round 10 learn_rate Learning Rate 2e-5 num_filters Dimensions of BERT hidden layers 768 optimizer Optimizer AdamW loss_function Loss Function CrossEntropyLoss

[0071] The test set is input into the trained model for testing. At the same time, the space science mission requirement data set is applied to other models for performance comparison. The accuracy P, recall R and the comprehensive index F1 for judging accuracy and recall are used as evaluation criteria. The results are shown in Table 2. It can be found that the classification model obtained after fine-tuning bert-base-uncased has good performance in accuracy, recall and F1 indicators, which proves that the space science mission requirement classification method based on BERT proposed in this invention is feasible.

[0072] Table 2 Model performance comparison results

[0073]

[0074]

[0075] Example 2

[0076] Embodiment 2 of the present invention provides a BERT-based space science mission requirement classification system, which is implemented based on the method of embodiment 1 and includes: a classification output module, which is used to input the space science mission requirement text into a pre-established and trained classification model, obtain the estimated probability of the requirement text in different categories, and select the category with the highest estimated probability to implement classification; the categories include: scientific objectives, performance requirements, payload requirements and engineering requirements;

[0077] The classification model adopts the BERT model, which uses attention mechanism, fully connected neural network layers and position encoding to learn the complex structure and non-sequential content of language.

[0078] Example 3

[0079] The embodiment of the present invention may also provide a computer program product, including a computer program. When the computer program is executed by a processor, the method steps in the embodiment 1 may be implemented.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A BERT-based space science mission requirements classification method, including: Inputting the space science mission requirement text into a pre-established and trained classification model to obtain the estimated probability of the requirement text in different categories, and selecting the category with the highest estimated probability for classification; the categories include: scientific objectives, performance requirements, payload requirements, and engineering requirements; The classification model adopts the BERT model, which uses attention mechanism, fully connected neural network layers and position encoding to learn the complex structure and non-sequential content of language.

2. The BERT-based space science mission requirements classification method according to claim 1 is characterized in that: The scientific objectives are clear, measurable, specific and practical research directions formulated for specific problems or hypotheses; The performance requirements refer to the various technical and operational parameters that need to be met in order to achieve scientific goals and ensure mission success, including probes, payloads, communications, data volume analysis, mission management, lifespan, and orbital parameters; The payload requirements refer to the requirements for scientific instruments or equipment in order to achieve specific scientific goals, including the functions, workflow and structural design of the payload; The engineering requirements refer to the quantitative requirements of various indicators proposed during the engineering development phase, including thermal control, power consumption, communication, size, weight, posture and data transmission indicators.

3. The BERT-based space science mission requirements classification method according to claim 1, characterized in that: The classification model includes 12 encoder layers and a linear classification layer connected in sequence, and the processing process includes: The space science mission requirement text is represented by tokens after passing through 12 encoder layers. The first token is input into the linear classification layer to obtain the estimated probability of the requirement text in different categories.

4. The BERT-based space science mission requirements classification method of claim 3, characterized in that: It also includes the training steps for the classification model, including: Step 1) Collect space science mission plan data and establish a data set; Step 2) Preprocess the data set to achieve data enhancement; Step 3) Divide the preprocessed data set into a training set, a test set, and a validation set. Divide the data in the training set by words and convert them into word vectors and input them into the model. Compare the predicted results with the true values ​​and update the model parameters. After each training iteration, use the validation set to verify the model training effect and export the model parameters with the highest current accuracy. Repeat this step until the training requirements are met and a trained classification model is obtained. Apply it to the test set to verify the model training effect.

5. The BERT-based space science mission requirements classification method according to claim 4 is characterized in that: The step 1) comprises: Collect space science mission plan documents, from which scientific goals, performance requirements, payload requirements, and engineering requirements are collected, and each type of requirement is labeled as 0, 1, 2, and 3 respectively to establish a data set.

6. The BERT-based space science mission requirements classification method according to claim 4, characterized in that: The step 2) comprises: Text splitting: Split text into multiple parallel or independent data based on the key characters "and", semicolon, comma and / or period; Noise addition: randomly add meaningless special symbols to part of the text instead of correct mathematical symbols; Synonym Replacement: Randomly swap proper nouns and acronyms in your text.

7. The BERT-based space science mission requirements classification method according to claim 4, characterized in that: The learning rate in the step 3) training is 2e-5, the optimizer is AdamW, and the loss function is CrossEntropyLoss.

8. A BERT-based space science mission requirements classification system, characterized by: include: A classification output module is used to input the space science mission requirement text into a pre-established and trained classification model, obtain the estimated probability of the requirement text in different categories, and select the category with the highest estimated probability for classification; the categories include: scientific objectives, performance requirements, payload requirements, and engineering requirements; The classification model adopts the BERT model, which uses attention mechanism, fully connected neural network layers and position encoding to learn the complex structure and non-sequential content of language.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images