Training method of large model in satellite communication field

Abstract

The invention discloses a method for training a large model in the field of satellite communication, and aims to solve the problem that a general large model is insufficient in processing capability in the professional field of satellite communication. The method comprises the specific steps that multi-modal professional corpora (structured and unstructured data) in the satellite communication field are collected; generating more than ten thousands of incremental pre-training and fine-tuning corpora through data preprocessing, performing data enhancement on the corpora by adopting a plurality of mixing strategies, and dividing a training set and a test set according to a general proportion; based on low-rank self-adaptive fine tuning technologies such as LoRA, incremental pre-training and supervised fine tuning are carried out on the general large model; a professional evaluation data set is constructed manually, a self-established evaluation data set is used for comparing semantic comprehension and generation capabilities of the satellite communication large model and the general model in the satellite communication field, and model training hyper-parameters are iterated according to an evaluation result; an RAG framework is fused, a self-built private database, a self-defined text partitioning method and a knowledge base calling strategy are integrated, knowledge base data are accurately called, and question and answer content conforming to rules and specifications is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of satellite communication technology and natural language processing technology, and in particular to a training method for a large model in the field of satellite communication. BACKGROUND

[0002] In recent years, large language models have made significant progress in the field of natural language processing and are widely used in text generation, question answering, translation and other tasks. However, in the professional field of satellite communication, the existing general large language models have obvious shortcomings. During the pre-training process, the general large model has insufficient coverage of professional corpus in the field of satellite communication, resulting in weak understanding and representation ability of the model for professional terms, technical principles, system design and other content in the field of satellite communication, which makes it difficult to meet the demand for high-precision and professional language processing in the field of satellite communication.

[0003] Currently, the training method for large models in specific fields usually has problems such as imperfect corpus construction, single data augmentation strategy, and weak model training pertinence, which limits the improvement of the professional performance of the trained model in the field of satellite communication and makes it difficult to effectively handle language tasks in complex professional scenarios such as satellite communication networks and space-ground integration. Therefore, there is an urgent need for a training method and system that can effectively improve the performance of large models in the field of satellite communication. SUMMARY

[0004] The purpose of the present application is to provide a training method for a large model in the field of satellite communication to solve the problem of insufficient professional processing capability of existing general large language models in the field of satellite communication and improve the understanding, representation and application ability of the model for professional knowledge in the field of satellite communication.

[0005] The technical solution adopted by the present application is as follows:

[0006] A training method for a large model in the field of satellite communication, comprising the following steps:

[0007] S1: Collect satellite communication technology knowledge corpus, including structured data and unstructured data, build a satellite communication field corpus, and perform data cutting, cleaning and deduplication to form a professional corpus containing satellite communication system technical principles, satellite communication system applications, industry standards and protocols and system operation test data contents, including pre-training data set and fine-tuning data set; wherein the fine-tuning data set is in the format of question and answer pair, and the pre-training data set is the answer part in the fine-tuning data set;

[0008] S2: Expand the pre-training data set generated in S1 by using a mixed data augmentation strategy, wherein the mixed data augmentation strategy includes random deletion, random insertion, random exchange and random synonym replacement;

[0009] S3: Based on the LoRA low-rank adaptive technology, an incremental pre-training and domain fine-tuning are performed on the general open-source large model, and the satellite communication large model is obtained by multiple iterations of the optimizer in the training process; wherein, in the incremental pre-training stage, the general data set and the satellite communication specific data set are mixed for training, so that the model retains the basic ability and at the same time injects domain knowledge, and the satellite communication specific data set includes the pre-training data set generated by S1 and the pre-training data set expanded by S2; in the domain fine-tuning stage, the fine-tuning data set is used for training, focusing on the vertical scene;

[0010] S4: From the expanded pre-training data set, a plurality of single-choice questions are prepared, each question sets four options, only one correct answer, forming a benchmark question bank;

[0011] S5: The benchmark question bank constructed by S4 is input into the satellite communication large model and the original base model, the performance of the satellite communication large model and the original base model is compared, the domain knowledge enhancement effect is evaluated, and the model training hyperparameters are adjusted according to the evaluation results;

[0012] S6: Combined with the satellite communication field data retrieval enhancement generation method based on the large language model, the large-scale and heterogeneous communication data are uniformly modeled, semantically inferred and efficiently answered, and the private document knowledge base in the satellite communication field is constructed to supplement the satellite communication large model.

[0013] Further, S1 specifically includes:

[0014] S11: Selecting unstructured data related to the technical principles of satellite communication system, satellite communication system applications, industry standards and protocols in public publications, including satellite communication field professional books, papers, patents and standards, and converting them into documents in a format suitable for large model reading and writing using a data conversion framework; at the same time, selecting structured data, including satellite communication system public test data set and private data set stored in system test running process, and converting structured data into large model reading and writing format text using the established mapping relationship dictionary;

[0015] S12: Constructing a data cutting, cleaning and deduplication preprocessing automation tool chain; in the data cutting stage, the text exceeding the set length is segmented according to semantic units, and the length of each text block is controlled, wherein the semantic units include paragraphs and chapters; in the data cleaning stage, regular expressions are used to remove image tags, special symbols and garbled characters in Markdown format text, and the text encoding format is unified; in the data deduplication stage, the text blocks are converted into vector representation, the text deduplication algorithm is adopted to calculate the similarity between the text blocks, and the similarity threshold is set, and the text blocks with similarity higher than the threshold are only kept one;

[0016] S13: Check the cleaned and deduplicated data for technical accuracy, including whether the description of satellite communication principles is correct and the parameter settings are reasonable; and check the text quality of the data, including the fluency of the sentences and the use of professional terms, and correct or exclude data with problems;

[0017] S14: From the data that passes the audit, select content closely related to the actual application scenarios of satellite communication, including satellite communication fault diagnosis cases, satellite network optimization schemes, and satellite communication and Internet of Things integration applications, to build a fine-tuning dataset in the form of question and answer pairs;

[0018] S15: Directly use the answer part of the fine-tuning dataset as the pre-training dataset.

[0019] Further, S2 specifically includes:

[0020] Random deletion: In the pre-training dataset, randomly delete individual words or short sentences with a set probability to simulate data loss in actual applications;

[0021] Random insertion: Randomly select words or short sentences and insert them into random positions in the original text of the pre-training dataset with a set probability;

[0022] Random exchange: Set an exchange probability and exchange the positions of adjacent words or short sentences in the original text of the pre-training dataset to change the syntax structure of the text;

[0023] Random synonym replacement: Use a professional satellite communication field synonym dictionary to replace words in the original text of the pre-training dataset with a set probability.

[0024] Further, S3 specifically includes:

[0025] S31: Based on low-rank adaptation technologies such as LoRA, use a general large model as the base, mix the general dataset and the satellite communication specific dataset into the model, use the optimizer for training, while retaining the basic language understanding ability of the model, inject satellite communication field knowledge into the model parameters, so that the model initially has the ability to process satellite communication field text; Wherein, the satellite communication specific dataset includes the pre-training dataset generated by S1 and the pre-training dataset expanded by S2;

[0026] S32: On the basis of incremental pre-training, use the fine-tuning dataset to supervise the fine-tuning of the model, and get the trained satellite communication large model; During the fine-tuning process, use the optimizer to perform supervised learning, so that the model is optimized for specific application scenarios and tasks in the satellite communication field, and the understanding and representation ability of the model for field terms and technical principles is strengthened, focusing on satellite communication vertical scenarios.

[0027] Furthermore, S4 specifically includes:

[0028] S41: From the expanded pre-training dataset, based on keyword extraction and semantic analysis techniques, content involving core knowledge points of satellite communication is randomly selected as the source of questions;

[0029] S42: Based on the selected question, and combining common technical misunderstandings and easily confused concepts, compile four options, of which only one is the correct answer; at the same time, add distractor options.

[0030] Furthermore, S5 specifically includes:

[0031] S51: Input the benchmark question bank constructed in S4 into the large satellite communication model and the original base model. The model selects the answers according to the question requirements. Count the number of questions answered correctly by the model and calculate the accuracy rate as the model's ability score in the field of satellite communication.

[0032] S52: Compare and analyze the scores of the large satellite communication model with those of the original base model, draw performance comparison charts, show the improvement of the model in general capabilities and satellite communication domain capabilities, quantitatively evaluate the impact of satellite communication domain knowledge enhancement on model performance, and adjust the hyperparameters of the large satellite communication model training based on the evaluation results.

[0033] The advantages of this invention compared to the prior art are:

[0034] This invention constructs a high-quality corpus in the field of satellite communications, employs diverse data augmentation strategies, builds a professional evaluation dataset, and combines a specific training framework and techniques to progressively train and optimize a general open-source large model, effectively improving the large model's professional processing capabilities in the satellite communications field. On the self-built evaluation dataset, the satellite communications large model scores significantly higher than the base model. The trained model performs excellently in professional tasks in the satellite communications field, meeting the high-precision requirements of practical applications in this area, and has significant practical application value and promotional significance. Attached Figure Description

[0035] Figure 1 This is a flowchart of a training method for a large-scale model in the field of satellite communication proposed in this invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments.

[0037] refer to Figure 1 As shown, this invention provides a training method for a large-scale model in the field of satellite communication, comprising the following steps:

[0038] S1: Construction of a Satellite Communication Corpus: First, targeted collection of satellite communication technology knowledge corpora is conducted. Due to the unique nature of satellite communication knowledge, large models, in addition to possessing the ability to recognize unstructured data such as language and text, also need to have the ability to recognize structured corpora. Therefore, the construction of the communication corpus should focus on covering structured knowledge data in the satellite communication field; simultaneously, a dictionary of structured data mapping relationships for satellite communication systems needs to be constructed to perform data format mapping and conversion. The communication corpus, constructed from the structured and unstructured data after mapping and conversion in the satellite communication field, is then segmented, cleaned, and deduplicated to form a high-quality professional corpus containing content such as the technical principles of satellite communication systems, satellite communication system applications, industry standards and protocols, and system operation test data. This corpus includes over ten thousand pre-trained datasets and fine-tuned datasets; the fine-tuned datasets are in question-answer pair format, and the pre-trained datasets contain the answer portions of the fine-tuned datasets.

[0039] S1 specifically includes the following sub-steps:

[0040] S11: Collect satellite communication technology knowledge corpus: 1) Select unstructured data such as professional books, papers, patents and standards in the field of satellite communication, including publicly published articles on the technical principles of satellite communication systems, applications of satellite communication systems, industry standards and protocols, etc., and use a data conversion framework to convert the training corpus into documents adapted to the reading and writing format of the large model; 2) Select structured data such as publicly available test datasets of satellite communication systems and private datasets stored during system testing and operation, and use the established mapping relationship dictionary to convert the structured data into text in the reading and writing format of the large model;

[0041] S12: Data Segmentation, Cleaning, and Deduplication: Construct an automated toolchain for data segmentation, cleaning, and deduplication preprocessing; in the data segmentation stage, long texts are divided into semantic units (e.g., paragraphs, chapters, etc.), and the length of each text block is controlled to facilitate subsequent processing; in the data cleaning stage, regular expressions are used to remove image tags, special symbols, garbled characters, etc. from Markdown and other formatted texts, and the text encoding format is unified; in the data deduplication stage, text blocks are converted into vector representations, and a text deduplication algorithm is used to calculate the similarity between text blocks, and a similarity threshold is set. Only one text block with a similarity higher than the threshold is retained to avoid duplicate data affecting the model training effect;

[0042] S13: Manual Review: A review team composed of satellite communication experts and natural language processing engineers will be formed to manually check the cleaned and deduplicated data; experts will focus on reviewing the technical accuracy of the corpus, such as whether the description of satellite communication principles is correct and whether the parameter settings are reasonable; engineers will check the text quality of the corpus, including sentence fluency and the standardization of the use of professional terminology, and correct or remove problematic corpus data.

[0043] S14: Fine-tuning Dataset Generation: From the approved data, select content closely related to actual satellite communication application scenarios, such as satellite communication fault diagnosis cases, satellite network optimization schemes, and integrated applications of satellite communication and the Internet of Things, and construct a fine-tuning dataset in question-answer pair format;

[0044] S15: Pre-training dataset generation: Directly use the answer portion of the fine-tuned dataset as the pre-training dataset.

[0045] S2: Satellite communication corpus data augmentation: Employing a hybrid data augmentation strategy including random deletion, random insertion, random swapping, and random synonym replacement, this expands the diversity of the pre-training dataset generated in S1 and enhances the model's generalization ability.

[0046] S2 specifically includes the following sub-steps:

[0047] Random deletion: In the pre-training dataset, single words or phrases are randomly deleted with a set probability to simulate the situation of missing data in real applications and enhance the model's ability to process incomplete information;

[0048] Random insertion: Randomly select words or phrases and insert them into random positions in the original text of the pre-training dataset with a set probability to increase the diversity of the text;

[0049] Random swapping: Set swapping probabilities and swap the positions of adjacent words or phrases in the original text of the pre-training dataset to change the word order structure of the text and make the model adapt to different language expressions;

[0050] Random synonym replacement: Using a professional thesaurus in the field of satellite communication, words in the original text of the pre-training dataset are replaced with synonyms with a set probability, enriching the model's understanding of different expressions of the same concept.

[0051] S3: Large-scale model training in the satellite communication domain: Based on low-rank adaptive techniques such as LoRA, and using an open-source large model as a foundation, incremental pre-training and domain fine-tuning are performed on the general open-source large model. In the incremental pre-training stage, a mixture of general datasets (such as the Alpaca dataset) and satellite communication-specific datasets (including the pre-training dataset generated in S1 and the pre-training dataset expanded in S2) is used to retain the basic capabilities of the model while injecting domain knowledge. In the domain fine-tuning stage, only the fine-tuning dataset in the satellite communication domain is used to focus on vertical scenarios. The training process uses an optimizer for multiple rounds of iterative training to enhance the model's understanding and representation of domain terms, resulting in a large-scale satellite communication model.

[0052] S3 specifically includes the following sub-steps:

[0053] S31: Incremental Pre-trained Model: Based on LoRA low-rank adaptation technology, using a general large model as a base, the model is input with a mixture of general datasets (such as the Alpaca dataset) and satellite communication-specific datasets (including the pre-trained dataset generated by S1 and the enhanced variant samples of the pre-trained dataset generated by S2). The model is trained using the AdamW optimizer. While retaining the model's basic language understanding ability, knowledge of the satellite communication domain is injected into the model parameters, enabling the model to initially have the ability to process text in the satellite communication domain.

[0054] S32: Supervised Fine-tuning Model: Based on incremental pre-training, the model is fine-tuned in a supervised manner using only the satellite communication domain fine-tuning dataset generated in S1. During the fine-tuning process, the AdamW optimizer is also used to optimize the model for specific application scenarios and tasks in the satellite communication domain through supervised learning. This strengthens the model's understanding and representation of domain terminology and technical principles, and improves the model's performance in vertical satellite communication scenarios.

[0055] S4: Construction of a professional evaluation dataset in the field of satellite communications: More than a thousand single-choice questions were manually compiled from the expanded pre-training dataset. Each question has four options (A, B, C, and D) (only one correct answer) to form a benchmark question bank for model performance evaluation.

[0056] S4 specifically includes the following sub-steps:

[0057] S41: Randomly select questions in the field of satellite communications: From the expanded pre-training dataset, based on keyword extraction and semantic analysis techniques, randomly select content involving core knowledge points of satellite communications as the source of questions;

[0058] S42: Manually Compiled Options and Correct Answers: Satellite communication experts will compile four options (A, B, C, and D) based on the selected questions, taking into account common technical misunderstandings and easily confused concepts. Only one option is correct. At the same time, reasonable distractor options will be set to enhance the test of the students' mastery of model knowledge.

[0059] S5: Evaluation of the Training Effect of the Large-Scale Satellite Communication Model: The benchmark question bank constructed in S4 is input into the large-scale satellite communication model and the original base model. The performance of the large-scale satellite communication model and the original base model are compared to evaluate the domain knowledge enhancement effect. Based on the evaluation results, the model training hyperparameter settings are adjusted. After the effect evaluation is completed, if the evaluation results do not meet the expectations, the model is retrained and fine-tuned to optimize the model performance. The optimized model is then evaluated again using the evaluation dataset until the model achieves the expected results.

[0060] S5 specifically includes the following sub-steps:

[0061] S51: Use the dataset compiled in step S4 to evaluate the model's capabilities in the field of satellite communications: Input the benchmark question bank (containing over a thousand single-choice questions) constructed in S4 into the large satellite communications model and the original base model. The model selects answers according to the question requirements. Count the number of questions the model answers correctly, calculate the accuracy rate as the model's capability score in the field of satellite communications, and evaluate the model's mastery and application ability of knowledge in the field of satellite communications.

[0062] S52: Compare and evaluate the performance of the baseline model and the fine-tuned model: Compare and analyze the scores of the satellite communication large model on the self-built evaluation dataset with the scores of the original base model, draw performance comparison charts, intuitively show the improvement of the model in general capabilities and satellite communication domain capabilities, quantitatively evaluate the impact of satellite communication domain knowledge enhancement on model performance, verify the effectiveness of the training method, and adjust the hyperparameters of the satellite communication large model training based on the evaluation results.

[0063] S6: Enhanced Knowledge Retrieval in Satellite Communications: Combining a data retrieval enhancement generation method based on a large language model in the satellite communications field, it improves semantic understanding and knowledge expression capabilities, performs unified modeling, semantic reasoning, and efficient responses for large-scale and heterogeneous communication data, and constructs a private document knowledge base in the satellite communications field to supplement the large satellite communications model.

[0064] S6 specifically includes the following sub-steps:

[0065] S61: Collect diverse documents from authoritative data sources in the satellite communications field. The collection process employs distributed crawling and API calls to ensure the timeliness and completeness of the data. The original documents undergo a unified format conversion, transforming various formats into a standardized plain text format for easier subsequent processing. This mainly includes removing formatting noise, preliminary identification of charts and graphs, and chapter segmentation.

[0066] S62: For long texts, a segmentation method based on semantic boundary detection is adopted to split the document into multiple semantically independent text blocks. Each block contains a complete semantic unit, such as a technical definition, protocol description or design scheme, which supports subsequent clustering analysis and knowledge extraction.

[0067] S7: Leveraging the enhanced intent recognition capabilities of the satellite communication big model after the improvements of S3 and S6, an intelligent-driven workflow + intelligent agent framework for typical satellite communication tasks is designed, which is "based on workflow as the execution framework and intelligent agents provide intelligent decision-making". It has natural language interactive question answering capabilities, context understanding capabilities, and automated execution capabilities. Through this architecture, various typical satellite communication system tasks such as knowledge question answering, traffic analysis, and fault detection can be completed.

[0068] S7 specifically includes the following sub-steps:

[0069] S71: Encapsulates typical tasks of satellite communication system knowledge question answering, traffic analysis, and fault detection into independent agents;

[0070] S72: Interact with the system through a unified API interface definition to achieve system modularization and standardization;

[0071] S73: Design a multi-task processing system based on workflow orchestration. In the system, design a task scheduling mechanism based on question type. When the user inputs a relevant question, the system automatically identifies the question type and routes it to the corresponding agent for processing.

[0072] S74: Multiple agents work together to ensure that the system can efficiently complete multiple typical tasks.

[0073] S8: Construct a private graph knowledge base from the graph content contained in the training corpus. Combine this with the private document knowledge base for the satellite communication domain constructed in S6. Utilize data retrieval enhancement and a multimodal output architecture to achieve the multimodal output process for user question answering.

[0074] S8 specifically includes the following sub-steps:

[0075] S81: Annotate and comment on the image information contained in the document to build a graph knowledge base corresponding to the text and images;

[0076] S82: Based on the retrieval-enhanced generation method framework, a complete private document knowledge base and graph knowledge base are loaded and constructed. Using a multimodal output framework, a multimodal output process with rich graphics and text is completed for various typical tasks of satellite communication systems.

[0077] This invention constructs a high-quality corpus in the field of satellite communications, employs diverse data augmentation strategies to build a professional evaluation dataset, and combines specific training framework techniques for progressive training and optimization to enhance the professional processing capabilities of large models. The trained model performs excellently in professional satellite communication tasks, meeting the high-precision requirements of practical applications in the field, and has significant practical application value and promotional significance.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a large-scale model in the field of satellite communication, characterized in that, Specifically, the following steps are included: S1: Collect satellite communication technology knowledge corpus, including structured and unstructured data, construct a corpus in the field of satellite communication, and perform data segmentation, cleaning and deduplication to form a professional corpus containing the technical principles of satellite communication systems, satellite communication system applications, industry standards and protocols and system operation test data, including pre-trained datasets and fine-tuned datasets; The fine-tuning dataset is in question-answer pair format, and the pre-training dataset is the answer portion of the fine-tuning dataset. S2: The pre-training dataset generated in S1 is augmented with a hybrid data augmentation strategy, which includes random deletion, random insertion, random swapping and random synonym replacement. S3: Based on LoRA low-rank adaptive technology, using a general open-source large model as a foundation, incremental pre-training and domain fine-tuning are performed on the general open-source large model. The training process uses an optimizer for multiple rounds of iterative training to obtain a large satellite communication model. In the incremental pre-training stage, a general dataset and a satellite communication-specific dataset are used for training, so that the model retains its basic capabilities while injecting domain knowledge. The satellite communication-specific dataset includes the pre-training dataset generated in S1 and the pre-training dataset expanded in S2. In the domain fine-tuning stage, the fine-tuning dataset is used for training, focusing on vertical scenarios. S4: Compile multiple single-choice questions from the expanded pre-training dataset, each with four options and only one correct answer, to form a benchmark question bank; S5: Input the benchmark question bank constructed in S4 into the large satellite communication model and the original base model, compare the performance of the large satellite communication model and the original base model, evaluate the domain knowledge enhancement effect, and adjust the model training hyperparameters based on the evaluation results; S6: Combining data retrieval enhancement generation methods based on large language models in the satellite communication field, we perform unified modeling, semantic reasoning, and efficient responses for large-scale and heterogeneous communication data, and construct a private document knowledge base in the satellite communication field to supplement the large satellite communication model.

2. The training method for a large-scale model in the field of satellite communication according to claim 1, characterized in that, S1 specifically includes: S11: Select unstructured data from publicly available publications related to the technical principles, applications, industry standards, and protocols of satellite communication systems, including professional books, papers, patents, and standards in the field of satellite communication, and convert them into documents adapted to the large model's read / write format using a data conversion framework; at the same time, select structured data, including publicly available test datasets of satellite communication systems and private datasets stored during system testing and operation, and convert the structured data into text in the large model's read / write format using an established mapping dictionary; S12: Construct an automated toolchain for data segmentation, cleaning, and deduplication preprocessing; in the data segmentation stage, text exceeding the set length is divided into semantic units, and the length of each text block is controlled, where semantic units include paragraphs and chapters; in the data cleaning stage, regular expressions are used to remove image tags, special symbols, and garbled characters from Markdown formatted text, and the text encoding format is unified; in the data deduplication stage, text blocks are converted into vector representations, a text deduplication algorithm is used to calculate the similarity between text blocks, and a similarity threshold is set, with only one text block whose similarity is higher than the threshold being retained; S13: Check the cleaned and deduplicated data, verify the technical accuracy of the data, including whether the description of satellite communication principles is correct and whether the parameter settings are reasonable; and check the text quality of the data, including the fluency of sentences and the standardization of the use of professional terminology, and correct or remove problematic data. S14: From the approved data, select content closely related to actual satellite communication application scenarios, including satellite communication fault diagnosis cases, satellite network optimization schemes, and integrated applications of satellite communication and the Internet of Things, and construct a fine-tuned dataset in question-and-answer pair format; S15: Directly use the answer portion of the fine-tuned dataset as the pre-training dataset.

3. The training method for a large-scale model in the field of satellite communication according to claim 1, characterized in that, S2 specifically includes: Random deletion: In the pre-training dataset, single words or phrases are randomly deleted with a set probability to simulate the situation of missing data in real-world applications; Random insertion: Randomly select words or phrases and insert them into random positions in the original text of the pre-training dataset with a set probability; Random swapping: Set swapping probabilities and swap the positions of adjacent words or phrases in the original text of the pre-training dataset to change the word order structure of the text; Random synonym replacement: Using a specialized thesaurus in the field of satellite communications, words in the original text of the pre-trained dataset are replaced with synonyms with a set probability.

4. The training method for a large-scale model in the field of satellite communication according to claim 1, characterized in that, S3 specifically includes: S31: Based on low-rank adaptation techniques such as LoRA, using a general large model as a base, the model is input with a mixture of general datasets and satellite communication-specific datasets. The model is trained using an optimizer. While retaining the model's basic language understanding ability, knowledge of the satellite communication domain is injected into the model parameters, enabling the model to initially have the ability to process text in the satellite communication domain. The satellite communication-specific dataset includes the pre-training dataset generated in S1 and the pre-training dataset expanded in S2. S32: Based on incremental pre-training, the model is supervised fine-tuned using a fine-tuning dataset to obtain a well-trained large-scale satellite communication model. During the fine-tuning process, an optimizer is used to optimize the model for specific application scenarios and tasks in the field of satellite communication through supervised learning, thereby strengthening the model's understanding and representation of domain terminology and technical principles, and focusing on vertical scenarios of satellite communication.

5. The training method for a large-scale model in the field of satellite communication according to claim 1, characterized in that, S4 specifically includes: S41: From the expanded pre-training dataset, based on keyword extraction and semantic analysis techniques, content involving core knowledge points of satellite communication is randomly selected as the source of questions; S42: Based on the selected question, and combining common technical misunderstandings and easily confused concepts, compile four options, of which only one is the correct answer; at the same time, add distractor options.

6. The training method for a large-scale model in the field of satellite communication according to claim 1, characterized in that, S5 specifically includes: S51: Input the benchmark question bank constructed in S4 into the large satellite communication model and the original base model. The model selects the answers according to the question requirements. Count the number of questions answered correctly by the model and calculate the accuracy rate as the model's ability score in the field of satellite communication. S52: Compare and analyze the scores of the large satellite communication model with those of the original base model, draw performance comparison charts, show the improvement of the model in general capabilities and satellite communication domain capabilities, quantitatively evaluate the impact of satellite communication domain knowledge enhancement on model performance, and adjust the hyperparameters of the large satellite communication model training based on the evaluation results.

Images