Concept map-based astronomical calculation reasoning instruction enhancement method

Abstract

This invention discloses a concept graph-based method for enhancing astronomical computational reasoning instructions, comprising the following steps: obtaining seed problems labeled with subdomains and topics as basic input; extracting knowledge points from the seed problems using a large language model to establish a knowledge framework covering subdomains, topics, and knowledge points; and transforming problem instances into conceptual representations; constructing a concept graph based on the knowledge framework; executing a dynamic random walk algorithm on the concept graph to output a complete concept combination; and using the complete concept combination to guide the large language model in generating astronomical computational problems and corresponding reasoning processes through example matching, data deduplication, and thought chain distillation. This invention, by constructing a concept graph and employing a random walk strategy, improves the diversity and logical rigor of the generated data, providing a new path for optimizing astronomical computational reasoning instructions.

Description

Technical Field

[0001] This invention relates to the fields of computer information processing, data processing, and other technical fields, and in particular to an enhanced method for astronomical computational reasoning instructions based on concept graphs. Background Technology

[0002] Large language models have demonstrated outstanding performance in data reasoning and computation tasks, but they face numerous challenges in computational reasoning within the vertical field of astronomy. Currently, mainstream research and practice generally employ instruction tuning to unleash the potential of large language models in astronomical computation tasks. This involves fine-tuning the model by constructing an astronomical computational reasoning dataset containing a large number of questions and solution paths. However, this method has significant limitations. Existing astronomical computational reasoning datasets are generally small in scale, and the limited number of samples makes it difficult for the model to fully learn the complex and diverse reasoning patterns in astronomical computations. This restricts the effectiveness of instruction tuning and, consequently, hinders the improvement of large language models' capabilities in this field.

[0003] The core reason for these limitations lies in the fundamental difference between astronomical computation problems and general mathematical problems. Astronomical computation problems heavily rely on specific domain concepts as the logical foundation, such as Kepler's laws and the conversion between apparent and absolute stars. When solving such problems, models not only need general mathematical computational capabilities but also the ability to accurately retrieve and internalize relevant physical theorems and astronomical constants. Current large language models lack a systematic understanding and precise application of this deep domain knowledge, making it difficult for them to construct correct problem-solving paths.

[0004] In summary, to address the problem that the existing astronomical computational reasoning datasets are relatively small and cannot fully unleash the potential of large language models, a concept graph-based method for enhancing astronomical computational reasoning instructions is proposed. Summary of the Invention

[0005] The main objective of this invention is to provide a concept graph-based method for enhancing astronomical computational reasoning instructions, aiming to address the problem that the existing astronomical computational reasoning datasets are too small to fully unleash the potential of large language models.

[0006] Based on the first main aspect of the present invention, a method for enhancing astronomical computational inference instructions based on a concept graph is provided, comprising the following steps:

[0007] We obtain seed questions labeled with subdomains and topics as basic input, use a large language model to extract knowledge points from the seed questions, build a knowledge framework covering subdomains, topics and knowledge points, and transform problem instances into conceptual representations.

[0008] A concept graph is constructed based on a knowledge framework. The nodes of the concept graph consist of a set of topics and a set of knowledge points. The concept graph has three types of edges: edges from topic to topic, edges from topic to knowledge point, and edges from knowledge point to knowledge point. The weight of the edges is obtained by counting their co-occurrence frequency.

[0009] A dynamic random walk algorithm is executed on the concept graph. The starting topic node is obtained on the topic set and walked. After completion, the knowledge point subgraph is walked. Finally, a complete concept combination containing the target topic set and the target knowledge point set is output.

[0010] Based on the complete concept combination, the large language model is guided to generate astronomical computation problems and corresponding reasoning processes by performing example matching, data deduplication, and mind chain distillation.

[0011] As a further preferred embodiment, in the aforementioned method, the steps for constructing the concept map are as follows:

[0012] If two nodes appear together in the seed problem of the same problem, then an edge is created between the two nodes;

[0013] The formal definition of an edge set is:

[0014] ;

[0015] in, Describe the set of edges. This represents two distinct nodes. Indicates the connection node and Edge weights.

[0016] As a further preferred embodiment, in the aforementioned method, obtaining the edge weights includes performing a logarithmic transformation on the co-occurrence count, specifically as follows:

[0017] ;

[0018] in, Indicates the connection node and Edge weights; Represents a node and The number of times each seed problem co-occurs; It represents a positive number and is used to avoid the anomaly of zero values ​​in logarithmic calculations; Represents a logarithmic function; This represents two distinct nodes.

[0019] As a further preferred option, in the aforementioned method, the concept map can be divided into three types of sub-maps, including topic sub-maps, topic-knowledge point association sub-maps, and knowledge point sub-maps.

[0020] As a further preferred embodiment, in the aforementioned method, the execution steps for outputting the complete concept combination are as follows:

[0021] Uniform random sampling is performed on the topic set to obtain the initial topic nodes;

[0022] The topic subgraph walk starts from the topic node and initiates a random walk within the topic subgraph. The probability of a node moving to another node is dynamically calculated based on the similarity weight between nodes.

[0023] After completing the topic subgraph walk, a single-step random walk is used to transition from the topic domain to the knowledge point domain, and a preliminary knowledge point set is generated through probability distribution rules.

[0024] Continue performing random walks of a preset execution length on the knowledge point subgraph, and output complete concept combinations by mining adjacent or related knowledge points.

[0025] As a further preferred embodiment, in the aforementioned method, the specific formula for the probability distribution rule is as follows:

[0026] ;

[0027] in, This represents the probability of moving from one node to another during a random walk in a topic subgraph or knowledge point subgraph. Indicates the connection node and Edge weights; This represents two distinct nodes; Indicate neighboring nodes; This represents the relationship between nodes in a topic subgraph or knowledge point subgraph. The set of neighboring nodes.

[0028] As a further preferred embodiment, in the aforementioned method, the execution steps of the example matching are as follows:

[0029] The Jaccard similarity coefficient is used to calculate the overlap between the seed problem and the complete concept combination on the set of knowledge points.

[0030] Instance matching is achieved by retrieving several instances that have the highest semantic relevance to the target as context;

[0031] The data deduplication process is as follows:

[0032] A similarity-based text filtering mechanism is introduced during data generation to remove generated samples that are similar to the original test set, thereby completing data deduplication, and natural language processing is used for evaluation.

[0033] The thought chain distillation involves using a large language model as the teacher model, training it to generate the complete intermediate reasoning process for astronomical calculation problems, and transferring logical capabilities to a small open-source model.

[0034] Based on a second key aspect of the present invention, a concept-graph-based astronomical computation inference instruction enhancement system for implementing the aforementioned method is provided, comprising:

[0035] The concept acquisition module is used to construct a training set of astronomical computing benchmarks and obtain seed problems from it.

[0036] The concept extraction module is used to automatically extract conceptual knowledge points from seed questions and build a knowledge framework by combining a large language model with prompting engineering.

[0037] The concept map building module is used to further construct concept maps by building knowledge frameworks;

[0038] The concept composition module is used to perform a dynamic random walk algorithm on the concept graph to obtain complete concept compositions;

[0039] The astronomical computational reasoning data generation module is used to guide the generation of astronomical computational problems based on complete concept combinations and large language models.

[0040] According to a third key aspect of the present invention, an electronic device includes: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0041] The memory stores a computer program that, when executed by the processor, causes the processor to perform the aforementioned concept-map-based astronomical computation inference instruction enhancement method.

[0042] Based on a fourth key aspect of the present invention, a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the aforementioned concept-graph-based astronomical computation inference instruction enhancement method.

[0043] Compared with existing technologies, this invention addresses the problems of limited astronomical computational reasoning datasets, strong dependence on original training examples, and insufficient reasoning ability of large language models in the astronomical vertical domain due to a lack of structured knowledge support. By extracting subdomains, topics, and knowledge points from seed questions to construct a structured concept graph, this invention quantifies the intrinsic relationships between astronomical concepts using co-occurrence weights between nodes. This provides a physically logical mapping foundation for subsequent generation tasks. Based on this design, it overcomes the limitations of traditional data augmentation methods, which can only transform the surface and are difficult to generate substantial and novel concept combinations. Its multi-level concept abstraction mechanism can effectively guide the model to focus on the core meta-information required for problem-solving, improving the scientific nature of the generated problems and the breadth of domain coverage.

[0044] Meanwhile, this invention designs a dynamic random walk strategy based on graph structure. By migrating and transitioning within the topic subgraph according to similarity weights to the knowledge point subgraph for in-depth exploration, this strategy enables diverse combinations of topics and knowledge points, expanding the breadth of concept exploration while ensuring tight semantic connections between the combinations. This design ensures that the generated astronomical calculation problems maintain internal logical consistency in their knowledge structure, avoiding issues such as concept stuffing or misuse of physical laws. It also strengthens the ability of large language models to construct correct problem-solving paths through instruction optimization, mitigating the problem of reasoning deviations caused by a lack of domain knowledge.

[0045] Finally, this invention lays the technical foundation for generating high-quality and diverse astronomical computation problems through concept graph construction and random walks. Combined with mechanisms such as instance matching, data deduplication, and thought chain distillation, it ensures the innovativeness, redundancy resistance, and transferability of the generated data and reasoning logic. Overall, this invention aligns with the concept compression and connection forging mechanisms in human cognitive learning, reducing the dependence on the number of original labeled samples and providing an efficient and scalable technical solution for instruction optimization of large-scale astronomical computational language models. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0047] Figure 1 The following is an execution flowchart of a concept graph-based astronomical computation inference instruction enhancement method according to one embodiment of the present invention;

[0048] Figure 2The diagram shows an overall flowchart of a concept-graph-based method for enhancing astronomical computational inference instructions, according to one embodiment of the present invention.

[0049] Figure 3 An example diagram of an astronomical concept graph structure for an astronomical computational reasoning instruction enhancement method based on a concept graph, according to one embodiment of the present invention, is shown.

[0050] Figure 4 The illustration shows a schematic diagram of the concept graph construction of a concept graph-based method for enhancing astronomical computational reasoning instructions, according to one embodiment of the present invention. Detailed Implementation

[0051] The preferred embodiments of the present invention will be described in detail below to provide a clearer understanding of the purpose, features, and advantages of the invention. It should be understood that the following embodiments are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the technical solution of the invention.

[0052] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known techniques associated with the invention may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0053] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0054] The following is a description of the specific meanings of technical terms, English abbreviations, and formula parameters that may be used in this invention:

[0055] Concept graph: A structured graph network constructed with topics and knowledge points as nodes and co-occurrence relationships between nodes as edges.

[0056] Co-occurrence: During the extraction of metadata for the same seed problem, both nodes are included in the metadata set of that problem.

[0057] Random walk: A node sampling algorithm based on graph structure, which explores closely related node combinations by starting from the initial node and moving along the path according to the transition probability calculated based on the edge weights.

[0058] Jaccard similarity coefficient: a statistic used to measure the similarity between two sets, defined as the ratio of the size of the intersection to the size of the union of the two sets.

[0059] BLEU-4: A commonly used evaluation metric in the field of machine translation and text generation. It assesses similarity by calculating the exact matching of n-tuples between the generated text and the reference text. The higher the value, the greater the surface overlap.

[0060] ROUGE-L: A text summarization evaluation metric that calculates the recall rate between the generated text and the reference text based on the longest common subsequence, used to measure text similarity.

[0061] Mind chain distillation: A knowledge distillation technique that uses a high-performance teacher model to generate a solution containing complete intermediate reasoning steps and then transfers its logical reasoning ability to a student model with fewer parameters.

[0062] DeepSeek-R1: A large language model with strong complex mathematical reasoning capabilities.

[0063] Large language models refer to deep learning models that contain massive parameters and are trained on large-scale text corpora, possessing text understanding and generation capabilities.

[0064] Instruction tuning: A fine-tuning paradigm for large language models, which improves the model's ability to follow human instructions by constructing samples consisting of instructions, inputs, and expected outputs for supervised training.

[0065] NASA Space Math is an astronomical mathematics teaching resource project led by NASA, aimed at basic education and public science popularization. In this invention, the original NASA Space Math data is mostly unstructured PDF documents or web page text. To construct a training set suitable for fine-tuning large language model instructions, this invention performs the following cleaning and standardization processing on the original data: converting unstructured text into a unified JSON format, including question descriptions, reasoning steps, and final answers, and removing samples with graph dependencies or vague descriptions. Based on the above-processed data as seed questions, an AstroCalcBench dataset is constructed using random walks and combinations of concept graphs.

[0066] Combination Figure 1 As shown, one embodiment of the present invention provides a method for enhancing astronomical computational inference instructions based on a concept graph, including the following steps 100-400:

[0067] Step 100: Obtain seed questions labeled with subdomains and topics as basic input, use the large language model to extract knowledge points from the seed questions, establish a knowledge framework covering subdomains, topics and knowledge points, and transform problem instances into conceptual representations;

[0068] Step 200: Construct a concept graph based on the knowledge framework. The nodes of the concept graph consist of a set of topics and a set of knowledge points. The concept graph has three types of edges: edges from topic to topic, edges from topic to knowledge point, and edges from knowledge point to knowledge point. The weight of the edges is obtained by counting their co-occurrence frequency.

[0069] Step 300: Execute a dynamic random walk algorithm on the concept graph to obtain the starting topic node on the topic set and walk on it. After completion, execute the knowledge point subgraph walk and finally output a complete concept combination containing the target topic set and the target knowledge point set.

[0070] Step 400: Based on the complete concept combination, the large language model is guided to generate astronomical calculation problems and corresponding reasoning processes by performing example matching, data deduplication, and mind chain distillation.

[0071] This invention proposes an astronomical computational reasoning instruction enhancement method based on concept graphs, also known as the AstroConcept method. The first step uses a large language model to extract topics and knowledge points from existing astronomical computational task seed samples. Through this process, specific problem instances are transformed into corresponding conceptual representations, greatly reducing the dependence on the original problem.

[0072] The second step is to construct a concept map to quantify and describe the relationships between different concepts, and on this basis, instruct the large language model to automatically generate new astronomical computation problems based on random combinations of the concept map.

[0073] Compared to traditional data augmentation-based methods, the concept graph-based astronomical computational reasoning instruction augmentation method of this invention can efficiently generate richer problem examples with diverse concept combinations.

[0074] In one feasible embodiment, combined with Figure 2 As shown, the following detailed explanation provides a method for enhancing astronomical computational reasoning instructions based on a concept map, which is a method provided by the present invention.

[0075] First, concept extraction is performed:

[0076] The concept-graph-based astronomical computation reasoning instruction enhancement method proposed in this invention uses seed problems as basic input, taking the training set in AstroCalcBench (an astronomical computation benchmark) as the seed problems. In this embodiment, the training set in AstroCalcBench is used as the seed problems. This dataset not only covers a wide range of astronomical computation problems but also labels each problem with a "subfield" and a "topic." To further explore the deep knowledge contained in these problems, this embodiment utilizes a large language model combined with prompting engineering to automatically extract finer-grained concept knowledge points from the seed problems—that is, the core meta-information required to answer these astronomical computation problems.

[0077] Specifically, "Subfield," "Topic," and "Knowledge Points" constitute the key meta-information system for astronomical computation problems. "Subfield" refers to different subfields of astronomy, such as "Earth," "Moon," "Sun," and "Planets," which helps to categorize and define the scope of the field. "Topic" corresponds to specific themes in astronomy textbooks or relevant chapters, such as "volume of cylinder," "area of ​​a circle," and "vectors," representing the knowledge modules emphasized in the problem. "Knowledge Points" represent more refined professional concepts and physical principles, such as "Eddington limit" and "Kepler's First Law," which are core knowledge units that must be understood and applied in solving astronomical computational reasoning problems.

[0078] This invention simulates a senior astronomy tutor in a large language model, requiring the model to extract 1 to 5 conceptual knowledge points from each seed question based on a pre-designed prompt template.

[0079] To ensure the diversity and quality of metadata, this invention makes full use of the different sub-domain problems covered in the astronomical computing benchmark dataset, avoiding the bias of a single domain.

[0080] Furthermore, for topics and knowledge points that appear very infrequently, i.e., topics and knowledge points that appear only once, this invention has screened and eliminated them to reduce data noise and improve the effectiveness and representativeness of the training data.

[0081] Ultimately, this invention successfully established a knowledge framework covering a set of sub-domains encompassing nine sub-domains, a set of over 400 topics, and a set of knowledge points comprising approximately 8,000 knowledge points.

[0082] This extraction and compression process aligns with the concept compression stage in the human cognitive learning mechanism proposed in existing technologies. Through efficient abstraction and induction, it extracts the core concepts that are crucial for solving problems, laying a solid foundation for the subsequent construction of concept maps and the generation of new problems.

[0083] Next, we construct the concept map:

[0084] Based on the subdomains, topics, and knowledge points extracted in the preceding steps, this invention further constructs a structured concept graph, whose nodes are composed of the extracted topic set ( ) and knowledge point set ( )composition.

[0085] in, Represents a set of topics. Representing a single topic, The cardinality of the set of topics. Represents a set of knowledge points. Representing a single knowledge point, The cardinality of the set of knowledge points.

[0086] Combination Figure 3 As shown, there are three types of edges in the concept graph: Topic to Topic, Topic to Knowledge Points, and Knowledge Points to Knowledge Points.

[0087] First, subfields represent the most macroscopic branches of research in astronomy. The diagram includes: Earth (involving geophysics, atmospheric calculations, etc.), Moon (e.g., tides, orbital mechanics), Sun (e.g., radiation, nuclear fusion), Planets (e.g., orbital parameters, gravity), Stars (e.g., luminosity, temperature), Universe (e.g., cosmological scales), Space Travel (e.g., rocket equations, orbital transfer), Astrobiology (e.g., habitable zone calculations), and Black Holes (e.g., Schwarzschild radius, accretion disks). These subfields help models quickly pinpoint the macroscopic scope of the problem, avoiding confusion between cross-disciplinary concepts.

[0088] Secondly, topics are more specific knowledge modules, corresponding to textbook chapters or mathematical tools. In the diagram: Dot and Cross Product (used in vector operations, such as force decomposition of celestial bodies), Scientific Notation (handling large astronomical numbers such as light-years and solar masses), Unit Conversion (such as astronomical units and parsecs), Volume (such as calculating the size of a planet using the volume of a sphere), Density (such as the average density of a planet), and Rates (such as orbital velocity and expansion rate). The topic layer connects mathematical calculation methods with astrophysical objects.

[0089] Finally, knowledge points are the most refined conceptual units, encompassing physical principles or measurement methods: Coordinate geometry (used to locate celestial coordinates), Concept of number density (e.g., the number density of particles in the interstellar medium), Understanding of atmospheric composition (e.g., refractive index, greenhouse effect calculations), Understanding of the Van Allen Belts (involving magnetic field trapping of particles), and Radiation measurement units (e.g., gray and sievert). These knowledge points are the core meta-information that must be internalized when solving problems.

[0090] Therefore, the overall diagram structure can be divided into three sub-diagrams: the theme diagram, the theme-knowledge point relationship diagram, and the knowledge point sub-diagram.

[0091] Combination Figure 4 As shown, in Figure 4 In the co-occurrence analysis section, the figure shows four example problems and their corresponding concept sets: Q1: {Solar, Flare, Energy}, Q2: {Density, Geometry}, Q3: {Eddington limit, Radiation}, and Q4: {Vectors, Velocity, Spacecraft}. Co-occurrence here refers to the simultaneous extraction of multiple concepts (topics or knowledge points) within the same astronomical calculation problem. By statistically analyzing the frequency of co-occurrence of concept pairs across different problems, the strength of the association between concepts can be quantified. The "Problem ID" and "Co-occurring Concepts" on the left side of the figure directly reflect this analysis process.

[0092] exist Figure 4 In the conceptual diagram construction section, some terms in the diagram, such as Flux, Stability, Mechanics, Stellar physics, Massive star, Astrophysics, Vectors, and Velocity, are all basic concepts that are known to those skilled in the art.

[0093] The specific composition method is as follows:

[0094] If two nodes and If two problems appear together in the same seed problem, then an edge is established between them.

[0095] The weight of an edge not only reflects whether nodes co-occur, but also captures the strength of the association between nodes by statistically analyzing their co-occurrence frequency. Co-occurrence refers to the frequency of occurrence of edges in a given problem instance. and All of these were extracted as metadata for the question.

[0096] The formal definition of an edge set is:

[0097] ;

[0098] in, Describe the set of edges. This represents two distinct nodes. Indicates the connection node and Edge weights;

[0099] Specifically, if two knowledge points or topics frequently appear in the same seed questions, their connection in the concept map is closer, reflecting the potential correlation of knowledge structures.

[0100] Co-occurrence count Recorded nodes and The original number of simultaneous occurrences. To improve the stability of the weights and prevent the influence of extreme data values, this invention performs a logarithmic transformation on the co-occurrence count.

[0101] Weights are defined as:

[0102] ;

[0103] in, Indicates the connection node and Edge weights; Represents a node and The number of times each seed problem co-occurs; It represents a very small positive number, used to avoid the anomaly of zero values ​​in logarithmic calculations; Represents a logarithmic function; This represents two distinct nodes.

[0104] The formation of concept maps can clearly show the internal connections and distribution of various themes and knowledge points in astronomical calculation problems, and provide a solid foundation for the generation of problems based on concept combinations.

[0105] This diagram illustrates the patterns of joint occurrences among astronomical knowledge points, which helps guide the model in selecting reasonable and relevant combinations of concepts when generating new questions, thereby improving the scientific rigor and diversity of the generated questions.

[0106] Next, we will combine the concepts:

[0107] Based on the conceptual diagram established above, this invention proposes an effective sampling method to obtain semantically related and diverse combinations of topics and knowledge points, thereby guiding the generation of new astronomical computation problems.

[0108] Therefore, this invention employs a graph-based dynamic random walk algorithm to simulate the process of humans exploring related concepts in a knowledge network. The specific execution process is as follows:

[0109] In the topic collection Perform uniform random sampling to obtain an initial topic node. This strategy ensures that the sampling is unbiased and prevents the algorithm from getting trapped in local optima of high-frequency topics too early, thus ensuring that the generated problems have broad representativeness in terms of domain coverage.

[0110] Theme sub-map navigation based on theme nodes Starting with the topic, initiate a random walk within the topic subgraph. This step aims to explore neighborhood concepts that are closely related to the starting topic.

[0111] During the journey, nodes Transfer to node probability Dynamic calculation based on similarity weights between nodes:

[0112] ;

[0113] in, This represents the probability of moving from one node to another during a random walk in a topic subgraph or knowledge point subgraph. Indicates the connection node and Edge weights; This represents two distinct nodes; Indicate neighboring nodes; This represents the relationship between nodes in a topic subgraph or knowledge point subgraph. The set of neighboring nodes.

[0114] After exploring the topic layer, a single-step random walk is used to transition from the topic domain to the knowledge point domain. This step generates a preliminary set of knowledge points according to the probability distribution rules determined in the previous step.

[0115] To expand the depth and breadth of knowledge combinations, random walks of 4 to 8 steps are performed on the knowledge point subgraph to mine adjacent or related knowledge points, ultimately outputting a set containing the target topic. and target knowledge point set A complete combination of concepts.

[0116] Finally, astronomical computational inference data is generated:

[0117] After obtaining the target topic-knowledge point combination, i.e. the complete concept combination, through graph random walk sampling, this invention uses it to guide the generation of new astronomical computation problems by a large language model.

[0118] This generation model draws on the idea of ​​contextual learning, using typical examples in simulated teaching scenarios to inspire the cognitive process of designing new questions. This results in generated content with high rationality and pedagogical value in terms of logical structure and knowledge depth. The specific implementation steps are as follows:

[0119] (1) Example match:

[0120] To ensure that the generated questions are similar in difficulty and meaning to the original questions, this invention designs a dynamic example filtering mechanism based on similarity.

[0121] Specifically, the Jaccard similarity coefficient is used to calculate the overlap between the seed problem and the target concept combination on the knowledge point set.

[0122] By retrieving several examples with the highest semantic relevance to the target as context, the model not only receives high-quality domain prior knowledge, but also effectively constrains the difficulty and form of the generated questions, thus achieving example-driven generation guidance.

[0123] (2) Data deduplication:

[0124] To prevent the model from overfitting to the original data during training, this invention introduces a purification procedure to remove data that overlaps with the original data.

[0125] This invention introduces a similarity-based text filtering mechanism during data generation to remove generated samples that are similar to the original test set. It uses the standard evaluation metrics BLEU-4 and ROUGE-L in natural language processing to calculate the maximum and average sequence overlap between the generated dataset and the test set.

[0126] The evaluation results show that the average BLEU-4 score of the generated data and the test set is 0.0681, and the average ROUGE-L score is 0.2411. These scores indicate that there is a significant difference in the distribution of the generated data compared with the test set in terms of macroscopic representation.

[0127] The maximum BLEU-4 score for the most similar question to the test set in the generated data is 0.3123, and the maximum ROUGE-L score is 0.3784.

[0128] Because different problems in the field of astronomical calculations inevitably contain technical terms and standard physics question formats, the maximum similarity score is still far below the empirical threshold for judging text copying in natural language processing.

[0129] The above quantitative indicators show that the data generated by the concept graph-based astronomical computation inference instruction enhancement method of this invention has a low degree of overlap with the test set, that is, there is no data leakage.

[0130] This step can effectively avoid duplicate content, enabling the model to generate new ways of expressing and structureing questions, thereby improving the innovation and diversity of generated questions.

[0131] (3) Mind chain distillation:

[0132] Existing research indicates that large model knowledge distillation has become an effective paradigm for obtaining high-quality synthetic data, and DeepSeek-R1 performs well for complex mathematical reasoning tasks.

[0133] Therefore, this invention uses this model to train the teacher model, enabling it to generate a complete intermediate reasoning process when solving complex computational problems, thereby transferring its logical reasoning ability to a small open-source model.

[0134] In this embodiment, the training and evaluation data used comes from the AstroCalcBench dataset independently constructed by this invention, wherein the original dataset is divided into a training set containing 1395 samples and a test set containing 457 samples.

[0135] Specifically, the raw data comes from NASA Space Math. The acquired raw data undergoes preprocessing, and the specific steps are as follows:

[0136] The raw data is mostly unstructured PDF documents or web page text. In order to construct a training set suitable for fine-tuning large language model instructions, this invention performs the following cleaning and standardization processing on the raw data:

[0137] First, the format is converted from unstructured text to a unified JSON format, which includes the question description, reasoning steps, and final answer.

[0138] Secondly, data cleaning was performed to remove samples that relied on charts or had vague descriptions.

[0139] Using the processed data as seed questions, and employing the concept graph-based astronomical computational reasoning instruction enhancement method proposed in this invention, a training set of approximately 10,000 items was finally constructed through random walks and combinations of concept graphs.

[0140] In one feasible embodiment, in order to verify the effectiveness of the method of the present invention in improving the astronomical computational reasoning ability of large language models, a comparative experiment was conducted using an astronomical computational benchmark dataset as the evaluation benchmark.

[0141] The experimental subjects were base models of different sizes and comparative models after various fine-tuning methods. As shown in Table 1, the concept graph-based astronomical computational reasoning instruction enhancement method achieved a quantitative performance improvement that was superior to the traditional simple fine-tuning and basic knowledge injection paradigm on backbone models with parameter scales ranging from 8B to 32B. This enhanced the computational accuracy and final answer validity of the model when performing long logical chain deductions.

[0142] Based on this, the effectiveness gain of this structured knowledge enhancement strategy shows a positive expansion trend without reaching saturation as the model capacity expands, objectively confirming the potential of a large parameter space to represent the deeper internalization of high-dimensional complex concept networks. This establishes a logically rigorous and highly reusable technical evolution path for comprehensively breaking through the underlying bottleneck of the computational reasoning dimension of the astronomical vertical large language model.

[0143] Overall, the concept graph-based method for enhancing astronomical computational reasoning instructions provides an effective path for the field of astronomical computing to enhance the capabilities of large language models through structured knowledge constructed using concept graphs, thereby improving reasoning and question-answering performance.

[0144]

[0145] In one feasible embodiment, in order to explore the specific role of each key component in the method of the present invention in the final performance of the model, an ablation experiment was designed and conducted in this embodiment.

[0146] Using LLaMA-3.1-8B as the backbone model, approximately 1.5k seed problems were used in the basic setup of the concept extraction stage.

[0147] After removing 50% of the seed data, the model's overall score dropped from 33.36 to 32.04, a relative decrease of 3.92%, demonstrating the scale effect of the seed problem: a larger seed dataset can support the model in mining richer and broader combinations of astronomical concepts, thus affecting the final training effect.

[0148] Secondly, the present invention sets the number of walking steps to 4, 6 and 8 steps respectively. As the number of walking steps increases from 4 to 8, the model performance shows an upward trend.

[0149] When the number of walk steps is set to 4, the model's total score drops to a minimum of 31.01, a decrease of 7.04% relative to the benchmark method. This indicates that a longer walk path is crucial for capturing deep knowledge connections in the concept graph, while a path that is too short cannot construct training samples with high logical density.

[0150] It is worth noting that when the number of steps is set to 8, its performance is still lower than the baseline strategy of the complete present invention. This indicates that the dynamic step size strategy adopted by the baseline method can better simulate the flexibility of human thinking compared to a single fixed step size, and further enriches the diversity of training data by generating inference paths of varying lengths.

[0151] The technical terms, principles, or means related to the technical solutions of the present invention mentioned in the above embodiments, which are not described in detail above, are all well-known technologies or common practices that are known to those skilled in the art.

[0152] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for enhancing astronomical computational reasoning instructions based on concept maps, characterized in that, Includes the following steps: We obtain seed questions labeled with subdomains and topics as basic input, use a large language model to extract knowledge points from the seed questions, build a knowledge framework covering subdomains, topics and knowledge points, and transform problem instances into conceptual representations. A concept graph is constructed based on a knowledge framework. The nodes of the concept graph consist of a set of topics and a set of knowledge points. The concept graph has three types of edges: edges from topic to topic, edges from topic to knowledge point, and edges from knowledge point to knowledge point. The weight of the edges is obtained by counting their co-occurrence frequency. A dynamic random walk algorithm is executed on the concept graph. The starting topic node is obtained on the topic set and walked. After completion, the knowledge point subgraph is walked. Finally, a complete concept combination containing the target topic set and the target knowledge point set is output. Based on the complete concept combination, the large language model is guided to generate astronomical computation problems and corresponding reasoning processes by performing example matching, data deduplication, and mind chain distillation.

2. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 1, characterized in that, The steps for constructing the concept map are as follows: If two nodes appear together in the seed problem of the same problem, then an edge is created between the two nodes; The formal definition of a boundary set is: ； in, Describe the set of edges. This represents two distinct nodes. Indicates the connection node and Edge weights.

3. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 1, characterized in that, The process of obtaining the edge weights involves performing a logarithmic transformation on the co-occurrence counts, as follows: ； in, Indicates the connection node and Edge weights; Represents a node and The number of times each seed problem co-occurs; It represents a positive number and is used to avoid the anomaly of zero values ​​in logarithmic calculations; Represents a logarithmic function; This represents two distinct nodes.

4. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 1, characterized in that, The concept map can be divided into three types of sub-maps: topic sub-maps, topic-knowledge point association sub-maps, and knowledge point sub-maps.

5. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 1, characterized in that, The execution steps for the complete concept combination output are as follows: Uniform random sampling is performed on the topic set to obtain the initial topic nodes; The topic subgraph walk starts from the topic node and initiates a random walk within the topic subgraph. The probability of a node moving to another node is dynamically calculated based on the similarity weight between nodes. After completing the topic subgraph walk, a single-step random walk is used to transition from the topic domain to the knowledge point domain, and a preliminary knowledge point set is generated through probability distribution rules. Continue performing random walks of a preset execution length on the knowledge point subgraph, and output complete concept combinations by mining adjacent or related knowledge points.

6. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 5, characterized in that, The specific formula for the probability distribution rule is as follows: ； in, This represents the probability of moving from one node to another during a random walk in a topic subgraph or knowledge point subgraph. Indicates the connection node and Edge weights; This represents two distinct nodes; Indicate neighboring nodes; This represents the relationship between nodes in a topic subgraph or knowledge point subgraph. The set of neighboring nodes.

7. The method for enhancing astronomical computational reasoning instructions based on concept graphs according to claim 1, characterized in that, The execution steps for the example matching are as follows: The Jaccard similarity coefficient is used to calculate the overlap between the seed problem and the complete concept combination on the set of knowledge points. Instance matching is achieved by retrieving several instances that have the highest semantic relevance to the target as context; The data deduplication process is as follows: A similarity-based text filtering mechanism is introduced during data generation to remove generated samples that are similar to the original test set, thereby completing data deduplication, and natural language processing is used for evaluation. The thought chain distillation involves using a large language model as the teacher model, training it to generate the complete intermediate reasoning process for astronomical calculation problems, and transferring logical capabilities to a small open-source model.

8. A concept-map-based astronomical computational inference instruction enhancement system for implementing the method of any one of claims 1-7, characterized in that, include: The concept acquisition module is used to construct a training set of astronomical computing benchmarks and obtain seed problems from it. The concept extraction module is used to automatically extract conceptual knowledge points from seed questions and build a knowledge framework by combining a large language model with prompting engineering. The concept map building module is used to further construct concept maps by building knowledge frameworks; The concept composition module is used to perform a dynamic random walk algorithm on the concept graph to obtain complete concept compositions; The astronomical computational reasoning data generation module is used to guide the generation of astronomical computational problems based on complete concept combinations and large language models.

9. An electronic device, comprising: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The feature is that the memory stores a computer program, which, when executed by the processor, causes the processor to execute the concept graph-based astronomical computation inference instruction enhancement method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the concept-map-based astronomical computational reasoning instruction enhancement method as described in any one of claims 1-7.

Images