Geological map understanding method combining knowledge guidance and few-sample learning

Abstract

The invention provides a geological map understanding method combining knowledge guidance and few-sample learning, and relates to the field of geological maps, comprising the following steps: constructing a data set composed of geological maps containing legend; constructing an expert knowledge base composed of geological experts and a geological legend sample set as a domain knowledge guide large model; adding a local visual enhancement prompt to the legend of each geological map in the data set; constructing prompts for a geological map understanding task, wherein the prompts comprise few-sample learning example guidance, result output specifications and the like; and constructing a feedback mechanism, feeding back the map with low accuracy of the output result of the large model, and returning to the prompt design step to redesign prompt. Through a method from local understanding to global understanding, domain knowledge is provided for a large model as assistance, understanding of legend in the map is enhanced, the problems of insufficient universality of the large model and dependence on large-scale annotation data are effectively relieved, and the efficiency and accuracy of geological map understanding are improved.

Description

Technical Field

[0001] This invention relates to the field of geological mapping, and more particularly to the field of multimodal technology. Specifically, it relates to a method for understanding geological maps using joint knowledge guidance and few-shot learning. Background Technology

[0002] Geological maps are a core data representation in geological exploration and research, containing rich information on rocks, minerals, stratigraphy, and geographical location. However, most geological maps contain complex symbols, spatial structures, and semantic information about geological objects, making automatic understanding and extraction challenging. Furthermore, geological maps possess multi-scale and multi-modal characteristics (symbols, colors, textures, etc.), requiring the overcoming of issues such as scarce labeled data and ambiguity in technical terminology. Simultaneously, complex spatial relationships of geological bodies (such as fault cutting and stratigraphic contact) demand strong reasoning capabilities from the model, while the ambiguity and regional differences of geological phenomena further increase the challenges of automated identification. With the rapid development of artificial intelligence and large-scale models, large-scale models and prompting engineering provide new research avenues for accurately extracting geological information from maps. Geological maps consist of various information such as stratigraphic units, structural features, surface morphology, symbols, and annotations. Human understanding not only requires high levels of geological expertise and reasoning ability but also needs to cope with the visual clutter caused by information density, and requires additional time and tool support.

[0003] In recent years, with the development of multimodal large models, many scholars have begun to apply them for information extraction from maps. Nevertheless, research on geological map understanding remains insufficient. While large models based on prompting engineering can analyze deep information in images, their recognition accuracy in specialized fields is low due to insufficient domain knowledge and data bias. When the GPT-4o model only interprets the legend in geological maps, the error rate reaches approximately 50% in two tasks, indicating weak local understanding capabilities. Therefore, simply using the GPT-4o model directly to interpret geological maps cannot effectively extract accurate geological knowledge.

[0004] Therefore, this invention proposes a Geological Map Understanding Method (GMUKF) combining knowledge guidance and few-shot learning. This method divides the task into four key steps: First, it injects a geological knowledge base into the GPT-4o model, combining geological expert knowledge and a sample set of geological map legends to provide a professional background for the large model, thus compensating for its lack of generality. Second, it incorporates a visual cueing mechanism, enhancing visual cues through legends in the map to achieve local understanding of the map. Third, it designs cue templates according to task requirements, combining few-shot learning to select a small number of examples as training samples to improve the standardization of the large model's output. Finally, it uses the BGE-large-zh-v1.5 model to evaluate the predicted text; if the accuracy is too low, a feedback mechanism returns to the cue design step to redesign more suitable cue, reducing the error rate in the predicted results. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a geological map understanding method that combines knowledge guidance and few-sample learning, which solves the problem of the lack of geological knowledge in existing large models, can provide professional support for the automatic parsing and understanding of geological maps, and enhance the reasoning ability of the model in geological tasks.

[0006] In a preferred embodiment, this invention provides a geological map understanding framework that combines knowledge guidance and few-shot learning. This framework comprises four modules: domain knowledge injection, visual cues mechanism, cues template design, and result output and evaluation. Its core method progresses from a local understanding of the legend to a global understanding of the main map. On the constructed geological map dataset, examples are introduced as training samples in the cues design to guide the large model to imitate and reason. Combined with the integration of visual cues, key local information in the geological map is annotated, effectively improving the ability to understand the geological map.

[0007] In a preferred embodiment, the present invention further provides a domain knowledge injection module, which is a key step in improving the understanding of geological maps. This module aims to provide structured geological background knowledge for large models, thereby effectively compensating for their lack of versatility. This process constructs a high-quality expert knowledge base through geological experts and a sample set of geological map legends, providing professional support for the automatic parsing and understanding of geological maps and enhancing the model's reasoning ability in geological tasks.

[0008] In a preferred embodiment, the present invention further provides a visual cueing mechanism module. This module helps the model focus on local knowledge and then extend to a global understanding by marking key areas in the map, thereby improving its understanding and reasoning capabilities in complex visual tasks. In a geological map understanding task, firstly, key areas in the map, such as geological legends and faults, are identified according to task requirements. Then, these areas are highlighted using intuitive markers (such as red boxes) to help the model focus on this specific visual information.

[0009] In a preferred embodiment, the present invention further provides a prompt template module. In the geological map understanding task, the task instructions are designed to guide the model to accurately extract geological entities and their relationships in the map and generate structured text. To ensure that the model effectively performs this task, the construction of the task instructions follows these steps: First, clarify the task objective, namely, to guide the model to identify geological entities (such as rock layers, minerals, stratigraphic ages, etc.) and their interrelationships (such as contact, superposition, penetration, etc.) in the map and output a structured description that conforms to geological standards; second, define the input content and expected output. The input is a geological map with prompt markers, and the output is structured text describing geological entities and their relationships, such as mineral composition, rock age, etc., in the format: "Entity Name: Entity Representation: ..., Relationship: ..."; then, set task constraints to ensure that the output conforms to the standard format for easy subsequent analysis; finally, use a few sample examples to help the model understand the task requirements.

[0010] In a preferred embodiment, the present invention further provides an output and evaluation module, which includes three parts: 1) Outputting structured results: Based on the first three steps, the result output stage combines domain knowledge guidance, visual cues enhancement, and few-shot learning to train the model to generate the expected results. 2) Result evaluation: In the result evaluation stage, experts write standard reference text for each geological map. The BGE-large-zh-v1.5 model converts the model's predicted text and the standard reference text into embedded vectors, and evaluates the semantic similarity between the two as the accuracy. 3) Feedback mechanism: The feedback mechanism optimizes the quality of the generated description, ensuring that the final output meets high standards. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This invention is a geological map understanding structure diagram that combines knowledge guidance and few-sample learning; Figure 2 This is a geological legend marking rule diagram of the present invention; Figure 3 This is a small-sample hint structure diagram for understanding geological maps in this invention; Figure 4This is a semantic similarity score diagram of the understanding of geological maps by different model combinations of the present invention; Figure 5 This is a graph showing the accuracy of the large model of this invention in understanding geological maps under six different marker types; Figure 6 This is a comparison chart of the accuracy of different prompt mark positions in this invention; Figure 7 This is a diagram showing the impact of different module combinations on the overall performance of this invention. Detailed Implementation

[0012] To better understand the purpose, system architecture, and functional implementation of this embodiment, the embodiments and features described herein can be combined with each other without conflict. The exemplary embodiments disclosed herein will be described below with reference to the accompanying drawings, which include specific technical details disclosed in this embodiment to aid understanding; however, these details should be considered exemplary rather than restrictive. Therefore, those skilled in the art should understand that various improvements and adjustments can be made to the embodiments described herein without departing from the scope and core ideas of the invention.

[0013] Example 1 Figure 1 This is a geological map understanding structure diagram based on a combined knowledge-guided and few-sample learning approach according to an embodiment of the present invention.

[0014] like Figure 1 As shown, the geological map understanding method 100, which combines knowledge guidance and few-shot learning, may include S110~S140.

[0015] In operation S110, domain knowledge is injected, consisting of an expert knowledge base constructed from geological experts and a set of legend samples.

[0016] When operating S120, the visual cueing mechanism enhances the attention of legends in geological maps by marking them with red boxes.

[0017] In operation S130, the prompt template design includes the writing of task instructions, and the prompts are used as training samples through example input and output.

[0018] In operation S140, the results are output and evaluated. This module is responsible for evaluating the results generated by the first two modules.

[0019] According to an embodiment of the present invention, in operation S110, the domain knowledge injection module provides external knowledge to the large model by constructing an expert knowledge base. This mainly consists of two steps: First, a dataset is constructed by selecting geological maps containing legends from geological reports and the Baidu database, and a geological map legend sample set is built, consisting of the "Regional Geological Map Legend" (GB / T958-2015) standard and approximately 6,000 legends from geological maps of 29 provinces and cities, thus enriching the domain knowledge.

[0020] Obtain multiple regional geological report documents , Let m represent the i-th geological report document, and m be the number of reports.

[0021] From each geological report document Geological maps containing legends are selected and collected using the following formula (1).

[0022] (1) in, Indicates from the report document The selected legend contains a collection of geological maps, and each geological map is... , where j is the report The number of geological maps that include legends; Several geological maps were collected from Baidu's database. The number of geological maps is m, and the formula is shown in equation (2) below.

[0023] (2) By all and The maps in the collection form the final geological map dataset. The formula is shown in equation (3) below.

[0024] (3) in, It is the set of geological maps with legends selected from the i-th geological report document. This is a collection of geological maps gathered from the Baidu database, where m represents the total number of regional geological report documents. Geological experts, based on their professional knowledge and experience, provide information such as lithological composition, legends, and sequence age tables, and construct expert knowledge bases. The formula is shown in equation (4) below.

[0025] (4) in, Indicates expert knowledge, This represents the i-th geological map. Symbols related to the legend (such as symbols for different lithologies, geological structures, etc.). The color represents the color associated with the legend (e.g., the color of different geological units), and n represents the size of the legend sample set, i.e., the total number of samples. A sample set of geological map legends was constructed, consisting of the "Regional Geological Map Legends" standard (GB / T 958-2015) and geological map legends from 29 provinces and cities, enriching the knowledge in the field. The formula is shown in the following equation (5): (5) in, This represents a sample set of legends constructed based on the "Regional Geological Map Legend" (GB / T 958-2015) standard. This represents a sample set constructed based on the legends of geological maps from 29 provinces and municipalities. It is the complete set of legend samples formed by the union of these two sets; The constructed domain knowledge is injected into the large model to provide expertise, as shown in the formula (6) below.

[0026] (6) in, It is the basic large model. It is a model that has been injected with domain knowledge. It is an injection operation.

[0027] Secondly, based on the input question, experts conduct a preliminary analysis using their experience. Considering factors such as the type of question, the geological phenomena involved, and the required level of expertise, they determine which type of knowledge is needed to answer the question. GPT-4o automatically retrieves the expert knowledge base and uses the necessary external knowledge, enabling the model to generate more accurate and professional answers, thereby improving its performance in geological map understanding tasks.

[0028] According to an embodiment of the present invention, in operation S120, the visual cueing mechanism module helps the model focus on local knowledge and then extend to the understanding of the whole by marking key areas in the map, thereby improving its understanding and reasoning ability in complex visual tasks. In the geological map understanding task, the key areas in the map, such as geological legends and faults, are first determined according to the task requirements. Then, these areas are highlighted with intuitive red boxes to help the model focus on these specific visual information. The image annotation tool is used to mark each legend of each geological map and its corresponding text description with red boxes as a whole, which serves as a visual cue for model training. The annotation of the red boxes is shown in the following formula (7).

[0029] (7) in, These are the coordinates of the i-th red box. These are the coordinates of the top-left corner of the box. The coordinates of the lower right corner of the box are used. The model is trained using images enhanced with visual cues (geological map images with red boxes). Each image is used as input, and the red box visual cue is used as the output target. The loss function for training is shown in Equation (8) below: (8) in, It is L2 loss. These are the predicted bounding box coordinates. These are the actual bounding box coordinates. After training, the model will automatically perform inference on new geological maps, identify and label legend areas, and generate visual cue markers.

[0030] like Figure 2 As shown, geological legends are laid out in three categories: horizontal, vertical, and intersecting. Each geological legend and its corresponding text description need to be labeled as a single unit. Legends are the parts of an image that explain the meaning of symbols, colors, lines, etc. Labeling legends helps the model better understand the specific meaning of each element in the image and associates legend data with the spatial location of the main map, thereby improving the overall understanding of the image. Legends are usually located at the edge of the image or in a dedicated area. Labeling legends does not interfere with the main content of the image, reducing visual distractions and annotation workload, and helping the model focus more on the main map information.

[0031] In an embodiment of the present invention, few-shot example-based instructions are employed in operation S130. Through these task instructions, the model can comprehensively understand geological maps and output structured information, providing support for subsequent reasoning and decision-making. Few-shot example construction is a key technology in few-shot learning, aiming to generate high-quality training examples from limited labeled samples to improve the model's generalization ability in data-scarce scenarios. This invention employs few-shot example construction technology, providing limited examples to guide large multimodal models such as GPT-4o to understand the task and generate expected outputs. Few-shot prompts consist of task instructions and input-output pairs, where examples are manually written and contain task-related questions and knowledge statements. The model infers task rules from the examples in the prompts without adjusting parameters. In this invention, as... Figure 3As shown, the constructed few-sample examples include task instructions and exemplary input-output pairs to standardize the model output format. For example, the output is required to describe the geological legend content in strips, such as "1. Sedimentary rocks: ..., 2. Basalt: ...", and provide detailed descriptions of the legends within the boxed markers, such as "Limestone: brownish-yellow background, brick-shaped representations...", as well as the spatial relationships of each legend in the main map, such as "Sedimentary rocks are located above basalt, adjacent to diabase and basalt...". This formatting requirement helps in the subsequent evaluation of the accuracy of the results.

[0032] In an embodiment of the present invention, in operation S140, the output and evaluation module includes three parts: 1) Outputting structured results: The result output stage, based on the first three steps, combines domain knowledge guidance, visual cue enhancement, and few-shot learning to train the model to generate expected results. In this stage, it is stipulated that the representation method of the geological map legend should be described first, transitioning from understanding the local legend to understanding the global main map, describing the positional relationship of each legend in the main map, etc. The structured results are output according to the task instructions and saved as the model prediction text. 2) Result evaluation: In the result evaluation stage, experts write standard reference text for each geological map. The BGE-large-zh-v1.5 model converts the model prediction text and the standard reference text into embedding vectors and evaluates the semantic similarity between the two as the accuracy. For example, text 1 "Sedimentary rocks are usually layered and mostly gray or brown in color." and text 2 "Sedimentary rocks are usually layered and mostly gray or brown in color." are converted into embedding vectors by the BGE-large-zh-v1.5 model, and the calculated cosine similarity is 0.96, indicating that the two texts are highly similar in content and expression. 3) Feedback mechanism: The role of the feedback mechanism is to optimize the quality of description generation and ensure that the final output meets high standards.

[0033] In this invention, a triple verification mechanism is established. If the reference text for the corresponding image is missing or the generated semantic similarity is below 0.8, resulting in a description quality that does not meet the standard, the system will allow three regenerations. If the requirements are still not met, the system will skip the semantic scoring process for that image and record it in the skip list. At this point, the system will return to the prompt design module to adjust the prompt template or reference text to ensure that the generated description meets the expected effect. This mechanism ensures that only high-quality descriptions are saved and avoids low-quality or erroneous output. The feedback mechanism provides users with clear prompts, indicating that the generated text does not meet the quality standard and may require optimization of the prompt template or reference text, thereby improving the output quality and reliability of the model.

[0034] Since the results of this invention are structured text, and the predicted descriptions of geological maps generated by the large model may be inconsistent each time, a semantic similarity evaluation method is chosen to measure the accuracy of the generated results. Specifically, the semantic similarity score between the predicted text output by the large model and the reference standard text is used as the evaluation standard for accuracy. The selected evaluation model maps the text to a dense vector in a high-dimensional embedding space to capture the semantic information of the text. By converting the predicted text and the reference text into fixed-dimensional embedding vectors and calculating the cosine similarity (CS) between them, the semantic similarity score can be obtained, as shown in Equation (9) below.

[0035] (9) in It is represented as cosine similarity, with a value of [0,1]. The closer the value is to 1, the more similar the texts are. and These represent the predicted text vector and the reference text vector generated by the model, respectively. Represented as a dot product, and They are vectors and In this invention, the semantic similarity score is used as the evaluation standard, namely the accuracy.

[0036] The large language models used in the experiments of this invention include GPT-4o, GPT-4-Vision-Preview, Text-Embedding-3-Large, Text-Embedding-3-Small, and BGE-large-zh-v1.5. GPT-4o, as an optimized version of GPT-4, can generate high-quality image descriptions and inferences by combining images and text through cross-modal learning. It is particularly suitable for handling the combination of complex visual information and text content, and efficiently completes image tasks with few-shot learning. GPT-4-Vision-Preview is a specially optimized visual enhancement version with the ability to directly process image input, supporting complex image parsing, image description generation, and visual question answering tasks. It is suitable for a wide range of image analysis applications; therefore, this invention will compare the performance of these two models in geological map understanding.

[0037] Text-Embedding-3-Large, as a large-scale model, possesses stronger semantic understanding capabilities and higher accuracy, making it suitable for complex cross-domain semantic similarity tasks. Text-Embedding-3-Small, on the other hand, is smaller in scale, computationally efficient, and can achieve rapid response in low-resource environments. The BGE-large-zh-v1.5 model is currently the preferred model in the field of Chinese semantic matching and embedding, suitable for most Chinese RAG and semantic matching scenarios. Therefore, these three models were selected for semantic similarity evaluation in the experiments of this invention.

[0038] To explore the impact of different model combinations on the understanding of geological maps, this invention selects GPT-4o and GPT-4-Vision-Preview as baseline models. These two models support image input and analysis. In the evaluation phase, semantic similarity is used as the evaluation criterion. The key is to convert text into embedding vectors and calculate cosine similarity. Therefore, Text-Embedding-3-Large, Text-Embedding-3-Small, and BGE-large-zh-v1.5 are selected as text embedding models, with a unified embedding dimension of 1024.

[0039] The two-stage models were combined into six model sets: g4o+blzv representing the combination of GPT-4o and BGE-large-zh-v1.5; g4o+te3l representing the combination of GPT-4o and Text-Embedding-3-Large; g4o+te3s representing the combination of GPT-4o and Text-Embedding-3-Small; g4vp+blzv representing the combination of GPT-4-Vision-Preview and BGE-large-zh-v1.5; g4vp+te3l representing the combination of GPT-4-Vision-Preview and Text-Embedding-3-Large; and g4vp+te3s representing the combination of GPT-4-Vision-Preview and Text-Embedding-3-Small. Experiments were conducted on these six model sets, using semantic similarity as the evaluation metric, to analyze their impact on geological map recognition. The results are as follows: Figure 4 As shown.

[0040] from Figure 4It can be seen that: 1) Although both GPT-4-Vision-Preview and GPT-4o are suitable for image parsing tasks, GPT-4o outperforms GPT-4-Vision-Preview in overall performance when combined with the evaluation model; 2) Among the two combinations with GPT-4-Vision-Preview and GPT-4o, the BGE-large-zh-v1.5 model with a unified embedding dimension of 1024 shows higher accuracy than the other two embedding models; 3) As the model combination changes from g4vp+te3s to g4o+blzv, the accuracy gradually increases from 0.8837 to 0.9354, and GPT-4o combined with BGE-large-zh-v1.5 achieves the highest performance accuracy of 93.54%. Therefore, based on the experimental results of this invention, the following experiments will select g4o+blzv as the final experimental model combination.

[0041] This invention studies a geological map understanding method constructed by combining domain knowledge injection, local visual enhancement red box prompts, and a few-shot learning module. GPT-4o is used as the geological map understanding model, and BGE-large-zh-v1.5 is used as the result evaluation model. To evaluate the effectiveness of the proposed method, it was compared with traditional deep learning image recognition methods. The following methods were selected: ResNet-50 (Residual Network), DenseNet-121 (Densely Connected Convolutional Network-121), Inception-ResNet-v2 (Inception-Residual Network version 2), EfficientNet-B0 and EfficientNet-B7 (Efficient Networks) optimized with composite scaling methods, ViT-Huge (Vision Transformer-Huge) based model, and Swin Transformer (Shifted Window Transformer-Base) with hierarchical feature extraction structure and local window attention mechanism. The method was also compared with classic text recognition methods PP-OCRv3 and PP-ChatOCRv4. Top-1 accuracy and recall@1 were used as metrics for accuracy in the experiments. The experimental results are shown in Table 1. It can be seen that the proposed geological map understanding method (Geological Map Understanding with joint knowledge guidance and...) is effective. Few-shot learning (GMUKF) significantly outperforms traditional deep learning methods, achieving an accuracy of 93.54%. Bold data in the table represents the best results, while underlined data represents the second-best results.

[0042] Table 1. Performance Comparison of Geological Map Understanding Methods and Deep Learning Methods for Geological Map Understanding Tasks

[0043] The GMUKF method improves accuracy by approximately 5% compared to the ViT-Huge method. Unlike the uninterpretable feature classification of deep learning models, the GMUKF method exhibits stronger generalization ability and interpretability. Cueing methods based on few-shot learning can effectively integrate external geological knowledge with large multimodal models, transferring the learning capabilities of large models to downstream tasks. Visual cueing methods simplify map interpretation and guide models to effectively decompose complex problems at the visual level.

[0044] The ablation experiments of this invention compared the effects of marking symbols, marking positions, and modules. In the visual cueing mechanism, to study the impact of different marking symbols on the understanding performance of geological maps, six different marking methods were used: red circles, red squares, red checkmarks, red triangles, red crosses, and no marking. Semantic similarity was used as the evaluation index, and the results are as follows: Figure 5 As shown.

[0045] from Figure 5 The results show that: 1) Marked images demonstrate significantly better understanding performance than unmarked images, indicating that adding specific markers (such as boxes, circles, etc.) enhances local information recognition and improves the model's global understanding ability. 2) Box marking significantly improves the model's understanding ability by clearly defining regions and key content, outperforming other marking methods. Boxes can completely "enclose" the local information that needs to be marked, while other markers are prone to "going out of bounds" or occlusion, leading to unclear region division and causing understanding interference. The performance of box marking reaches 0.9354, an improvement of approximately 33% compared to unmarked prompts.

[0046] This invention proposes using red boxes as cues only for the legend in geological maps. To compare the impact of the cue location on map comprehension tasks, a geological map was randomly selected, and experiments were conducted comparing its legend, main map, and map without cues. The results are as follows: Figure 6 As shown.

[0047] from Figure 6 It can be seen that: 1) When only the legend is labeled, the generated structured description has the highest accuracy, reaching 91.53%, which is 8.76% higher than without labeling. This result indicates that the legend, as auxiliary information of geological images, can provide sufficient context for the model, thus effectively helping it to accurately interpret the geological content in the main map. 2) When the legend and its description are labeled in the main map, the accuracy is the lowest, at only 79.78%, which is 2.99% lower than without labeling. The results show that labeling the legend and its description in the main map may interfere with the model's analysis of the legend, thereby affecting its ability to understand the main map.

[0048] The research methodology employed in this study is localized enhanced visual cues (marked with red boxes) + few-shot learning + domain knowledge injection. To explore the contribution of each module to the experimental results, the performance of each module was evaluated on a geological map dataset, and the results are shown in Table 2 and... Figure 7 As shown in the figure. The baseline model represents no cueing strategy; 1 represents only adding a visual cueing module; 2 represents only using a few-shot learning strategy; 3 represents only adding a domain knowledge injection module; 1+2, 1+3, and 2+3 represent module combinations. Bold data represents the best results, and underlined data represents the second-best. From... Figure 7 It can be seen that the addition of modules 1, 2, and 3 all improved the performance of the baseline model, with module 2 showing a particularly significant improvement. Any combination of two modules further enhances performance compared to a single module, validating the effectiveness of the three-module design.

[0049] Table 2 Impact of different module combinations on overall performance (1: marked with red boxes)

[0050] Table 2 shows that: 1) Compared to the baseline model, the accuracy gradually improves with the addition of each module, indicating that each module has a positive impact on the overall method; 2) When only visual cues are added, the accuracy is improved by approximately 25% compared to the baseline method, indicating that this method can guide the large model to focus on local information, help expand to overall knowledge, analyze geological legends from a fine-grained visual level, and thus understand the overall map; 3) When only the few-shot learning method is added, the accuracy is improved by 27.5% compared to the baseline method, indicating that this method can efficiently adapt to data scarcity and task complexity by utilizing pre-trained knowledge and domain knowledge, while the baseline model's performance is limited due to lack of training; 4) When only the domain knowledge injection module is added, the accuracy is improved by 25.2% compared to the baseline method, indicating that injecting geological domain knowledge into the large model as external knowledge assistance can effectively improve the large model's domain understanding ability. After combining the three modules, the experimental accuracy reached the optimal level of 93.54%, which is an improvement of approximately 33% compared to the basic model.

[0051] Table 3 Impact of different module combinations on overall performance (1: marked with red circles)

[0052] Furthermore, in Table 3, this invention replaces Module 1 with a red circle cue marker to explore its impact on overall performance. Experimental results show that the accuracy of the red circle cue marker is improved by 29.3% compared to the baseline model, but this improvement is lower than that of the red square cue marker. Moreover, the accuracies of Modules 1+2, 1+3, and 1+2+3 are all lower than the corresponding accuracies of the red square cue marker. Therefore, the GMUKF method, which combines domain knowledge injection, red square visual cues, and few-shot learning, performs best in geological map understanding tasks.

[0053] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A geological map understanding method combining knowledge guidance and few-shot learning, characterized in that, Includes the following steps: S1. Construct an expert knowledge base by using geological expert knowledge and geological map legend sample sets, and inject structured geological knowledge into the multimodal large model; S2. Construct a visual cueing mechanism to enhance visual attention to the legend of each geological map in the dataset, marking it with a red box to improve the model's understanding of the local area; S3. Construct prompts for geological map understanding tasks, including few-shot learning example guidance and result output specifications, so as to obtain a descriptive output of geological map understanding; S4. Construct a feedback mechanism: Based on the above steps, generate structured text results, use a semantic embedding model to calculate the semantic similarity between the structured text results and the expert reference text as the accuracy rate, and return to the prompt design step to redesign the prompts for those with low accuracy rates in the large model output results to achieve the expected results.

2. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The construction of the expert knowledge base in step S1 includes: A dataset was constructed by selecting geological maps with legends from geological reports and the Baidu database; Integrating the "Regional Geological Map Legend" standard (GB / T 958-2015) and sample sets of geological legends from 29 provinces and municipalities; The system dynamically retrieves and calls relevant knowledge base content based on the type of question input.

3. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The structured geological description in step S1 includes: The colors and textures of the symbols in the illustrations indicate their meaning; The spatial topological relationships of various geological bodies in the main image; Description of geological processes involving fault cutting and stratigraphic contact.

4. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The multimodal large model mentioned in step S1 is GPT-4o. It receives geological maps marked with red boxes, combines the knowledge base content to parse the semantics of the map legend, generates structured text descriptions based on a few sample examples, and parses the domain knowledge related to the task.

5. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The key area mentioned in step S2 includes geological legends and faults. The preset marking symbol is a red box. The red box is used to mark the legend and its corresponding text description in one piece, and different marking rules are adapted for legends that are arranged horizontally, vertically, or crosswise.

6. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The construction of the task instructions in step S3 includes: clearly identifying the target as geological entities and their interrelationships, wherein the geological entities include rock layers, minerals, and stratigraphic ages, and the interrelationships include contact, superposition, and penetration; defining the input as a marked geological map and the output as structured text in the format "entity name: entity representation: ..., relationship: ...".

7. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The few-sample example input-output pair mentioned in step S3 includes: examples of striped descriptions of legend content, examples of detailed descriptions of marked areas, and examples of descriptions of spatial relationships of geological entities in the main map.

8. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The semantic embedding model mentioned in step S4 is the BGE-large-zh-v1.5 model. The semantic similarity is obtained by calculating the cosine similarity between the predicted text vector and the reference text vector. The formula for calculating the cosine similarity is shown in the following formula (1): (1) in, It is represented as cosine similarity, with a value of [0,1]. The closer the value is to 1, the more similar the texts are. and These represent the predicted text vector and the reference text vector generated by the model, respectively. Represented as a dot product, and They are vectors and In the model, the semantic similarity score was used as the evaluation standard, which is the accuracy.

9. The geological map understanding method based on joint knowledge guidance and few-shot learning according to claim 1, characterized in that, The logic for generating the structured text results in step S4 is as follows: first, describe the representation of the legend, and then extend the description of the spatial positional relationship of geological entities in the main map based on the legend information, so as to achieve the transition from local understanding to global understanding.

10. A method for combining models for understanding geological maps, characterized in that, Includes the following steps: The GPT-4o model was used as the geological map interpretation model. The BGE-large-zh-v1.5 model was adopted as the semantic similarity evaluation model. By combining the above models, structured text generation and accuracy assessment of geological maps can be achieved, with the highest accuracy rate of the model combination reaching 93.54%.

Images