Method for venue knowledge extraction based on apg adaptive prompt generation

Abstract

The application relates to the field of artificial intelligence and natural language processing technology, and provides a venue knowledge extraction method based on APG adaptive prompt generation, which comprises the following steps: inputting initialization parameters, including an initial prompt for a knowledge extraction task, a training text dataset containing standard entity and relationship annotation, an optimizer large language model, and an executor large language model; performing an APG iterative optimization cycle; if an iteration termination condition is met, the iteration is terminated, and a candidate prompt with the highest current performance score is output as an optimized prompt; if the iteration termination condition is not met, the iteration is continued; and the optimized prompt is applied to an unstructured text corpus of a target venue, full-quantity knowledge extraction is performed by an LLM-Executor, and a domain knowledge graph of the target venue is constructed. The application solves the core problems that artificial design static prompt templates have poor generalization ability and adaptability, and are low in engineering efficiency and difficult to obtain an optimal solution.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a method for extracting venue knowledge based on APG adaptive prompting. Background Technology

[0002] Building high-quality domain knowledge graphs is the cornerstone of realizing smart venue services. The key lies in accurately and efficiently extracting entities, attributes, and relationships from massive amounts of unstructured text. Current mainstream methods rely on pre-trained large language models combined with manually designed prompts for extraction, but they still face significant challenges when applied to highly specialized venue domains.

[0003] First, manually designed static cue templates suffer from severe deficiencies in generalization ability and domain adaptability. Venue texts typically contain a large number of technical terms, complex backgrounds, and diverse expressions. Fixed cue templates struggle to fully cover these complexities, resulting in low accuracy and recall rates for extraction results. In particular, traditional methods are almost ineffective in capturing implicit semantic relationships such as symbolism, reflection, and embodiment, which carry deep cultural and craft logic. The extraction quality falls far short of the requirements for subsequent deep intelligent applications.

[0004] Secondly, the optimization process of prompting engineering heavily relies on the experience of domain experts and repeated trial and error, resulting in low efficiency and difficulty in obtaining a globally optimal solution. To improve the extraction effect of specific types of text, experts need to spend a lot of time adjusting prompt words. This process is highly subjective, and the final prompts are often only applicable to local scenarios, failing to guarantee stable performance on the entire dataset. This problem is particularly prominent when dealing with different venues or diverse text types.

[0005] Finally, existing methods lack the ability to self-evolve and automatically adapt to new data and new requirements. When new text needs to be processed or newly defined entity relationship types need to be extracted, the time-consuming and subjective manual tuning process must be restarted. This makes the construction, expansion, and maintenance of knowledge graphs costly and seriously hinders their dynamic updates and continuous application. Summary of the Invention

[0006] To address the above issues, this invention introduces an automatic prompt generation iterative framework that coordinates the optimizer and executor's dual large language models, a comprehensive performance scoring formula, a dynamic optimization trajectory update mechanism based on an elite retention strategy, and explicit iteration termination conditions. This achieves a fundamental shift from manual parameter tuning to algorithm optimization, significantly improves the semantic depth of the knowledge graph, ensures stable and efficient convergence of the optimization process, balances computational resources and performance gains, and ultimately completes an efficient expansion from algorithm optimization to large-scale production.

[0007] According to an embodiment of the present invention, a method for extracting venue knowledge based on APG adaptive prompting is provided.

[0008] In a first aspect of the invention, a method for extracting venue knowledge based on APG adaptive prompting is provided. The method includes: Step S01: Input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Step S02: Execute the APG iterative optimization loop. If the iteration termination condition is met, terminate the iteration and output the candidate prompt with the highest current performance score as the optimized prompt. Step S03: If the iteration termination condition is not met, return to step S02 to continue the iteration; Step S04: Apply the optimized prompts to the unstructured text corpus of the target venue, and have the LLM-Executor perform full knowledge extraction to construct the domain knowledge graph of the target venue.

[0009] Further, step S02 is as follows: Step S021: The LLM-Optimizer optimizes the current trajectory. Generate a set of candidate hints for knowledge extraction. ; Step S022: For each candidate hint in the candidate hint set The LLM-Executor applies it to the training dataset. Perform knowledge extraction operations to generate corresponding knowledge graph subsets. ; Step S023: Divide each knowledge graph subset With training dataset The candidate suggestion is calculated by comparing it with the standard annotation in the standard annotation. Performance score ; Step S024: Based on the performance score set Update and optimize the trajectory .

[0010] Further, the specific steps of step S021 are as follows: Construct a meta-hint, which includes a task description, examples of high-performing hints from historical iterations and their performance scores, and instructions to generate new, improved hints; input the meta-hint into the LLM-Optimizer to generate... Candidate prompts that differ in sentence structure, focus, or explicitness of instruction. .

[0011] Further, the specific steps of step S022 are as follows: For For each text sample in the sample, candidate prompts will be provided. This text sample is concatenated to form the complete input; the complete input is then submitted to the LLM-Executor, which uses candidate suggestions... The instructions in the document extract triples from text samples, represented as entities, relations, entities or entities, attributes, and attribute values; and aggregate them from... The triples extracted from all text samples constitute a subset of the knowledge graph. .

[0012] Furthermore, the performance score calculation formula mentioned in step S023 is as follows: , in, and Representing subsets of the knowledge graph Compared to the accuracy and recall of standard annotations, This indicates the recall rate for symbols, reflections, and representations of implicit relationships. and These are the preset weighting coefficients.

[0013] Furthermore, the specific strategy for step S024 is as follows: All candidate suggestions and their performance scores from this iteration. Add to history; from the complete history, retain the top-performing scores. The prompts and their scores constitute the updated optimization trajectory. This is used to guide the generation of candidate hints for the next iteration.

[0014] Furthermore, the iteration termination condition is: the number of iterations. Reaching the preset maximum number of iterations Or the best performance score obtained from the previous iteration Best performance score compared to the previous iteration The increase is less than the preset threshold. .

[0015] In a second aspect of the invention, an apparatus for extracting venue knowledge based on APG adaptive prompting is provided. The apparatus includes: Initialization module: used to input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Iteration module: Used to execute the APG iterative optimization loop. If the iteration termination condition is met, the iteration is terminated and the candidate prompt with the highest current performance score is output as the optimized prompt; if the iteration termination condition is not met, the iteration is returned to step S02 to continue the iteration. The knowledge graph construction module applies the optimized prompts to the unstructured text corpus of the target venue, and the LLM-Executor performs full knowledge extraction to construct the domain knowledge graph of the target venue.

[0016] In a third aspect of the invention, an electronic device is provided. The electronic device includes a memory and a processor, the memory storing a computer program, the processor executing the program to implement the method according to a first aspect of the invention.

[0017] In a fourth aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method according to a first aspect of the invention.

[0018] This invention achieves a fundamental shift from manual parameter tuning to algorithm optimization by introducing an automatic prompt generation iterative framework that coordinates the optimizer and executor's dual large language models, a comprehensive performance scoring formula, a dynamic optimization trajectory update mechanism based on an elite retention strategy, and explicit iteration termination conditions. This significantly improves the semantic depth of the knowledge graph, ensures stable and efficient convergence of the optimization process, balances computational resources and performance gains, and ultimately completes an efficient expansion from algorithm optimization to large-scale production.

[0019] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description.

[0020] The beneficial effects of this invention are: 1. By introducing an automatic prompt generation iterative framework that integrates two large language models, the optimizer and the executor, the prompt design is transformed from relying on human experience into a closed algorithmic process that can automatically search, evaluate, and optimize. The optimizer LLM is responsible for analyzing the characteristics of historical high-performance prompts and generating diverse candidate prompt variants. The executor LLM is responsible for faithfully executing each candidate prompt and verifying its extraction effect. Through multiple rounds of iterative optimization, the system can automatically find the optimal extraction prompt that is highly adapted to the text features of a specific venue domain in the prompt design space. This achieves a fundamental shift from manual parameter tuning to algorithm optimization, and completely solves the core problems of insufficient generalization ability and adaptability of manually designed static prompt templates, as well as low engineering efficiency and difficulty in obtaining the optimal solution. 2. By using a comprehensive performance scoring formula to provide precise navigation for the optimizer's search direction, it not only focuses on the extraction quality of entities and explicit relationships, but also guides the optimization process to consciously explore hints that can effectively capture the cultural logic behind the text by introducing an emphasis on deep semantic relationships, thus achieving a revolutionary improvement in the semantic depth of knowledge graphs; 3. By designing a dynamic optimization trajectory update mechanism based on an elite retention strategy, the system can retain the best-performing hints and their scores as successful experiences in each iteration and use them as a reference for generating new hints in the next round. This simulates the experience accumulation process in intelligent optimization algorithms, enabling the search to converge quickly to the high-performance region and effectively avoid getting trapped in local optima. It achieves stable and efficient convergence of the optimization process and solves the problems of blind direction and slow convergence in traditional trial-and-error methods. 4. By setting clear iteration termination conditions, the process automatically terminates when the preset maximum number of iterations is reached or the performance improvement tends to stabilize, achieving the best balance between computational resource investment and performance benefits, and ensuring the practicality and economy of the algorithm. 5. By applying the high-performance prompts obtained from the final optimization to the full text corpus and using batch parallel processing technology, we have achieved efficient expansion from algorithm optimization to large-scale production, which has completely changed the construction and maintenance mode of knowledge graphs. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. Wherein: Figure 1 A flowchart of a method for extracting venue knowledge based on APG adaptive prompting according to an embodiment of the present invention is shown; Figure 2 A partial knowledge graph entity relationship diagram according to an embodiment of the present invention is shown; Figure 3 A schematic diagram of an automatically captured implicit cultural association map is shown according to an embodiment of the present invention; Figure 4 A schematic diagram illustrating the evaluation results of knowledge graph extraction on a test set according to an embodiment of the present invention is shown. Figure 5 A block diagram of a device for extracting venue knowledge based on APG adaptive prompting according to an embodiment of the present invention is shown. Figure 6 A schematic diagram of a device for extracting venue knowledge based on APG adaptive prompting generation according to an embodiment of the present invention is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] According to an embodiment of the present invention, a method for venue knowledge extraction based on APG adaptive prompt generation is proposed. By introducing an automatic prompt generation iterative framework with the collaboration of two large language models of optimizer and executor, a comprehensive performance scoring calculation formula, a dynamic optimization trajectory update mechanism based on an elite retention strategy, and a clear iteration termination condition, a fundamental shift from manual parameter tuning to algorithm optimization is achieved. This significantly improves the semantic depth of the knowledge graph, ensures stable and efficient convergence of the optimization process, balances computational resources and performance gains, and ultimately completes the efficient expansion from algorithm optimization to large-scale production.

[0024] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.

[0025] Figure 1 This is a schematic flowchart of a venue knowledge extraction method based on APG adaptive prompting according to an embodiment of the present invention. The method includes: Step S01: Input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Step S02: Execute the APG iterative optimization loop. If the iteration termination condition is met, terminate the iteration and output the candidate prompt with the highest current performance score as the optimized prompt. Step S03: If the iteration termination condition is not met, return to step S02 to continue the iteration; Step S04: Apply the optimized prompts to the unstructured text corpus of the target venue, and have the LLM-Executor perform full knowledge extraction to construct the domain knowledge graph of the target venue.

[0026] It should be noted that although the operation of the method of the present invention has been described in a specific order in the above embodiments and figures, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0027] To provide a clearer explanation of the above-described method for extracting venue knowledge based on APG adaptive prompts, a specific embodiment will be used for illustration below. However, it is worth noting that this embodiment is only for better illustrating the present invention and does not constitute an improper limitation of the present invention.

[0028] The following example will further illustrate the method of venue knowledge extraction based on APG adaptive prompts.

[0029] This example demonstrates how a museum can construct a knowledge graph encompassing bronzes and porcelains, with the data source being an 850,000-word archaeological report and collection archive.

[0030] Step S01: Input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, and executor large language model.

[0031] In this embodiment, 5000 text entries are sampled from the full dataset, and entities and relationships such as cultural relics, dynasties, craftsmanship, patterns, and excavation sites are manually labeled to form a training set. .set up ="Extract entities and relationships from text", using GPT-4 as the dual-role model, parameters =0.5, =0.3, K=5, =15, =0.01.

[0032] Step S02: Execute the APG iterative optimization loop. If the iteration termination condition is met, terminate the iteration and output the candidate prompt with the highest current performance score as the optimized prompt.

[0033] The core innovation of the APG (Automatic Prompt Generation) framework lies in transforming prompt word design from a manual process relying on expert experience and repeated trial and error into an automated, iteratively optimized algorithmic problem. This framework comprises two core roles: the LLM-Optimizer and the LLM-Executor. The optimizer's task is to analyze historical optimization trajectories and generate new, potentially better, candidate prompt variants; the executor then uses each candidate prompt to perform an extraction task on the training data to verify its actual effectiveness.

[0034] Through iterative steps in S02, the system can automatically search within a broad cue design space, performing generation, evaluation, feedback, and regeneration to find the optimal extraction cue that highly adapts to the text features of a specific venue. This fundamentally solves the problems of insufficient generalization and adaptability of manually designed static prompt templates, as well as low engineering efficiency and difficulty in obtaining optimal solutions.

[0035] In each iteration In the process, the following sub-steps are executed sequentially: Step S021: The LLM-Optimizer optimizes the current trajectory. Generate a set of candidate hints for knowledge extraction. The specific steps are as follows: Construct a meta-hint containing a task description, examples of high-performing hints from previous iterations along with their performance scores, and instructions to generate new, improved hints; input the meta-hint into the LLM-Optimizer to generate... Candidate prompts that differ in sentence structure, focus, or explicitness of instruction. .

[0036] Meta-hints are key to guiding effective innovation in the LLM-Optimizer. They utilize historical experience from the optimization process (optimization trajectory). This is encapsulated as a natural language instruction. Meta-hints typically consist of several parts: a clear task definition, displaying one or more historically best-performing hint examples and their scores, and explicit generation instructions, such as analyzing the advantages of these high-scoring hints and generating K new hints that may improve in terms of instruction explicitness, entity coverage completeness, or relation description accuracy. It attempts to generate new hints that more explicitly list the target entity type in the instructions or emphasize the extraction of specific relations, thereby achieving intelligent exploration of the hint space.

[0037] Step S022: For each candidate hint in the candidate hint set The LLM-Executor applies it to the training dataset. Perform knowledge extraction operations to generate corresponding knowledge graph subsets. The specific steps are as follows: for For each text sample in the sample, candidate prompts will be provided. This text sample is concatenated to form the complete input; the complete input is then submitted to the LLM-Executor, which uses candidate suggestions... The instructions in the document extract triples from text samples, represented as entities, relations, entities or entities, attributes, and attribute values; and aggregate them from... The triples extracted from all text samples constitute a subset of the knowledge graph. .

[0038] For the training dataset For each annotated text in the system, the system will provide candidate suggestions. As a prefix to the instruction, it is concatenated with the text to form a complete prompt that conforms to the input format of the large language model. For example, if the candidate prompt is to extract the cultural relic, its dynasty, and its craftsmanship from the following text, where the text is "This Tang Dynasty sancai pottery figurine uses a low-temperature glaze technique," then the concatenated input would be: Extract the cultural relic, its dynasty, and its craftsmanship from the following text: "This Tang Dynasty sancai pottery figurine uses a low-temperature glaze technique." After receiving this input, the LLM-Executor understands and parses it according to the prompt instruction, and outputs a structured triple, such as: Sancai pottery figurine, dynasty, Tang Dynasty; Sancai pottery figurine, craftsmanship, low-temperature glaze. (Traversal) After processing all samples, all extracted triples constitute a subset of the knowledge graph under the candidate prompt. This subset will be compared with the manually annotated standard answer, and its quality directly determines the performance score of the candidate suggestion.

[0039] Step S023: Divide each knowledge graph subset With training dataset The candidate suggestion is calculated by comparing it with the standard annotation in the standard annotation. Performance score Its formula is: , in, and Representing subsets of the knowledge graph Compared to the accuracy and recall of standard annotations, This indicates the recall rate for symbols, reflections, and representations of implicit relationships. and These are the preset weighting coefficients.

[0040] Performance Score Formula This is the guiding core of our method for achieving high-quality, deep semantic knowledge extraction, and it is not simply about using... Instead of using the harmonic mean of precision and recall, targeted enhancements were implemented. The formula consists of three parts: the first two terms... Used to measure the quality of extraction of explicit entities and relations, through weights. The emphasis can be adjusted between accuracy and comprehensiveness. (Third item) This is a key innovation of the formula, specifically designed to evaluate the ability of candidate cues to extract deep, implicit semantic relationships such as symbolism, reflection, and embodiment. In museum texts, such relationships, such as "the ding symbolizes power" and "cloud and thunder patterns reflect nature worship," carry important cultural and historical information, but traditional extraction methods are prone to overlooking them. By setting weights... This method consciously guides the system to find clues that can effectively capture such deep logic during the optimization process, thereby significantly improving the semantic depth and reasoning value of the constructed knowledge graph and solving the problem of capturing implicit semantic relationships in the background technology.

[0041] Step S024: Based on the performance score set Update and optimize the trajectory The specific strategy is as follows: All candidate suggestions and their performance scores from this iteration. Add to history; from the complete history, retain the top-performing scores. The prompts and their scores constitute the updated optimized trajectory. This is used to guide the generation of candidate hints for the next iteration.

[0042] Update and optimize trajectory An elite-retention strategy is employed, a key mechanism to ensure the optimization process continuously converges towards the high-performance region. After each iteration, the system records all attempted candidate suggestions and their performance scores; subsequently, the system filters out the top performers from the entire historical record. These are the elite hints, used to construct a new generation of optimization trajectories. This trajectory will be provided to the LLM-Optimizer in the next iteration as a model and source of inspiration for generating new candidate suggestions. This mechanism simulates the selection process in evolutionary algorithms, ensuring that excellent suggestion features are preserved and inherited, while avoiding the optimization process being skewed by low-quality suggestions, thus efficiently and stably searching for globally better suggestion solutions.

[0043] In this embodiment, the system enters the APG iterative optimization process. The following is the specific iterative evolution process from the first to the third round: First round of iteration: Generate candidate hints: historical trajectory at this time Includes only initial prompts "Extract entities and relationships from text." The system build meta-prompts you to input "LLM-Optimizer," requesting the generation of... =5 candidate suggestions. One typical candidate suggestion generated by LLM-Optimizer. The output format is: "Please extract the artifact entities and their explicit relationships from the following museum text, and output them in the form of triples." Extraction will be performed by LLM-Executor. The algorithm concatenates the text with 5000 texts from the training set. For example, when faced with the text "This Tang Dynasty sancai pottery figurine uses a low-temperature glaze technique, and its exquisite cloud and thunder patterns reflect the ancients' worship of nature," the LLM-Executor successfully extracts the explicit triples (sancai pottery figurine, dynasty, Tang Dynasty) and (sancai pottery figurine, technique, low-temperature glaze), but misses implicit semantic relationships (such as reflect, symbolize). Finally, all triples are aggregated to form a subset of the knowledge graph. .

[0044] Calculation performance score: Compared to the standard answer, the explicit relation performed well in terms of P and recall R, but the implicit semantic relation resulted in poor recall. =0, according to the formula The calculation shows that the final score for this prompt is 0.42.

[0045] Second round of iteration: Generate candidate hints: The LLM-Optimizer reads the trajectory containing the first round of high-scoring hints and their scores. The meta-hints guide it to focus on "improving the ability to capture implicit relationships." The LLM-Optimizer adjusts its strategy to generate candidate hints that include explicit feature requirements. "Please extract the physical artifact, craftsmanship, and dynasty from the text. Please pay special attention to extracting any cultural connotations or relationships mentioned in the text." Extraction: LLM-Executor usage Perform a full extraction. Faced with the same example text, LLM-Executor not only extracted the explicit triples, but also successfully extracted the implicit semantic triples (cloud and thunder pattern, reflection, nature worship).

[0046] Computational performance score: Based on comparison, due to the successful capture of some implicit relationships, The score improved from 0 to 0.45. After substituting into the scoring formula, the performance score of this candidate suggestion improved to 0.58.

[0047] Updated and optimized trajectory: All hints and scores from the second round are incorporated into the history, and the top performers across all stages are selected. =5 hints, updated and optimized trajectory as follows .

[0048] Third round of iteration: Generating candidate suggestions: LLM-Optimizer is based on It was discovered that directly enumerating implicit relational verbs in the prompt words significantly improved the score. Therefore, this approach was further refined to generate more structurally rigorous candidate prompts. "As a professional archaeologist, please accurately extract the names, dynasties, techniques, and decorations of the artifacts from the text. Pay special attention to extracting statements that indicate the connotation of the artifacts, such as 'reflects,' 'symbolizes,' and 'embodies,' and organize them into (entity, relationship, entity) triplets." Extraction is performed by injecting this hint into the LLM-Executor. By assigning the model the role of a "professional archaeologist" and standardizing the output format, the standardization of the extraction and the accuracy of capturing implicit relationships are greatly improved.

[0049] Computational performance score: Based on the evaluation, the explicit relations of this subset are... The value reached 0.85, indicating a latent relationship recall rate. The score was significantly improved to 0.78. The final calculated overall performance score was 0.66.

[0050] The core code is as follows: import json def calculate_performance_score(kg_subset, ground_truth, alpha=0.5,beta=0.3): """ #Calculate prompt word score according to the proprietary performance scoring formula of this invention """ # 1. Calculate the precision (P) and recall (R) for explicit entities and relations. tp_explicit, fp_explicit, fn_explicit = 0, 0, 0 # 2. Calculate the recall of symbols, reflections, and manifestations of implicit relationships. tp_hidden, fn_hidden = 0, 0 hidden_relation_keywords = ["symbol", "reflect", "embody", "implication"] for item in kg_subset: rel = item.get("relation") if item in ground_truth: if any(k in rel for k in hidden_relation_keywords): tp_hidden += 1 else: tp_explicit += 1 else: fp_explicit += 1 for item in ground_truth: rel = item.get("relation") if item is not in kg_subset: if any(k in rel for k in hidden_relation_keywords): fn_hidden += 1 else: fn_hidden += 1 # Calculate the dominant F1-Score P = tp_explicit / (tp_explicit + fp_explicit) if (tp_explicit + fp_explicit) > 0 else 0 R = tp_explicit / (tp_explicit + fn_explicit) if (tp_explicit + fn_explicit) > 0 else 0 f1_explicit = (2 P R) / (P + R) if (P + R) > 0 else 0 # Calculate the recall rate R_hidden for hidden relationships R_hidden = tp_hidden / (tp_hidden + fn_hidden) if (tp_hidden + fn_hidden) > 0 else 0 # Final performance score formula: Score = alpha F1 + beta R_hidden score = alpha f1_explicit + beta R_hidden return score def apg_iteration_loop(initial_prompt, train_dataset, max_iters=15, K=5, M=5, epsilon=0.01): """ #APG Adaptive Hints for Generating Iterative Loop Main Function """ # Initialize optimized trajectory history history_trajectory = [{"prompt": initial_prompt, "score": 0.0}] best_score_history = [0.0] for t in range(1, max_iters + 1): # Generate K candidate hints based on the current optimization trajectory using LLM-Optimizer top_m_elites = sorted(history_trajectory, key=lambda x: x["score"],reverse=True)[:M] meta_prompt = build_meta_prompt(top_m_elites, K) candidate_prompts = call_llm_optimizer(meta_prompt) # Returns a list of K new prompt words current_iteration_results = [] # Traverse the candidate suggestions, and have the LLM-Executor extract and calculate the scores. for prompt in candidate_prompts: kg_subset = [] For sample in train_dataset: # Concatenation suggestions and text input actuators executor_input = f"Prompt: {prompt}\nText: {sample['text']}" triplets = call_llm_executor(executor_input) kg_subset.extend(triplets) # Calculate the performance score for this prompt score = calculate_performance_score(kg_subset, sample['label']) current_iteration_results.append({"prompt": prompt, "score": score}) #Update historical optimization trajectory based on elite retention strategy history_trajectory.extend(current_iteration_results) # Extract the best score of this round best_candidate = max(current_iteration_results, key=lambda x: x["score"]) current_best_score = best_candidate["score"] #Determine the termination condition of the iteration score_improvement = current_best_score - best_score_history[-1] best_score_history.append(current_best_score) print(f"The {t}th iteration is complete. Current best score: {current_best_score:.4f}, Improvement: {score_improvement:.4f}") if t >= max_iters or (t > 1 and score_improvement < epsilon): print("Termination condition satisfied, iteration terminated.") optimized_prompt = max(history_trajectory, key=lambda x: x["score"])["prompt"] return optimized_prompt return max(history_trajectory, key=lambda x: x["score"])["prompt"] Step S03: If the iteration termination condition is not met, return to step S02 to continue the iteration.

[0051] The iteration terminates when either of the following two conditions is met: 1. Number of iterations Reaching the preset maximum number of iterations ; 2. The best performance score obtained in the current iteration Best performance score compared to the previous iteration The increase is less than the preset threshold. ; These two termination conditions are complementary: Condition 1 is the maximum number of iterations, i.e. This is a safety measure to prevent the optimization process from looping indefinitely and to ensure that the algorithm can end within a controllable time. Condition 2 is the performance convergence threshold, i.e. This means that when the performance improvement from the best suggestion found by the system after one iteration is negligible, the optimization process is considered to have converged and can be stopped. Condition 2 reflects the economy of the optimization process; once the performance improvement reaches a plateau, it automatically terminates, avoiding unnecessary consumption of computational resources, making this method highly practical.

[0052] In this embodiment, after 8 rounds of iteration, the optimization yielded the desired result. "As a professional archaeologist, please extract the following information precisely from the text:" Name of the cultural relic and the dynasty to which it belongs; production techniques and core decorative patterns; location of excavation or collection.

[0053] Special attention should be paid to extracting statements that demonstrate the cultural connotations and symbolic meaning of cultural relics, such as "the ding symbolizes ritual power" and "the cloud and thunder pattern reflects nature worship," and organizing them into (entity, relationship, entity) tripartite groups.

[0054] Step S04: Apply the optimized prompts to the unstructured text corpus of the target venue, and have the LLM-Executor perform full knowledge extraction to construct the domain knowledge graph of the target venue.

[0055] High-performance prompts obtained through previous iterations and optimizations , Integrating the achievements of domain adaptation and task optimization, this approach is suitable for efficient and high-quality full-scale knowledge extraction from massive, unlabeled, unstructured text corpora within target venues, such as all collection archives and research papers. By employing a batch processing and parallel computing strategy—submitting large amounts of text in batches to a locally deployed LLM executor and utilizing multiple computing instances to process different batches simultaneously—it significantly reduces the time required to build large-scale domain knowledge graphs. This enables a rapid transition from small-sample optimization to large-scale production, thereby fundamentally changing the inefficient traditional reliance on manual maintenance of knowledge graphs.

[0056] In this embodiment, some knowledge graph entity relationship graphs are as follows: Figure 2 As shown, the automatically captured implicit cultural association graph is visualized as follows: Figure 3 As shown.

[0057] like Figure 4 As shown in Figure 1 and Table 1, the system performs stably on a test set containing 15,500 triples, with weighted synthesis... The accuracy rate reached 86.40%, indicating that the knowledge graph extraction system has high accuracy and practicality in processing unstructured text from museums.

[0058] For explicit relationships such as "dynasty," "material," and "place of excavation," which are well-defined and clearly expressible, the model's precision and recall both reached 88%–94%. This indicates that the system can capture the basic attribute information of cultural relics very accurately.

[0059] Even for implicit relationships like "symbol" and "reflection"—which lack obvious characteristic words and require understanding within a deeper cultural context—the system still achieved a recall rate of approximately 78.5%. This demonstrates that by combining vector retrieval with large-scale model inference, the system successfully overcomes the bottleneck of conventional models in extracting deep cultural connotations.

[0060] Table 1

[0061] The domain knowledge graph constructed based on this method identifies five core entity types: cultural relics, exhibits, creators, figures, dynasties, periods, exhibition halls, locations, themes, and schools of thought. These types are meticulously designed for the knowledge system of the venue domain, comprehensively covering the core elements of venue-related knowledge. The graph has a clear structured model, which not only facilitates the organization and storage of knowledge but also lays a solid foundation for subsequent intelligent applications, such as precise retrieval and relational reasoning. This transforms the knowledge graph from merely a collection of data into a knowledge system with a clear semantic model.

[0062] Regarding entity relationships, the graph not only includes explicit relationships reflecting objective facts such as creation, belonging, collection, creation date, and style, but also, through algorithmic optimization, extracts implicit semantic relationships such as reflection, symbolism, embodiment, and craftsmanship. For example, the system of inscribing craft names on artifacts reflects the standardization of military industry in the Qin Dynasty, and the kneeling archer terracotta warrior's posture embodies shooting manuals. These relationships reveal the historical, cultural, technological, and social logic behind the artifacts, significantly enhancing the semantic level and knowledge depth of the graph, thereby supporting deeper knowledge discovery, complex question answering, and academic research. This is a significant advantage of this method compared to traditional extraction techniques.

[0063] In terms of deployment, this method runs entirely in a localized environment. Both the LLM-Optimizer, responsible for prompt optimization, and the LLM-Executor, which performs knowledge extraction, utilize locally deployable open-source large language models such as Llama and Qwen. The entire process is completed on the venue's own servers or computing clusters. All training data, intermediately generated prompts, and the final constructed knowledge graph are processed entirely within the venue's internal network, eliminating any risk of data leakage. This effectively solves the data privacy and security compliance issues that may arise from using commercial cloud-based model APIs, making it particularly suitable for handling sensitive information such as museum archives and unpublished research findings.

[0064] Based on the same inventive concept, this invention also proposes a device for extracting venue knowledge based on APG adaptive prompts. The implementation of this device can be found in the implementation of the method described above; repeated details will not be repeated. Figure 5 As shown, the device 100 includes: Initialization module 101: Used to input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Iteration module 102: Used to execute the APG iterative optimization loop. If the iteration termination condition is met, the iteration is terminated and the candidate prompt with the highest current performance score is output as the optimized prompt; if the iteration termination condition is not met, the iteration is returned to step S02 to continue the iteration. Graph Construction Module 103: This module applies the optimized prompts to the unstructured text corpus of the target venue, and the LLM-Executor performs full knowledge extraction to construct the domain knowledge graph of the target venue.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0066] like Figure 6As shown, the device includes a central processing unit (CPU), which can perform various appropriate actions and processes based on computer program instructions stored in read-only memory (ROM) or loaded from storage units into random access memory (RAM). The RAM can also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0067] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0068] The processing unit executes the various methods and processes described above, such as method steps S01 to S04. For example, in some embodiments, method steps S01 to S04 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of method steps S01 to S04 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute method steps S01 to S04 by any other suitable means (e.g., by means of firmware).

[0069] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0070] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0071] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0072] Furthermore, although the operations are described in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0073] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for extracting venue knowledge based on APG adaptive prompting, characterized in that, The method includes: Step S01: Input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Step S02: Execute the APG iterative optimization loop. If the iteration termination condition is met, terminate the iteration and output the candidate prompt with the highest current performance score as the optimized prompt. Step S03: If the iteration termination condition is not met, return to step S02 to continue the iteration; Step S04: Apply the optimized prompts to the unstructured text corpus of the target venue, and have the LLM-Executor perform full knowledge extraction to construct the domain knowledge graph of the target venue.

2. The method for extracting venue knowledge based on APG adaptive prompting as described in claim 1, characterized in that, The steps in step S02 are as follows: Step S021: The LLM-Optimizer optimizes the current trajectory. Generate a set of candidate hints for knowledge extraction. ; Step S022: For each candidate hint in the candidate hint set The LLM-Executor applies it to the training dataset. Perform knowledge extraction operations to generate corresponding knowledge graph subsets. ; Step S023: Divide each knowledge graph subset With training dataset The candidate suggestion is calculated by comparing it with the standard annotation in the standard annotation. Performance score ; Step S024: Based on the performance score set Update and optimize the trajectory .

3. The method for extracting venue knowledge based on APG adaptive prompting according to claim 2, characterized in that, The specific steps of step S021 are as follows: Construct a meta-hint, which includes a task description, examples of high-performing hints from historical iterations and their performance scores, and instructions for generating new, improved hints; input the meta-hint into the LLM-Optimizer to generate... Candidate prompts that differ in sentence structure, focus, or explicitness of instruction. .

4. The method for extracting venue knowledge based on APG adaptive prompting as described in claim 2, characterized in that, The specific steps of step S022 are as follows: For For each text sample in the sample, candidate prompts will be provided. This text sample is concatenated to form a complete input. Submit the complete input to the LLM-Executor, which then uses the candidate suggestions... The instructions in the document extract triples from text samples, represented as entities, relations, entities or entities, attributes, and attribute values; and aggregate them from... The triples extracted from all text samples constitute a subset of the knowledge graph. .

5. The method for extracting venue knowledge based on APG adaptive prompting according to claim 2, characterized in that, The performance score calculation formula mentioned in step S023 is as follows: ， in, and Representing subsets of the knowledge graph Compared to the accuracy and recall of standard annotations, This indicates the recall rate for symbols, reflections, and representations of implicit relationships. and These are the preset weighting coefficients.

6. The method for extracting venue knowledge based on APG adaptive prompting according to claim 2, characterized in that, The specific strategy for step S024 is as follows: All candidate suggestions and their performance scores from this iteration. Add to history; From the complete historical record, retain the top-performing scores. The prompts and their scores constitute the updated optimization trajectory. This is used to guide the generation of candidate hints for the next iteration.

7. The method for extracting venue knowledge based on APG adaptive prompting as described in claim 1, characterized in that, The iteration termination condition is: the number of iterations. Reaching the preset maximum number of iterations Or the best performance score obtained from the previous iteration Best performance score compared to the previous iteration The increase is less than the preset threshold. .

8. A device for extracting venue knowledge based on APG adaptive prompts, characterized in that, The device implements the method as described in any one of claims 1 to 7, comprising: Initialization module: used to input initialization parameters, including: initial prompts for the knowledge extraction task, training text dataset containing standard entity and relation annotations, optimizer large language model, executor large language model; Iteration module: Used to execute the APG iterative optimization loop. If the iteration termination condition is met, the iteration is terminated and the candidate prompt with the highest current performance score is output as the optimized prompt; if the iteration termination condition is not met, the iteration is returned to step S02 to continue the iteration. The knowledge graph construction module applies the optimized prompts to the unstructured text corpus of the target venue, and the LLM-Executor performs full knowledge extraction to construct the domain knowledge graph of the target venue.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images