A method and device for creating a knowledge graph in the field of unmanned system testing

Abstract

The application discloses a kind of for the creation method and device of knowledge graph in the field of unmanned system test, the method comprises: using large language model to the unstructured data to be extracted is labeled processing, generation has labeled structured data;Knowledge extraction model is used to the knowledge extraction processing of the structured data, generates knowledge triple;Wherein, the knowledge triple is used to indicate the combination formed by first entity, inter-entity relationship and second entity;Based on several knowledge triples, the knowledge graph of target field is constructed.Thereby, using large language model to data is labeled, effectively reduce the demand of manual intervention, solve the problem of fund pressure of project implementation, improve the capital utilization rate;By adjusting data processing method and establishing reliable trust mechanism, the consistency of directly using large model for knowledge extraction is low and the problem of poor explainability is solved, improve the efficiency and automation degree of knowledge graph construction process.

Description

Technical Field

[0001] This invention belongs to the technical field of knowledge graph creation, and particularly relates to a method and apparatus for creating a knowledge graph for unmanned system testing. Background Technology

[0002] Traditional knowledge graph construction methods typically rely on datasets obtained from the internet and extract knowledge by training natural language processing (NLP) models. In practical applications, the extracted results often need to be deambiguously resolved using knowledge fusion techniques, and the deambiguously resolved information is then stored in a knowledge base. However, this process faces multiple challenges, the core of which lies in obtaining high-quality triples with canonical relationships, rather than simply combining triples into a graph structure for storage. First, while internet datasets are a common way to obtain high-quality relation triples, open-source datasets often suffer from insufficient quality and coverage. Although paid datasets offer greater assurance in terms of quality, they are expensive and may lack suitable options in certain innovative fields. Furthermore, relying on existing datasets to build knowledge graphs often limits the breadth of content and makes it difficult to reflect the dynamic changes in emerging fields in a timely manner. Second, while self-made datasets can be customized by combining automated tools with manual annotation, they often suffer from uneven quality. Even with significant investment in manual labor, the model performance may still be unsatisfactory, especially in fields with high accuracy requirements, such as medical diagnosis and financial analysis. While relying entirely on manual methods to collect relation triples without using an NLP model can improve accuracy, it significantly increases labor costs and leads to inefficiency. This is especially true when dealing with large-scale data, making it almost impossible to meet application requirements.

[0003] The development of large-scale modeling techniques has brought new knowledge extraction schemes, enabling knowledge extraction under zero-sample or few-sample conditions, and even allowing the training of specific large-scale models for knowledge graph construction. However, because large-scale models are based on generative logic, the reliability and accuracy of their results remain uncertain, especially when dealing with complex texts, making it difficult to guarantee high-quality knowledge graph construction.

[0004] In summary, traditional methods and large-scale models each have their advantages and disadvantages, primarily involving trade-offs between construction costs and data accuracy, making it difficult to achieve a balance. Furthermore, another significant challenge is that while knowledge extraction models perform well when processing text with simple structures, their accuracy drops significantly when dealing with long texts rich in information and complex in structure, further impacting the completeness and accuracy of knowledge graphs. Therefore, an optimization method is urgently needed to achieve a balance between acquiring high-quality relation triples and controlling construction costs, thereby effectively promoting the construction and application of knowledge graphs. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, embodiments of the present invention provide a method and apparatus for creating knowledge graphs in the field of unmanned system testing. This method combines a large language model and a knowledge extraction model, which not only improves the accuracy and data quality of knowledge extraction but also effectively reduces the need for manual intervention, thereby achieving high efficiency and automation in the knowledge graph construction process.

[0006] According to a first aspect of the present invention, a method for creating a knowledge graph for the field of unmanned system testing is provided; comprising: using a large language model to annotate unstructured data to be extracted to generate annotated structured data; using a knowledge extraction model to perform knowledge extraction processing on the structured data to generate knowledge triples; wherein the knowledge triples are used to indicate combinations formed by a first entity, inter-entity relations, and a second entity; and constructing a knowledge graph of a target domain based on a plurality of the knowledge triples.

[0007] Optionally, the method for creating a knowledge graph further includes: using preprocessed and unlabeled unstructured data as candidate training samples, and obtaining prompt words corresponding to the knowledge graph to be built from the candidate training samples; wherein, the prompt words are words used to guide the model to extract knowledge; based on the prompt words, using a large language model to label the candidate training samples to generate quasi-training samples; and based on several quasi-training samples, using an NLP model to train and generate a knowledge extraction model.

[0008] Optionally, the prompt words corresponding to the knowledge graph to be built are obtained from the candidate training samples; including: determining the ontology graph of the knowledge graph to be built based on the target domain requirements corresponding to the candidate training samples; wherein the ontology graph is used to indicate the entity types, entity relationship types and entity constraint ranges involved in the target domain; and constructing the prompt words corresponding to the knowledge graph to be built based on the ontology graph.

[0009] Optionally, preprocessed and unlabeled unstructured data can be used as candidate training samples, including: performing text cleaning on the unlabeled unstructured data to generate cleaned data; performing standardization on the cleaned data to generate standard data; and converting the standard data into a preset format to obtain candidate training samples.

[0010] Optionally, based on the prompt words, a large language model is used to annotate the candidate training samples to generate quasi-training samples; this includes: based on the prompt words, using a large language model to sequentially perform data decomposition and completion processing on the candidate training samples to generate several simple sentences; for any one of the several simple sentences: perform standard verification processing on the simple sentence; if the verification passes, use the large language model to identify key knowledge points in the simple sentence, and generate annotated structured data based on the identification results; if the structured data does not meet preset requirements, adjust the structured data based on user adjustment requests to obtain quasi-training samples and revision logs; if the structured data meets preset requirements, use the structured data as quasi-training samples.

[0011] Optionally, based on a user adjustment request, the structured data is adjusted to obtain quasi-training samples; this includes: adjusting the structured data based on the user adjustment request to generate adjusted structured data; wherein the adjusted structured data includes several rows of data; each row of data includes at least one triplet sample; each element in each triplet sample has a corresponding element attribute; determining the evaluation index corresponding to each element attribute in each row of data in the adjusted structured data; for any element attribute: determining the subjective weight corresponding to the element attribute based on the analytic hierarchy process; determining the objective weight corresponding to the element attribute based on information entropy; adjusting the objective weight and the subjective weight using adjustment parameters to generate the comprehensive weight corresponding to the element attribute; applying the corresponding comprehensive weight to the standard value corresponding to the element attribute to obtain a comprehensive score; for any row of data: summing the comprehensive scores corresponding to each element attribute in the row of data to generate a row data score; obtaining quasi-training samples based on the row data scores corresponding to each row of data in the several rows of data.

[0012] Optionally, the method further includes: standardizing each element attribute in the adjusted structured data to generate a standardized matrix formed by standard values; calculating the information entropy corresponding to each standard value in the adjusted structured data based on the standardized matrix; and determining the objective weight corresponding to the element attribute based on the information entropy corresponding to each standard value.

[0013] Optionally, the method further includes: generating a judgment matrix based on the importance parameter of each element attribute in the adjusted structured data; when the consistency of the judgment matrix meets a preset condition, normalizing the judgment matrix to obtain a weight vector; and determining the subjective weight corresponding to each element attribute based on the weight vector.

[0014] Optionally, a knowledge extraction model is generated by training an NLP model based on several quasi-training samples; including: for any quasi-training sample among the several quasi-training samples: converting the quasi-training sample into BERT format and GPLinker format respectively; using the deep representation capability of BERT and the relation linking layer of GPLinker, performing dual training on the quasi-training sample for entity recognition and entity relation extraction to generate a knowledge extraction model.

[0015] According to a second aspect of the present invention, an apparatus for creating a knowledge graph in the field of unmanned system testing is also provided, comprising: an annotation processing module for annotating unstructured data to be extracted using a large language model to generate annotated structured data; a knowledge extraction module for extracting knowledge from the structured data using a knowledge extraction model to generate knowledge triples; wherein the knowledge triples are used to indicate combinations formed by a first entity, inter-entity relations, and a second entity; and a construction module for constructing a knowledge graph of a target domain based on a plurality of the knowledge triples.

[0016] According to a third aspect of the present invention, an electronic device is also provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method as described in the first aspect.

[0017] According to a fourth aspect of the present invention, a computer-readable medium is also provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0018] Compared with existing technologies, the present invention provides a method for creating a knowledge graph, comprising: labeling unstructured data to be extracted using a large language model to generate labeled structured data; extracting knowledge from the structured data using a knowledge extraction model to generate knowledge triples; wherein the knowledge triples are used to indicate combinations formed by a first entity, inter-entity relations, and a second entity; and constructing a knowledge graph of a target domain based on several of the knowledge triples. Thus, using a large language model to label the data effectively reduces the need for manual intervention, solves the financial pressure problem of project implementation, and improves capital utilization; by adjusting the data processing method and establishing a reliable trust mechanism, the problems of low consistency and poor interpretability of directly using large models for knowledge extraction are solved, improving the efficiency and automation of the knowledge graph construction process. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a method for creating a knowledge graph in the field of unmanned system testing, provided by an embodiment of the present invention. Figure 2 This is a flowchart illustrating a method for generating a knowledge extraction model in one embodiment of the present invention. Figure 3 This is a schematic diagram of the ontology and corresponding prompt words provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of a knowledge graph creation device for the field of unmanned system testing, provided as an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] Traditional NLP algorithms rely heavily on large amounts of high-quality training data and have limited capabilities when processing long texts. Large-scale models, on the other hand, possess excellent long-text processing capabilities and can achieve good knowledge extraction results even with few or no samples. However, their results lack reliable trust mechanisms, which may affect the accuracy and consistency of extraction. Therefore, this invention proposes an optimized method and system for knowledge graph construction based on a combination of large-scale models and NLP. By optimizing the data processing flow and establishing a trust mechanism, the quality and efficiency of knowledge graph construction are improved.

[0022] like Figure 1 The diagram shown is a flowchart illustrating a method for creating a knowledge graph for unmanned system testing, according to an embodiment of the present invention.

[0023] A method for creating a knowledge graph for unmanned system testing includes at least the following steps: S101, using a large language model to label the unstructured data to be extracted, generating labeled structured data; S102, use a knowledge extraction model to perform knowledge extraction processing on structured data to generate knowledge triples; wherein, knowledge triples are used to indicate the combination formed by the first entity, the relationship between entities, and the second entity; S103 constructs a knowledge graph for the target domain based on several knowledge triples.

[0024] Specifically, Step 1: Import the unstructured data to be extracted into the system.

[0025] Ensure that the input unstructured data conforms to the software system's format requirements (such as CSV, JSON, etc.) to facilitate subsequent processing and model application. The system will perform preliminary validation of the input unstructured data, checking its integrity and consistency to ensure that the data can smoothly enter the processing flow.

[0026] Step 2: The large language model decomposes, completes, and annotates the input unstructured data line by line.

[0027] The system processes the input data line by line. First, it decomposes the unstructured data using a large language model, breaking down complex long sentences into multiple independent simple sentences. This operation can effectively improve the efficiency and accuracy of subsequent knowledge extraction, providing each simple sentence with a clear grammatical structure and information points, thus facilitating the better identification of entities and relationships within it.

[0028] Step 3: Use a knowledge extraction model to extract knowledge from the structured data.

[0029] After processing long and complex sentences, the system will perform relation extraction using a pre-trained knowledge extraction model (such as a joint architecture of BERT and GPLinker). The system automatically identifies and extracts entities, relations, and attributes from the sentences, generating high-quality knowledge triples. Each triple will include information such as [entity 1, relation, entity 2], ensuring the completeness and accuracy of the information.

[0030] Step 4: Deduplicate the extracted knowledge triples to eliminate duplicate entity and relation information.

[0031] To ensure the cleanliness and consistency of the generated knowledge graph, the system will deduplicate the extracted knowledge triples, eliminating duplicate entity and relation information. Simultaneously, all attribute values ​​will be uniformly formatted to ensure consistent data format across the knowledge graph, avoiding data redundancy or query difficulties caused by format differences.

[0032] Step 5: Import the processed knowledge triples into the knowledge graph database.

[0033] Finally, the processed knowledge triples will be imported into a knowledge graph database for efficient knowledge storage and management. The database will support subsequent queries, reasoning, updates, and expansions, while ensuring data scalability and maintainability, providing support for the continuous optimization and iteration of the knowledge graph.

[0034] This embodiment of the method fully leverages the advantages of large language models in processing long texts and extracting knowledge from few samples, overcoming the shortcomings of traditional NLP algorithms, such as strong dependence on training data and high costs of manual annotation. By improving data processing methods and establishing a reliable trust mechanism, this invention aims to improve the accuracy and data quality of knowledge extraction, effectively reduce the need for manual intervention, and thus achieve high efficiency and automation in the knowledge graph construction process; it also improves the quality of knowledge graph construction. Furthermore, this method promotes the integration of cross-domain knowledge, providing users with richer knowledge resources and intelligent support.

[0035] like Figure 2 The diagram shown is a flowchart illustrating a method for generating a knowledge extraction model in one embodiment of the present invention.

[0036] Methods for generating knowledge extraction models include at least the following steps: S201, preprocessed and unlabeled unstructured data is used as candidate training samples, and prompt words corresponding to the knowledge graph to be built are obtained from the candidate training samples; where prompt words are used to guide the model to extract knowledge; S202, based on prompt words, uses a large language model to annotate candidate training samples and generate quasi-training samples; S203, based on several quasi-training samples, uses an NLP model for training to generate a knowledge extraction model.

[0037] In S201, S1. Collect unlabeled unstructured data from different sources in the target domain to determine the diversity and relevance of the data. For example, collect unlabeled unstructured data from multiple academic journals, industry reports, social media, and open datasets. When the target domain is unmanned system testing, use methods such as web crawling, API interfaces, or manual collection to gather relevant unstructured data from unmanned system test reports, academic papers, and technical literature.

[0038] S2, perform text cleaning on the unlabeled unstructured data to generate cleaned data; perform standardization on the cleaned data to generate standard data; convert the standard data into a preset format and save it to obtain candidate training samples.

[0039] When cleaning unlabeled unstructured data, it can not only remove duplicate document content, but also remove irrelevant noise information (such as advertisements, links, special characters, etc.), thus retaining the core content that is meaningful to the data annotation.

[0040] Annotation processing includes, for example, word segmentation and indexing.

[0041] Converting standard data to CSV, JSON, or TXT formats ensures the consistency and parsability of the standard data.

[0042] Preprocessing unstructured data ensures the quality of the underlying data used to build knowledge graphs, thus providing reliable input for model training and enhancing the comprehensiveness and applicability of knowledge graphs.

[0043] S3, based on the target domain requirements corresponding to the candidate training samples, conducts in-depth analysis of the candidate training samples to determine the ontology graph of the knowledge graph to be built; the ontology graph is used to indicate and describe the entity types, entity relationship types, and entity constraint scope involved in the target domain. For example, in the field of unmanned system testing, the ontology graph can include basic concepts such as "test project", "unmanned system model", "test environment", "performance index", and "test result", and the relationships between them need to be clearly defined (such as "test result belongs to test project", "performance index is related to unmanned system model", etc.).

[0044] S4. Based on the ontology graph, construct prompt words corresponding to the knowledge graph to be built; the prompt words are used to help the model accurately understand and extract key information in the target domain.

[0045] like Figure 3 The diagram shown is a schematic representation of the ontology and corresponding prompt words provided in one embodiment of the present invention.

[0046] For example: (1) Instruction: This part tells the large model how to process the input data, and specifically explains the task objectives and execution requirements. The nested content instructs the system to extract entities and output them in the given schema format. Detailed explanation is as follows: (a) Instruction: "You are a graph entity knowledge structuring expert. Based on the schema description of the input entity type, extract the corresponding entity instance and its attribute information from the text. Attributes that do not exist are not output; if an attribute has multiple values, return a list, and output in parsable JSON format." This part defines the system's goal: Based on the given schema (i.e., ontology graph structure), extract the corresponding entities and their attributes from the input text and return a compliant JSON format. If an attribute does not exist, it is not output; if an attribute has multiple values, a list is returned.

[0047] (b) Schema: This section provides the specific entity type and its attribute information, which the system uses for data extraction. For example, "entity_type": "Target under test", "attributes": { "indicator": "indicator to be tested"} indicates that the system needs to extract the entity "Target under test" from the text and extract its "indicator" attribute (and mark its value as "indicator to be tested"). If there is no indicator information in the text, the system will not output any attributes.

[0048] (c) input: This part represents the actual input text from which the system needs to extract information. The input content is replaced by actual data during the execution process.

[0049] (2) Sample Section: This section provides a concrete example to help the system understand how to extract actual data and return it in JSON format as required. Detailed explanation follows: (a) Input: This is a long text describing a test process for the smoothness of a robot's movement on unpaved roads. This text will serve as the input to the system, from which the system needs to extract the relevant entity types and attributes.

[0050] (b) Output: The output section shows how the system should return the extracted information given the input. In this example, the system extracted the "robot" entity and extracted multiple attribute values ​​related to "metrics" from the input text, such as "unpaved road smoothness" and "position and speed fluctuation curves".

[0051] (3) Output Format Requirements: This section specifies the exact output format, clarifies the JSON structure, and specifies the requirement that no additional content should be output, facilitating software recognition and application. Details are as follows: (a) ID field: A placeholder {ID}, which should be replaced with a unique identifier in practice.

[0052] (b) Target under test: In the output JSON, "Target under test" is a description of an entity and a list of "metrics" for that entity.

[0053] Therefore, the precise design of the ontology graph provides strong support for the organization, storage, and association of unstructured data, thereby helping to improve the accuracy and consistency of knowledge extraction model training; while the reasonable design of prompt words enables large language models to process candidate training samples efficiently and accurately, which is conducive to the optimization of the subsequent annotation process, and thus provides a clear direction for knowledge extraction.

[0054] In S202, a large language model is integrated with pre-designed prompt words to develop the system. The system can efficiently decompose, complete, label, and score uploaded unstructured data. This enables automated processing of unstructured data, improving the efficiency and accuracy of unstructured data processing.

[0055] For example, preprocessed and unlabeled unstructured data is imported into the system as candidate training samples. Based on the prompt words, the system uses a large language model to sequentially decompose and complete the candidate training samples, generating several simple sentences. For any one of the simple sentences, a standard verification process is performed. If the verification passes, the large language model is used to identify key knowledge points in the simple sentence, and labeled structured data is generated based on the identification results. If the structured data does not meet the preset requirements, the structured data is adjusted based on the user's adjustment request to obtain quasi-training samples and revision logs. If the structured data meets the preset requirements, the structured data is used as a quasi-training sample.

[0056] Specifically, the system uses a large language model to automatically break down unstructured data into multiple independent and concise simple sentences. For any given simple sentence: the system performs integrity identification and fills in missing information to ensure the completeness of each simple sentence. The system then performs standard validation on the simple sentences; this validation includes checking the consistency of data format, the completeness of content, and the accuracy of the decomposed text. If the validation passes, all simple sentences are saved in an appropriate format (such as CSV, JSON, etc.), thus providing a high-quality data foundation for subsequent annotation and knowledge extraction tasks.

[0057] The system also provides users with real-time progress feedback and displays the data processing status through a visual interface. The data processing status includes the number of lines of unstructured data, the progress of disassembly and completion, and potential error or warning messages. Furthermore, the visual interface allows users to review, revise, and score the automatically labeled data. Users can manually adjust entity names, relationship types, attribute values, etc., in the labeling results according to actual needs, and assign weights to each line of data to ensure data quality and reliability. Users can also modify inaccurately completed or labeled data to improve the accuracy and usability of the final output data.

[0058] For example: S1, the system provides a file upload interface.

[0059] To ensure the system can flexibly handle data files of various formats (such as text files, CSV, JSON, etc.), a file upload interface is designed and provided, supporting batch uploads. Users can use this interface to quickly import data files into the system, ensuring that data can be efficiently transmitted and stored before proceeding to subsequent processing steps.

[0060] S2 integrates a large language model and provides data decomposition, completion, and annotation functions.

[0061] Based on the needs of the target domain, select a suitable large language model (such as GPT, Spark, etc.). Combine this with pre-designed prompts to process the uploaded data automatically. After data upload, the system can automatically perform the following tasks: By using designed prompts, the system can identify the grammatical structure of sentences and simplify the complexity of the original text. The system breaks down long and complex sentences into multiple independent simple sentences, ensuring that each sentence is actionable. During the breakdown process, if missing information is found, the system will automatically complete it based on context or existing annotations. This completion process not only fills in the gaps in the data but also ensures that each broken-down sentence is grammatically and semantically complete and coherent, providing high-quality foundational data for subsequent processing. The system not only fills in the gaps in unstructured data but also ensures that each broken-down sentence is complete and coherent.

[0062] The system extracts meaningful information (such as entities and relationships between entities) from unstructured data. This process uses a large language model to identify key knowledge points in the text and transform them into labeled structured data. As a result, the system can accurately label each entity and the relationships between them, providing high-quality labeled results for the subsequent training of NLP models.

[0063] S3 features a visual interface that allows users to review, revise, and score data automatically labeled by the system.

[0064] Users can manually adjust entity names, relationship types, attribute values, and other elements in the annotation results according to their actual needs, while assigning weights to each row of data to ensure data quality and reliability. Thus, users can modify the annotated structured data through a visual interface to improve the accuracy and usability of the final output data.

[0065] For example, based on a user's adjustment request, the structured data is adjusted to generate adjusted structured data; wherein, the adjusted structured data includes several rows of data; each row of data includes at least one triplet sample; each element in each triplet sample has a corresponding element attribute; the evaluation index corresponding to each element attribute in each row of data in the adjusted structured data is determined; for any element attribute: the subjective weight corresponding to the element attribute is determined based on the analytic hierarchy process; the objective weight corresponding to the element attribute is determined based on information entropy; the objective weight and the subjective weight are adjusted using adjustment parameters to generate a comprehensive weight corresponding to the element attribute; the corresponding comprehensive weight is applied to the standard value corresponding to the element attribute to obtain a comprehensive score; for any row of data: the comprehensive scores corresponding to each element attribute in the row of data are summed to generate a row data score; based on the row data scores corresponding to each row of data in the several rows of data, quasi-training samples are obtained.

[0066] For example, the system displays annotated structured data by task, and users can view the annotation results through a visual interface. The system also provides convenient editing tools, allowing users to revise or score the displayed annotated structured data. Users can modify entity names, relationships between entities, and other relevant content to ensure the accuracy of the annotation results. To improve the quality of the annotated structured data, users can also use the Analytic Hierarchy Process (AHP) and entropy to assign weights to each row of data, ensuring that the weighting of the data is more scientific and reasonable. After the user completes the revision of the annotated data, the system will prompt the user to confirm the changes. All revisions are properly recorded and a revision log is generated for subsequent auditing and tracking. This process ensures the traceability of data quality and ensures that each revision is recorded in detail.

[0067] The subjective and objective weights corresponding to the element attributes are obtained through the following methods: Based on the importance parameters of each element attribute in the adjusted structured data, a judgment matrix is ​​generated; when the consistency of the judgment matrix meets the preset conditions, the judgment matrix is ​​normalized to obtain a weight vector; and the subjective weight corresponding to each element attribute is determined based on the weight vector.

[0068] The attributes of each element in the adjusted structured data are standardized to generate a standardized matrix formed by standard values; the information entropy corresponding to each standard value in the adjusted structured data is calculated based on the standardized matrix; and the objective weight corresponding to each attribute is determined based on the information entropy corresponding to each standard value.

[0069] More specifically, revising or scoring structured data involves the following steps: S1 allows for the optional modification of entity names and adjustment of relationships between entities, thereby not only accurately expressing the meaning of entities but also ensuring the rationality and consistency of entity relationships.

[0070] S2, determine the evaluation index corresponding to each element attribute in each row of data in the adjusted structured data. Assume each row of data has... Each attribute has a different aspect of the data, such as "entity name", "relationship type", and "confidence level".

[0071] S3. Determine the subjective weights corresponding to element attributes based on the Analytic Hierarchy Process (AHP).

[0072] First, construct the judgment matrix.

[0073] Based on expert advice or experience, the importance of each element attribute in the adjusted structured data is compared pairwise, and a system is built. Judgment matrix ,in Indicates the first The element attribute is relative to the first The importance of element properties.

[0074] Second, consistency check.

[0075] For the judgment matrix Perform consistency checks to ensure the rationality of the judgment logic. The conformity ratio (CR) is typically used as a metric. If the consistency check is passed, the decision matrix will pass; otherwise, it needs to be adjusted.

[0076] Finally, calculate the subjective weights.

[0077] For matrix Perform normalization processing to obtain the weight vector of each element attribute in the adjusted structured data. This allows us to determine the subjective weight corresponding to each element attribute. Subjective weights are used to indicate the importance that experts subjectively assign to each element attribute.

[0078] S4, Determine the objective weights corresponding to element attributes based on information entropy. First, standardize the data.

[0079] Each element attribute in the adjusted structured data is standardized to eliminate dimensional differences between different element attributes, and a standardization matrix is ​​calculated. ,in Indicates the first The first row of data The standard value of an element's attribute.

[0080] Secondly, calculate the information entropy.

[0081] Calculate the entropy value of each element attribute in the adjusted structured data using a normalized matrix. The formula is: Equation (1); in, For the first Line number The proportion of each element's attribute values It is a constant. This is the number of rows in the adjusted structured data.

[0082] Finally, calculate the objective weights.

[0083] Based on information entropy value Calculate the weight of each element attribute : Equation (2); These weights represent the objective importance of each element's attribute.

[0084] S5, balancing subjective and objective weights. Subjective weighting... and objective weight Perform a weighted average or other comprehensive method to obtain the final comprehensive weight of each element attribute. The formula is as follows: Equation (3); in, This is an adjustment parameter used to balance the influence of subjective and objective weights, typically set to around 0.5. The final result... Vectors are used to calculate the overall weight of each row of data.

[0085] S6, calculate the weight of each row of data. This is based on the attribute values ​​of each row of data and the final overall weight. Calculate the weighted score for each row of data. The formula is as follows: Equation (4); in, Indicates the first The final weight score of the row data is used for sorting, filtering, or further processing.

[0086] High-quality labeled data is obtained through review and revision, and then exported as quasi-training samples in standard formats (such as CSV, JSON, etc.) for further processing and application.

[0087] This embodiment leverages a large-scale generative model, which possesses powerful natural language processing capabilities, but this does not guarantee the complete reliability of the results. Through convenient editing tools, users can effectively review and revise the annotation results generated by the large model, thereby improving data quality. This process not only ensures the acquisition of high-quality training data but also significantly saves labor costs, improves overall efficiency, and provides a solid foundation for subsequent knowledge extraction and knowledge graph construction.

[0088] In S203, for example, for any one of the plurality of quasi-training samples: the quasi-training sample is converted into BERT format and GPLinker format respectively; using the deep representation capability of BERT and the relation linking layer of GPLinker, the quasi-training sample is subjected to dual training of entity recognition and entity relation extraction to generate a knowledge extraction model.

[0089] For example: S1 uses revised, high-quality training data to build an efficient NLP knowledge extraction model. This model should combine domain knowledge with the capabilities of large models in handling long texts and few samples to accurately meet the needs of specific domains.

[0090] S2, during its construction process, performs targeted optimizations on different levels and modules of the model, including entity recognition, relation extraction, and attribute extraction, to enhance the overall performance and stability of the model.

[0091] S3 is used to train the model, leveraging BERT's deep semantic understanding and GPLinker's relational linking capabilities to achieve more accurate knowledge extraction.

[0092] First, the quasi-training samples are converted into the input format for BERT and GPLinker, including preprocessing operations such as word segmentation, encoding, and label conversion, to ensure that the data can be correctly interpreted by the model. Second, a pre-trained BERT model is loaded, and a joint extraction architecture is built based on it. This combines the relation linking layers of BERT and GPLinker to achieve dual-task optimization for entity recognition and relation extraction, enabling the model to better capture complex information in the text. Finally, multiple rounds of training are performed using several quasi-training samples, with independent optimization for both entity recognition and relation extraction tasks. A multi-task loss function is used to adjust the weights to gradually improve model performance.

[0093] S4. After training, evaluate the model performance on the validation set, using multiple evaluation metrics (such as precision, recall, and F1 score) to detect the accuracy and stability of entity and relation extraction.

[0094] The NLP knowledge extraction model constructed through the above steps in this embodiment is highly efficient and reliable, accurately identifying key information in text and providing strong support for the automated construction of knowledge graphs. This process not only significantly improves the accuracy of knowledge extraction but also lays the foundation for the model's multi-domain adaptability, helping to address potential new tasks and domain requirements in the future.

[0095] The following section provides a detailed description of a knowledge graph creation method for unmanned system testing, based on specific application scenarios.

[0096] S1. Perform text cleaning on the unlabeled unstructured data to generate cleaned data; perform standardization on the cleaned data to generate standard data; convert the standard data into a preset format to obtain candidate training samples.

[0097] S2, based on the target domain requirements corresponding to the candidate training samples, determine the ontology graph of the knowledge graph to be built; wherein, the ontology graph is used to indicate the entity types, entity relationship types and entity constraint ranges involved in the target domain; based on the ontology graph, construct the prompt words corresponding to the knowledge graph to be built; wherein, the prompt words are words used to guide the model to extract knowledge.

[0098] S3, based on prompt words, uses a large language model to sequentially decompose and complete the candidate training samples to generate several simple sentences; for any simple sentence among these sentences, a standard verification process is performed; if the verification passes, the large language model is used to identify key knowledge points in the simple sentence, and labeled structured data is generated based on the recognition results.

[0099] S4, if the structured data does not meet the preset requirements, the structured data is adjusted based on the user's adjustment request to generate adjusted structured data. The adjusted structured data includes several rows of data; each row includes at least one triplet sample; each element in each triplet sample has a corresponding element attribute; the evaluation index corresponding to each element attribute in each row of the adjusted structured data is determined; for any element attribute: the subjective weight corresponding to the element attribute is determined based on the analytic hierarchy process (AHP); the objective weight corresponding to the element attribute is determined based on information entropy; the objective weight and the subjective weight are adjusted using adjustment parameters to generate a comprehensive weight corresponding to the element attribute; the corresponding comprehensive weight is applied to the standard value corresponding to the element attribute to obtain a comprehensive score; for any row of data: the comprehensive scores corresponding to each element attribute in the row are summed to generate a row data score; based on the row data scores corresponding to each row of data in the several rows, quasi-training samples and revision logs are obtained; if the structured data meets the preset requirements, the structured data is used as quasi-training samples.

[0100] S5. For any one of the several quasi-training samples: convert the quasi-training sample into BERT format and GPLinker format respectively; use the deep representation capability of BERT and the relation linking layer of GPLinker to perform dual training on the quasi-training sample for entity recognition and entity relation extraction to generate a knowledge extraction model.

[0101] In step S4, the objective weights are obtained as follows: each element attribute in the adjusted structured data is standardized to generate a standardized matrix formed by standard values; based on the standardized matrix, the information entropy corresponding to each standard value in the adjusted structured data is calculated; and based on the information entropy corresponding to each standard value, the objective weights corresponding to the element attributes are determined.

[0102] Subjective weights are obtained as follows: a judgment matrix is ​​generated based on the importance parameters of each element attribute in the adjusted structured data; when the consistency of the judgment matrix meets the preset conditions, the judgment matrix is ​​normalized to obtain a weight vector; and the subjective weight corresponding to each element attribute is determined based on the weight vector.

[0103] This implementation method achieves comprehensive optimization of the knowledge graph construction process, significantly improving data quality and ensuring the accuracy and reliability of extracted knowledge triples. It also effectively reduces labor costs, making the overall construction process more efficient and economical. Through automated data processing and intelligent annotation and revision, this implementation method reduces reliance on manual intervention and improves work efficiency.

[0104] Furthermore, this implementation method, by integrating a large-model-driven knowledge extraction approach, significantly enhances its ability to process complex text and cope with diverse information needs. This optimized method is not only applicable to current knowledge management and information retrieval fields but also provides stronger technical support for future intelligent applications, laying a solid foundation for the widespread application and continuous development of knowledge graphs.

[0105] like Figure 4 The diagram shown is a structural schematic of a knowledge graph creation device for the field of unmanned system testing provided by an embodiment of the present invention.

[0106] A knowledge graph creation apparatus for the field of unmanned system testing, the apparatus 400 comprising at least: an annotation processing module 401, used to annotate unstructured data to be extracted using a large language model to generate annotated structured data; a knowledge extraction module 402, used to extract knowledge from the structured data using a knowledge extraction model to generate knowledge triples; wherein the knowledge triples are used to indicate combinations formed by a first entity, inter-entity relations, and a second entity; and a construction module 403, used to construct a knowledge graph of a target domain based on several of the knowledge triples.

[0107] In a preferred embodiment of this invention, the apparatus further includes: an acquisition module, configured to use preprocessed and unlabeled unstructured data as candidate training samples, and acquire prompt words corresponding to the knowledge graph to be built from the candidate training samples; wherein the prompt words are words used to guide the model to extract knowledge; an annotation module, configured to annotate the candidate training samples using a large language model based on the prompt words to generate quasi-training samples; and a model training module, configured to train an NLP model based on several quasi-training samples to generate a knowledge extraction model.

[0108] In a preferred embodiment of this example, the acquisition module includes: a determination unit, used to determine the ontology graph of the knowledge graph to be built based on the target domain requirements corresponding to the candidate training samples; wherein the ontology graph is used to indicate the entity types, entity relationship types and entity constraint ranges involved in the target domain; and a construction unit, used to construct prompt words corresponding to the knowledge graph to be built based on the ontology graph.

[0109] In a preferred embodiment of this example, the acquisition module further includes: a text cleaning unit for cleaning the unlabeled unstructured data to generate cleaned data; a standardization unit for standardizing the cleaned data to generate standard data; and a conversion unit for converting the standard data into a preset format to obtain candidate training samples.

[0110] In a preferred embodiment of this example, the annotation module includes: a generation unit, configured to, based on the prompt words, sequentially perform data decomposition and completion processing on the candidate training samples using a large language model to generate several simple sentences; an identification unit, configured to, for any one of the several simple sentences, perform standardization verification processing on the simple sentence; if the verification passes, perform key knowledge point identification processing on the simple sentence using a large language model, and generate annotated structured data based on the identification results; and a determination unit, configured to, when the structured data does not meet preset requirements, adjust the structured data based on user adjustment requests to obtain quasi-training samples and revision logs; and when the structured data meets preset requirements, use the structured data as quasi-training samples.

[0111] In a preferred embodiment of this example, the determining unit includes: an adjustment subunit, used to adjust the structured data based on a user adjustment request to generate adjusted structured data; wherein the adjusted structured data includes several rows of data; each row of data includes at least one triplet sample; each element in each triplet sample has a corresponding element attribute; a first determining subunit, used to determine the evaluation index corresponding to each element attribute of each row of data in the adjusted structured data; a first generating subunit, used for any element attribute to: determine the subjective weight corresponding to the element attribute based on the analytic hierarchy process; determine the objective weight corresponding to the element attribute based on information entropy; adjust the objective weight and the subjective weight using adjustment parameters to generate the comprehensive weight corresponding to the element attribute; apply the corresponding comprehensive weight to the standard value corresponding to the element attribute to obtain a comprehensive score; a second generating subunit, used for any row of data to: sum the comprehensive scores corresponding to each element attribute in the row of data to generate a row data score; and an obtaining subunit, used to obtain quasi-training samples based on the row data scores corresponding to each row of data in the several rows of data.

[0112] In a preferred embodiment of this example, the determining unit further includes: a standardization subunit, used for The system performs standardization processing on each element attribute in the adjusted structured data to generate a standardized matrix formed by standard values; a calculation subunit is used to calculate the information entropy corresponding to each standard value in the adjusted structured data based on the standardized matrix; and a second determination subunit is used to determine the objective weight corresponding to the element attribute based on the information entropy corresponding to each standard value.

[0113] In a preferred embodiment of this example, the determining unit further includes: a third generating subunit, used to generate a judgment matrix based on the importance parameters of each element attribute in the adjusted structured data; a normalization processing unit, used to normalize the judgment matrix to obtain a weight vector when the consistency of the judgment matrix meets a preset condition; and a third determining subunit, used to determine the subjective weight corresponding to each element attribute based on the weight vector.

[0114] In a preferred embodiment of this example, the model training module includes: a format conversion unit, used to convert any one of the plurality of quasi-training samples into BERT format and GPLinker format respectively; and a dual training unit, used to perform dual training on the quasi-training samples for entity recognition and entity relation extraction using the deep representation capabilities of BERT and the relation linking layer of GPLinker, thereby generating a knowledge extraction model.

[0115] The aforementioned apparatus can execute a method for creating a knowledge graph in the field of unmanned system testing, as provided in an embodiment of the present invention. It possesses corresponding functional modules and beneficial effects for executing such a method. Technical details not described in detail in this embodiment can be found in the method for creating a knowledge graph in the field of unmanned system testing, as provided in an embodiment of the present invention.

[0116] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the knowledge graph creation method for the field of unmanned system testing described in the present invention.

[0117] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0118] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0119] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to the following embodiments of this application described in the "Exemplary Methods" section above.

[0120] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0121] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0122] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0123] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0124] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0125] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0126] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0128] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for creating a knowledge graph for unmanned system testing, characterized in that, include: Preprocessed and unlabeled unstructured data are used as candidate training samples, and prompt words corresponding to the knowledge graph to be built are obtained from the candidate training samples; wherein, the prompt words are used to guide the model to extract knowledge; the unstructured data are used to indicate relevant unstructured data collected from unmanned system test reports, academic papers and technical documents by means of web crawling, API interface or manual collection. Based on the prompt words, the candidate training samples are labeled using a large language model to generate quasi-training samples; Based on several quasi-training samples, an NLP model is used for training to generate a knowledge extraction model. The large language model is used to annotate the unstructured data to be extracted, generating annotated structured data; The structured data is processed by a knowledge extraction model to generate knowledge triples; wherein, the knowledge triples are used to indicate the combination formed by a first entity, the relationship between entities, and a second entity; Based on several of the aforementioned knowledge triples, a knowledge graph for the target domain is constructed. The step of obtaining prompt words corresponding to the knowledge graph to be built from the candidate training samples includes: Based on the target domain requirements corresponding to the candidate training samples, the ontology graph of the knowledge graph to be built is determined; wherein, the ontology graph is used to indicate the entity types, entity relationship types and entity constraint ranges involved in the target domain; based on the ontology graph, prompt words corresponding to the knowledge graph to be built are constructed. The step of annotating the candidate training samples using a large language model based on the prompt words to generate quasi-training samples includes: Based on the prompt words, the candidate training samples are sequentially processed by data decomposition and completion using a large language model to generate several simple sentences. For any one of the simple sentences, a standard verification process is performed. If the verification passes, the simple sentence is processed by key knowledge point identification using the large language model, and labeled structured data is generated based on the identification results. If the structured data does not meet the preset requirements, the structured data is adjusted based on the user's adjustment request to obtain quasi-training samples and revision logs; if the structured data meets the preset requirements, the structured data is used as quasi-training samples.

2. The method according to claim 1, characterized in that, The step of using preprocessed and unlabeled unstructured data as candidate training samples includes: The unlabeled unstructured data is cleaned to generate cleaned data; The cleaned data is then standardized to generate standard data; The standard data is converted into a preset format to obtain candidate training samples.

3. The method according to claim 1, characterized in that, The step of adjusting the structured data based on user adjustment requests to obtain quasi-training samples includes: Based on the user's adjustment request, the structured data is adjusted to generate adjusted structured data; wherein, the adjusted structured data includes several rows of data; each row of data includes at least one triplet sample; each element in each triplet sample has a corresponding element attribute; Determine the evaluation index corresponding to each element attribute in each row of data in the adjusted structured data; For any element attribute: determine the subjective weight corresponding to the element attribute based on the analytic hierarchy process; determine the objective weight corresponding to the element attribute based on information entropy; adjust the objective weight and the subjective weight using adjustment parameters to generate the comprehensive weight corresponding to the element attribute; apply the corresponding comprehensive weight to the standard value corresponding to the element attribute to obtain a comprehensive score. For any row of data: sum the comprehensive scores corresponding to each element attribute in the row of data to generate a row data score; Based on the row data score corresponding to each row of data in the aforementioned rows, quasi-training samples are obtained.

4. The method according to claim 3, characterized in that, Also includes: The attributes of each element in the adjusted structured data are standardized to generate a standardized matrix composed of standard values. Based on the standardization matrix, calculate the information entropy corresponding to each standard value in the adjusted structured data; Based on the information entropy corresponding to each standard value, the objective weight corresponding to the element attribute is determined.

5. The method according to claim 3, characterized in that, Also includes: A judgment matrix is ​​generated based on the importance parameters of each element attribute in the adjusted structured data; When the consistency of the judgment matrix meets the preset condition, the judgment matrix is ​​normalized to obtain the weight vector. The subjective weight corresponding to each element attribute is determined based on the weight vector.

6. The method according to claim 1, characterized in that, The step of training a knowledge extraction model using an NLP model based on several quasi-training samples to generate a model includes: For any one of the aforementioned quasi-training samples: convert the quasi-training sample into BERT format and GPLinker format respectively; By leveraging BERT's deep representation capabilities and GPLinker's relational linking layer, the quasi-training samples are subjected to dual training for entity recognition and entity relation extraction, thereby generating a knowledge extraction model.

7. A device for creating knowledge graphs in the field of unmanned system testing, characterized in that, include: The acquisition module is used to take preprocessed and unlabeled unstructured data as candidate training samples and obtain prompt words corresponding to the knowledge graph to be built from the candidate training samples; wherein, the prompt words are words used to guide the model to extract knowledge; the unstructured data are used to indicate relevant unstructured data collected from unmanned system test reports, academic papers and technical documents by means of web crawling, API interface or manual collection. The acquisition module includes: a determination unit, used to determine the ontology graph of the knowledge graph to be built based on the target domain requirements corresponding to the candidate training samples; wherein the ontology graph is used to indicate the entity types, entity relationship types and entity constraint ranges involved in the target domain; and a construction unit, used to construct prompt words corresponding to the knowledge graph to be built based on the ontology graph. The annotation module is used to annotate the candidate training samples based on the prompt words using a large language model to generate quasi-training samples. The annotation module is further used to perform data decomposition and completion processing on the candidate training samples sequentially using the prompt words and a large language model to generate several simple sentences. For any one of the several simple sentences: the simple sentence undergoes standardization verification; if the verification passes, the simple sentence undergoes key knowledge point identification processing using the large language model, and annotated structured data is generated based on the identification results; if the structured data does not meet preset requirements, the structured data is adjusted based on user adjustment requests to obtain quasi-training samples and revision logs; if the structured data meets preset requirements, the structured data is used as a quasi-training sample. The model training module is used to train an NLP model based on several quasi-training samples to generate a knowledge extraction model. The annotation processing module is used to use a large language model to annotate the unstructured data to be extracted, generating annotated structured data. The knowledge extraction module is used to perform knowledge extraction processing on the structured data using a knowledge extraction model to generate knowledge triples; wherein, the knowledge triples are used to indicate the combination formed by a first entity, the relationship between entities, and a second entity; The module is used to construct a knowledge graph of the target domain based on several of the aforementioned knowledge triples.

8. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-6.

Images