Large model question answering method and device based on knowledge graph, electronic equipment and medium
By applying a large-scale model question-and-answer method based on knowledge graphs in the field of metrology detection, the problems of data dispersion and slow response in traditional data acquisition systems are solved, data sorting and automatic question-and-answer are realized, and the accuracy and efficiency of reply are improved.
Patent Information
- Application Number
- CN202411964521.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-29
AI Technical Summary
The traditional metrological detection data acquisition system lacks professional data storage and processing methods, resulting in data dispersion or missing, and cannot achieve real-time response, resulting in poor timeliness, low reply accuracy and slow response.
The big model question and answer method based on knowledge graph is adopted to obtain the relevant data sets and ontology models of metrological detection, generate the knowledge graph, and use the knowledge graph to optimize the big model to achieve real-time response and accurate response of users' query information.
It realizes effective data sorting and automatic large-scale question and answers in the field of metrology and detection, improves the accuracy and efficiency of reply, and can respond in real time based on user retrieval.
Smart Images
Figure CN119903221A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large-model question-answering method, device, electronic device and medium based on a knowledge graph. Background Art
[0002] At present, in the field of measurement and testing, traditional data acquisition systems lack professional data storage and processing methods, resulting in scattered or missing data, which is not conducive to subsequent data analysis and other aspects. In addition, they cannot respond in real time according to user searches, and it takes a lot of time to search for answers. It is difficult to obtain answers simply and quickly, resulting in problems such as poor timeliness, low reply accuracy and slow response. Summary of the invention
[0003] The main purpose of the embodiments of the present application is to propose a large-model question-answering method, device, electronic device and medium based on a knowledge graph, which can realize effective data organization and large-model automatic question-answering in the field of measurement and detection, and improve the accuracy and efficiency of responses.
[0004] On the one hand, the embodiment of the present application proposes a large model question answering method based on a knowledge graph, and the method comprises the following steps:
[0005] Obtain metrology and testing related data sets and ontology models;
[0006] Generate a knowledge graph according to the measurement and detection related data set and the ontology model;
[0007] Get user query information;
[0008] The knowledge graph is used to optimize the big model, and the optimized big model is used to output corresponding target reply data according to the user query information.
[0009] In some embodiments, generating a knowledge graph according to the metrology detection related data set and the ontology model specifically includes:
[0010] Performing data preprocessing on the measurement and detection related data set to obtain a preprocessed measurement and detection related data set;
[0011] Dividing the preprocessed measurement detection related data set to obtain a first data set and a second data set;
[0012] Constructing a data annotation model, and training the data annotation model using the first data set;
[0013] Using the trained data annotation model to annotate the second data set to obtain a corresponding third data set;
[0014] According to the ontology model, data mapping is performed using the third data set to generate the knowledge graph.
[0015] In some embodiments, the constructing of the data annotation model and training the data annotation model using the first data set specifically includes:
[0016] In response to a labeling operation on each first sample data in the first data set, determining entity and relationship labeling information corresponding to each first sample data;
[0017] Binding each of the first sample data with the corresponding entity and relationship annotation information to generate a corresponding entity and relationship information data set;
[0018] The data annotation model is trained using the entity and relationship information dataset.
[0019] In some embodiments, the step of using the trained data annotation model to annotate the second data set to obtain a corresponding third data set specifically includes:
[0020] Input each second sample data in the second data set into the trained data annotation model to determine the model annotation information corresponding to each second sample data;
[0021] In response to the labeling operation on each of the second sample data, determining the manual labeling information corresponding to each of the second sample data;
[0022] Each of the second sample data is bound with the corresponding model annotation information and the manual annotation information to generate the third data set.
[0023] In some embodiments, the optimizing the large model by using the knowledge graph specifically includes:
[0024] Build the corresponding visual language model;
[0025] The knowledge graph is called to optimize the visual language model and adjust the model parameters corresponding to the visual language model.
[0026] In some embodiments, the method further comprises:
[0027] The big model is configured with rules, data query rules and data reply rules corresponding to the big model are determined, and the data query rules and the data reply rules are managed.
[0028] In some embodiments, the using of the optimized large model to output corresponding target reply data according to the user query information specifically includes:
[0029] Inputting the user query information into the optimized big model, using the optimized big model to query relevant data from the knowledge graph according to the data query rules and the user query information, and obtaining preliminary response data;
[0030] The optimized large model is used to process the preliminary response data according to the data response rules to generate the corresponding target response data.
[0031] On the other hand, the embodiment of the present application proposes a large model question-answering device based on a knowledge graph, the device comprising:
[0032] The first module is used to obtain the data sets and ontology models related to metrology and detection;
[0033] The second module is used to generate a knowledge graph based on the measurement and detection related data set and the ontology model;
[0034] The third module is used to obtain user query information;
[0035] The fourth module is used to optimize the big model using the knowledge graph, and use the optimized big model to output corresponding target reply data according to the user query information.
[0036] On the other hand, an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the large model question and answer method described above when executing the computer program.
[0037] On the other hand, an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the large model question and answer method described above.
[0038] The embodiments of the present application include at least the following beneficial effects: The present application provides a large model question-answering method, device, electronic device and medium based on a knowledge graph, which obtains data sets and ontology models related to metrology detection, generates a knowledge graph based on the data sets and ontology models related to metrology detection, obtains user query information, optimizes the large model using the knowledge graph, and outputs corresponding target reply data based on the user query information using the optimized large model. The present application can achieve effective data organization and large model automatic question-answering in the field of metrology detection, and responds in real time based on user retrieval through the large model, thereby improving the accuracy and efficiency of replies. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a flowchart of a large model question answering method based on knowledge graph provided in an embodiment of the present application;
[0040] Figure 2 It is a structural schematic diagram of a large model question-answering device based on a knowledge graph provided in an embodiment of the present application;
[0041] Figure 3 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.
[0043] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0044] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.
[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0046] Reference Figure 1 , Figure 1 It is an optional flowchart of a large model question answering method based on a knowledge graph provided in an embodiment of the present application. The method may include but is not limited to steps S101 to S104:
[0047] Step S101, obtaining measurement and detection related data sets and ontology models;
[0048] Step S102, generating a knowledge graph based on the measurement and detection related data set and the ontology model;
[0049] Step S103, obtaining user query information;
[0050] Step S104, use the knowledge graph to optimize the big model, and use the optimized big model to output corresponding target response data based on user query information.
[0051] In some embodiments, relevant data are collected from multiple sources such as standards, certificate reports, etc. in the field of metrology and testing, and the collected data are cleaned, including removing duplicate data, processing missing values, correcting erroneous data, etc. The data is converted into a format suitable for knowledge graph construction, such as JSON, XML, etc., to obtain the above-mentioned metrology and testing related data sets.
[0052] In some embodiments, step S102 may include but is not limited to steps S201 to S205:
[0053] Step S201, performing data preprocessing on a measurement detection related data set to obtain a preprocessed measurement detection related data set;
[0054] Step S202, dividing the preprocessed measurement and detection related data set to obtain a first data set and a second data set;
[0055] Step S203, constructing a data annotation model, and using the first data set to train the data annotation model;
[0056] Step S204, using the trained data annotation model to annotate the second data set to obtain a corresponding third data set;
[0057] Step S205: Perform data mapping using the third data set according to the ontology model to generate a knowledge graph.
[0058] In some embodiments, illustratively, the metrology detection related data set is divided into two parts in a ratio of 2:8 to obtain a first data set and a second data set.
[0059] In some embodiments, ontology technology is used to further define and classify entities and relationships to generate an ontology model in the field of metrology and detection. The ontology model provides a basic classification framework for information in the field of metrology and detection, and also provides a certain degree of logical description for the correlation between information in the field of metrology and detection, so that information resources in the field of metrology and detection can be organized into an organic whole based on the framework of the ontology description.
[0060] The ontology model consists of three main components, which are described as follows:
[0061] Classes: different types of data that exist;
[0062] Relationship: connects the attributes of two classes;
[0063] Attributes: Describe the properties of a single class.
[0064] When constructing the ontology model, first, identify the measurement and testing domain class (the only type of thing in the data). The measurement and testing domain class can include but is not limited to certificates, reports, standards, laboratories, testing methods and test equipment.
[0065] Then, identify the relationship and attributes. In order to simplify the identification process and improve identification efficiency, both relationship and entity attributes can be regarded as attributes. Use the measurement and testing domain classes defined previously to view the data and list all attributes of each class. For example, in the certificate class, the corresponding attributes can include but are not limited to reporting unit, preparer, and certificate number, etc. Some attributes are relationships connecting two classes. For example, "certificate has preparer" is a relationship attribute, which is a relationship connecting the certificate class and the preparer class. Other attributes, such as "date of issuance of certificate" are entity attributes, which only describe one class instead of connecting two classes together.
[0066] In some embodiments, the ontology model is used as a basic framework, and a third data set is added to create a knowledge graph. Optionally, first, based on the third data set and the ontology model, the nodes and edges of the knowledge graph are constructed, the nodes represent entities in the field of metrology and detection, and the edges represent the relationships between entities. Finally, a graph database (such as Neo4j) is used to store and display the knowledge graph.
[0067] In some embodiments, step S203 may include but is not limited to steps S301 to S303:
[0068] Step S301, in response to a labeling operation on each first sample data in a first data set, determining entity and relationship labeling information corresponding to each first sample data;
[0069] Step S302, binding each first sample data with the corresponding entity and relationship annotation information to generate a corresponding entity and relationship information data set;
[0070] Step S303: training the data annotation model using the entity and relationship information dataset.
[0071] In some embodiments, optionally, each first sample data in the first data set is manually labeled to determine the entity and relationship labeling information corresponding to each first sample data, and the entity and relationship labeling information includes key information such as the number of the measurement and testing certificate, the actual measured value of the test item, the point, the labeled number, the specification model of the test equipment, etc.
[0072] Then, each first sample data is bound to the corresponding entity and relationship annotation information to generate a corresponding entity and relationship information data set, and the entity and relationship information data set is used to train a convolutional neural network to obtain a corresponding data annotation model, which is used for subsequent automatic annotation of the second data set.
[0073] In some embodiments, step S204 may include but is not limited to steps S401 to S403:
[0074] Step S401, inputting each second sample data in the second data set into the trained data annotation model to determine the model annotation information corresponding to each second sample data;
[0075] Step S402, in response to the labeling operation on each second sample data, determining the manual labeling information corresponding to each second sample data;
[0076] Step S403: Bind each second sample data with the corresponding model annotation information and manual annotation information to generate a third data set.
[0077] In some embodiments, each second sample data in the second data set is semi-automatically labeled. The semi-automatic labeling technology combines the advantages of manual and automatic labeling, can effectively improve the labeling efficiency and quality, and reduce the burden of manual labeling.
[0078] Specifically, first, each second sample data in the second data set is automatically labeled using a data labeling model to determine entity and relationship labeling information output by the model corresponding to each second sample data, that is, the above-mentioned model labeling information.
[0079] Then, based on the model annotation information of each second sample data, each second sample data is manually annotated, the entity and relationship annotation information of each second sample data is improved, and the manual annotation information corresponding to each second sample data is determined. Finally, each second sample data is bound to the corresponding model annotation information and manual annotation information to generate the above-mentioned third data set.
[0080] In step S104 of some embodiments, the large model is optimized using the knowledge graph. Specifically, a corresponding visual language model is constructed, the knowledge graph is called, the visual language model is optimized, and the model parameters corresponding to the visual language model are adjusted.
[0081] In some embodiments, the visual language model is a VLM (Vision Language Models) model. The VLM model is a model that combines visual and language information and is designed to process multimodal data, including images, videos, and text. They can automatically generate descriptive text for images, understand natural language questions and provide answers based on image content, generate images or videos, and perform automatic translation, language learning, and other applications. Among them, the VLM model may include but is not limited to CLIP, BLIP, BLIP2, Flamingo, LLaVA, MiniCPT, InstructBLIP, and mPLUG-owl.
[0082] In some embodiments, the knowledge graph is used to fine-tune the visual language model, optimize the model parameters corresponding to the visual language model, and improve the question prediction effect and response accuracy of the visual language model.
[0083] In some embodiments, step S104 may include but is not limited to steps S501 to S502:
[0084] Step S501, inputting the user query information into the optimized big model, using the optimized big model to query relevant data from the knowledge graph according to the data query rules and the user query information, and obtaining preliminary response data;
[0085] Step S502, using the optimized large model to process the preliminary response data according to the data response rules to generate corresponding target response data.
[0086] In some embodiments, the large model question answering method may further include step S601:
[0087] Step S601, configure rules for the big model, determine data query rules and data reply rules corresponding to the big model, and manage the data query rules and data reply rules.
[0088] In some embodiments, data query rules are configured, for example, based on user query information, all key data related to the user query information in the knowledge graph is obtained, and all key data is returned in a specified format (such as JSON format) to obtain preliminary response data.
[0089] In some embodiments, data reply rules are configured, for example, to remove duplicate information in the preliminary reply data and to organize the cleaned preliminary reply data in a specific reply format to obtain target reply data.
[0090] Reference Figure 2 , Figure 2: is an optional structural diagram of a large model question-answering device based on a knowledge graph provided in an embodiment of the present application. The device is used to implement the above-mentioned large model question-answering method. The device may include:
[0091] The first module is used to obtain the data sets and ontology models related to metrology and detection;
[0092] The second module is used to generate a knowledge graph based on the measurement and detection related data sets and ontology models;
[0093] The third module is used to obtain user query information;
[0094] The fourth module is used to optimize the big model using the knowledge graph, and use the optimized big model to output the corresponding target response data based on the user query information.
[0095] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0096] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned large model question-answering method when executing the computer program. The electronic device can be any smart terminal including a tablet computer.
[0097] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0098] See also Figure 3 , Figure 3 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0099] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0100] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the large model question-answering method of the embodiments of this application;
[0101] Input / output interface 903, used to implement information input and output;
[0102] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);
[0103] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0104] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0105] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned large model question and answer method is implemented.
[0106] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0107] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0108] The embodiments of the present application provide a large-model question-and-answer method, device, electronic device, and medium based on a knowledge graph, which can achieve effective data organization and large-model automatic question-and-answer in the field of metrology and detection, and respond to user searches in real time through a large model, thereby improving the accuracy and efficiency of responses.
[0109] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0110] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0111] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0112] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0113] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0114] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A large model question answering method based on knowledge graph, characterized in that: The method comprises the following steps: Obtain metrology and testing related data sets and ontology models; Generate a knowledge graph according to the measurement and detection related data set and the ontology model; Get user query information; The knowledge graph is used to optimize the big model, and the optimized big model is used to output corresponding target reply data according to the user query information.
2. The large model question answering method according to claim 1, characterized in that: The generating of a knowledge graph according to the measurement detection related data set and the ontology model specifically includes: Performing data preprocessing on the measurement and detection related data set to obtain a preprocessed measurement and detection related data set; Dividing the preprocessed measurement detection related data set to obtain a first data set and a second data set; Constructing a data annotation model, and training the data annotation model using the first data set; Using the trained data annotation model to annotate the second data set to obtain a corresponding third data set; According to the ontology model, data mapping is performed using the third data set to generate the knowledge graph.
3. The large model question answering method according to claim 2, characterized in that: The constructing of the data annotation model, and training the data annotation model using the first data set, specifically includes: In response to a labeling operation on each first sample data in the first data set, determining entity and relationship labeling information corresponding to each first sample data; Binding each of the first sample data with the corresponding entity and relationship annotation information to generate a corresponding entity and relationship information data set; The data annotation model is trained using the entity and relationship information dataset.
4. The large model question answering method according to claim 2, characterized in that: The step of using the trained data annotation model to annotate the second data set to obtain a corresponding third data set specifically includes: Input each second sample data in the second data set into the trained data annotation model to determine the model annotation information corresponding to each second sample data; In response to the labeling operation on each of the second sample data, determining the manual labeling information corresponding to each of the second sample data; Each of the second sample data is bound with the corresponding model annotation information and the manual annotation information to generate the third data set.
5. The large model question answering method according to claim 1, characterized in that: The optimization of the large model by using the knowledge graph specifically includes: Build the corresponding visual language model; The knowledge graph is called to optimize the visual language model and adjust the model parameters corresponding to the visual language model.
6. The large model question answering method according to claim 2, characterized in that: The method further comprises: The big model is configured with rules, data query rules and data reply rules corresponding to the big model are determined, and the data query rules and the data reply rules are managed.
7. The large model question answering method according to claim 6, characterized in that: The method of using the optimized large model to output corresponding target reply data according to the user query information specifically includes: Inputting the user query information into the optimized big model, using the optimized big model to query relevant data from the knowledge graph according to the data query rules and the user query information, and obtaining preliminary response data; The optimized large model is used to process the preliminary response data according to the data response rules to generate the corresponding target response data.
8. A large model question-answering device based on knowledge graph, characterized in that: The device comprises: The first module is used to obtain the data sets and ontology models related to metrology and detection; The second module is used to generate a knowledge graph based on the measurement and detection related data set and the ontology model; The third module is used to obtain user query information; The fourth module is used to optimize the big model using the knowledge graph, and use the optimized big model to output corresponding target reply data according to the user query information.
9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the large model question-answering method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the large model question-answering method according to any one of claims 1 to 7 is implemented.