A knowledge graph strategy evaluation method and related device

Abstract

The present application discloses a method for evaluating a strategy of a knowledge graph and a related device, which relates to the field of artificial intelligence, including: obtaining a question, an annotated answer to the question, a text content containing the annotated answer, and an interpretable path representing the guiding information for answering the question; constructing a knowledge graph based on the text content through a preset knowledge graph construction strategy; obtaining a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy; inputting the knowledge graph, the predicted answer, the standard answer, and the interpretable path into a pre-built strategy analysis model, and obtaining a prediction result through the strategy analysis model, which is a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy. Which strategy among the strategies of the knowledge graph obtained through the model may need to be optimized without human involvement. This makes the maintenance and update of the knowledge graph simple.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a knowledge graph strategy evaluation method and related devices. Background Art

[0002] With the rapid development of artificial intelligence technology, knowledge graphs have become an important tool for organizing and understanding massive amounts of information. By representing structured data in the form of graphs, knowledge graphs can reveal the relationships between entities, thereby supporting complex queries and intelligent applications. Although knowledge graphs have shown great potential in many fields, there are still some challenges and limitations in setting up knowledge graph construction strategies and knowledge graph query strategies.

[0003] The knowledge graph construction strategy refers to the strategy of extracting, integrating and constructing the knowledge graph from the text content; the knowledge graph query strategy refers to the strategy of retrieving information, discovering the relationship between entities and obtaining answers to questions in the knowledge graph. If there is an error in the knowledge graph construction strategy, the knowledge graph obtained by the knowledge graph construction strategy may have errors; if there is an error in the knowledge graph query strategy, the answer obtained from the knowledge graph through the knowledge graph query strategy is likely to be wrong.

[0004] In related technologies, domain experts are needed to discover whether there are problems with the knowledge graph construction strategy or the knowledge graph query strategy. The lack of an automated mechanism makes it difficult to maintain and update the knowledge graph. Summary of the invention

[0005] In view of the above problems, this application provides a knowledge graph strategy evaluation method and related devices to achieve the purpose of automatically detecting whether there are problems with the knowledge graph construction strategy or the knowledge graph query strategy. The specific scheme is as follows:

[0006] The first aspect of the present application provides a method for evaluating a knowledge graph strategy, comprising:

[0007] Obtaining a question, an annotated answer to the question, text content containing the annotated answer, and an interpretable path representing guidance information for answering the question;

[0008] A knowledge graph is constructed based on the text content using a preset knowledge graph construction strategy;

[0009] Obtaining a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy;

[0010] The knowledge graph, the predicted answer, the standard answer and the explainable path are input into a pre-built strategy analysis model, and a prediction result is obtained through the strategy analysis model. The prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

[0011] In a possible implementation, the knowledge graph construction strategy includes a semantic segmentation strategy, a sharding strategy, a triple extraction strategy, and a graph construction strategy. The steps of constructing a knowledge graph based on the text content using a preset knowledge graph construction strategy include:

[0012] Acquire multiple semantic units from the text content by using the semantic segmentation strategy;

[0013] Segmenting the text content into a plurality of text segments by combining the segmentation strategy with the plurality of semantic units;

[0014] Extracting triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and a relationship between the two entities;

[0015] The plurality of triples are constructed into a knowledge graph through the graph construction strategy.

[0016] In a possible implementation, the knowledge graph query strategy includes: text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy. The step of obtaining a predicted answer from the knowledge graph based on the question through the preset knowledge graph query strategy includes:

[0017] Converting the question into a question vector by using the text vectorization strategy;

[0018] identifying a plurality of query key vectors from the question vector by using the chunk matching strategy;

[0019] Obtaining subgraphs corresponding to a plurality of query key vectors from the knowledge graph through the subgraph query strategy;

[0020] Filtering out a plurality of target subgraphs from the plurality of subgraphs by using the branch reduction and fusion strategy;

[0021] Sorting the multiple target subgraphs by using the rerank reordering strategy to obtain a sorting result;

[0022] The prediction answer is obtained from the sorting results through the large model summary strategy.

[0023] In a possible implementation, the step of inputting the knowledge graph, the predicted answer, the standard answer, and the explainability path into a pre-built strategy analysis model, and obtaining a prediction result through the strategy analysis model includes:

[0024] The multiple semantic units, the multiple text fragments, the multiple triples, the knowledge graph, the question vector, the multiple query key vectors, the multiple subgraphs, the multiple target characters, the sorting results, the predicted answers, the labeled answers and the explainable paths are input into the strategy analysis model, and the prediction results are obtained through the strategy analysis model. The first probability includes the probability of problems with the semantic segmentation strategy, the probability of problems with the sharding strategy, the probability of problems with the triple extraction strategy, and the probability of problems with the graph construction strategy; the second probability includes the probability of problems with the text vectorization strategy, the probability of problems with the chunk matching strategy, the probability of problems with the subgraph query strategy, the probability of problems with the branch reduction and fusion strategy, the probability of problems with the rerank reordering strategy, and the probability of problems with the large model summary strategy.

[0025] In a possible implementation, the method further includes:

[0026] Determining a target text content in which the predicted answer is different from the standard answer from the plurality of text contents;

[0027] For each strategy problem, determine the target number of target text contents in multiple target texts whose probability is greater than or equal to the preset threshold corresponding to the strategy problem; the strategy problem is any one of semantic segmentation strategy, sharding strategy, triple extraction strategy, graph construction strategy, text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy;

[0028] For each strategic problem, the ratio of the number of targets corresponding to the strategic problem to the number of all target text contents is determined as the optimization probability corresponding to the strategic problem.

[0029] The second aspect of the present application provides a knowledge graph strategy evaluation device, including:

[0030] A first acquisition module is used to acquire a question, a marked answer to the question, text content containing the marked answer, and an interpretable path representing guidance information for solving the question;

[0031] A construction module, used to construct a knowledge graph based on the text content through a preset knowledge graph construction strategy;

[0032] A second acquisition module is used to obtain a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy;

[0033] The third acquisition module is used to input the knowledge graph, the predicted answer, the standard answer and the explainability path into a pre-built strategy analysis model, and obtain the prediction result through the strategy analysis model. The prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

[0034] In a possible implementation, the knowledge graph construction strategy includes a semantic segmentation strategy, a sharding strategy, a triple extraction strategy, and a graph construction strategy, and the construction modules include:

[0035] A first acquisition unit, configured to acquire a plurality of semantic units from the text content by using the semantic segmentation strategy;

[0036] A segmentation unit, configured to segment the text content into a plurality of text segments by combining the segmentation strategy with the plurality of semantic units;

[0037] An extraction unit, configured to extract triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and a relationship between the two entities;

[0038] A construction unit is used to construct the plurality of triples into a knowledge graph through the graph construction strategy.

[0039] A third aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement a method for evaluating a strategy of a knowledge graph according to the first aspect or any implementation of the first aspect.

[0040] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0041] The memory is used to store computer programs;

[0042] The processor is used to execute the computer program so that the electronic device can implement the strategy evaluation method of the knowledge graph of the above-mentioned first aspect or any implementation method of the first aspect.

[0043] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement a method for evaluating the strategy of the knowledge graph of the above-mentioned first aspect or any implementation of the first aspect.

[0044] By means of the above technical scheme, the present application provides a method for evaluating the strategy of a knowledge graph, obtaining questions, annotated answers to questions, text content containing annotated answers, and an interpretable path representing the guiding information for answering questions; constructing a knowledge graph based on text content through a preset knowledge graph construction strategy; obtaining predicted answers from the knowledge graph based on questions through a preset knowledge graph query strategy; inputting the knowledge graph, predicted answers, standard answers, and interpretable paths into a pre-built strategy analysis model, and obtaining a prediction result through the strategy analysis model, which is a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy. Which strategy among the strategies of the knowledge graph obtained through the model may need to be optimized without human involvement. This makes the maintenance and updating of the knowledge graph simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0046] Figure 1 A schematic diagram of a system architecture provided for this application;

[0047] Figure 2 A schematic diagram of an optional hardware structure of the terminal 100 provided in this application;

[0048] Figure 3 A schematic diagram of the structure of the server 200 provided in this application;

[0049] Figure 4 A schematic diagram of a flow chart of a knowledge graph strategy evaluation method provided in this application;

[0050] Figure 5 A schematic diagram of the structure of a knowledge graph strategy evaluation device provided in this application;

[0051] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0056] Among them, an application or interface can be installed on the terminal 100. The above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the interface, such as instructions for triggering a strategy for evaluating the knowledge graph, and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters, such as which strategy of the knowledge graph needs to be optimized, and return the processing results to the terminal 100.

[0057] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0058] Next describe Figure 1 The product form of the mid-terminal 100;

[0059] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0060] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0061] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0062] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0063] Among them, the input device 132 can receive input data and the like.

[0064] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 may be used to display interfaces, processing results, etc.

[0065] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0066] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0067] Among them, the memory 120 can be used to store software codes related to the evaluation method of the knowledge graph strategy, the processor 170 can execute the steps of the evaluation method of the knowledge graph strategy, and can also schedule other units (such as the above-mentioned input unit 130 and the display unit 140) to implement corresponding functions.

[0068] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0069] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0070] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0071] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0072] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0073] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0074] Next describe Figure 1 The product form of the server 200;

[0075] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0076] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0077] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0078] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0079] Among them, the memory 204 can be used to store software codes related to the evaluation method of the knowledge graph strategy, the processor 202 can execute the steps of the evaluation method of the knowledge graph strategy of the chip, and can also schedule other units to implement corresponding functions.

[0080] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0081] Reference Figure 4 , Figure 4 A schematic diagram of a flow chart of a knowledge graph strategy evaluation method provided in an embodiment of the present application, such as Figure 4 As shown, a knowledge graph strategy evaluation method provided in an embodiment of the present application may include steps S401 to S404, and these steps are described in detail below.

[0082] Step S401: Obtain a question, a marked answer to the question, text content containing the marked answer, and an interpretable path representing guidance information for solving the question.

[0083] The above terms are explained below.

[0084] A question refers to a query request raised by a user, with the aim of obtaining specific information or answers from the knowledge graph.

[0085] The annotated answer to a question refers to the answer to the question obtained by the researcher from the text content.

[0086] The text content is in text format, table format or PDF format. The amount of text content can be determined based on actual conditions and is not limited in this application.

[0087] An explainability path is a series of steps or logical reasoning or data flow or statements that explain how to obtain an annotated answer from the text content based on the question.

[0088] The following examples illustrate the above terms.

[0089] Question: A 20-year-old female always dozes off during the day and sometimes falls asleep suddenly. What disease could she have?

[0090] Text content: It is recommended that you sleep no less than 8 hours a day. Psychological stress and anxiety may lead to poor sleep quality. It is recommended to relieve stress through exercise, meditation, and consultation with a psychologist. Malnutrition or lack of certain nutrients, such as iron, vitamin D, or B vitamins, may also lead to drowsiness. Therefore, it is recommended to eat a balanced diet and supplement nutrients when necessary. Drowsiness may be caused by a variety of reasons, including but not limited to ascending aortic aneurysm and primary carnitine deficiency. In addition, lethargy may be related to refractory hypertension.

[0091] The labeled answers to the above question are: ascending aortic aneurysm, primary carnitine deficiency, or resistant hypertension.

[0092] Possible explanations for the above problem include: falling asleep suddenly and being sick.

[0093] In summary, different problems correspond to different interpretability paths.

[0094] Step S402: construct a knowledge graph based on the text content using a preset knowledge graph construction strategy.

[0095] Exemplarily, a knowledge graph construction strategy refers to a strategy for extracting, integrating, and constructing a knowledge graph from text content.

[0096] Exemplarily, the knowledge graph construction strategy may be determined by experts.

[0097] Step S403: Obtain a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy.

[0098] Exemplarily, a knowledge graph query strategy refers to a strategy for retrieving information in a knowledge graph, discovering relationships between entities, and obtaining answers to questions.

[0099] Exemplarily, the knowledge graph construction strategy may be determined by experts.

[0100] Step S404: Input the knowledge graph, the predicted answer, the standard answer and the explainable path into a pre-built strategy analysis model, and obtain a prediction result through the strategy analysis model, wherein the prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

[0101] The construction process of the strategy analysis model is explained below.

[0102] The sample knowledge graph, the labeled answers to the sample questions, the predicted answers to the sample questions, and the explainable paths corresponding to the sample questions are taken as input, and the labeling probabilities of problems with the knowledge graph construction strategy and the labeling probabilities of problems with the knowledge graph query strategy are taken as training targets to train a strategy analysis model.

[0103] Exemplarily, the process of obtaining the sample knowledge graph of the sample question is shown in step S401 to step S402, and the process of obtaining the predicted answer of the sample question is shown in step S403. The interpretability path corresponding to the sample question can be manually annotated.

[0104] In an optional implementation, for sample questions, the probability of marking that there is a problem with the knowledge graph construction strategy is 1 or 0, and the probability of marking that there is a problem with the knowledge graph query strategy is 1 or 0.

[0105] It is understandable that if the first probability is higher than the second probability, the R&D personnel will focus the review on the optimization of the knowledge graph construction strategy; if the second probability is higher than the first probability, the R&D personnel will focus the review on the optimization of the knowledge graph query strategy.

[0106] It can be understood that, for text content where the predicted answer is the same as the marked answer, the first probability and the second probability are both 0; for text content where the predicted answer is different from the marked answer, the first probability or the second probability is not 0.

[0107] Exemplarily, the process of training the strategy analysis model involves at least one of the techniques in machine learning, such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and method-based learning.

[0108] Exemplarily, the strategy analysis model may be any one of a neural network model, a logistic regression model, a linear regression model, a support vector machine (SVM), an Adaboost, an XGboost, and a Transformer-Encoder model.

[0109] Exemplarily, the neural network model can be any one of a recurrent neural network-based model, a convolutional neural network-based model, and a Transformer-encoder-based classification model.

[0110] Exemplarily, the policy analysis model may be a deep hybrid model of a recurrent neural network-based model, a convolutional neural network-based model, and a Transformer-encoder-based classification model.

[0111] Exemplarily, the strategy analysis model can be any one of an attention-based deep model, a memory network-based deep model, and a deep learning-based short text classification model.

[0112] The short text classification model based on deep learning is a recurrent neural network (RNN) or a convolutional neural network (CNN) or a variant based on a recurrent neural network or a convolutional neural network.

[0113] For example, some simple domain adaptation modifications can be made to the pre-trained model to obtain a strategy analysis model.

[0114] Exemplarily, “simple domain adaptation” includes but is not limited to re-pre-training an already pre-trained model using large-scale unsupervised domain corpus, and / or compressing an already pre-trained model by model distillation.

[0115] The embodiment of the present application provides a method for evaluating the strategy of a knowledge graph, obtaining questions, annotated answers to the questions, text content containing annotated answers, and an interpretable path that represents the guidance information for answering the questions; constructing a knowledge graph based on the text content through a preset knowledge graph construction strategy; obtaining a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy; inputting the knowledge graph, the predicted answer, the standard answer, and the interpretable path into a pre-built strategy analysis model, and obtaining a prediction result through the strategy analysis model, which is a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy. Which strategy among the strategies of the knowledge graph obtained by the model may need to be optimized without human involvement. This makes the maintenance and updating of the knowledge graph simple.

[0116] In an optional implementation, the knowledge graph construction strategy includes but is not limited to: semantic segmentation strategy, sharding strategy, triple extraction strategy, and graph construction strategy. The specific implementation of step S402 includes the following steps A1 to A4.

[0117] Step A1: Acquire multiple semantic units from the text content by using the semantic segmentation strategy.

[0118] Semantic segmentation strategy refers to the method and process of decomposing text content into semantic units with clear semantic meanings during the construction of knowledge graph. These semantic units can represent key information in the text content, such as entities, concepts, events or actions, and they are the basic building blocks for building knowledge graphs. The purpose of semantic segmentation strategy is to better understand and process text content, so as to accurately identify and extract information for building knowledge graphs.

[0119] Exemplarily, the method for implementing the semantic segmentation strategy includes the following steps B1 to B5.

[0120] Step B1: Segment the text content to obtain multiple keywords.

[0121] Exemplarily, NLP (Natural Language Processing) technology may be used for segmentation to obtain multiple keywords.

[0122] Step B2: Perform part-of-speech tagging for multiple keywords.

[0123] Exemplarily, the parts of speech of a keyword include, but are not limited to: noun, verb, and adjective.

[0124] Step B3: Based on the part of speech of each keyword, the dependency relationship between sentences in the text content is analyzed to determine the grammatical structure.

[0125] Exemplarily, a dependency parser or a NER tool may be used to obtain the dependency relationship between sentences.

[0126] Step B4: Based on the part of speech and grammatical structure of each keyword, identify entities from the text content.

[0127] Exemplarily, the entity may be a person's name, a place name, an organization name, or a disease.

[0128] Step B5: Segment the text content into multiple semantic units based on each entity.

[0129] Exemplarily, the semantic unit includes an entity and an entity type to which the entity belongs. Exemplarily, the entity type to which the entity belongs includes, but is not limited to: a person's name, a place name, an organization name, and a disease.

[0130] Step A2: Segment the text content into multiple text segments by combining the segmentation strategy with the multiple semantic units.

[0131] Exemplarily, after obtaining multiple semantic units, the fragmentation strategy organizes these semantic units into more meaningful text fragments according to certain logical relationships, preparing for subsequent triple extraction and knowledge graph construction.

[0132] Exemplarily, a text segment includes multiple semantic units.

[0133] The following example illustrates the relationship between semantic units and text fragments.

[0134] Assume that the text content is: Apple was founded by Steve Jobs, Steve Wozniak and Ronald Wayne in California on April 1, 1976. The company was originally named Apple Computer, Inc., and was not renamed Apple Inc. until January 9, 2007.

[0135] The multiple semantic units obtained through the semantic segmentation strategy are: Apple, Steve Jobs, Steve Wozniak, Ronald Wayne, California, April 1, 1976, Apple Computer, January 9, 2007.

[0136] Through the fragmentation strategy, two text segments are obtained, namely: Segment 1 "Apple was founded by Steve Jobs, Steve Wozniak and Ronald Wayne in California on April 1, 1976"; Segment 2 "The company was originally named Apple Computer, Inc., and was not renamed Apple Inc. until January 9, 2007."

[0137] In summary, the sharding strategy provides context for extracting entities and relationships between entities in triples.

[0138] Step A3: extracting triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and the relationship between the two entities.

[0139] Exemplarily, a triple includes two entities and a relationship between the two entities, which is described below with an example.

[0140] As shown in the above text content, 7 triplets can be obtained, namely: Triple 1 (Apple Inc., founder, Steve Jobs), Triple 2 (Apple Inc., founder, Steve Wozniak), Triple (Apple Inc., founder, Ronald Wayne), Triple 4 (Apple Inc., date of establishment, April 1, 1976), Triple (Apple Inc., location of establishment, California), Triple 6 (Apple Computer Inc., changed its name to Apple Inc.), Triple 7 (Apple Computer Inc., date of name change, January 9, 2007).

[0141] Exemplarily, triples may be stored in a structured format, such as RDF (Resource Description Framework).

[0142] Step A4: construct the plurality of triples into a knowledge graph through the graph construction strategy.

[0143] Exemplarily, multiple triples are constructed into a knowledge graph, which can be represented as a directed graph, in which nodes represent entities and edges represent relationships between entities.

[0144] Exemplarily, the knowledge graph constructed by the above 7 triples is as follows:

[0145] Apple

[0146] ├── Founder──> Steve Jobs

[0147] ├── Founder──> Steve Wozniak

[0148] ├── Founder──> Ronald Wayne

[0149] ├── Date of Establishment──> April 1, 1976

[0150] ├── Established in California

[0151] └── Former name──> Apple Computer

[0152] └── Name change date──> January 9, 2007

[0153] In an optional implementation, the knowledge graph query strategy includes but is not limited to: text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy. Step S403 may include the following steps C1 to C6.

[0154] Step C1: Convert the question into a question vector using the text vectorization strategy.

[0155] For example, word embedding technology can be used to convert each word in the question into a vector in a high-dimensional space.

[0156] For example, the question vector of the question can be obtained using a Bag of Words (BOW) model, an N-gram model, or a One-Hot Encoding.

[0157] Step C2: identifying multiple query key vectors from the question vector using the chunk matching strategy.

[0158] Chunk matching strategy is a method for identifying multiple query key vectors from the question vector, which usually involves identifying key information in the question, such as entities, attributes, or relations, and mapping them to corresponding concepts in the knowledge graph.

[0159] Exemplarily, there are multiple ways to implement step C2. The embodiment of the present application provides but is not limited to the following method, which includes the following steps D1 to D8.

[0160] Step D1: Split the question vector into word vectors corresponding to individual words.

[0161] Step D2: Use the NER (Named Entity Recognition) model to identify entities in the word vector.

[0162] For example, entities include names of people, places, organizations, etc.

[0163] Step D3: Match the identified entities with the entities in the knowledge graph and determine their corresponding nodes in the knowledge graph to obtain matching results.

[0164] Exemplarily, the identified entities are entity linked with entities in the knowledge graph. This process is also called entity resolution or entity alignment. The purpose of entity linking is to determine the corresponding node in the knowledge graph for the entity in the question.

[0165] Step D4: According to the matching results, determine the word vector most relevant to the question vector as the query key vector.

[0166] For example, if the question is “In which year was Apple founded?”, the query key vectors may be the query key vectors corresponding to “Apple” and “founding year” respectively.

[0167] Step C3: obtaining subgraphs corresponding to multiple query key vectors from the knowledge graph through the subgraph query strategy.

[0168] Exemplarily, the subgraph contains entities and relations directly related to the query key vector. For example, a subgraph related to "Apple" is retrieved from the knowledge graph constructed from the above 7 triples. This subgraph may contain information such as the founder, products, and founding time of Apple.

[0169] Step C4: Filter out multiple target subgraphs from the multiple subgraphs by using the branch reduction and fusion strategy.

[0170] Exemplarily, the most relevant target subgraph is screened out from the retrieved multiple subgraphs, that is, a preset number of subgraphs having the highest similarity or correlation with the query key vector are determined as target subgraphs.

[0171] For example, multiple subgraphs corresponding to "Apple Inc." may be retrieved, but only the subgraph containing "founding time" information is the most relevant, so this subgraph is determined to be the target subgraph.

[0172] Step C5: sorting the multiple target subgraphs by using the rerank reordering strategy to obtain a sorting result.

[0173] Exemplarily, the rerank reordering strategy may include a rerank model, which may be a model based on deep learning, such as Cross-Encoder, FlashRank, T5 Seq2Seq, etc. The scores of multiple target subgraphs may be obtained through the rerank model, and the scores of the multiple target subgraphs may be sorted to obtain a sorting result.

[0174] Exemplarily, the multiple screened target subgraphs are sorted to determine which target subgraph is most likely to contain the correct answer.

[0175] For example, if there are multiple target subgraphs related to "Apple" and "founding date", they may be ranked according to the centrality of the entities in the target subgraph or the strength of the relationship.

[0176] Step C6: Obtain a prediction answer from the ranking results through the large model summary strategy.

[0177] Exemplarily, the large model summarization strategy may include large language models, such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), etc.

[0178] Exemplarily, the implementation of step C6 includes the following steps E1 to E4.

[0179] Step E1: Convert the target subgraph in the ranking result into an input format that can be processed by a large language model.

[0180] Step E2: Convert the target subgraph into an embedded representation through a large language model.

[0181] Step E3: Compute the similarity between the question vector and the embedded representation of each target subgraph.

[0182] Exemplarily, the similarity is calculated by methods such as cosine similarity.

[0183] Step E4: Select the answers in the Top-K target subgraphs with the highest similarity as the final predicted answers.

[0184] For example, the highest-ranked target subgraph in the above example contains the "founded" relationship between "Apple Inc." and "1976", so the large model summary strategy obtains the predicted answer that "Apple Inc." was founded in 1976 and returns the predicted answer to the user.

[0185] In an optional implementation, the specific implementation of step S404 is: input the multiple semantic units, the multiple text fragments, the multiple triples, the knowledge graph, the question vector, the multiple query key vectors, the multiple subgraphs, the multiple target characters, the sorting results, the predicted answers, the labeled answers and the interpretability paths into the strategy analysis model, and obtain the prediction results through the strategy analysis model. The first probability includes the probability of problems with the semantic segmentation strategy, the probability of problems with the sharding strategy, the probability of problems with the triple extraction strategy, and the probability of problems with the graph construction strategy; the second probability includes the probability of problems with the text vectorization strategy, the probability of problems with the chunk matching strategy, the probability of problems with the subgraph query strategy, the probability of problems with the branch reduction and fusion strategy, the probability of problems with the rerank reordering strategy, and the probability of problems with the large model summary strategy.

[0186] It can be understood that the strategy problem is any one of the semantic segmentation strategy, sharding strategy, triple extraction strategy, graph construction strategy, text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy. There are multiple text contents in step S401, and the strategy analysis model determines the target text content whose predicted answer is different from the marked answer from the multiple text contents; the optimization probability of each strategy problem can be directly output through multiple target text contents. That is, the first probability and the second probability are the optimization probabilities.

[0187] In an optional implementation, the method for determining the optimization probability of the present application further includes the following steps F1 to F3.

[0188] Step F1: determining a target text content in which the predicted answer is different from the standard answer from the plurality of text contents.

[0189] Step F2: for each strategic question, determining a target number of target text contents in a plurality of target texts whose probability is greater than or equal to a preset threshold corresponding to the strategic question.

[0190] Exemplarily, the preset thresholds corresponding to different policy issues may be the same or different, and may be determined based on actual conditions.

[0191] Step F3: For each strategic problem, determine the ratio of the number of targets corresponding to the strategic problem to the number of all target text contents as the optimization probability corresponding to the strategic problem.

[0192] Understandably, there is a lack of automated strategy evaluation and optimization mechanisms in the construction and query process of knowledge graphs, which means that even if there are performance bottlenecks or inaccurate query results, it is difficult for the system to self-adjust to improve efficiency and accuracy.

[0193] The interpretability of knowledge graphs is particularly important in industries such as finance, healthcare, and risk control that require high transparency in decision-making. However, many related technologies cannot provide clear explanations, that is, they cannot explain why specific data points are linked together or why specific query results are returned. This lack of transparency limits users' trust in the system and its widespread application.

[0194] Based on this, the present application provides a method for evaluating knowledge graph strategies, which can improve efficiency and accuracy, and can provide an explainable and automated strategy optimization mechanism.

[0195] The above introduces a knowledge graph strategy evaluation method provided in an embodiment of the present application. The following will introduce a device for executing the above-mentioned knowledge graph strategy evaluation method.

[0196] See also Figure 5 , Figure 5 A schematic diagram of a knowledge graph strategy evaluation device provided in an embodiment of the present application. Figure 5 As shown, the evaluation device of the knowledge graph strategy includes:

[0197] A first acquisition module 501 is used to acquire a question, a marked answer to the question, text content containing the marked answer, and an interpretable path representing guidance information for solving the question;

[0198] A construction module 502 is used to construct a knowledge graph based on the text content using a preset knowledge graph construction strategy;

[0199] A second acquisition module 503 is used to obtain a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy;

[0200] The third acquisition module 504 is used to input the knowledge graph, the predicted answer, the standard answer and the explainability path into a pre-built strategy analysis model, and obtain a prediction result through the strategy analysis model, wherein the prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

[0201] In an optional implementation, the knowledge graph construction strategy includes a semantic segmentation strategy, a sharding strategy, a triple extraction strategy, and a graph construction strategy, and the construction modules include:

[0202] A first acquisition unit, configured to acquire a plurality of semantic units from the text content by using the semantic segmentation strategy;

[0203] A segmentation unit, configured to segment the text content into a plurality of text segments by combining the segmentation strategy with the plurality of semantic units;

[0204] An extraction unit, configured to extract triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and a relationship between the two entities;

[0205] A construction unit is used to construct the plurality of triples into a knowledge graph through the graph construction strategy.

[0206] In an optional implementation, the knowledge graph query strategy includes: text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy, and the second acquisition module includes:

[0207] A conversion unit, configured to convert the question into a question vector by using the text vectorization strategy;

[0208] an identification unit, configured to identify a plurality of query key vectors from the question vector by using the chunk matching strategy;

[0209] A second acquisition unit is used to acquire subgraphs corresponding to a plurality of query key vectors from the knowledge graph by using the subgraph query strategy;

[0210] A screening unit, configured to screen out a plurality of target subgraphs from the plurality of subgraphs by using the branch reduction and fusion strategy;

[0211] A sorting unit, used to sort the multiple target subgraphs by using the rerank reranking strategy to obtain a sorting result;

[0212] The third acquisition unit is used to obtain a prediction answer from the sorting result through the large model summary strategy.

[0213] In an optional implementation, the third acquisition module is specifically used to: input the multiple semantic units, the multiple text fragments, the multiple triples, the knowledge graph, the question vector, the multiple query key vectors, the multiple subgraphs, the multiple target characters, the sorting results, the predicted answers, the labeled answers and the explainable paths into the strategy analysis model, and obtain the prediction results through the strategy analysis model, the first probability includes the probability of problems with the semantic segmentation strategy, the probability of problems with the fragmentation strategy, the probability of problems with the triple extraction strategy, and the probability of problems with the graph construction strategy; the second probability includes the probability of problems with the text vectorization strategy, the probability of problems with the chunk matching strategy, the probability of problems with the subgraph query strategy, the probability of problems with the branch reduction and fusion strategy, the probability of problems with the rerank reordering strategy, and the probability of problems with the large model summary strategy.

[0214] In an optional implementation, the method further includes:

[0215] A first determination module is used to determine a target text content in which the predicted answer is different from the standard answer from the plurality of text contents;

[0216] The second determination module is used to determine, for each strategy problem, a target number of target text contents in a plurality of target texts whose probability is greater than or equal to a preset threshold corresponding to the strategy problem; the strategy problem is any one of a semantic segmentation strategy, a sharding strategy, a triple extraction strategy, a graph construction strategy, a text vectorization strategy, a chunk matching strategy, a subgraph query strategy, a branch reduction and fusion strategy, a rerank reordering strategy, and a large model summary strategy;

[0217] The third determination module is used to determine, for each policy problem, a ratio of the number of targets corresponding to the policy problem to the number of all target text contents as the optimization probability corresponding to the policy problem.

[0218] The present application also provides an electronic device in an embodiment. Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0219] like Figure 6As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0220] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0221] Also provided in an embodiment of the present application is a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements a method for evaluating a strategy of any knowledge graph provided in an embodiment of the present application.

[0222] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement an evaluation method for any knowledge graph strategy provided in an embodiment of the present application.

[0223] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0224] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0225] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0226] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A knowledge graph strategy evaluation method, characterized in that: include: Obtaining a question, an annotated answer to the question, text content containing the annotated answer, and an interpretable path representing guidance information for answering the question; A knowledge graph is constructed based on the text content using a preset knowledge graph construction strategy; Obtaining a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy; The knowledge graph, the predicted answer, the standard answer and the explainable path are input into a pre-built strategy analysis model, and a prediction result is obtained through the strategy analysis model. The prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

2. The evaluation method of the knowledge graph strategy according to claim 1 is characterized in that: The knowledge graph construction strategy includes a semantic segmentation strategy, a sharding strategy, a triple extraction strategy, and a graph construction strategy. The steps of constructing a knowledge graph based on the text content using the preset knowledge graph construction strategy include: Acquire multiple semantic units from the text content by using the semantic segmentation strategy; Segmenting the text content into a plurality of text segments by combining the segmentation strategy with the plurality of semantic units; Extracting triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and a relationship between the two entities; The plurality of triples are constructed into a knowledge graph through the graph construction strategy.

3. The evaluation method of the knowledge graph strategy according to claim 2 is characterized in that: The knowledge graph query strategy includes: text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy. The step of obtaining a predicted answer from the knowledge graph based on the question through the preset knowledge graph query strategy includes: Converting the question into a question vector by using the text vectorization strategy; identifying a plurality of query key vectors from the question vector by using the chunk matching strategy; Obtaining subgraphs corresponding to a plurality of query key vectors from the knowledge graph through the subgraph query strategy; Filtering out a plurality of target subgraphs from the plurality of subgraphs by using the branch reduction and fusion strategy; Sorting the multiple target subgraphs by using the rerank reordering strategy to obtain a sorting result; The prediction answer is obtained from the sorting results through the large model summary strategy.

4. The method for evaluating the strategy of the knowledge graph according to claim 3 is characterized in that: The step of inputting the knowledge graph, the predicted answer, the standard answer, and the explainable path into a pre-built strategy analysis model and obtaining a prediction result through the strategy analysis model comprises: The multiple semantic units, the multiple text fragments, the multiple triples, the knowledge graph, the question vector, the multiple query key vectors, the multiple subgraphs, the multiple target characters, the sorting results, the predicted answers, the labeled answers and the explainable paths are input into the strategy analysis model, and the prediction results are obtained through the strategy analysis model. The first probability includes the probability of problems with the semantic segmentation strategy, the probability of problems with the sharding strategy, the probability of problems with the triple extraction strategy, and the probability of problems with the graph construction strategy; the second probability includes the probability of problems with the text vectorization strategy, the probability of problems with the chunk matching strategy, the probability of problems with the subgraph query strategy, the probability of problems with the branch reduction and fusion strategy, the probability of problems with the rerank reordering strategy, and the probability of problems with the large model summary strategy.

5. The method for evaluating the strategy of the knowledge graph according to claim 4 is characterized in that: Also includes: Determining a target text content in which the predicted answer is different from the standard answer from the plurality of text contents; For each strategy problem, determine the target number of target text contents in multiple target texts whose probability is greater than or equal to the preset threshold corresponding to the strategy problem; the strategy problem is any one of semantic segmentation strategy, sharding strategy, triple extraction strategy, graph construction strategy, text vectorization strategy, chunk matching strategy, subgraph query strategy, branch reduction and fusion strategy, rerank reordering strategy, and large model summary strategy; For each strategic problem, the ratio of the number of targets corresponding to the strategic problem to the number of all target text contents is determined as the optimization probability corresponding to the strategic problem.

6. A knowledge graph strategy evaluation device, characterized in that: include: A first acquisition module is used to acquire a question, a marked answer to the question, text content containing the marked answer, and an interpretable path representing guidance information for answering the question; A construction module, used to construct a knowledge graph based on the text content through a preset knowledge graph construction strategy; A second acquisition module is used to obtain a predicted answer from the knowledge graph based on the question through a preset knowledge graph query strategy; The third acquisition module is used to input the knowledge graph, the predicted answer, the standard answer and the explainability path into a pre-built strategy analysis model, and obtain the prediction result through the strategy analysis model. The prediction result includes a first probability that there is a problem with the knowledge graph construction strategy and a second probability that there is a problem with the knowledge graph query strategy.

7. The device for evaluating the strategy of the knowledge graph according to claim 6, characterized in that: The knowledge graph construction strategy includes semantic segmentation strategy, sharding strategy, triple extraction strategy, and graph construction strategy. The construction modules include: A first acquisition unit, configured to acquire a plurality of semantic units from the text content by using the semantic segmentation strategy; A segmentation unit, configured to segment the text content into a plurality of text segments by combining the segmentation strategy with the plurality of semantic units; An extraction unit, configured to extract triples from the plurality of text segments by using the triple extraction strategy, wherein the triples include two entities and a relationship between the two entities; A construction unit is used to construct the plurality of triples into a knowledge graph through the graph construction strategy.

8. A computer program product, characterized in that It includes computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the evaluation method of the knowledge graph strategy as described in any one of claims 1 to 5.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the knowledge graph strategy evaluation method as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the knowledge graph strategy evaluation method as described in any one of claims 1 to 5.

Images