Machine learning model training method and device, equipment and program product

Abstract

The invention provides a training method and device of a machine learning model, equipment and a program product. The method comprises the following steps: extracting a sample embedding vector from a sample time sequence knowledge graph, and calling a generator to perform path reasoning to obtain a reasoning path; performing difference calculation on the reasoning path and a real path of a sample time sequence knowledge graph to determine a first sub-loss function of a generator, and training the generator based on the first sub-loss function; based on the reasoning path, calling a discriminator to carry out prediction processing to obtain a prediction probability of a real path; performing difference calculation on the actual probability and the prediction probability of the real path to determine a second sub-loss function of the discriminator, and training the discriminator based on the second sub-loss function; and constructing the trained generator and discriminator into a machine learning model. According to the method and the device, the accuracy of outputting answers when the inference problem is solved by the trained machine learning model can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a training method, device, equipment and program product for a machine learning model. Background Art

[0002] In the era of information explosion, we are faced with the challenge of acquiring knowledge from massive amounts of information. Related technologies for intelligent question answering are usually implemented through single-hop queries. This method directly obtains answers through a single reasoning path and cannot solve the logical reasoning process behind complex problems. Related technologies obtain answers through the knowledge graph of the problem, but rely on model reasoning based on symbolic rules. When the rules are inaccurate or incomplete, the model may make incorrect inferences, increasing the complexity and time cost of machine learning model training. Related technologies currently do not have a more efficient and accurate method to train machine learning models for solving reasoning problems. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus, device and program product for training a machine learning model, which can improve the accuracy of the answers output by the trained machine learning model when solving reasoning problems.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for training a machine learning model, the method comprising:

[0006] Extract sample embedding vectors from sample time series knowledge graphs, call the generator to perform path reasoning, and obtain the reasoning path;

[0007] Determine a first sub-loss function of the generator by calculating the difference between the inference path and the real path of the sample time series knowledge graph, and train the generator based on the first sub-loss function;

[0008] Based on the reasoning path, calling the discriminator to perform prediction processing to obtain the prediction probability of the true path;

[0009] Determining a second sub-loss function of the discriminator by calculating a difference between an actual probability of the true path and the predicted probability, and training the discriminator based on the second sub-loss function;

[0010] The trained generator and discriminator are constructed as a machine learning model.

[0011] The present application embodiment provides a training device for a machine learning model, comprising:

[0012] A generator training module is used to extract a sample embedding vector from a sample time series knowledge graph, call the generator to perform path reasoning, and obtain a reasoning path; by performing a difference calculation between the reasoning path and the real path of the sample time series knowledge graph, determine the first sub-loss function of the generator, and train the generator based on the first sub-loss function;

[0013] A discriminator training module is used to call the discriminator to perform prediction processing based on the reasoning path to obtain the predicted probability of the real path; determine the second sub-loss function of the discriminator by calculating the difference between the actual probability of the real path and the predicted probability, and train the discriminator based on the second sub-loss function;

[0014] A model building module is used to build the trained generator and discriminator into a machine learning model.

[0015] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A memory for storing computer executable instructions or computer programs;

[0017] The processor is used to implement the training method of the machine learning model provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the training method of the machine learning model provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the training method of the machine learning model provided in the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] The sample embedding vector is extracted from the sample time series knowledge graph, and the path reasoning is performed through the generator based on the obtained sample embedding vector to obtain the reasoning path. The first sub-loss function of the generator is determined based on the reasoning path and the true path. The generator is trained through the first sub-loss function, and the generated reasoning path is predicted by the discriminator to obtain the predicted probability that the reasoning path is the true path. The second sub-loss function of the discriminator is determined based on the actual probability and the predicted probability of the true path. The discriminator is trained through the second sub-loss function. The quality of the generated reasoning path is evaluated through the interaction between the generator and the discriminator, and the parameters of the generator and the discriminator are updated. The efficiency of the machine learning model constructed by the trained generator and discriminator in solving reasoning problems and the quality of the output answers to the questions are improved through the update of parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of an application mode of the training method of the machine learning model provided in the embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the structure of the server provided in the embodiment of the present application;

[0024] Figure 3A It is a first flow chart of the training method of the machine learning model provided in the embodiment of the present application;

[0025] Figure 3B It is a second flow chart of the training method of the machine learning model provided in the embodiment of the present application;

[0026] Figure 4 It is a third flow chart of the training method of the machine learning model provided in the embodiment of the present application;

[0027] Figure 5 It is a structural diagram of the machine learning model provided in the embodiment of the present application.

[0028] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of superiority or inferiority of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0030] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0031] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0032] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0033] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0034] The relevant data collection and processing in the embodiments of this application (for example, timing issues to be responded to) should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0035] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0036] 1) Multi-hop reasoning problems: Problems that require multiple reasoning steps to answer, which involve starting from the original question and gradually inferring the answer by reading and understanding multiple related information sources. Such problems usually require the system to understand and relate information scattered in different places and perform complex logical reasoning.

[0037] 2) Reasoning path: A directed path that starts from one or more entities in the knowledge graph, goes through a series of reasoning steps, and finally reaches the answer entity. The construction of the reasoning path is based on the entities and relationships existing in the knowledge graph, and is completed by applying reasoning rules or algorithms. The core purpose of the reasoning path is to connect the entities in the question (usually the entities mentioned in the question) and other entities in the knowledge graph in order to find relevant information that can answer the question. When constructing the reasoning path, the question-answering model may consider multiple possible paths, evaluate the rationality and possibility of each path, and then select the path that is most likely to lead to the correct answer.

[0038] 3) Question-Answer Pair (QA Pair): refers to a question and its corresponding answer that are associated in the knowledge graph. This association is usually based on the relationship between entities or between entities and attributes in the knowledge graph.

[0039] 4) Wikidata: An open knowledge base that contains a lot of structured data from different sources, including questions and answers, facts, items, concepts, and categories. It also contains a lot of time-related information, such as dates, events, news articles, etc., which is used to build knowledge graphs.

[0040] 5) Generator: It is a neural network in the generative adversarial network, which receives a random noise vector as input and converts it into data with practical significance. Its main purpose is to generate specific data or objects. In the embodiment of the present application, the main task of the generator is to generate an accurate answer based on the input question.

[0041] 6) Discriminator: It is a neural network in the generative adversarial network. It distinguishes whether the input data comes from the real data distribution or the synthetic data generated by the generator. Its main task is to judge whether the generated answer is true and accurate. In the embodiment of the present application, the role of the discriminator is to distinguish the answer generated by the generator from the real answer, so as to evaluate the performance of the generator.

[0042] 7) Adversarial relationship: The competitive relationship between the generator and the discriminator. The generator tries to generate reasoning paths that can confuse the discriminator. The discriminator distinguishes between the generated paths and the real paths. This adversarial relationship promotes the improvement of the generator through mutual confrontation during the training process, enabling it to generate paths that are more difficult for the discriminator to recognize.

[0043] 8) Hits@1: A way to measure the accuracy of the model for the query. In the sorted list returned by the model, is the first item ranked first the correct answer? If the first item is correct, it is recorded as a successful hit, otherwise it is recorded as a miss.

[0044] 9) Hits@10: A way to measure the recall of a model for a query. In the sorted list returned by the model, does the top ten items contain at least one correct answer? If at least one of the top ten items is correct, it is counted as a successful hit. Hits@10 measures the recall of a model for a query, that is, the proportion of correct answers that the model can retrieve among all possible answers.

[0045] 10) Knowledge graph: It is a structured form of knowledge representation that organizes the relationships between entities in the form of a graph and represents various knowledge and information in the world through the relationships between entities.

[0046] 11) Temporal Knowledge Graph (TKG): It is an extension of the traditional knowledge graph. It contains not only static information of entities and relationships, but also dynamic information about their changes over time. The introduction of temporal information can better capture dynamic changes, enrich semantic representation, enhance reasoning capabilities, and cope with uncertainty.

[0047] 12) Reward-guided Policy Algorithm (REINFORCE): It is an algorithm that learns policies by maximizing expected returns and updates parameters using policy gradients. The agent's policy is represented as a probability distribution, which determines the likelihood of each action taken by the agent in a given state. The policy can be parameterized and is usually represented using a neural network. The goal of the policy gradient is to maximize the expected return under the policy.

[0048] In related technologies, when solving problems, they are limited to single-hop queries, and the answers to the problems are only directly obtained through a single reasoning path. There is a lack of reasoning and search for the logic behind complex problems. When the symbolic rules that assist machine learning models in reasoning are inaccurate or incomplete, the model may make erroneous inferences, reducing the efficiency and quality of the answers output by the machine learning model to solve reasoning problems.

[0049] The embodiments of the present application provide a method for training a machine learning model, a training device for a machine learning model, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of the answers output by the trained machine learning model when solving reasoning problems.

[0050] The following describes an exemplary application of the electronic device provided in the embodiments of the present application. The electronic device provided in the embodiments of the present application can be implemented as various types of terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, smart phones, smart speakers, smart watches, smart TVs, and vehicle-mounted terminals, and can also be implemented as servers. The following describes an exemplary application when the electronic device is implemented as a terminal or a server.

[0051] See also Figure 1 , Figure 1 This is a schematic diagram of an application mode of a training method for a machine learning model provided in an embodiment of the present application, for example, to implement a training application supporting a machine learning model. Figure 1 The system involves a server 200, a network 300, a terminal device 400 and a database 500. The terminal device 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0052] In some embodiments, the user may be a technician in this field or an ordinary user with query needs. The server 200 is a server for training a trained machine learning model. The terminal device 400 is a terminal operated by the user, such as a computer or a mobile phone. The terminal device 400 is installed with a pre-configured question-and-answer application that can be run, and the database 500 stores a time-series knowledge graph dataset.

[0053] For example, the server 200 constructs a time series knowledge graph dataset based on the questions to be answered, stores it in the database 500, trains the machine learning model to be trained based on the time series knowledge graph dataset, obtains the trained machine learning model, and the trained machine learning model can be stored in the server 200. The terminal device 400 is used to receive the questions to be answered sent by the user, send them to the server 200 through the network 300, the server 200 calls the trained machine learning model for processing, outputs the answers to the questions to be answered, and feeds them back to the terminal device 400 through the network 300, and the terminal device 400 displays the answers to the questions to be answered to the user.

[0054] In some embodiments, the training method of the machine learning model of the embodiment of the present application can also be applied in the following application scenarios:

[0055] 1. Application scenarios of search engines. The training method of the machine learning model provided in the embodiments of the present application can assist the search engine to better understand the user's query intention, provide more accurate and relevant search results, and improve the user experience.

[0056] 2. Application scenario of voice assistant: users provide questions to be answered through the voice interaction system. The training method of the machine learning model provided in the embodiment of the present application can understand and respond to users' complex questions and provide accurate answers and solutions.

[0057] 3. Application scenario of educational tutoring. The training method of the machine learning model provided in the embodiment of the present application can provide students with a personalized learning experience, test students' learning outcomes in the form of questions and answers, quickly and accurately solve problems, and provide educational support.

[0058] The embodiments of the present application can be implemented through database technology. In short, a database can be regarded as an electronic file cabinet where electronic files are stored. Users can add, query, update, delete, etc. data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application program.

[0059] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as Structured Query Language (SQL), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0060] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0061] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the server provided in the embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 410, a memory 450 and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0062] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0063] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0064] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0065] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0066] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0067] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Authentication (WiFi), and Universal Serial Bus (USB).

[0068] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 A training device 455 of a machine learning model stored in a memory 450 is shown, which may be software in the form of a program and a plug-in, etc., and includes the following software modules: a generator training module 4551, a discriminator training module 4552, and a model building module 4553. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0069] In some embodiments, the terminal or server can implement the training method of the machine learning model provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0070] The training method of the machine learning model provided in the embodiments of the present application will be explained in combination with the exemplary application and implementation of the electronic device provided in the embodiments of the present application.

[0071] The following describes the electronic device management method provided by the embodiment of the present application. As mentioned above, the electronic device implementing the electronic device management method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution subject of each step will not be repeatedly described below.

[0072] See also Figure 3A , Figure 3A This is a first flow chart of the training method of the machine learning model provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained, Figure 3A The executive body is Figure 1 Server 200 in.

[0073] In step 301, a sample embedding vector is extracted from the sample time series knowledge graph, and the generator is called to perform path reasoning to obtain a reasoning path.

[0074] For ease of understanding, the application scenario of the training method of the machine learning model provided in the embodiment of the present application is explained. The machine learning model can be a question-and-answer model for solving reasoning problems. The question-and-answer model is a technology that simulates human language comprehension ability and is used to process and answer questions raised by users. The core function is to provide accurate answers through understanding of the problem and retrieval of knowledge. It is applied to scenarios such as human-computer interaction, intelligent customer service, information retrieval, and knowledge management. The question-and-answer model is trained based on a large amount of text data and annotated question-and-answer pairs. The question-and-answer pair refers to a question and a corresponding answer that are associated in the knowledge graph. This association is usually based on the relationship between entities and entities in the knowledge graph, or the relationship between entities and attributes. The time series knowledge graph is an extension of the traditional knowledge graph. It not only contains static information about entities and relationships, but also includes dynamic information about them changing over time.

[0075] For example, a corresponding sample time series knowledge graph is constructed according to the first data set, the sample time series knowledge graph includes entities and facts, and the sample time series knowledge graph is divided. The machine learning model to be trained is called to extract features from the time series knowledge graph to obtain a sample embedding vector of the sample knowledge graph. The sample embedding vector is a four-tuple vectorized representation of facts. The four-tuple vectorized representation of facts includes topics, relationships, objects, and timestamps (expressed as start time and end time). A large number of sample time series knowledge graphs are randomly divided in proportion to obtain training sets, validation sets, and test sets for the machine learning model. The sample embedding vector is the current state and strategy of the current node in the time series knowledge graph. The current state is the feature vector of the current node and the context information of the current node. The strategy is a strategy network that generates an inference path. The generator is called based on the current state and strategy to construct an inference path to obtain an inference path.

[0076] In some embodiments, before executing step 301, a sample time series knowledge graph is also selected through the following processing: based on the first data set, multiple first time series knowledge graphs are constructed, and the first data set is a question set provided by the knowledge base; multiple first time series knowledge graphs are denoised to obtain multiple second time series knowledge graphs after denoising; multiple second time series knowledge graphs are normalized for time information to obtain multiple third time series knowledge graphs; and a sample time series knowledge graph is selected from the multiple third time series knowledge graphs.

[0077] For example, the first data set is a data set for time series prediction. In an embodiment of the present application, the first data set may be a Wiki database, through which a knowledge graph is constructed. The Wiki database contains a lot of structured data and a lot of time-related information from different sources, which are used to construct a knowledge graph. Based on the first data set, multiple first time series knowledge graphs are constructed. The first time series knowledge graph contains a large number of entities and facts, covering multiple natural language questions, which require time reasoning to answer. Before calling the machine learning model, the time series knowledge graph is preprocessed, and the preprocessing includes: denoising, unifying time information, and selecting sample time series knowledge graphs. The first time series knowledge graph features include: entities (Entities, E), relationships (Relations, R), and timestamps (Timestamps, T).

[0078] Denoising is performed on multiple first time series knowledge graphs to remove missing or abnormal values ​​in the first time series knowledge graphs to ensure data quality, which can be represented by the following formula (1), which is described in detail below.

[0079] X cleaned =x∈X|isValid(x) (1)

[0080] Where X is the original first data set, isValid(x) is a validation function used to determine whether the data x is valid. cleaned It is multiple second temporal knowledge graphs after removing noise.

[0081] Unify the representation of time information of multiple second time series knowledge graphs, standardize the time information, and standardize the time information into a unified representation to obtain multiple third time series knowledge graphs, which are convenient for machine learning model processing and can be represented by the following formula (2), which is described in detail below.

[0082] t normalized =(tt min ) / (t max -t min ) (2)

[0083] Among them, t is the original time value, t min and t max are the earliest and latest time values ​​in the second temporal knowledge graph, t normalized It is a normalized time value in the range of [0, 1].

[0084] A sample time series knowledge graph is selected from multiple third time series knowledge graphs according to a preset ratio, and the third time series knowledge graph is divided into a training set, a validation set, and a test set to facilitate model training and evaluation. The preset ratios of the training set, the validation set, and the test set are 70%, 15%, and 15%, respectively.

[0085] In the embodiment of the present application, the time series knowledge graph is denoised and the representation is standardized, and the removal of missing or outliers can ensure the validity of the data and assist the machine learning model in efficiently processing and understanding the data. A sample time series knowledge graph is selected, and based on the sample time series knowledge graph, the machine learning model to be trained is called for feature extraction. By using the sample time series knowledge graph containing time information, the machine learning model can better understand the relationship between the question background and the data, thereby improving the accuracy of the answer.

[0086] In some embodiments, see Figure 5 , Figure 5 is a schematic diagram of the structure of the machine learning model provided in the embodiment of the present application, Figure 5 The machine learning model 501 in the example is composed of an encoder 5011, a generator 5012, a discriminator 5013, and a decoder 5014. The encoder 5011 is called to perform feature extraction processing to obtain a sample embedding vector. Before calling the generator for path reasoning, the parameters of the generator and the discriminator in the machine learning model are also initialized. The initialized activation function can use a linear activation function, an S-curve activation function, or a hyperbolic tangent activation function, which can be represented by the following formula (3), which is described in detail below.

[0087]

[0088] Among them, W is the initialized weight matrix, n in is the number of input units, n out is the number of output units. U(a, b) represents a uniform distribution on the interval [a, b], ensuring the rationality of the initial weights of the generator and the discriminator.

[0089] In some embodiments, see Figure 3B , Figure 3B It is a second flow chart of the training method of the machine learning model provided in the embodiment of the present application; Figure 3A Step 301 can be performed by executing Figure 3B Steps 3011 to 3013 in the above code implement the processing performed by the generator that calls the machine learning model, which is described in detail below.

[0090] In step 3011, based on the sample embedding vector and the sample time series knowledge graph, the generator is called to determine the node features and context information features of the first node.

[0091] For example, based on the sample embedding vector and the sample time series knowledge graph, the current state of the first node is determined. The current state is the node feature of the first node and the context information feature of the first node, including the representation of the first node, the history of the context, and the time information, which can be represented by the following formula (4), which is described in detail below.

[0092] s t =Feature(a t ,context) (4)

[0093] Among them, s t is the current state at time step t, a t is the first node, context is context information, including previously visited nodes and other related information, Feature (a t , context) is the feature extraction function, which converts the first node and context information into a feature vector s t .

[0094] In step 3012, node prediction processing is iteratively performed based on the node features and context information features to obtain multiple second nodes.

[0095] Here, the input of each node prediction process includes: the node feature and context information feature of the first node and the node feature and context information feature of the predicted second node, and the output of each node prediction process is a second node.

[0096] For example, node prediction processing is performed iteratively based on the node feature of the first node and the context information feature, and the prediction processing is based on the strategy of the first node embedded in the feature to obtain multiple second nodes. The strategy is a strategy network for generating an inference path, and the strategy network is used to predict the node based on the current state s of the first node. t The probability distribution of outputting the next node selection, that is, the probability of selecting the second node, can be represented by the following formula (5), which is described in detail below.

[0097] π(s t )=P(a t+1 |θ G ) (5)

[0098] Among them, π(s t ) is the policy network in state s t The probability distribution of the second node output under t+1 |θ G ) indicates that in the generator parameter θ G Next, according to the current state s of the first node t Select the second node a t+1 The conditional probability, θ Gare the parameters of the generator.

[0099] In step 3013, the first node and each second node are combined into an inference path.

[0100] For example, the generator in the machine learning model is called to combine the first node and each second node into an inference path, starting from a given starting node a1, and extracting the initial state s1 of the feature and context information of the starting node, that is, according to the above formula (4), s1 = Feature (a t , context), for each time step t, according to the current state s t , the policy network π outputs the probability distribution π(s) of selecting the next node according to formula (5) t ), from the probability distribution π(s t ) to sample or select the next node a t+1 , construct a reasoning path containing multiple nodes. The reasoning path can be represented by the following formula (6), which is described in detail below.

[0101] τ=(a1,a2,…,a T ) (6)

[0102] Among them, τ is the reasoning path consisting of a series of nodes, a T is the Tth node in the path. By repeatedly applying the strategy, starting from the initial node a1, the next node is selected according to the current state until the complete reasoning path τ is constructed.

[0103] The second node selects the initial node with the largest probability distribution value output as the next node, which can be represented by the following formula (7), which is described in detail below.

[0104]

[0105] in, is the function that takes the maximum value of the probability distribution, and then updates the current state according to the above formula (4), repeating the above steps until the preset path length T is reached or other termination conditions are met. The final complete reasoning path consists of a series of nodes (a1, a2, ..., a T ), the execution of the inference path can be done by Figure 5 Generator 5012 in is implemented.

[0106] In an embodiment of the present application, the calling generator performs path reasoning based on the current state and strategy of the first node, and provides answers to questions to be answered by generating multiple reasoning paths, thereby solving the limitations of a single reasoning path, improving the accuracy of the answer, and increasing the richness of the results.

[0107] Continue to see Figure 3AIn step 302, the first sub-loss function of the generator is determined by calculating the difference between the inference path and the real path of the sample time series knowledge graph, and the generator is trained based on the first sub-loss function.

[0108] For example, the difference between the nodes in the true path and the nodes in the generated reasoning path is calculated. The difference between the nodes can be represented by the difference corresponding to the predicted probabilities corresponding to the nodes, and each cross entropy loss of the nodes between the true path and the reasoning path is obtained. The mean of the cross entropy loss of each node is used as the first sub-loss function corresponding to the generator, and the generator is trained based on the first sub-loss function.

[0109] In step 303, based on the inference path, the discriminator is called to perform prediction processing to obtain the predicted probability of the true path.

[0110] Here, the predicted probability of the true path is the predicted probability that the type of the inference path is the true path.

[0111] For example, the discriminator is called to predict the reasoning path. The discriminator is used to distinguish the reasoning path generated by the generator from the real path in the sample time series knowledge graph, and obtain the first prediction probability that the reasoning path is the real path in the sample time series knowledge graph. The prediction processing of the discriminator can be done by Figure 5 The discriminator 5013 in is implemented using a recurrent neural network and can be represented by formulas (8) and (9), which are described in detail below.

[0112] h t =σ(W h ·a t +U h ·h t--1 +b h ) (8)

[0113] o t =σ(W o ·h t +b o ) (9)

[0114] Among them, h t is the hidden state at time step t, a t is the representation of the tth node in the path, W h , U h 、b h are the weight and bias parameters of the recurrent neural network, o t is the output vector, σ is the activation function, and time steps are a series of equally spaced units into which time is divided.

[0115] For example, the discriminator receives an inference path as input, which can be the inference path generated by the generator or the real path in the sample time series knowledge graph. The neural network structure of the discriminator processes the received inference path, and outputs a scalar value as the first prediction probability for each node in the time step recursive processing path, which can be represented as formula (10), which is described in detail below.

[0116] D(τ)=P(real|τ;θ G ) (10)

[0117] Where D(τ) is the output scalar value of the discriminator, that is, the first predicted probability that the path τ is the true path real, and θ G are the parameters of the discriminator.

[0118] In an embodiment of the present application, by calling the discriminator in the machine learning model to perform prediction processing on the reasoning path, the predicted probability that the reasoning path is the real path in the sample time series knowledge graph is obtained, thereby effectively distinguishing high-quality reasoning paths, ensuring the reliability of the answers generated by the machine learning model, and providing a basis for the adversarial reinforcement process between the generator and the discriminator.

[0119] In step 304, a second sub-loss function of the discriminator is determined by calculating the difference between the actual probability and the predicted probability of the true path, and the discriminator is trained based on the second sub-loss function.

[0120] For example, the cross entropy loss between the actual probability corresponding to the true path and the predicted probability corresponding to the inference path is calculated, and the second sub-loss function corresponding to the discriminator is determined based on the cross entropy loss of the path. The true path and the generated path are distinguished by the second sub-loss function, and the discriminator is trained by maximizing the actual probability of the true path and minimizing the predicted probability of the generated path. It can be represented as formula (11), which is described in detail below.

[0121] L D = -[logD(τ real )+logD(1-D(τ gen ))] (11)

[0122] Among them, τ real is the true path, τ gen is the inference path generated by the generator, logD(τ real ) is the logarithmic probability that the real path is correctly identified as real by the discriminator, logD(1-D(τ gen )) is the logarithmic probability that the generated path is correctly identified as false by the discriminator, using the back-propagation and gradient descent algorithms, according to the second sub-loss function L D Update the parameters θ of the discriminator G .

[0123] For example, based on the first sub-loss function and the predicted probability corresponding to the inference path, the reward function of the generator is determined, and the predicted probability is used as the reward function value of the generator for updating the parameters of the generator. The reward function reflects the quality of the generated path. If the value of the authenticity probability score is high, it means that the discriminator believes that the path is close to the true path, and thus gives the generator a higher reward function value. Otherwise, the reward function value is low, which can be represented by formula (12), which is described in detail below.

[0124] r(τ)=D(τ) (12)

[0125] Here, r(τ) is the reward function value of the inference path τ, and D(τ) is the predicted probability of the true path. Based on the policy gradient method, the parameters of the generator are updated according to the policy update formula of the generator. The policy gradient method can be the reward-guided policy algorithm (REINFORCE). The reward-guided policy algorithm is an unbiased estimation policy gradient method that uses the cumulative rewards of the sampled path to update the policy parameters. The policy gradient update formula of the generator can be represented as formula (13), which is described in detail below.

[0126]

[0127] Among them, θ G are the parameters of the generator, α is the learning rate, is the generator in state s t Next select action a t The probability of , r(τ) is the reward of the reasoning path τ, that is, D(τ). The calculation of the reward function provides feedback to the generator, and the policy gradient update uses this feedback to adjust the parameters of the generator.

[0128] For example, the first sub-loss function and the second sub-loss function are used as the loss functions of the machine learning model. The parameters of the generator are updated according to the first sub-loss function and the predicted probability, and the parameters of the discriminator are updated according to the second sub-loss function.

[0129] In an embodiment of the present application, by determining the loss functions of the generator and the discriminator, the parameters of the generator and the discriminator are iteratively updated. The updating process of the parameters of the generator and the discriminator is a generative adversarial process, which gradually improves the reasoning efficiency and the quality of the reasoning answers generated by the model.

[0130] Continue to see Figure 3A , in step 305, the trained generator and discriminator are constructed as a machine learning model.

[0131] For example, the generator and discriminator in the machine learning model are trained based on the iteration of the first sub-loss function and the second sub-loss function. When the current sub-losses corresponding to the generator and the discriminator respectively meet the pre-configured conditions, the training is stopped to obtain the trained machine learning model, and the machine learning model is used to generate the answer to the question to be answered. The pre-configured conditions are pre-set conditions for judging whether the machine learning model has achieved the expected training effect, for example: the iteration reaches the pre-configured number of times or the loss value changes tend to be stable.

[0132] In some embodiments, the discriminator and the generator may already exist in the machine learning model, and the discriminator and the generator in the overall machine learning model are trained to obtain a constructed machine learning model.

[0133] In some embodiments, step 305 can be implemented by the following method: determining the updated first sub-loss function of the trained generator, and determining the updated second sub-loss function and the current number of iterations of the trained discriminator, the updated first sub-loss function and the second sub-loss function are used as the loss functions used in the next update process; when the update process meets the preconfigured conditions, the trained generator and discriminator are constructed as a machine learning model.

[0134] For example, the corresponding parameters are updated iteratively based on the loss functions of the generator and the discriminator to obtain an updated machine learning model, and the updated loss function is used as the loss function for the next iterative processing to determine the current number of iterations, and to determine whether the current number of iterations or the current sub-loss meets the pre-configured conditions, that is, the steps of iterative generation path, discriminator training, and generator parameter update are repeated, and the machine learning model is checked every several iterations to determine whether it meets the pre-configured conditions (balance and convergence criteria) to determine whether training needs to continue. During the iterative training process, the update steps of the generator and the discriminator can maintain a certain ratio, such as 1:1 or 1:5, etc., to prevent excessive updates of either party from easily causing the training of the other party to stagnate or fail. Based on the learning curves of the generator and the discriminator, it can be determined whether the two are overfitting or underfitting. The update step number is the number of times the parameters of the generator and the discriminator are updated in each iteration.

[0135] In some embodiments, the preconfigured conditions satisfied by the training process include at least one of the following:

[0136] Condition 1: The current number of iterations corresponding to the machine learning model reaches the first preconfigured number of iterations.

[0137] For example, the current number of iterations corresponding to the machine learning model reaches a first preconfigured number of iterations. The first preconfigured number of iterations is a pre-set maximum number of iterations, for example, 10,000 or 100,000. Training stops when this number of iterations is reached.

[0138] Condition 2: within a second preconfigured number of iterations, a first change rate of the first current sub-loss and a second change rate of the second current sub-loss are both less than a change rate threshold.

[0139] For example, the rate of change of the loss function corresponding to each iteration is obtained within a second preconfigured number of times, and the first rate of change of the first current sub-loss and the second rate of change of the second current sub-loss are both less than the rate of change threshold. For example: if the loss function changes less than a certain threshold (such as 0.001) in 500 consecutive iterations, the machine learning model is considered to have converged, and the training of the machine learning model is stopped.

[0140] In some embodiments, after the trained generator and discriminator are constructed as a machine learning model, the following processing is performed: based on a sub-dataset of the sample time series knowledge graph, the trained machine learning model is performance evaluated to obtain a performance evaluation result; when the performance evaluation result does not meet the pre-configured evaluation result, the hyperparameters of the machine learning model are adjusted to obtain an adjusted machine learning model.

[0141] For example, after the trained generator and discriminator are built into a machine learning model, the trained machine learning model is called to perform performance evaluation based on a validation set obtained by dividing the sample time series knowledge graph to obtain a performance evaluation result. When the performance evaluation result does not meet the pre-configured evaluation result, the hyperparameters of the machine learning model are adjusted, and the hyperparameters of the machine learning model include a learning rate, an optimizer, or a network structure to obtain an adjusted machine learning model.

[0142] For example, the learning rate is a key parameter that affects the convergence speed and stability of the machine learning model. If the learning rate is too low, the convergence speed of the machine learning model will be very slow. The learning rate is dynamically adjusted using a predefined strategy, which can be represented as formula (18), which is described in detail below.

[0143] α t =α0·e -λt (18)

[0144] Among them, α t is the learning rate of the tth iteration, α0 is the initial learning rate, λ is the decay rate, and t is the number of iterations.

[0145] For example, the adjustment of network structure includes: changing the number of layers, number of neurons, activation function, regularization method. Common methods of adjusting network structure include: increasing or decreasing the number of layers and neurons, increasing the complexity of the model or reducing overfitting, changing the activation function or other nonlinear activation functions, and applying regularization to prevent overfitting. Choosing a suitable optimizer helps optimize model performance, for example: including stochastic gradient descent, stochastic average gradient descent, etc.

[0146] In some embodiments, the performance evaluation of the trained machine learning model is also performed in at least one of the following ways:

[0147] Method 1: Based on the second data set, call the trained machine learning model to perform question and answer processing to obtain predicted answers, test the predicted answers through preconfigured scoring criteria, and obtain performance evaluation results. The second data set is a question set for testing the machine learning model.

[0148] For example, based on the second data set, the trained machine learning model is called to perform question-answering processing to obtain a predicted answer. The test data set is obtained by dividing the sample time series knowledge graph. The predicted answer is tested by the preconfigured scoring criteria to obtain the performance evaluation result. The evaluation results are Hits@1 and Hits@10 standards. Hits@1 is whether the first-ranked item in the sorted list returned by the machine learning model is the correct answer. If the first-ranked item is correct, it is recorded as a successful hit, otherwise it is recorded as a miss, which measures the accuracy of the machine learning model for the query. Hits@10 is whether the top ten items in the sorted list returned by the machine learning model contain at least one correct answer. If at least one of the top ten items is correct, it is recorded as a successful hit.

[0149] Method 2: Based on the second data set, call the trained machine learning model to perform question and answer processing to obtain predicted answers, and determine the performance evaluation results of the machine learning model based on the labeled true answers and predicted answers corresponding to the second data set. The performance evaluation results include at least one of the following: a first accuracy rate, a first precision rate, and a first recall rate.

[0150] For example, based on the labeled true answer and the predicted answer corresponding to the second data set, the performance evaluation result of the machine learning model is determined, and the performance evaluation result includes at least one of the following: a first accuracy rate, a first precision rate, and a first recall rate. The first accuracy rate is the ratio of the number of samples correctly predicted by the machine learning model to the total number of samples, which is used to measure the overall correctness of the machine learning model and can be represented by formula (14), which is described in detail below.

[0151] Accuracy=(TP+TN) / (TP+TN+FP+FN) (14)

[0152] Among them, Accuracy is the first accuracy, TP is the number of samples whose true label is positive and predicted to be positive, TN is the number of samples whose true label is negative and predicted to be negative, FP is the number of samples whose true label is negative but predicted to be positive, and FN is the number of samples whose true label is positive but predicted to be negative.

[0153] The first precision rate is the proportion of samples predicted to be positive by the machine learning model that are actually positive. It is used to measure the accuracy of the correct answer given by the machine learning model and can be represented by formula (15), which is described in detail below.

[0154] Precision=TP / TP+FP (15)

[0155] Among them, Precision is the first accuracy, TP is the number of samples whose true label is positive and predicted to be positive, and FP is the number of samples whose true label is negative but predicted to be positive.

[0156] The first recall rate is the proportion of samples with true positive labels that are correctly predicted as positive by the machine learning model. It is used to measure the comprehensiveness of the model's answer to the question and can be represented by formula (16), which is explained in detail below.

[0157] Recall=TP / TP+FN (16)

[0158] Among them, Recall is the first recall rate, TP is the number of samples whose true label is positive and predicted to be positive, and FN is the number of samples whose true label is positive but predicted to be negative.

[0159] For example, after the machine learning model is trained and its performance is evaluated, the trained model is used to predict or generate new data or answers. Before generating the answers, the questions to be answered need to be preprocessed. The specific steps of preprocessing may include: text cleaning and standardization, word segmentation and feature extraction. Text cleaning and standardization include removing special characters, punctuation marks, unifying uppercase and lowercase letters, etc.; word segmentation is to divide the text into a sequence of words or subwords; feature extraction is to convert the text into numerical features that the model can understand. After generating the answers to the questions to be answered, the output needs to be formatted to ensure that the format of the answers meets the expected output requirements, and the generated answers need to be further analyzed or processed to verify their quality and accuracy. If the answers do not meet expectations, it may be necessary to adjust the model, optimize the data preprocessing steps, or adjust the model hyperparameters to optimize or adjust the quality or applicability of the answers.

[0160] In the embodiment of the present application, the time series knowledge graph is denoised and the expression form is standardized to ensure the validity of the data, assist the machine learning model in efficiently processing and understanding the data, and dynamically represent the knowledge through the sample time series knowledge graph containing time information, and update it over time. The generator of the machine learning model is called to perform path reasoning, and answers are provided by generating multiple reasoning paths. The discriminator is called to evaluate the reasoning path, effectively distinguish high-quality reasoning paths, and improve the efficiency and quality of the model's generation of answers through the adversarial process between the generator and the discriminator. The performance of the trained machine learning model is evaluated and the hyperparameters are adjusted to ensure the quality of the answers generated by the model, so that the trained machine learning model can efficiently and accurately handle complex reasoning tasks.

[0161] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0162] In the current era of information explosion, we are faced with the challenge of obtaining the required knowledge from massive amounts of information. Knowledge graphs, as a structured form of knowledge representation, are widely used in information retrieval, intelligent question answering and other fields. Traditional knowledge graph question answering methods are usually limited to single-hop queries, and can only directly obtain answers through a single reasoning path, ignoring the multi-hop reasoning process behind complex problems. The multi-hop knowledge graph question answering method aims to obtain deeper answers from the knowledge graph through multi-step jumps and reasoning. It uses the reinforcement learning framework and interactive learning with the environment to enable the intelligent agent to perform multi-step reasoning on the knowledge graph and finally obtain accurate answers. By continuously interacting with the environment, the intelligent agent learns how to perform effective reasoning and search on the knowledge graph, realizing the function of multi-hop knowledge graph question answering.

[0163] In practical applications, related technologies still have problems such as the sparsity of knowledge graph data, reduced reasoning efficiency, insufficient model generalization ability, and error propagation, which limit the effectiveness and reliability in practical applications. Symbolic rules help guide question-answering models to perform effective reasoning, but if the rules are inaccurate or incomplete, the model may make incorrect inferences or ignore potential correct paths, reducing the accuracy of the reasoning to get the answer to the question.

[0164] The training method of the machine learning model can be a training method for a question-answering model. The embodiment of the present application takes the scenario of training a question-answering model and generating answers to questions to be answered by the trained question-answering model as an example to illustrate the training method of the machine learning model provided by the present application.

[0165] The embodiment of the present application constructs a time series knowledge graph, initializes the parameters of the generator and discriminator in the question-answering model based on the features of the time series knowledge graph, calls the generator to construct the reasoning path, calls the discriminator to evaluate the generated reasoning path, determines the loss function of the discriminator and the reward function of the generator based on the probability that the evaluation results in the true path, and updates the parameters of the discriminator and generator in the question-answering model. When the iteration of the parameters of the generator and the discriminator reaches the preset conditions, the training is determined to be completed, and the trained question-answering model is obtained, which improves the efficiency and accuracy of the output answer while reducing the time cost of model training.

[0166] The following is a description with reference to the accompanying drawings. Figure 4 , Figure 4 : is a third flow chart of the training method of the machine learning model provided in the embodiment of the present application. The execution subject may be a terminal device, a server, or a combination of the two. The embodiment of the present application takes the server as the execution subject as an example. Figure 4 The steps shown are explained in detail.

[0167] In step 401, a time series knowledge graph is constructed, feature extraction is performed on the time series knowledge graph, a question-answering model is called based on the embedded features of the time series knowledge graph, and the parameters of the generator and discriminator in the question-answering model are initialized.

[0168] For example, a time series knowledge graph is constructed based on a sample dataset. The sample dataset is a dataset for time series prediction (CronQuestions). The knowledge graph is constructed through the Wikidata database (Wikidata). Wikidata contains a lot of structured data and a lot of time-related information from different sources, which is used to build the knowledge graph. The dataset of the constructed time series knowledge graph contains a large number of entities and facts, covering multiple natural language questions, which require time reasoning to answer. Each fact is represented in the form of a quadruple, which includes a subject, a relationship, an object, and a timestamp (expressed as a start time and an end time), and the dataset corresponding to the time series knowledge graph is divided into a training set, a validation set, and a test set. The above sets contain 350,000, 30,000, and 30,000 question-answer pairs, respectively.

[0169] For example, the time series knowledge graph is initialized. The initialization process includes pre-loading the data set, feature extraction, and data set partitioning. The data set of the time series knowledge graph contains event and relationship information in time, which is suitable for time series knowledge graph tasks. The data set features include: entities (Entities, E), relations (Relations, R), and timestamps (Timestamps, T).

[0170] The data set is loaded and preprocessed to remove missing or outliers in the data set to ensure data quality, which can be represented by the following formula (1), which is described in detail below.

[0171] X cleaned =x∈X|isValid(x) (1)

[0172] Among them, X is the original sample data set, sValid(x) is a verification function used to determine whether the data x is valid. cleaned is the dataset after noise is removed.

[0173] After loading and preprocessing, the time information of the data set after noise removal is standardized, and the time information is standardized into a unified representation form to facilitate question-answering model processing, which can be represented by the following formula (2), which is described in detail below.

[0174] t normalized =(tt min ) / (t max -t min ) (2)

[0175] Among them, t is the original time value, t min and t max are the earliest and latest time values ​​in the data set, respectively, normalized It is a normalized time value in the range of [0, 1].

[0176] Finally, the dataset of the time series knowledge graph is divided into training set, validation set and test set for model training and evaluation. The ratio of the training set, validation set and test set is 70%, 15% and 15% respectively. The training set, validation set and test set are subsets randomly divided from the original sample dataset in proportion. The training set is used to train the generator and discriminator in the question-answering model; the validation set is used to verify whether the trained question-answering model meets the expected standards; the test set is used to test the quality of the answers generated by the trained question-answering model.

[0177] For example, feature extraction is performed on the time series knowledge graph, and the question answering model is called based on the embedded features of the time series knowledge graph. In some embodiments, see Figure 5 , Figure 5 is a schematic diagram of the structure of the machine learning model provided in the embodiment of the present application, Figure 5 The structural diagram of the model in Figure 5The question-answering model 501 in the example is composed of an encoder 5011 and a generator 5012. The model 501 is composed of an encoder 5011, a generator 5012, a discriminator 5013 and a decoder 5014. The encoder 5011 is called to perform feature extraction on the input knowledge graph to obtain the corresponding embedded features, and the parameters of the generator and the discriminator in the question-answering model are initialized. The initialized activation function can use a linear activation function, an S-shaped curve activation function (sigmoid) or a hyperbolic tangent activation function (tanh). The weights of the generator and the discriminator parameters are initialized to make the variances of the input and output as similar as possible to avoid the phenomenon of gradient disappearance or explosion. This can be represented by the following formula (3), which is described in detail below.

[0178]

[0179] Among them, W is the initialized weight matrix, n in is the number of input units, n out is the number of output units. U(a, b) represents a uniform distribution on the interval [a, b]. Through the above steps, the initial weights of the generator and the discriminator are ensured to be reasonable, and the data set is processed and divided to improve the training stability and performance of the model.

[0180] Continue to see Figure 4 ,In step 402, based on the embedded features in the temporal knowledge graph, a generator is called to build an inference path containing multiple nodes.

[0181] For example, based on the embedded features in the temporal knowledge graph, the embedded features are the current state and strategy, the current state is the feature vector of the current node and the current node context information, the strategy is the strategy network for generating the reasoning path, and the generator is called based on the current state and strategy to construct the reasoning path.

[0182] The feature vector of the current node context information includes the node representation, context history, and time information, which can be represented by the following formula (4), which is described in detail below.

[0183] s t =Feature(a t ,context) (4)

[0184] Among them, s t is the current state at time step t, a t is the first node, context is context information, including previously visited nodes and other related information, Feature (a t , context) is the feature extraction function, which converts the first node and context information into a feature vector s t .

[0185] The policy network is used to t The probability distribution of outputting the next node can be represented by the following formula (5), which is described in detail below.

[0186] π(s t )=P(a t+1 |θ G ) (5)

[0187] Among them, π(s t ) is the policy network in state s t The probability distribution of the output under t+1 |θ G ) indicates that the parameter θ G Next, according to the current state s t Select the next node a t+1 The conditional probability distribution of G are the parameters of the generator.

[0188] For example, the generator selects the next node according to the current state and the policy network and constructs the reasoning path, which can be represented by the following formula (6), which is described in detail below.

[0189] τ=(a1,a2,…,a T ) (6)

[0190] Among them, τ is the reasoning path consisting of a series of nodes, a T is the Tth node in the path. By repeatedly applying the strategy, starting from the initial node a1, the next node is selected according to the current state until the complete path τ is constructed.

[0191] For example, starting from a given starting node a1, extract the initial state s1 of the feature and context information of the starting node, that is, according to the above formula (4), we get s1 = Feature (a t , context), for each time step t, according to the current state s t , the policy network π outputs the probability distribution π(s) of selecting the next node according to formula (5) t ), from the probability distribution π(s t ) to sample or select the next node a t+1 , the next node selects the initial node with the largest probability distribution value as the next node, which can be represented by the following formula (7), which is described in detail below.

[0192]

[0193] in, is the function that takes the maximum value of the probability distribution, and then updates the current state according to the above formula (4), repeating the above steps until the preset path length T is reached or other termination conditions are met. The final complete reasoning path consists of a series of nodes (a1, a2, ..., a T ), the reasoning path can be Figure 5 Generator 5012 in is obtained.

[0194] Continue to see Figure 4 In step 403, the discriminator is called based on the reasoning path to perform path evaluation processing to obtain the authenticity probability score corresponding to the reasoning path.

[0195] For example, the discriminator in the question-answering model is called to evaluate the reasoning path generated by the generator. The discriminator is used to distinguish the generated reasoning path from the real path in the sample data set, and obtain the authenticity probability score corresponding to the reasoning path. The authenticity probability score represents the numerical value of the probability that the reasoning path is the real path.

[0196] In some embodiments, the evaluation of the reasoning path is performed by Figure 5 The discriminator 5013 in is implemented. The discriminator 5013 uses a recurrent neural network to determine the authenticity of the input path and can capture the time dependency and sequential pattern in the path. The discriminator can be represented by formulas (8) and (9), which are described in detail below.

[0197] h t =σ(W h ·a t +U h ·h t-1 +b h ) (8)

[0198] o t =σ(W o ·h t +b o ) (9)

[0199] Among them, h t is the hidden state at time step t, a t is the representation of the tth node in the path, W h , U h 、b h are the weight and bias parameters of the recurrent neural network, o t is the output vector, σ is the activation function, and time steps are a series of equally spaced units into which time is divided.

[0200] For example, the discriminator receives an inference path as input, which can be the inference path generated by the generator or the real path in the sample data set. The neural network structure of the discriminator processes the path, recursively processes each node in the path through time steps (a series of equally spaced units that divide time), and obtains a scalar value as the authenticity probability score of the corresponding path, which represents the probability of whether the path is real (from training data) or generated (from the generator). The output of the discriminator can be represented by formula (10), which is explained in detail below.

[0201] D(τ)=P(real|τ;θ G ) (10)

[0202] Among them, D(τ) is the output of the discriminator, that is, the probability that the path τ is the real path real, θ D are the parameters of the discriminator.

[0203] For example, after calculating the authenticity probability score, the authenticity of the generated path is measured by the cross entropy loss function. The loss function is used to distinguish the true path from the generated path. The discriminator is trained by maximizing the probability of the true path and minimizing the probability of the generated path. The cross entropy loss function can be represented by formula (11), which is described in detail below.

[0204] L D = -[logD(τ real )+log(1-D(τ gen ))] (11)

[0205] Among them, τ real is the true path, τ gen is the path generated by the generator, logD(τ real ) is the logarithmic probability that the real path is correctly identified as real by the discriminator, log(1-D(τ gen )) is the logarithmic probability that the generated path is correctly identified as false by the discriminator, using backpropagation and gradient descent algorithms, according to the loss function L D Update the parameters θ of the discriminator D .

[0206] In step 404, based on the authenticity probability score of the discriminator, a reward function of the generator is determined, and the parameters of the generator are updated based on the reward function.

[0207] For example, the reward function of the generator is determined according to the evaluation result of the discriminator on the path, and the authenticity probability score output by the discriminator is used as the reward function value for updating the parameters of the generator, which can be represented by formula (12), which is described in detail below.

[0208] r(τ)=D(τ) (12)

[0209] Here, r(τ) is the reward for path τ, D(τ) is the authenticity probability score of the discriminator for path τ, and the reward function reflects the quality of the generated path. If the value of the authenticity probability score is high, it means that the discriminator believes that the path is close to the true path, thereby giving the generator a higher reward function value. Otherwise, the reward function value is low.

[0210] Based on the policy gradient method, the parameters of the generator are updated according to the policy update formula of the generator. The policy gradient method can be a reward-guided policy algorithm (REINFORCE). The reward-guided policy algorithm is an unbiased estimation policy gradient method that uses the cumulative rewards of the sampling path to update the policy parameters. The policy gradient update formula of the generator can be represented as formula (13), which is described in detail below.

[0211]

[0212] Where: θ G are the parameters of the generator, α is the learning rate, is the generator in state s t Next select action a t The probability of , r(τ) is the reward function of path τ, that is, D(τ). The calculation of the reward function provides feedback to the generator. The policy gradient update uses this feedback to adjust the parameters of the generator. Through continuous iteration of this process, the generator learns how to generate more realistic data and improve the quality of the generated reasoning path.

[0213] In step 405, when the parameter iterations of the generator and the discriminator reach preset conditions, a trained question-answering model is obtained.

[0214] For example, when the question-answering model is initialized, the learning rate of the question-answering model and the number of iterations of the preset value are also set. The training iterates the parameters between the generator and the discriminator until the preset conditions are met, and the training of the question-answering model is stopped to obtain the trained question-answering model. The learning rate controls the pace of parameter updates during the training process and determines the magnitude of the model's optimization of parameters according to the loss function in each iteration. The value of the learning rate is usually between 0 and 1, and can be adjusted according to different applications and model structures. The preset value condition is the balance between the generator and the discriminator and the convergence of the training. The balance standard refers to the ability of the generator and the discriminator to reach a certain equilibrium state. In this state, the generator can generate realistic samples so that the discriminator cannot easily distinguish between the generated samples and the real samples. The balance standard can be measured by the stability of the loss function and the judgment of the quality of the generated samples, which is explained in detail below.

[0215] When the loss function changes of the generator and discriminator tend to be stable, the loss functions of the generator and discriminator no longer fluctuate or decrease significantly, and the loss function value of the discriminator is close to 0.5, indicating that the discriminator's ability to distinguish between real samples and generated samples is roughly the same.

[0216] When the quality of samples generated by the generator is significantly improved, the samples are close to the real data distribution visually or through other quantitative indicators. Some evaluation indicators can be used to quantitatively evaluate the quality of generated samples, such as the Inception Score (IS) or Fréchet Inception Distance (FID).

[0217] For example, during the training of the question-answering model, the update steps of the generator and the discriminator can maintain a certain ratio, such as 1:1 or 1:5, to prevent excessive updates of either party from causing stagnation or failure of the training of the other party. Based on the learning curves of the generator and the discriminator, it can be determined whether the two are overfitting or underfitting.

[0218] For example, the convergence criterion means that the training process reaches a certain termination condition and further training can be stopped. There are usually several criteria to judge whether the training has converged, which are described in detail below.

[0219] Preconfigure the number of iterations and set a maximum number of iterations in advance, such as 10,000 or 100,000. Stop training after reaching this number of iterations.

[0220] Get the rate of change of the loss function corresponding to each iteration. If the rate of change of the loss function of the generator and discriminator in multiple consecutive iterations is small, it can be considered to have converged. For example, if the loss function changes less than a certain threshold (such as 0.001) in 500 consecutive iterations, the model is considered to have converged.

[0221] When the parameter iterations of the generator and discriminator reach the preset conditions, the training is stopped. That is, the steps of iterative path generation, discriminator training, and generator parameter update are repeated, and the balance and convergence criteria of the question-answering model are checked every several iterations to determine whether training needs to be continued. For example, the preset conditions are 20,000 iterations or the loss change is less than 0.001 for 500 consecutive times. When the pre-configured conditions are met, it indicates that the question-answering model is tending to balance at this time, and the training is stopped to obtain the trained question-answering model for the output of subsequent answers.

[0222] In step 406, the performance of the trained question-answering model is evaluated based on the validation set, the parameters of the question-answering model are adjusted based on the performance evaluation results, and the question-answering model with adjusted parameters is called to obtain the answer to the question to be answered.

[0223] For example, based on the validation set obtained during initialization, the performance of the trained question-answering model is evaluated. The hyperparameters of the question-answering model are adjusted based on the performance evaluation indicators, and the question-answering model with adjusted hyperparameters is called to output the answer to the question to be answered. The performance evaluation indicators are the accuracy, precision, recall and evaluation value (F1 value) of the question-answering model or the Hits@1 and Hits@10 standards. The hyperparameters of the question-answering model include the learning rate, optimizer or network structure. Hits@1 is whether the first-ranked item in the sorted list returned by the question-answering model is the correct answer. If the first-ranked item is correct, it is recorded as a successful hit, otherwise it is recorded as a miss, which measures the accuracy of the question-answering model for the query. Hits@10 is whether the top ten items in the sorted list returned by the question-answering model contain at least one correct answer. If at least one of the top ten items is correct, it is recorded as a successful hit, which measures the proportion of correct answers that the question-answering model can retrieve among all possible answers.

[0224] The accuracy rate is the ratio of the number of samples correctly predicted by the question-answering model to the total number of samples. It is used to measure the overall correctness of the question-answering model and can be expressed as formula (14), which is described in detail below.

[0225] Accuracy=(TP+TN) / (TP+TN+FP+FN) (14)

[0226] Among them, Accuracy is the accuracy rate, TP is the number of samples whose true label is positive and predicted to be positive, TN is the number of samples whose true label is negative and predicted to be negative, FP is the number of samples whose true label is negative but predicted to be positive, and FN is the number of samples whose true label is positive but predicted to be negative.

[0227] Precision is the proportion of samples predicted as positive by the question-answering model that are actually positive. It is used to measure the accuracy of the model’s correct answers and can be expressed as formula (15), which is described in detail below.

[0228] Precision=TP / TP+FP (15)

[0229] Among them, Precision is the accuracy, TP is the number of samples whose true label is positive and predicted to be positive, and FP is the number of samples whose true label is negative but predicted to be positive.

[0230] The recall rate is the proportion of samples with true positive labels that are correctly predicted as positive by the model. It is used to measure the comprehensiveness of the model's answers and can be expressed as formula (16). The details are as follows.

[0231] Recall=TP / TP+FN (16)

[0232] Among them, Recall is the recall rate, TP is the number of samples whose true label is positive and predicted to be positive, and FN is the number of samples whose true label is positive but predicted to be negative.

[0233] The evaluation value (F1 value) is the harmonic mean of precision and recall, which is used to comprehensively consider the accuracy and coverage of the question-answering model. It can be represented by formula (17), which is explained in detail below.

[0234] F1=(Precision×Recall) / (Precision+Recall)*2 (17)

[0235] Among them, F1 is the evaluation value, Precision is the accuracy, and Recall is the recall rate.

[0236] For example, based on the performance evaluation indicators, the learning rate, optimizer or network structure of the model is adjusted to optimize the model performance. The learning rate is a key parameter that affects the convergence speed and stability of the model. If the learning rate is too high, the model may not converge or oscillate around the optimal solution; if the learning rate is too low, the model converges very slowly. The learning rate is dynamically adjusted using a predefined strategy, which can be represented as formula (18), which is described in detail below.

[0237] α t =α0·e -λt (18)

[0238] Among them, α t is the learning rate of the tth iteration, α0 is the initial learning rate, λ is the decay rate, and t is the number of iterations.

[0239] For example, adjustments to the network structure include changing the number of layers, the number of neurons, the activation function, the regularization method, etc. Common methods include: increasing or decreasing the number of layers and neurons, increasing the complexity of the model or reducing overfitting, changing the activation function or other nonlinear activation functions, and applying regularization to prevent overfitting. Fine-tuning the model in the above way to optimize model performance and selecting a suitable optimizer will help optimize model performance, such as stochastic gradient descent, stochastic average gradient descent, etc., to optimize performance on the validation set and improve generalization ability.

[0240] For example, after model training and evaluation, the trained model is used to predict or generate new data or answers. Before generating answers, the input data needs to be preprocessed. The specific steps of preprocessing may include: text cleaning and standardization, word segmentation and feature extraction. Text cleaning and standardization include removing special characters, punctuation marks, and unifying upper and lower case. Word segmentation is to divide the text into a sequence of words or subwords. Feature extraction is to convert the text into numerical features that the model can understand.

[0241] In some embodiments, see Figure 5 , Figure 5 It is a structural diagram of the machine learning model provided in an embodiment of the present application; after the input data preprocessing is completed, the machine learning model 501 is called to generate the answer, the encoder 5011 inputs the preprocessed data into the machine learning model 501 so that the model can understand and process it, the generator 5012 infers or predicts each input, and generates a corresponding answer, the discriminator 5013 evaluates the generated answer, and the decoder 5014 converts the numerical value or probability output by the model into a readable answer form, such as text, classification label, etc.

[0242] After generating the answer to the question to be answered, you need to format the output to ensure that the format of the answer meets the expected output requirements, and further analyze or process the generated answer to verify its quality and accuracy. If the answer does not meet expectations, you may need to adjust the model, optimize the data preprocessing steps, or adjust the model hyperparameters to optimize or adjust the quality or applicability of the answer.

[0243] The training method of the question-answering model provided in the embodiment of the present application has the following beneficial effects:

[0244] By constructing a temporal knowledge graph, taking into account the temporal changes of knowledge, dynamically representing knowledge, updating it over time, solving the problem of insufficient static knowledge representation, and assisting in providing more accurate and timely answers. The generator of the question-answering model is called for path reasoning, and answers are provided by generating multiple reasoning paths, so that the limitations of a single reasoning path are solved, the accuracy of the answer is improved, and the richness of the results is increased. The discriminator is called to evaluate the reasoning path, effectively distinguishing high-quality reasoning paths, and ensuring the reliability of the answers generated by the question-answering model. According to the authenticity probability score determined by the evaluation, the loss function of the discriminator and the reward function of the generator are determined. According to the framework of adversarial reinforcement learning, the reasoning efficiency is gradually improved through iterative training before the preset conditions are met, which can improve the efficiency of solving complex reasoning, while ensuring the quality of the generated reasoning answers, so that the trained question-answering model can efficiently handle complex reasoning tasks, which not only improves the performance of the question-answering model, but also enhances the reliability, adaptability and user acceptance of the system.

[0245] The following is a description of an exemplary structure of a training device 455 for a machine learning model provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2As shown, the software modules in the training device 455 of the machine learning model stored in the memory 450 may include: a generator training module 4551, which is used to extract a sample embedding vector from a sample time series knowledge graph, call the generator to perform path reasoning, and obtain an inference path; determine the first sub-loss function of the generator by performing a difference calculation between the inference path and the real path of the sample time series knowledge graph, and train the generator based on the first sub-loss function; a discriminator training module 4552, which is used to call the discriminator for prediction processing based on the inference path to obtain a predicted probability of the real path; determine the second sub-loss function of the discriminator by performing a difference calculation between the actual probability of the real path and the predicted probability, and train the discriminator based on the second sub-loss function; a model construction module 4553, which is used to construct the trained generator and discriminator into a machine learning model.

[0246] In some embodiments, before extracting the sample embedding vector from the sample time series knowledge graph, the generator training module 4551 is also used to construct multiple first time series knowledge graphs based on the first data set, where the first data set is a question set provided by the knowledge base; denoising the multiple first time series knowledge graphs to obtain multiple denoised second time series knowledge graphs; standardizing the time information of the multiple second time series knowledge graphs to obtain multiple third time series knowledge graphs; and selecting a sample time series knowledge graph from the multiple third time series knowledge graphs.

[0247] In some embodiments, the generator training module 4551 is also used to call the generator to determine the node features and context information features of the first node based on the sample embedding vector and the sample time series knowledge graph; iteratively perform node prediction processing based on the node features and context information features to obtain multiple second nodes; the input of each node prediction processing includes: the node features and context information features of the first node and the node features and context information features of the predicted second node, and the output of each node prediction processing is a second node; the first node and each second node are combined into an inference path.

[0248] In some embodiments, the model building module 4553 is also used to determine the updated first sub-loss function of the trained generator, and to determine the updated second sub-loss function and the current number of iterations of the trained discriminator. The updated first sub-loss function and the second sub-loss function are used as the loss functions used in the next update process. When the update process meets the preconfigured conditions, the trained generator and discriminator are constructed as a machine learning model.

[0249] In some embodiments, the preconfigured conditions include at least one of the following: the current number of iterations corresponding to the machine learning model reaches a first preconfigured number of iterations; within a second preconfigured number of iterations, the first change rate of the first current sub-loss function and the second change rate of the second current sub-loss function are both less than a change rate threshold.

[0250] In some embodiments, after the trained generator and discriminator are constructed as a machine learning model, the model construction module 4553 is also used to perform performance evaluation on the trained machine learning model based on a sub-dataset of the sample time series knowledge graph to obtain a performance evaluation result; when the performance evaluation result does not meet the pre-configured evaluation result, the hyperparameters of the question-answering model are adjusted to obtain an adjusted machine learning model.

[0251] In some embodiments, the model building module 4553 is also used to perform at least one of the following processing: based on the second data set, calling the trained machine learning model to perform question and answer processing to obtain a predicted answer, testing the predicted answer through a preconfigured scoring standard to obtain a performance evaluation result, and the second test set includes a question set for testing the machine learning model; based on the second data set, calling the trained machine learning model to perform question and answer processing to obtain a predicted answer, and determining the performance evaluation result of the machine learning model based on the labeled true answer and the predicted answer corresponding to the second data set, and the performance evaluation result includes at least one of the following: a first accuracy rate, a first precision rate, and a first recall rate.

[0252] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the training method of the machine learning model described in the embodiment of the present application.

[0253] The present application embodiment provides a computer-readable storage medium, in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will execute the training method of the question-answering model provided in the present application embodiment, for example, Figure 3A The training method of the machine learning model is shown.

[0254] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various electronic devices including one or any combination of the above memories.

[0255] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0256] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0257] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0258] In summary, through the embodiments of the present application, the generator in the machine learning model is called to perform path reasoning based on the sample embedding vector, and the discriminator in the machine learning model is called to predict the generated reasoning path. Through the interaction between the generator and the discriminator, the quality of the generated reasoning path is evaluated, and the parameters of the generator and the discriminator are updated. The efficiency of the machine learning model in generating answers when solving reasoning problems is improved, the performance of the trained machine learning model is evaluated and the hyperparameters are adjusted to improve the accuracy of the generated answers.

[0259] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A training method for a machine learning model, characterized in that: The method comprises: Extract sample embedding vectors from sample time series knowledge graphs, call the generator to perform path reasoning, and obtain the reasoning path; Determine a first sub-loss function of the generator by calculating the difference between the inference path and the real path of the sample time series knowledge graph, and train the generator based on the first sub-loss function; Based on the reasoning path, calling the discriminator to perform prediction processing to obtain the prediction probability of the true path; Determining a second sub-loss function of the discriminator by calculating a difference between an actual probability of the true path and the predicted probability, and training the discriminator based on the second sub-loss function; The trained generator and discriminator are constructed as a machine learning model.

2. The method according to claim 1, characterized in that Before extracting the sample embedding vector from the sample time series knowledge graph, the method further includes: Based on a first data set, construct multiple first time-series knowledge graphs, where the first data set is a question set provided by a knowledge base; De-noising the plurality of first time series knowledge graphs to obtain a plurality of second time series knowledge graphs after denoising; Performing normalization processing on the time information of the plurality of second time series knowledge graphs to obtain a plurality of third time series knowledge graphs; A sample time series knowledge graph is selected from the multiple third time series knowledge graphs.

3. The method according to claim 1, characterized in that The calling generator performs path reasoning to obtain a reasoning path, including: Based on the sample embedding vector and the sample time series knowledge graph, calling the generator to determine the node features and context information features of the first node; Iteratively perform node prediction processing based on the node features and the context information features to obtain multiple second nodes; The input of each node prediction process includes: the node feature and context information feature of the first node and the node feature and context information feature of the predicted second node, and the output of each node prediction process is a second node; The first node and each of the second nodes are combined into the reasoning path.

4. The method according to claim 1, characterized in that: The step of constructing the trained generator and the discriminator into a machine learning model comprises: Determine the updated first sub-loss function of the trained generator, and determine the updated second sub-loss function and the current iteration number of the trained discriminator, wherein the updated first sub-loss function and the second sub-loss function are used as loss functions used in the next update process; When the updating process satisfies preconfigured conditions, the trained generator and discriminator are constructed as a machine learning model.

5. The method according to claim 4, characterized in that The pre-configured condition includes at least one of the following: The current number of iterations corresponding to the machine learning model reaches a first preconfigured number of iterations; Within a second preconfigured number of iterations, a first change rate of the first sub-loss function and a second change rate of the second sub-loss function are both less than a change rate threshold.

6. The method according to any one of claims 1 to 5, characterized in that: After constructing the trained generator and the discriminator as a machine learning model, the method further includes: Based on the sub-dataset of the sample time series knowledge graph, the performance of the trained machine learning model is evaluated to obtain a performance evaluation result; When the performance evaluation result does not meet the preconfigured evaluation result, the hyperparameters of the machine learning model are adjusted to obtain the adjusted machine learning model.

7. The method according to claim 6, characterized in that The sub-dataset based on the sample time series knowledge graph performs performance evaluation on the trained machine learning model to obtain a performance evaluation result, including: Perform at least one of the following actions: Based on the second data set, calling the trained machine learning model to perform question-answering processing to obtain a predicted answer, testing the predicted answer by a preconfigured scoring standard to obtain a performance evaluation result, wherein the second test set includes a question set for testing the machine learning model; Based on the second data set, call the trained machine learning model to perform question and answer processing to obtain a predicted answer, and determine a performance evaluation result of the machine learning model based on the labeled true answer corresponding to the second data set and the predicted answer, wherein the performance evaluation result includes at least one of the following: a first accuracy rate, a first precision rate, and a first recall rate.

8. A training device for a machine learning model, characterized in that: The device comprises: A generator training module is used to extract a sample embedding vector from a sample time series knowledge graph, call the generator to perform path reasoning, and obtain a reasoning path; by performing a difference calculation between the reasoning path and the real path of the sample time series knowledge graph, determine the first sub-loss function of the generator, and train the generator based on the first sub-loss function; A discriminator training module is used to call the discriminator to perform prediction processing based on the reasoning path to obtain the predicted probability of the real path; determine the second sub-loss function of the discriminator by calculating the difference between the actual probability of the real path and the predicted probability, and train the discriminator based on the second sub-loss function; A model building module is used to build the trained generator and discriminator into a machine learning model.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, used to implement the training method of the machine learning model described in any one of claims 1 to 7 when executing the computer executable instructions or computer program stored in the memory.

10. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the training method of the machine learning model described in any one of claims 1 to 7 is implemented.

11. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the training method of the machine learning model described in any one of claims 1 to 7 is implemented.

Images