Intelligent Go telling method and system based on multi-modal retrieval enhancement

Abstract

The invention belongs to the field of multi-modal large model application, and provides an intelligent Go telling method and system based on multi-modal retrieval enhancement, and the technical scheme is as follows: obtaining historical chess and corresponding comments, and extracting feature embedding of the historical chess and the corresponding comments; training the composition feature extraction network based on the similarity between the historical composition feature embedding and the corresponding comment feature embedding to obtain a trained composition feature extraction network; extracting feature embedding of the current chess and historical chess features based on the trained chess feature extraction network; calculating the similarity between the current chess feature embedding and each historical chess feature embedding, obtaining similar chess according to a similarity calculation result, and searching corresponding comments according to indexes of the similar chess; searching corresponding comments according to the AI prediction result of the current chess and the index of the similar chess to form cue words; and obtaining a speaking game analysis result of the current chess game according to the cue word and the large language model. And the comment generation credibility of the pre-trained large language model is improved.

Description

Technical Field

[0001] The present invention belongs to the field of multimodal large model applications, and in particular relates to an intelligent Go game interpretation method and system based on multimodal retrieval enhancement. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Due to the extreme complexity of Go, machines cannot play by listing all possible moves, and traditional Go programs can only reach the advanced beginner level. In recent years, with the development of deep learning, Go engines based on reinforcement learning, especially AlphaZero and KataGo, have developed chess skills that surpass human players. However, these Go engines can only give win rates and predictions, and cannot directly enable users to deeply understand the decisions in the game. Users still need to listen to professionals' explanations to understand the current situation and changes in the game.

[0004] To solve this problem, large language models based on retrieval-enhanced generation technology have made it possible for intelligent Go game commentary. Large language models (LLM) are trained with massive amounts of data, can understand basic knowledge and professional terms in the field of Go, and use hundreds of billions of parameters to generate original output for tasks such as answering questions and completing sentences. Currently, large language models, such as GPT-4o, can play against humans and generate analytical comments on the current moves and the game based on prompt words such as winning rate and land occupation.

[0005] Although the pre-trained large language model can understand Go terminology and game rules, it faces a huge challenge: AI Hallucinations, that is, the model will produce completely fabricated information without any known facts. This usually leads to the risk of generating wrong or false information, which will have a serious impact on vertical fields, especially professional applications such as Go game commentary.

[0006] An effective way to address this challenge is to use multimodal retrieval-augmented generation technology. Retrieval-augmented generation (RAG) technology expands the already powerful natural language generation of large language models to access the internal knowledge base of specific fields or organizations, retrieve relevant information from authoritative and predetermined knowledge sources, and enhance the relevance and accuracy of text generated in vertical fields. Go can essentially be seen as a pattern recognition game: from eyes and qi to fixed patterns and land occupation, professional players rely on a rich set of concepts to express the structure on the board. Based on the existing Go games with a large number of human annotations, retrieving the board and natural language annotations as two modalities can provide a more accurate source of information for the large language model and enhance the credibility of the generated text. However, Go game datasets are usually sparse, and each move may not necessarily retrieve exactly the same game in the existing data, and the credibility of the generated comments is low. Summary of the invention

[0007] In order to solve at least one of the technical problems existing in the above-mentioned background technology, the present invention provides an intelligent Go game commentary system based on multimodal retrieval enhancement for beginners. For the current game, it searches the internal database of the Go professional field, constructs prompt words based on the retrieved similar games and comments, and improves the credibility of the comments generated by the pre-trained large language model.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] The first aspect of the present invention provides an intelligent Go game interpretation method based on multimodal retrieval enhancement, comprising the following steps:

[0010] Obtain historical chess games and corresponding comments, and extract feature embeddings of historical chess games and corresponding comments;

[0011] The chess game feature extraction network is trained based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain a trained chess game feature extraction network;

[0012] Extract feature embedding of the current chess game and features of historical chess games based on the trained chess game feature extraction network;

[0013] Calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games based on the similarity calculation results, and search for corresponding comments based on the index of similar chess games;

[0014] The AI ​​prediction result of the current chess game and the index of similar chess games are searched for the corresponding comments to form prompt words;

[0015] The analysis results of the current chess game are obtained based on the prompt words and the large language model.

[0016] Furthermore, the chess game feature extraction network is trained based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain the trained chess game feature extraction network, including:

[0017] Extract the first chess game feature embedding based on the pre-trained KataGo residual neural network;

[0018] Output the first chess game feature embedding to a trainable neural network Φ to obtain a second chess game feature embedding;

[0019] A loss function of the global model is constructed based on the similarity between the second chess game feature embedding and the corresponding comment feature embedding. When the global model loss function converges to a predetermined accuracy or reaches a predetermined number of training rounds, a trained neural network Φ is obtained.

[0020] Furthermore, the loss function of the global model is:

[0021]

[0022]

[0023] Among them, τ is the temperature parameter of the neural network, and the key-value pair of historical chess games-comments is (B i ,C i ), is the feature embedding of the second chess game, is the comment feature embedding; C j Any comment in the collection of comments.

[0024] Furthermore, after obtaining the historical chess games and the current chess games, the historical chess games and the current chess games are converted into the SGF standard format.

[0025] Furthermore, the large language model adopts GPT-4o or Tongyi Qianwen 2.5.

[0026] Furthermore, the AI ​​prediction result of the current chess game is obtained through KataGo analysis, including winning rate analysis and next step prediction.

[0027] The second aspect of the present invention provides an intelligent Go game interpretation system based on multimodal retrieval enhancement, comprising:

[0028] The chess game feature extraction network training module is configured to obtain historical chess games and corresponding comments, extract feature embeddings of the historical chess games and the corresponding comments; train the chess game feature extraction network based on the similarity between the feature embeddings of the historical chess games and the feature embeddings of the corresponding comments, and obtain a trained chess game feature extraction network;

[0029] A feature extraction module is configured to extract feature embedding of the current chess game and feature embedding of the historical chess game based on the trained chess game feature extraction network;

[0030] The prompt word generation module is configured to calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games according to the similarity calculation results, search for corresponding comments according to the index of the similar chess games; and form prompt words with the AI ​​prediction result of the current chess game and the corresponding comments searched for the index of the similar chess games;

[0031] The game comment generation module is configured to obtain the game analysis results of the current chess game based on the prompt words and the large language model.

[0032] A third aspect of the present invention provides a computer-readable storage medium.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned intelligent Go game interpretation method based on multimodal retrieval enhancement.

[0034] A fourth aspect of the present invention provides a computer device.

[0035] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the above-mentioned intelligent Go game interpretation method based on multimodal retrieval enhancement are implemented.

[0036] A fifth aspect of the present invention provides a program product.

[0037] A program product, which is a computer program product, includes a computer program, and when the computer program is executed by a processor, it implements the steps in the intelligent Go game interpretation method based on multimodal retrieval enhancement as described above.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] Based on the transmission of chess game data between the front-end and back-end servers, the present invention retrieves similar chess games and existing comments from the vector database, combines the Go AI prediction results, and generates a commentary analysis of the current chess game by a large language model. At the same time, based on the designed retriever, the chess game embedding model is trained to guide the retrieval process of similar chess games, and finally an intelligent Go commentary system based on multimodal retrieval enhanced generation is realized, which improves the accuracy of retrieval and realizes the credibility of comment generation.

[0040] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0042] Figure 1 This is a framework diagram of an intelligent Go game interpretation system based on multimodal retrieval enhancement provided by an embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the structure of the retriever training and reasoning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] As mentioned in the background technology of this application, the Go game data set is usually sparse, and each move may not necessarily be able to retrieve the exact same game in the existing data, and the credibility of the generated comments is low. Based on the transmission of the game data between the front and back ends of the server, the present invention retrieves similar games and existing comments from the vector database, combines the Go AI prediction results, and generates the analysis of the current game by the large language model. At the same time, based on the designed retriever, the retrieval process of similar games is guided by training the game embedding model, and finally an intelligent Go game commentary system based on multimodal retrieval enhancement generation is realized, which improves the accuracy of retrieval and realizes the credibility of comment generation.

[0048] Embodiment 1

[0049] like Figure 1 and Figure 2 As shown, this embodiment provides an intelligent Go game interpretation method based on multimodal retrieval enhancement, comprising the following steps:

[0050] Step 1: Obtain historical chess games and corresponding comments, and extract feature embeddings of historical chess games and corresponding comments;

[0051] In this embodiment, the historical chess games and corresponding comments are represented as follows: the existing chess games B = {B1, B2, ..., B N} and corresponding comments C = {C1, C2, ..., C N};

[0052] For a given chess game-comment key-value pair (B i , C i ), assuming It is a pre-trained KataGo residual neural network The C-dimensional vector space of the encoded H×W feature plane is embedded in the chess game features through a trainable neural network Φ

[0053] Assume that the corresponding comment C i Embedding of features obtained from pre-trained text encoder and and Same dimensions.

[0054] Step 2: Train the chess game feature extraction network based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain the trained chess game feature extraction network;

[0055] The similarity between chess games and comments can be expressed as:

[0056]

[0057] Among them, the data pair (B i , C i ) is a chess game-comment pair,

[0058] The loss function of the training process can be defined as:

[0059]

[0060] Among them, τ is the temperature parameter of the neural network, C j is any comment in comment set C;

[0061] During training, sim(B i , C i ) is B i and all comments C in batch j Calculate the similarity respectively.

[0062] Repeat the above training steps until the global model loss function converges to a predetermined accuracy or reaches a predetermined number of training rounds, and obtain the trained neural network Φ.

[0063] Step 3: Get the current chess game and pre-process the current chess game;

[0064] In this embodiment, after obtaining the current chess game, the current chess game played by the user is converted into a standard format SGF (SmartGameFormat).

[0065] Step 4: Extract feature embedding of the current game and features of historical games based on the trained game feature extraction network;

[0066] Let the existing chess game B = {B1, B2, ..., B N} The feature vector embedding is obtained by sequentially using the pre-trained KataGo deep neural network and the trained neural network Φ The vector database FAISS is used to store the game index and the corresponding comments C = {C1, C2, ..., C N}.

[0067] In the reasoning process based on the multimodal retriever, for the new chess game B input by the front end q , vector feature embedding is obtained through the pre-trained KataGo residual neural network and the trained multi-layer neural network Φ

[0068] Step 5: Calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games based on the similarity calculation results, and search for corresponding comments based on the index of similar chess games;

[0069] In this embodiment, the current chess game feature embedding and vector storage E are calculated. B The similarity formula for each feature vector embedding in is:

[0070]

[0071] Among them, i∈{1, 2, ..., N}, take the top k maximum similarity TopK(sim(B q , B i )) and get a similar chess game R B = {B r}, r∈{1, 2, ..., k}. Find the corresponding comment R according to the index of similar chess games c ={C r}, r∈{1, 2, ..., k} is an important component of the prompt word.

[0072] Step 6: Use the prediction result of the current chess game through the KataGo network and the index of similar chess games to find the corresponding comments to form a prompt word;

[0073] For the new chess game B passed in by the front end q , assuming that the result of KataGo analysis and prediction is P q , the prompt word can be expressed as Prompt(P q , R C ).

[0074] Step 7: Obtain the analysis result of the current chess game based on the prompt words and the large language model.

[0075] In this embodiment, the pre-trained large language model G is used as the generator, and the analysis output for the current chess game is:

[0076] Analysis=G(Prompt(P q , R C )),

[0077] The pre-trained large language model G can use any large language model, such as GPT-4o, Tongyi Qianwen 2.5, etc.

[0078] Through the above scheme, the intelligent Go commentary function is realized by using a multimodal retrieval-enhanced generation model, retrieving similar games and existing comments from the vector database, combining the Go AI prediction results, and generating a commentary analysis of the current game by a large language model. In addition, the present invention designs a retriever, which guides the retrieval process of similar games by training a game embedding model, and finally realizes an intelligent Go commentary system based on multimodal retrieval-enhanced generation, which improves the accuracy of retrieval and realizes the credibility of commentary generation.

[0079] Embodiment 2

[0080] This embodiment provides an intelligent Go game interpretation system based on multimodal retrieval enhancement for beginners, including a server and a user.

[0081] The user terminal is configured to: process the current chess game played by the user and send it to the server terminal;

[0082] In this embodiment, the user terminal is configured to: send the current chess game of the user to the server terminal in a standard format SGF (SmartGameFormat) through an API;

[0083] The server includes a KataGo analysis module, a chess game feature extraction network training module, a feature extraction module, a prompt word generation module and a game comment generation module;

[0084] The KataGo analysis module is configured to receive a current chess game played by a user, analyze the current chess game based on KataGo, and obtain a KataGo chess game analysis result;

[0085] After receiving the chess game in SGF format through the API, the KataGo analysis module processes the current chess game and sends it to the KataGo server deployed in the cloud for win rate analysis and next step prediction, etc., and obtains the return result of KataGo;

[0086] The chess game feature extraction network training module is configured to obtain historical chess games and corresponding comments, extract feature embeddings of the historical chess games and the corresponding comments; train the chess game feature extraction network based on the similarity between the historical chess game feature embeddings and the corresponding comment feature embeddings to obtain a trained chess game feature extraction network;

[0087] The feature extraction module is configured to extract feature embedding of the current chess game and feature embedding of the historical chess game based on the trained chess game feature extraction network;

[0088] The prompt word generation module is configured to calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games according to the similarity calculation result, search for corresponding comments according to the index of the similar chess games; and form prompt words by combining the AI ​​prediction result of the current chess game and the corresponding comments searched for the index of the similar chess games;

[0089] The present invention searches for similarity between the current chess game and the chess games pre-stored in the trained multimodal retriever and vector database, queries the chess game comments corresponding to the chess game with the greatest similarity through index, and combines the chess game comments corresponding to the chess game with the greatest similarity and the KataGo chess game analysis results into prompt words, which helps to improve the credibility of comments generated by the pre-trained large language model.

[0090] The game comment generation module is configured to obtain the game analysis results of the current game based on the prompt words and the large language model, and return them to the user-side Go game web page interface for display.

[0091] The implementation process of each module in this embodiment is consistent with the content of embodiment 1.

[0092] Embodiment 3

[0093] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the intelligent Go game interpretation method based on multimodal retrieval enhancement as described above are implemented.

[0094] Embodiment 4

[0095] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the intelligent Go game interpretation method based on multimodal retrieval enhancement as described above are implemented.

[0096] Embodiment 5

[0097] This embodiment provides a program product, which is a computer program product, including a computer program. When the computer program is executed by a processor, the steps in the intelligent Go game interpretation method based on multimodal retrieval enhancement as described above are implemented.

[0098] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0099] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0102] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent Go game interpretation method based on multimodal retrieval enhancement, characterized in that: The steps include: Obtain historical chess games and corresponding comments, and extract feature embeddings of historical chess games and corresponding comments; The chess game feature extraction network is trained based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain a trained chess game feature extraction network; Extract feature embedding of the current chess game and features of historical chess games based on the trained chess game feature extraction network; Calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games based on the similarity calculation results, and search for corresponding comments based on the index of similar chess games; The AI ​​prediction result of the current chess game and the index of similar chess games are searched for the corresponding comments to form prompt words; The analysis results of the current chess game are obtained based on the prompt words and the large language model.

2. The intelligent Go game interpretation method based on multimodal retrieval enhancement as claimed in claim 1, characterized in that: The chess game feature extraction network is trained based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain the trained chess game feature extraction network, including: Extract the first chess game feature embedding based on the pre-trained KataGo residual neural network; Output the first chess game feature embedding to a trainable neural network Φ to obtain a second chess game feature embedding; A loss function of the global model is constructed based on the similarity between the second chess game feature embedding and the corresponding comment feature embedding. When the global model loss function converges to a predetermined accuracy or reaches a predetermined number of training rounds, a trained neural network Φ is obtained.

3. The intelligent Go game interpretation method based on multimodal retrieval enhancement as claimed in claim 2, characterized in that: The loss function of the global model is: Among them, τ is the temperature parameter of the neural network, and the key-value pair of historical chess games-comments is (B i ,C i ), is the feature embedding of the second chess game, is the comment feature embedding; C j Any comment in the collection of comments.

4. The intelligent Go game interpretation method based on multimodal retrieval enhancement as claimed in claim 1, characterized in that: After obtaining the historical chess games and the current chess game, the historical chess games and the current chess game are converted into the SGF standard format.

5. The intelligent Go game interpretation method based on multimodal retrieval enhancement as claimed in claim 1, characterized in that: The large language model adopts GPT-4o or Tongyi Qianwen 2.

5.

6. The intelligent Go game interpretation method based on multimodal retrieval enhancement as claimed in claim 1, characterized in that: The AI ​​prediction results of the current chess game are obtained through KataGo analysis, including winning rate analysis and next step prediction.

7. An intelligent Go game interpretation system based on multimodal retrieval enhancement, characterized in that: include: A chess game feature extraction network training module is configured to obtain historical chess games and corresponding comments, and extract feature embeddings of the historical chess games and corresponding comments; The chess game feature extraction network is trained based on the similarity between the historical chess game feature embedding and the corresponding comment feature embedding to obtain a trained chess game feature extraction network; A feature extraction module is configured to extract feature embedding of the current chess game and feature embedding of the historical chess game based on the trained chess game feature extraction network; The prompt word generation module is configured to calculate the similarity between the feature embedding of the current chess game and the feature embedding of each historical chess game, obtain similar chess games according to the similarity calculation results, search for corresponding comments according to the index of the similar chess games; and form prompt words with the AI ​​prediction result of the current chess game and the corresponding comments searched for the index of the similar chess games; The game comment generation module is configured to obtain the game analysis results of the current chess game based on the prompt words and the large language model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of an intelligent Go game interpretation method based on multimodal retrieval enhancement as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of an intelligent Go game interpretation method based on multimodal retrieval enhancement as described in any one of claims 1-6 are implemented.

10. A program product, the program product being a computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of an intelligent Go game interpretation method based on multimodal retrieval enhancement as described in any one of claims 1 to 6 are implemented.

Images