Multi-modal interactive chess teaching method and system based on large model

Abstract

The invention relates to the technical field of artificial intelligence, and provides a multimodal interactive chess teaching method and system based on a language large model, and the method comprises the steps: obtaining a chess game image, recognizing a chess game, combining the chess level of a chess player, generating a chess game description through a large language model, and controlling a voice interaction mechanism to feed back the chess game description; based on the chess game, the teaching difficulty and the chess player style, analyzing to obtain a recommended walking method of the chess learner, and controlling the chessboard lighting interaction mechanism and / or the voice interaction mechanism to feed back the recommended walking method so as to complete chess piece falling of the chess learner; judging whether chess playing is finished or not, and if not, returning to analyze the chess playing position of the robot; and if the chess piece falling position is finished, giving a winner, updating the chess player style and the chess art level of the chess player according to the chess piece falling position of each step of chess, the chess game after the chess piece falling, the emotional state of the chess player and the winner, and adjusting the teaching difficulty of the chess player by utilizing a large language model. And each chess learner can improve the chess art under the condition of being suitable for the rhythm of himself / herself.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a multi-modal interactive chess teaching method and system based on a large model. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of education, chess education is regarded as an effective means of quality education, which has a positive impact on improving students' comprehensive quality. In particular, Go, a highly strategic and competitive intellectual sport, plays an important role in cultivating logical thinking, decision-making ability and psychological quality. With the rapid development of artificial intelligence technology, AI has made major breakthroughs in the field of Go. Among them, AlphaGo's victory marks that AI has reached an unprecedented level in the field of Go and has opened up a new path for Go education. However, existing Go AI applications are mainly concentrated in high-level games and competitions, and support for popular education and personal learning is still limited.

[0004] At present, the traditional method of Go teaching mainly relies on the personal experience of Go teachers, lacking systematic scientific analysis and personalized guidance. The teaching content is often fixed and difficult to adapt to the learning progress and style of different Go learners. In addition, although Go learning resources are abundant, they lack effective integration and intelligent processing, making it difficult for Go learners to obtain targeted guidance and suggestions from them. At the same time, Go learners often face the problem of lack of immediate feedback and high-level playing partners during the self-study process. In addition, most of the Go teaching software and online platforms on the market lack natural language interaction capabilities, cannot provide in-depth tactical analysis and strategy suggestions, and are difficult to adjust the teaching content and difficulty according to the specific situation of Go learners. These limitations not only limit the potential of Go in improving logical thinking, concentration, patience, decision-making ability, etc., but also affect the intelligentization of Go teaching, personalized learning experience and the popularization of Go education. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a multimodal interactive chess teaching method and system based on a large model, which relies on the technical advantages of the large language model to realize personalized Go teaching. Through the large language model and multimodal interaction method, the teaching content and difficulty can be flexibly adjusted according to the chess learner's chess level, mood and style, and a learning strategy can be tailored for each chess learner, thereby effectively improving learning efficiency and promoting the widespread dissemination and popularization of Go education.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a multi-modal interactive chess teaching method based on a large model, comprising:

[0008] After obtaining the chess game image and identifying the Go chess game, the robot's chess position and reasons for making moves are obtained through large language model analysis in combination with the teaching difficulty and the learner's style, and the robot's chess position is fed back through a certain interaction mode to complete the robot's move, and the reasons for the move are fed back through the voice interaction mechanism; after obtaining the chess game image and identifying the Go chess game, the robot generates a chess game description through a large language model in combination with the learner's chess skill level, controls the voice interaction mechanism to feed back the chess game description, and waits for the learner to complete the move;

[0009] Determine whether the game is over. If not, return to reacquire the chess game image and analyze the robot's chess position. If it is over, give the winner. Based on the position of each move, the Go game after the move, the learner's emotional state and the winner, use the large language model to update the learner's style and chess skill level. Determine the interaction mode based on the learner's chess skill level. And adjust the teaching difficulty of the learner based on the learner's current winning rate.

[0010] Furthermore, the large language model adopts a multimodal large language model GPT-4o, which is obtained by pre-training and fine-tuning through a Go game database; the pre-training adopts a masked language model and a next sentence prediction task; and the fine-tuning is to fine-tune the large language model for application tasks based on the pre-training.

[0011] Furthermore, the application task is to generate a chess game description based on the chess game and the chess player's chess skill level; generate the chess player's style and chess skill level based on the position of each chess move, the chess game after the move, the chess player's emotional state and the winner; or, based on the chess game, teaching difficulty and chess player's style, analyze the next move and the reason for the move.

[0012] Furthermore, if the chess player is a beginner, the interaction modes are chessboard lighting interaction and voice interaction;

[0013] If the chess player is an amateur, the interaction mode is chessboard lighting interaction or machine move / drop interaction;

[0014] If the chess learner is a professional chess player, the interaction mode will be machine-pickup / drop interaction.

[0015] Furthermore, the teaching difficulty is adjusted by:

[0016] D new =D current +α·(P correct -P target )

[0017] Among them, D new is the adjusted teaching difficulty, D current is the teaching difficulty before adjustment, α is the learning rate, P correct is the winning rate of chess learners so far, P target is the target win rate.

[0018] Furthermore, the step of identifying the Go game includes: graying the game image, adaptive filtering, contrast enhancement, perspective transformation and interpolation, and then using a convolutional neural network model to identify the chess piece coordinates and chess piece types in the game image to obtain the Go game.

[0019] The second aspect of the present invention provides a multi-modal interactive chess teaching system based on a large model, which comprises:

[0020] The multimodal interactive teaching module is configured to: obtain a chess game image and identify the Go chess game, then analyze the robot's chess position and move reason through a large language model in combination with the teaching difficulty and the learner's style, feedback the robot's chess position through a determined interactive mode, complete the robot's move, and feedback the move reason through a voice interaction mechanism; obtain a chess game image and identify the Go chess game, then generate a chess game description through a large language model in combination with the learner's chess skill level, control the voice interaction mechanism to feedback the chess game description, and wait for the learner to complete the move;

[0021] The learner information update module is configured to: determine whether the game is over. If not, return to reacquire the chess game image and analyze the robot's chess position; if it is over, give the winner, and use the large language model to update the learner's style and chess skills based on the position of each move, the Go game after the move, the learner's emotional state and the winner, and determine the interaction mode based on the learner's chess skills; and adjust the learning difficulty of the learner based on the learner's winning rate to date.

[0022] Furthermore, the large language model adopts a multimodal large language model GPT-4o, which is obtained by pre-training and fine-tuning through a Go game database; the pre-training adopts a masked language model and a next sentence prediction task; and the fine-tuning is to fine-tune the large language model for application tasks based on the pre-training.

[0023] The third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the multimodal interactive chess teaching method based on a large model as described above.

[0024] The fourth aspect of the present invention provides a computer device, comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein when the processor executes the program, the steps in the multimodal interactive chess teaching method based on a large model as described above are implemented.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] Large model personalized teaching strategy: The present invention adopts a large language model to more deeply analyze the behavior, style and chess skill level of chess learners. It dynamically adjusts the teaching content and difficulty according to the learning progress, level, emotional state and style of chess learners, and combines multimodal interaction methods to provide personalized learning experience, ensuring that every chess learner can improve his chess skills at a pace that suits him.

[0027] Multimodal interaction improves learning engagement: The present invention significantly improves the engagement and learning experience of chess learners through multimodal interaction methods, such as voice, visual and tactile feedback. In contrast, the existing technology only provides a single mode or limited interaction methods, and fails to fully utilize the potential of multimodal interaction to enhance learning motivation.

[0028] Intelligent game analysis enhanced by large models: The present invention uses a large language model to conduct in-depth analysis of the Go game and provide accurate game descriptions for Go learners. This is relatively rare in the existing technology. Most existing technologies lack such in-depth analysis capabilities or require human assistance to provide similar analysis.

[0029] Real-time feedback and adaptability of large model optimization: The large language model in the present invention can process the interactive data of chess learners in real time, provide feedback on the chess learners' chess playing style and chess skills, and dynamically adjust the teaching content and difficulty based on the feedback. This adaptability is not available in existing technologies, which require more manual intervention or cannot respond to the needs of chess learners in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 A multi-modal interactive chess teaching system and device architecture diagram based on a large model according to an embodiment of the present disclosure;

[0032] Figure 2 A diagram of a Go teaching language and lighting interaction device according to an embodiment of the present disclosure;

[0033] Figure 3This is a diagram of a Go teaching device for picking up and dropping pieces according to an embodiment of the present disclosure;

[0034] Figure 4 The present invention is a schematic diagram of the hardware platform of a large-model-based multi-modal interactive chess teaching system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0036] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0037] Embodiment 1

[0038] This embodiment provides a multimodal interactive chess teaching method based on a large model.

[0039] This embodiment provides a multimodal interactive chess teaching method based on a large model. By acquiring and analyzing Go game data, a rich chess game database is constructed and maintained. This database contains a variety of historical chess games and tactics, chess games played by chess learners, and related chess game teaching materials, etc., which are used to train and optimize the pre-trained large language model so that it can understand and generate natural language content related to chess.

[0040] This embodiment provides a large-model-based multimodal interactive chess teaching method, which uses a multimodal large language model, such as GPT-4o, for pre-training and fine-tuning, so that the large language model can understand and process images, text descriptions, and voice commands of Go games; and by designing specific pre-training tasks, such as masked language model (MLM) and next sentence prediction (NSP) tasks, the large language model learns the coherence and strategy analysis of the game.

[0041] This embodiment provides a multimodal interactive chess teaching method based on a large model. Through a large language model and a multimodal interactive method, it is possible to dynamically adjust the teaching content and difficulty according to the learning progress, level, emotional state and style of the chess learners, thereby providing a personalized learning experience.

[0042] This embodiment provides a multimodal interactive chess teaching method based on a large model, which integrates an adaptive teaching module and a large language model. This integration not only enhances the ability to understand and generate Go teaching content, but also ensures that the teaching content can meet the needs of different Go learners, thereby achieving truly personalized teaching.

[0043] This embodiment provides a multi-modal interactive chess teaching method based on a large model, such as Figure 1 As shown, the following steps are included:

[0044] Step 1: Obtain and process chess game history data.

[0045] Step 101, multi-source data collection of historical Go games: collect historical Go game data from various Go competition records, online Go game platforms, etc.; at the same time, use OCR technology and computer vision technology to extract historical Go game data from historical Go books and Go explanation videos.

[0046] Among them, some of the chess game history data are historical chess games in SGF format containing chess game comments, and this part of the data does not need to perform step 102.

[0047] Among them, the collected chess game historical data includes: the coordinates of each move, the type of move, the description text of the move position, the reason for the move (reason for the move), the game after the move, the game comments (game description), the difficulty of the game, the emotional state of the players, and the final outcome.

[0048] Among them, the text describing the position of the chess piece can be: coordinate indication type (such as placing the black piece at the coordinate (A, 4) of the chessboard), relative position type (such as placing the white piece at the empty intersection on the right side of the black piece), or based on a fixed pattern or layout description type (such as, according to the fixed pattern of the star position small flying hanging corner, the black piece is placed at the star position hanging corner in the upper right corner), etc.

[0049] For example, the description of the chess position can be: After the small flying piece guards the corner at the star position, it then moves to the side. The reason for the move is: through reasonable spacing, expand your sphere of influence on this side, build a wider shape, and have the opportunity to transform this area into a real number of points (territory) in the future, like the once popular "three-star" layout, which continuously places pieces on one side of the chessboard and then moves to the side, intending to create a grand shape, induce the opponent to break in and destroy it, and then use the thick position to attack and profit, and surround more space.

[0050] Among them, the chess game records whether there is a chess piece and the type of chess piece at each of the 361 intersections on the chessboard.

[0051] Among them, the example of chess game description: This is a middle stage of the game, with White playing first. White has formed a certain solid area in the upper right and lower left corners, while Black is eyeing the game covetously with several external positions. The distribution of pieces on both sides is relatively balanced, and they are in the stage of testing each other and looking for breakthrough points.

[0052] Step 102, data conversion: Develop or use existing conversion tools to convert chess game information (text description of the position of each chess move, move coordinates, move type, move reason, the chess game after the move, chess game comments (game description), game difficulty, chess player emotional state and final victory or defeat) into SGF format.

[0053] Among them, the SGF file uses a tree structure to represent the game: a complete game is divided into multiple nodes, a node represents a chess move (a chess step), and each node is associated with a description text of the position of the move, the coordinates of the move, the type of move, the reason for the move, the game after the move, the game commentary (game description), the game difficulty (can be used as teaching difficulty), and the emotional state of the chess player; the points are connected by branches, and the branches represent the direction from one chess game to the next; each SGF file is associated with the final victory or defeat result.

[0054] Among them, data conversion involves writing scripts or using specialized libraries to handle the format conversion of chess game information to ensure data consistency and readability.

[0055] Among them, data conversion converts the chessboard positions from the traditional coordinate system to a numerical range. For example, the 361 intersections of the chessboard are mapped to values ​​​​from 0 to 360. This conversion helps the large language model process and learn the relative positions of the chessboard space.

[0056] Step 103, data cleaning and labeling: The converted SGF file is cleaned, including: checking the legality of the moves, the accuracy of the wins and losses, and the appropriateness of the game description, ensuring the accuracy of the move information, and removing format errors or incomplete records; then, the SGF file is labeled, including the move type of each node (such as capture, capture, and robbery), the game stage (such as opening, middle game, closing game, etc.) and the styles and skills of the two players associated with each SGF file to obtain a data set.

[0057] Step 104, data enhancement: Enhance the dataset by simulating different chess game variations or using known chess game theories, including: randomly changing part of the chess game, or generating new chess game variants based on specific strategies to improve the generalization ability of the large language model.

[0058] Step 105, data integration and storage: Store the data set in a database suitable for large-scale data processing, such as a relational database or a NoSQL database, for subsequent model training and analysis; for large-scale data sets, it is necessary to use a distributed file system or a cloud database to optimize storage and access efficiency; and then obtain a Go game database.

[0059] Through the above steps, a structured, cleaned and optimized Go game dataset can be prepared to provide high-quality input for large language model training. This ensures that the large language model can learn effective features from the dataset, thereby providing accurate predictions and personalized teaching content.

[0060] Step 2: Pre-training and fine-tuning of large language models.

[0061] Step 201, large language model selection: adopt an advanced multimodal large language model, such as GPT-4o.

[0062] Among them, GPT-4o can process a variety of data types including text and images, providing a basis for multimodal understanding of Go games.

[0063] Step 202: pre-training task design.

[0064] Pre-training tasks are designed, including masked language model (MLM) and next sentence prediction (NSP) tasks, which are aimed at training the large language model to understand the Go game, game descriptions, and the voice instructions (piece position descriptions) of the learner. The MLM task randomly masks some words in the input text (game description or piece position description) to let the large language model predict these words; while the NSP task constructs a sample consisting of two sentences to let the large language model determine whether the second sentence is the next sentence of the first sentence; at the same time, the MLM task randomly masks some pieces in the input game to let the large language model predict these pieces; at the same time, the NSP task constructs a sample consisting of two games to let the large language model determine whether the second game is the next step of the first game, which is very helpful for understanding the coherence of the Go game.

[0065] Suppose there is a text description sentence of a Go game: "White has an advantage at [MASK]." In the MLM task, the large language model needs to predict the masked word "corner". The output of the large language model is a probability distribution P(w i |w1,...,w i-1 ,w i+1 w1,...,w N ), where w i is the masked word, and w1,...,w i-1 ,w i+1 w1,...,w N It is the unobscured word in the sentence.

[0066] Suppose two sentences are given. The first sentence is "Black launched an offensive in the center.", and the second sentence is "This forced White to strengthen its defense." In the NSP task, the large language model needs to determine whether the second sentence is the next sentence of the first sentence. The large language model outputs a probability P(s2|s1), which indicates the probability that the second sentence follows the first sentence.

[0067] During the pre-training process, the large language model sees a large number of chess game images and text pairs as well as the voice instructions of the chess learners. The large language model needs to learn how to extract useful features from these data and make predictions on the MLM and NSP tasks. The training goal of the large language model is to minimize the loss function L = L MLM +L NSP , where L MLM is the loss of the MLM task, L NSP It is the loss of NSP mission.

[0068] Step 203, model fine-tuning: Based on the pre-training, a fine-tuning dataset is constructed for specific application tasks, and the large language model is fine-tuned. The parameters of the large language model are adjusted to minimize the loss function of the specific task and improve the performance of the large language model in Go game analysis and teaching.

[0069] In this embodiment, the application tasks include:

[0070] (1) Application Task 1: Generate the chess player’s chess skill level and style by analyzing the position of each chess move, the chess situation after the move, the emotional state of the chess player and the winner, and the large language model;

[0071] (2) Application Task 2: Based on the chess game and the chess player’s chess skill level, the large language model is used to generate a detailed description of the chess game;

[0072] (3) Application Task 3: Based on the chess game, teaching difficulty, and learner’s style, the large language model analyzes and provides specific recommendations for the next move, chess position information, and reasons for the move.

[0073] When executing application tasks, the large language model performs multimodal data fusion: the Go game, text description (game description) and the voice command of the learner are integrated. This requires the large language model to be able to process and understand information of different modalities at the same time, improving the generalization ability of the large language model and in-depth understanding of the Go game. For example, the game shows that the white chess piece is placed in the star position in the upper left corner, and the text description is "White chess took the offensive at the star position." The large language model integrates the information of these two modalities by learning a joint representation R = f (I, T), where I is the game image feature and T is the text (voice command text or game description) feature.

[0074] Step 204, performance evaluation and optimization: Evaluate the performance of the large language model on different fine-tuning datasets through cross-validation to ensure the generalization ability of the large language model. At the same time, according to the feedback and learning effect of chess learners, further adjust the parameters of the large language model to achieve continuous optimization of the large language model.

[0075] Among them, the data sets used for pre-training and fine-tuning of the large language model all come from the Go game database in step 1.

[0076] Step 3, obtaining existing information of chess learners: in response to the user registration instruction, the external display interaction processing module controls the external display interaction mechanism to display the registration interface, receives the interaction information fed back by the chess learners, and obtains the teaching difficulty and interaction mode selected by the user.

[0077] The interactive information fed back by chess learners is the basic information of chess learners, including: name, age, gender, contact information, competition information, honors obtained, chess learner's style and chess level, etc.

[0078] Step 4, according to the teaching difficulty, interaction mode, chess learner style and chess skill level obtained in step 3 or step 5, complete the multi-modal interactive AI chess game;

[0079] Step 401, game start: prepare to start the game, the chess learner or AI interaction processing module is in standby state, the chessboard detection camera and the multimodal interaction device are ready, and the order of playing chess is confirmed.

[0080] Step 402, real-time chess game data acquisition: the chessboard detection camera collects chess game images, and converts the chess game images into digital signals and sends them to the computing host.

[0081] Among them, the chessboard detection camera uses a high-resolution camera.

[0082] Step 403, real-time chess game data processing, chess game status update: after the computing host receives the digital signal from the chessboard detection camera, it updates the chess game to ensure that the current state of the chess game is consistent with the actual chess piece positions.

[0083] (1) Preprocessing: For Go game images, the image preprocessing method enhanced by deep learning is used to improve the quality of the game images and provide clear input for the CNN model.

[0084] Among them, deep learning enhanced image preprocessing methods include: grayscale, adaptive filtering, contrast enhancement, etc.

[0085] (2) Perspective transformation and interpolation: Automatically calibrate and process chessboard images from multiple perspectives to achieve more accurate perspective transformation and obtain a bird's-eye view. Improve the resolution and quality of chessboard images through bilinear interpolation. Explore deep learning-based methods to automatically correct chessboard image distortion, such as using a deep network to predict transformation parameters and achieve perspective conversion.

[0086] (3) CNN model recognition: Develop a customized convolutional neural network (CNN) model to recognize chess piece coordinates and chess piece type (black or white) in chess game images.

[0087] Among them, the CNN model achieves accurate recognition of chess pieces by learning the mapping relationship between image features and chessboard states. Convolutional neural network training involves a large amount of chess game image data, and may use data enhancement techniques such as rotation and scaling to improve the generalization ability of the model.

[0088] In this embodiment, a high-resolution camera is used to capture chess game images in real time, and a convolutional neural network (CNN) is used to perform image recognition and processing.

[0089] The image is obtained by using convolutional neural network (CNN) to identify the position of the chess piece. For each chess piece, the probability of its position on the chessboard is calculated, which can be expressed by the following formula:

[0090]

[0091] Among them, P(position) is the probability that the chess piece is at a specific position, z is the eigenvalue of the position, N is the total number of positions on the chessboard, and e is the base of the natural logarithm.

[0092] In the context of Go, the local perception ability of the CNN model can capture local patterns and features in the chessboard image. The Go board is a two-dimensional structure, and the state of each position is related to the surrounding positions. Therefore, CNN is more suitable for processing the local structure of Go. In addition, CNN uses parameter sharing to reduce the number of model parameters, improve training efficiency and generalization ability. When processing Go board images, CNN can ignore the absolute position of the board and pay more attention to the local structure and relative relationship of the board.

[0093] (4) SGF format conversion: The chess game recognized by the CNN model is converted into the SGF format to ensure the compatibility of the data with the Go software and database, facilitating further analysis and storage.

[0094] Step 404, multi-modal interactive AI chess game: after obtaining the chess game image and identifying the chess game status, combined with the chess learner's chess learning information data (teaching difficulty, chess learner's style), use the large language model to analyze and generate recommended moves, chess position information and reasons for moves, and complete the robot's chess round through the corresponding interactive mode.

[0095] For Go beginners, a large language model is used to analyze the game and learner information to generate basic game descriptions and simple recommended moves for beginners. Based on the generated recommended moves and chess position information, the voice interaction mechanism and the chessboard lighting interaction mechanism are controlled to provide multimodal prompts of the chess position to the learner, guiding the learner to complete the AI ​​chess round. The recommended moves are highlighted by the chessboard lighting interaction mechanism to provide visual prompts for beginners, allowing them to understand the game and recommended moves more intuitively. The voice interaction mechanism provides verbal guidance and explains the reasons for the recommended moves to beginners in simple and easy-to-understand language, helping them understand the importance of each step and the logic behind it. The reasons for the moves start with the teaching of basic rules and gradually introduce the introduction of basic tactics. As learners master the basic concepts, the difficulty gradually increases to adapt to their progress and needs.

[0096] For amateur Go players: The large language model deeply analyzes the game, providing more complex tactical analysis and multiple possible moves. Depending on the personal preferences of the learner, you can choose to use the board lighting interaction to highlight recommended moves, or use the machine move / drop interaction to simulate the opponent's moves. The voice interaction provides more in-depth tactical analysis and encourages amateurs to explore different moves. The reasons for the moves include discussions of tactical variants and game reviews to improve amateurs' tactical understanding and application capabilities.

[0097] For professional Go players: The large language model provides advanced tactical analysis, including the opponent's possible response and prediction of advanced tactics. The machine's move / drop interaction mechanism simulates the opponent's moves, providing professional Go players with a close-to-real-combat experience. The voice interaction mechanism provides real-time tactical advice and opponent analysis to help professional Go players make decisions during the game. The move reasoning focuses on in-depth research on advanced tactics and the use of psychological tactics to improve the competitive level of professional Go players.

[0098] In summary, the interaction method can be flexibly selected according to the needs and preferences of chess learners to complete multimodal interactive AI chess games and realize AI chess teaching.

[0099] For chess learners of different levels, AI can adjust the teaching difficulty to suit different levels of chess learners. By adding the teaching difficulty parameter to the large language model, the convolutional neural network model can provide different levels of difficulty for chess players. For example, for beginners, AI will adopt more basic and direct moves and simplify tactics; for amateur players, it will adopt intermediate tactics, such as fixed patterns and layout strategies; for professional players, it will adopt advanced tactics and strategies, including complex fixed pattern changes and endgame techniques.

[0100] Step 405: Acquire a chess game image, and identify the chess piece coordinates and chess piece type (black or white) in the chess game image through the method of step 403 to obtain the chess game.

[0101] Step 406: Generate a description of the chess game based on the chess game and the chess player's chess skill level through a large language model, and provide feedback through a voice interaction mechanism.

[0102] When generating chess game descriptions, the large language model chooses to use different languages ​​and explanation depths based on the chess player's chess skill level: for beginners, the game description may be simpler and more direct, focusing on explaining the basic rules; for amateur players, the game description contains more tactical analysis, such as fixed pattern selection; for professional players, the game description may contain more tactical analysis and discussion of changes, as well as in-depth analysis of advanced tactics and strategies.

[0103] Step 407, AI recommends moves: Based on the game, teaching difficulty, and player style, the current game is analyzed through a large language model, and moves are recommended to the learner. Feedback can be given through the chessboard lighting interaction mechanism or through voice feedback, and encouragement or explanation of the strategic significance of the recommended moves (reasons for the moves) are given to help learners deeply understand Go strategies.

[0104] This step is similar to step 404, except that step 404 is to complete the robot's move, while step 407 is to complete the chess learner's move.

[0105] Among them, the large language model adjusts the complexity of recommended moves according to the difficulty of teaching: for beginners, the recommended moves may focus more on basic rules and simple tactics; for amateur players, the recommended moves involve intermediate tactics, such as pattern construction; for professional players, the recommended moves include deeper tactics and strategies, recommend advanced tactics, and explain subsequent changes; for amateur or professional players, you can choose not to provide feedback on the recommended moves for learners.

[0106] During the game, AI uses speech synthesis technology to make suggestions on where to place the chess piece, such as "It is recommended to place the chess piece at position C10" or "You have made great progress, keep it up! Next, I suggest you make some arrangements in the middle area of ​​the board," thereby enhancing the learning motivation of chess learners.

[0107] It should be noted that on a 19-way Go board, the coordinate system usually consists of letters and numbers to identify specific positions on the board. Letters from A to T (or S) represent horizontal columns, and numbers from 1 to 19 represent vertical rows. Therefore, C10 represents the intersection of the 3rd column (column C) and the 10th row. This coordinate system helps to describe the position of chess pieces in the game, facilitating communication and analysis. For example, in Go teaching or playing, the instructor may instruct the student or opponent to place the chess piece at the C10 position. This precise coordinate indication is essential for understanding the game and executing the strategy.

[0108] Step 408: The chess learner completes the move.

[0109] As an implementation method, the chess player can respond with voice "C10" to execute the recommended move, or can respond with voice to the move he or she decides, such as "C11". Then, based on the chess game and the voice response (the description of the move position, or the description of the move position and the chess game description), the move position is generated through a large language model, and then the machine's lifting / dropping mechanism makes corresponding actions on the chessboard.

[0110] As another implementation, the chess learner directly moves the chess pieces.

[0111] Step 409: After the learner has finished making his move, determine whether the game is over; if it is over, give the winner; otherwise, return to step 402, and the learner and the robot continue the AI ​​game, and continue to provide real-time analysis and recommendations.

[0112] Among them, the voice interaction processing module uses advanced voice recognition technology to convert the learner's voice commands into text, and can provide natural voice feedback, so that learners can communicate through natural language and obtain Go teaching content and strategy suggestions; it can also explain the strategic significance of its recommended moves through voice, enhancing learners' understanding of Go strategy.

[0113] Among them, the chessboard lighting interaction processing module: according to the results of AI analysis, the light source on the chess box can be illuminated from top to bottom to the designated position of the chessboard, such as C10, to visually prompt the chess learner. In order to further enhance the interactive experience, the chessboard lighting interaction mechanism can use dynamic lighting effects, such as gradient or flashing, to attract the attention of the chess learner and enhance the memory effect. This visual cue is crucial to help chess learners concentrate and understand the tactics in the game. The module dynamically adjusts the changes in the light on the chessboard, such as the gradient of color or brightness, which can indicate the urgency or importance of the move, thereby providing intuitive visual feedback. This intuitive visual cue helps chess learners quickly identify the moves recommended by AI, while also increasing the interactivity and fun of the game. The chessboard lighting interaction processing module can also confirm the validity of the move or provide immediate feedback through light changes after the learner completes a move.

[0114] Among them, the machine move / drop interactive processing module not only enhances the learner's sense of participation, but also helps learners improve their skills in practice by providing real-time feedback and personalized move recommendations. The implementation of this module takes into account the complexity of Go and the special needs of teaching, ensuring that every learner can improve their skills at a pace that suits them.

[0115] Among them, the external display interactive processing module: The external display can display the real-time analysis of the chess game and the AI's strategic suggestions. When the AI ​​recommends move C10, the position can be highlighted on the display and tactical analysis can be provided. A view of the chess game is provided on the external display to help learners understand the layout of the chess game and potential moves from different angles. In addition, the display can also be used to display cultural and historical knowledge related to Go, increasing learners' understanding of Go culture.

[0116] Step 5: Real-time information feedback for chess learners.

[0117] During the game, the player's facial expressions and body language (such as personal posture, hand movements, etc.) are captured through the player detection camera, and computer vision technology is used to analyze his or her emotional state in real time.

[0118] First, by analyzing the position of each chess piece in the game, the situation after each move, the emotional state, and the final outcome, a large language model is used to predict the changes in the learner's style and chess skill level. The learner's style and chess skill level are updated to predict future performance and learning needs, thereby achieving personalized customization of teaching content.

[0119] Then, adjust the interaction mode according to the chess skills of the learners.

[0120] Finally, calculate the winning rate of the chess learner so far and adjust the difficulty of the chess learner's teaching content:

[0121] D new =D current +α·(P correct -P target )

[0122] Among them, D new is the new teaching difficulty, D current is the current difficulty, α is the learning rate, P correct is the proportion of correct answers given by chess learners (the winning rate so far), P target is the target win rate.

[0123] Through the above steps, the big language model of Go teaching can not only analyze the game, but also dynamically adjust the difficulty of the teaching content according to the learning progress of the learners, providing Go learners with a personalized, interactive and fruitful learning experience, so as to realize personalized Go teaching and improve learning effects.

[0124] In this embodiment, the teaching module can ensure that each chess learner can improve his chess skills at a pace that suits him or her, while enjoying personalized teaching content. This strategy not only improves learning efficiency, but also enhances the learning motivation and satisfaction of chess learners.

[0125] In this embodiment, a teaching experience adapted to the needs of chess learners of different levels is provided, thereby improving the learning efficiency and chess skills of the learners. This innovative interactive design can stimulate the interest of chess learners and help them make rapid progress on the road of Go.

[0126] Embodiment 2

[0127] This embodiment provides a multi-modal interactive chess teaching system based on a large model. Figure 2 , Figure 3 and Figure 4 As shown, it specifically includes: a chessboard detection camera 1, a chess learner detection camera 4, a chessboard lighting interaction mechanism 2, a voice interaction mechanism 3, a local controller 5, a standard Go chessboard 6, a machine lifting / dropping interaction mechanism 7, a Go robot body 8, a power supply, a computing host, a cloud server, an external display interaction mechanism, and a chess piece box.

[0128] Chessboard detection camera: captures the position and status of chess pieces on the board and provides clear chess game image input.

[0129] The chess player's posture detection camera monitors the chess player's game behavior and posture, captures the chess player's facial expression and body language, and analyzes the chess player's game status in real time. It is used for information input of the adaptive teaching module to predict the chess player's chess style and chess skill level based on the chess player's feedback and behavior.

[0130] Go robot body: Made of high-strength materials to ensure the stability and durability of the body. Modular design facilitates maintenance and upgrades to adapt to the development and changes of technology.

[0131] Standard Go board: The standard size of the Go board meets international standards, providing a standard game platform.

[0132] Piece box: stores black / white Go pieces, divided into black chess box and white chess box.

[0133] The voice interaction mechanism includes: a microphone array: used to capture the user's voice commands and support far-field voice recognition; a speaker: provides clear voice feedback for teaching guidance and interaction.

[0134] The interactive mechanism of chessboard lighting includes: infrared lighting components: providing uniform lighting to ensure that the chess pieces on the chessboard are clearly visible to assist image recognition and improve the accuracy of chess piece recognition; rotating components: achieving dynamic lighting effects through rotating components to enhance visual experience. The lighting effects also highlight the key points of teaching and help learners understand the chess game.

[0135] The machine's interactive mechanism for lifting / dropping chess pieces includes: X, Y, and Z three-axis moving parts of the robot arm: the three-axis moving parts are used to achieve precise positioning of the robot arm on the chessboard, and to achieve flexible movement and positioning of the robot arm on the chessboard, simulating the lifting and dropping operations of human arms; lifting / dropping parts: a precisely designed clamping mechanism to ensure stable clamping and placement of chess pieces.

[0136] Power supply: Provides stable power supply for the entire teaching system. It can use built-in batteries or external power adapters to support long-term operation of the teaching system, including overcharge, over-discharge, overheating and short-circuit protection to ensure safe power use and extend battery life.

[0137] Local controller: It is equipped with a high-performance processor for executing algorithms and data processing. It is equipped with sufficient RAM to ensure smoothness when processing large-scale chess game data. It uses high-speed storage devices such as SSD to increase data reading and writing speed and reduce latency. It is equipped with voice interaction processing module, chessboard lighting interaction processing module, machine lifting / dropping interaction processing module, and external display interaction processing module.

[0138] Computing host: Equipped with a high-performance processor and large-capacity random access memory (RAM), the system is highly efficient and smooth when executing complex algorithms and processing large-scale chess game data. The high-speed computing power of the processor supports real-time chess game analysis and dynamic adjustment of teaching strategies, and can run multiple applications at the same time, including image recognition, natural language processing, and machine learning models. The computing host has a built-in pre-trained and fine-tuned large language model that can understand and generate natural language content related to chess, generate Go teaching strategies, provide accurate chess game descriptions, and recommend moves and chess position information.

[0139] Cloud Server: Dynamically allocate computing resources based on demand, process large-scale data sets and complex machine learning tasks, synchronize and back up chess game data and user learning progress to the cloud, and ensure data security and cross-device access.

[0140] External display interaction mechanism: provides a high-resolution touch screen display for displaying chess game analysis, teaching content and interactive interface.

[0141] The computing host is configured with a multimodal interactive teaching module and a chess learner information updating module.

[0142] The multimodal interactive teaching module is as follows: according to the generated recommended moves and chess position information, the multimodal prompts provided to the learner are guided to complete the AI ​​chess round by controlling the voice interaction mechanism and the chessboard lighting interaction mechanism. In addition, according to the generated recommended moves and chess position information prompts, the system sends chess position instructions to the robot through the robot's machine lifting / dropping interaction mechanism, and the robot's machine lifting / dropping interaction mechanism autonomously completes the AI ​​chess round. The system can flexibly select the interaction mode according to the needs and preferences of the learner, complete the multimodal interactive AI chess game, and realize AI chess teaching. The chess game image is acquired and analyzed through image recognition technology, and the coordinates and types of chess pieces in the chess game are identified using advanced image processing technologies such as grayscale, adaptive filtering, contrast enhancement and perspective transformation. Then, the large language model is used to understand and parse the learner's natural language input, including questions and feedback, to generate personalized teaching content. The module can generate a detailed description of the chess game and provide feedback to the learner through a voice interaction mechanism. At the same time, based on the chess game, teaching difficulty and learner's style, it analyzes and provides recommended moves. These moves are fed back to the learner in the form of visual prompts through the chessboard lighting interaction mechanism, or provided in the form of verbal guidance through the voice interaction mechanism. In addition, the module also provides multimodal feedback and interaction methods, including real-time voice feedback, visual prompts and tactile feedback, to simulate the real game experience. The teaching difficulty is dynamically adjusted according to the learner's winning rate and target winning rate to ensure that the learner always learns at an appropriate challenge level. The module can also monitor the learner's emotional state and adjust the teaching strategy according to emotional changes. Finally, the module tracks the learner's learning progress and uses a large language model to update the learner's chess skill level and style based on the position of each move, the chess game after the move, the learner's emotional state and the winner, and continuously optimizes the teaching content to provide a more effective learning experience.

[0143] The chess learner information update module is configured to: determine whether the game is over. If not, return to reacquire the chess game image and analyze the robot's chess position; if it is over, give the winner, and use the large language model to update the chess learner's style and chess skill level based on the chess piece placement position of each move, the chess game after the move, the chess learner's emotional state and the winner, and determine the interaction mode based on the chess learner's chess skill level; and adjust the teaching difficulty of the chess learner based on the chess learner's current winning rate.

[0144] Furthermore, the method for adjusting the teaching difficulty is as follows:

[0145] D new =D current +α·(P correct -P target )

[0146] Among them, D newis the adjusted teaching difficulty, D current is the teaching difficulty before adjustment, α is the learning rate, P correct is the winning rate of chess learners so far, P target is the target win rate.

[0147] Furthermore, the chess game recognition step includes: after graying, adaptive filtering, contrast enhancement, perspective transformation and interpolation of the chess game image, a convolutional neural network model is used to recognize the chess piece coordinates and chess piece types in the chess game image to obtain the chess game.

[0148] It should be noted here that each module in this embodiment corresponds to each step in Example 1 one by one, and the specific implementation process is the same, which will not be repeated here.

[0149] Embodiment 3

[0150] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the multimodal interactive chess teaching method based on a large model as described in the first embodiment above are implemented.

[0151] Embodiment 4

[0152] This embodiment provides a computer device, including a computer-readable storage medium (volatile memory and non-volatile storage medium), a processor, a communication interface (i.e., a network interface), and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor, the communication interface, and the computer-readable storage medium may be connected via a bus or other means. The communication interface is used to receive and send data, and when the processor executes the program, the steps in the multimodal interactive chess teaching method based on a large model as described in the first embodiment above are implemented.

[0153] Among them, any reference to memory, storage, database or other media provided by the present application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multimodal interactive chess teaching method based on a large language model, characterized in that: include: After obtaining the chess game image and identifying the Go chess game, the robot's chess position and reasons for making moves are obtained through large language model analysis in combination with the teaching difficulty and the learner's style, and the robot's chess position is fed back through a certain interaction mode to complete the robot's move, and the reasons for the move are fed back through the voice interaction mechanism; after obtaining the chess game image and identifying the Go chess game, the robot generates a chess game description through a large language model in combination with the learner's chess skill level, controls the voice interaction mechanism to feed back the chess game description, and waits for the learner to complete the move; Determine whether the game is over. If not, return to reacquire the chess game image and analyze the robot's chess position. If it is over, give the winner. Based on the position of each move, the Go game after the move, the learner's emotional state and the winner, use the large language model to update the learner's style and chess skill level. Determine the interaction mode based on the learner's chess skill level. And adjust the teaching difficulty of the learner based on the learner's current winning rate.

2. A multimodal interactive chess teaching method based on a language large model as claimed in claim 1, characterized in that: The large language model adopts the modal large language model GPT-4o, which is obtained by pre-training and fine-tuning through a Go game database; the pre-training adopts a masked language model and a next sentence prediction task; the fine-tuning is to fine-tune the large language model for application tasks based on the pre-training.

3. A multimodal interactive chess teaching method based on a language large model as claimed in claim 2, characterized in that: The application task is: to generate a chess game description based on the chess game and the chess learner's chess skill level; to generate the learner's style and chess skill level based on the position of each chess move, the chess game after the move, the learner's emotional state and the winner; or, based on the chess game, teaching difficulty and learner's style, analyze the next move and the reason for the move.

4. A multimodal interactive chess teaching method based on a language large model as claimed in claim 1, characterized in that: If the chess player is a beginner, the interaction modes are chessboard lighting interaction and voice interaction. If the chess player is an amateur, the interaction mode is chessboard lighting interaction or machine move / drop interaction; If the chess learner is a professional chess player, the interaction mode will be machine-pickup / drop interaction.

5. A multimodal interactive chess teaching method based on a language large model as claimed in claim 1, characterized in that: The method for adjusting the teaching difficulty is as follows: D new =D current +α·(P correct -P target ) Among them, D new is the adjusted teaching difficulty, D current is the teaching difficulty before adjustment, α is the learning rate, P correct is the winning rate of chess learners so far, P target is the target win rate.

6. A multimodal interactive chess teaching method based on a language large model as claimed in claim 1, characterized in that: The step of identifying the Go game includes: graying, adaptive filtering, contrast enhancement, perspective transformation and interpolation of the game image, and then using a convolutional neural network model to identify the chess piece coordinates and chess piece types in the game image to obtain the Go game.

7. A multimodal interactive chess teaching system based on a large language model, characterized in that: include: The multimodal interactive teaching module is configured to: obtain a chess game image and identify the Go chess game, then analyze the robot's chess position and move reason through a large language model in combination with the teaching difficulty and the learner's style, feedback the robot's chess position through a determined interactive mode, complete the robot's move, and feedback the move reason through a voice interaction mechanism; obtain a chess game image and identify the Go chess game, then generate a chess game description through a large language model in combination with the learner's chess skill level, control the voice interaction mechanism to feedback the chess game description, and wait for the learner to complete the move; The learner information update module is configured to: determine whether the game is over. If not, return to reacquire the chess game image and analyze the robot's chess position; if it is over, give the winner, and use the large language model to update the learner's style and chess skills based on the position of each move, the Go game after the move, the learner's emotional state and the winner, and determine the interaction mode based on the learner's chess skills; and adjust the learning difficulty of the learner based on the learner's winning rate to date.

8. A multi-modal interactive chess teaching system based on a language large model as claimed in claim 7, characterized in that: The large language model adopts the multimodal large language model GPT-4o, which is obtained by pre-training and fine-tuning through a Go game database; the pre-training adopts a masked language model and a next sentence prediction task; the fine-tuning is to fine-tune the large language model for application tasks based on the pre-training.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of a multimodal interactive chess teaching method based on a large language model as described in any one of claims 1 to 6 are implemented.

10. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored in the computer-readable storage medium and executable on the processor, characterized in that: When the processor executes the program, the steps in the multimodal interactive chess teaching method based on a large language model as described in any one of claims 1-6 are implemented.

Images