Method, System, Electronic Device and Storage Medium for Implementing Game Interaction

Abstract

The embodiment of the present application provides a method, system, electronic device and storage medium for realizing game interaction. The electronic device part includes: a game generation module for receiving game generation input, and calling a multimodal large model based on the game generation input to generate game content; a game initialization module for initializing the display content and game status information of a card device and a chessboard device based on the game content, and displaying and / or voice broadcasting the game rules; a game control module for: identifying the card content and game elements in the chessboard display area and their positions in the chessboard display area in real time during the game, and making game event determinations based on the identification results, game status information and game rules to generate event determination results and interaction results; and controlling the game process based on the interaction results. The system part includes the electronic device, the card device and the chessboard device. It can bring a new, highly customized and intelligent interactive game experience to players.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and specifically relates to a method, system, electronic device, and storage medium for realizing game interaction based on a multimodal large model. Background Art

[0002] Traditional tabletop games often have fixed rules, gameplay, and content, lacking flexibility and the ability of personalized customization. Players are prone to boredom after playing multiple times, and the preparation work of the game, such as card making and chessboard drawing, is relatively cumbersome. In addition, the game process either relies on players' memory and control, resulting in relatively simple game rules; or requires other physical props and recording tools, resulting in a cumbersome game process. Summary of the Invention

[0003] With the rapid development of artificial intelligence technology, especially the powerful content generation ability demonstrated by multimodal large models, applying them to the field of tabletop games is expected to solve many pain points of traditional tabletop games and bring players a brand-new, highly customized, and intelligent interactive game experience.

[0004] In view of this, the embodiments of this application provide a method, system, electronic device, and storage medium for realizing game interaction to solve the above technical problems. The embodiments of this application can be applied to: home entertainment, which can provide family members with rich and diverse, customizable family board game time at any time, enhancing family interaction and the happy atmosphere; commercial board game venues, such as board game bars and cafes, can use this invention to attract more customers, provide unique game services, and enhance competitiveness; education and training, educational tabletop games can be designed, and through artificial intelligence, knowledge questions and answers, task challenges, etc. adapted to the game scenario can be generated to make the learning process more interesting, and it can be applied to scenarios such as school education and enterprise training.

[0005] In a first aspect, an embodiment of the present application provides an electronic device, including: a game generation module, configured to receive game generation inputs, call a multimodal large model based on the game generation inputs, and generate game content through the multimodal large model, where the game content includes game rules, a card set, board display information, and interactive audio, and the card set includes multiple types of cards; a game initialization module, configured to initialize the display content of the card device and the board device and the game state information based on the game content, and display and / or voice broadcast the game rules; a game control module, configured to, during the game process: real-time identify the card content and game elements in the board display area and their positions in the board display area to obtain a first identification result; determine game events based on the first identification result, the game state information, and the game rules to generate an event determination result and an interactive result; control the game process based on the interactive result, including: updating at least one of the game state information, playing interactive audio, updating the display content of the card device, and updating the display content of the board device, where the interactive audio includes at least one of music, sound effects, and voice conversations.

[0006] In some embodiments, the game control module is further configured to: during the game process, real-time identify player voice inputs to obtain a second identification result; where the game control module determines the game events based on the first identification result, the second identification result, the game state information, and the game rules.

[0007] In some embodiments, the game generation module is further configured to call the multimodal large model using prompt words to convert the game content into a format that the game control module can process through the multimodal large model.

[0008] In some embodiments, the game generation module is specifically configured to: store the text file organized from the game rules in a specific format for display and / or voice broadcast; extract the information of the cards in the card set and convert it into an instruction set that can be transmitted to the card device for display, including the encoding format of the display content of each card device; convert the board display information into coordinates and instructions for display, where the instructions include image resource call instructions.

[0009] In some embodiments, the game control module is further configured to: real-time identify player voice inputs; if a question intention is recognized, generate a reply message from a game knowledge base or according to the current game state information and feedback it to the player.

[0010] In some embodiments, the game generation input includes at least one of a player name, the number of players, the game type, the story background, the main characters of the game, and the main plot of the game; and / or the types of cards in the card set include at least one of character cards, skill cards, resource cards, event cards, mission cards, and reward cards; and / or the board display information includes at least one game element among the spatial layout of the game, area division, boundaries and restrictions, terrain, resources, obstacles, special areas, and events; and / or the game rules include the interaction rules between characters and the mission completion conditions; and / or the game state information includes the game stage, the missions completed by the players, the quantity and distribution of the players' resources, and the usage history of the cards.

[0011] In a second aspect, an embodiment of the present application provides an electronic device, including: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to execute a game interaction implementation method, including: receiving game generation input, and invoking a multimodal large model based on the game generation input to generate game content through the multimodal large model, the game content including game rules, a card set, board display information, and interactive audio, and the card set including various types of cards; initializing the display content of a card device, a board device, and game state information based on the game content, and displaying and / or voice-broadcasting the game rules; during the game process: real-time recognizing the card content and game elements in the board display area and their positions in the board display area to obtain a first recognition result; making a game event determination based on the first recognition result, the game state information, and the game rules to generate an event determination result and an interaction result; and controlling the game process based on the interaction result, including: updating at least one of the game state information, playing interactive audio, updating the display content of the card device, and updating the display content of the board device, and the interactive audio includes at least one of music, sound effects, and voice conversations.

[0012] In some embodiments, executing the game interaction implementation method further includes: during the game process, real-time recognizing a player's voice input to obtain a second recognition result; wherein the game event determination is made based on the first recognition result, the second recognition result, the game state information, and the game rules.

[0013] In some embodiments, before initializing the display content of the card device, the board device, and the game state information based on the game content, it further includes: using a prompt to invoke the multimodal large model to convert the game content into a format that can be processed by the game control module through the multimodal large model.

[0014] In a third aspect, an embodiment of the present application provides a game interaction implementation system, comprising: the electronic device mentioned above; a plurality of card devices, the card devices being wirelessly connected to the electronic device for receiving and displaying card information; and a chessboard device for displaying chessboard display information.

[0015] In some embodiments, the card device includes: an electronic ink screen for displaying the card information; a wireless communication module for wirelessly communicating with the electronic device; and a battery module for supplying power to the electronic ink screen and the wireless communication module.

[0016] In a fourth aspect, an embodiment of the present application provides a method for realizing game interaction, comprising: receiving a game generation input, calling a multimodal large model based on the game generation input, so as to generate game content through the multimodal large model, wherein the game content includes game rules, a card set, chessboard display information and interactive audio, and the card set includes multiple types of cards; initializing the display content and game status information of a card device and a chessboard device based on the game content, and displaying and / or voice broadcasting the game rules; during the game: real-time identification of card content and game elements in a chessboard display area and their positions in the chessboard display area to obtain a first recognition result; performing game event determination based on the first recognition result, the game status information and the game rules to generate event determination results and interactive results; controlling the game process based on the interactive result, including: updating at least one of the game status information, playing interactive audio, updating the display content of the card device, and updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogues.

[0017] In a fifth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned game interaction implementation method.

[0018] The game interaction implementation method, system, electronic device and storage medium provided in the embodiments of the present application generate game content based on game-generated inputs with the help of a multimodal large model, monitor the game process using visual and voice recognition technology, act as a referee and provide interactive support, and dynamically adjust the display of game elements and audio playback according to the game process. This can bring players a new, highly customized and intelligently interactive game experience.

[0019] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0021] Figure 1 Fig. shows a schematic block diagram of a game interaction implementation system provided by an embodiment of the present application.

[0022] Figure 2 Fig. shows a schematic block diagram of a card device provided by an embodiment of the present application.

[0023] Figure 3 Fig. shows a schematic block diagram of an electronic device provided by an embodiment of the present application.

[0024] Figure 4 Fig. shows a flowchart of a game interaction implementation method provided by an embodiment of the present application.

[0025] Figure 5 Fig. shows a structural block diagram of an exemplary electronic device capable of implementing the embodiments of the present application. Detailed Embodiments

[0026] The following will describe in detail the embodiments of the present application. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the present application and should not be construed as a limitation of the present application.

[0027] To enable those skilled in the art to better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present application.

[0028] In the embodiments of the present application, it should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0029] Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

[0030] In the description of the embodiments of the present application, words such as "example" or "for example" are used to give examples, explanations or descriptions. Any embodiment or design solution described as "for example" or "for instance" in the embodiments of the present application is not to be construed as being more preferred or having more advantages than another embodiment or design solution. The use of words such as "example" or "for example" is intended to present relative concepts in a clear manner.

[0031] In addition, "a plurality of" in the embodiments of the present application means two or more. In view of this, "a plurality of" in the embodiments of the present application can also be understood as "at least two". "At least one" can be understood as one or more, for example, understood as one, two or more. For example, including at least one means including one, two or more, and does not limit which ones are included. For example, including at least one of A, B, and C, then what can be included are A, B, C, A and B, A and C, B and C, or A and B and C.

[0032] It should be noted that in the embodiments of the present application, "and / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / ", unless otherwise specified, generally represents an "or" relationship between the associated objects before and after.

[0033] It should be pointed out that "connection" in the embodiments of the present application can be understood as electrical connection. The connection of two electrical components can be a direct or indirect connection between the two electrical components. For example, when A is connected to B, it can be either a direct connection between A and B or an indirect connection between A and B through one or more other electrical components.

[0034] The embodiments of the present application provide a game interaction implementation system, as Figure 1 shown, the game interaction implementation system 100 includes an electronic device 10, a plurality of card devices 20, and a chessboard device 30.

[0035] In the embodiments of the present application, the electronic device 10 serves as a game console device. The electronic device 10 may include various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0036] In the embodiments of the present application, the chessboard device 30 may include devices or apparatuses with display functions such as projectors, display screens, etc. Although Figure 1 the chessboard device 30 and the electronic device 10 shown in are separate cubic frames, in actual implementation, the chessboard device 30 and the electronic device 10 may be integrated together, that is, the electronic device 10 includes the chessboard device 30.

[0037] In the embodiments of the present application, as Figure 1 shown, a plurality of card devices 20 are respectively compared to card devices 201 to 20 n . The card device 20 is wirelessly communicatively connected to the electronic device 10. The wireless connection method may include short-range wireless communication technologies such as Bluetooth, ZigBee, Wi-Fi, UWB (Ultra Wideband), Z-Wave (Z-Wave short-range wireless communication technology), etc. The wireless connection method may also include mobile cellular technologies such as 4G, 5G, etc. Short-range communication technologies are preferred. The card device 20 can display visual information such as graphics and text. In the embodiments of the present application, the card device 20 is used to display the card information sent by the electronic device 10. When the card devices 201 to 20 n display the information of different types of cards, a card set including multiple types of cards can be formed. The types of cards may include but are not limited to at least one of character cards, skill cards, resource cards, event cards, task cards, and reward cards. That is to say, a card set composed of specific cards can be formed by making the card device 20 display different card information.

[0038] Please refer to Figure 2As shown, a card device 20 in a typical implementation may include an e-ink screen 21, a wireless communication module 22, and a battery module 23. The e-ink screen 21 has low power consumption. After the display content is refreshed, it will stay on the screen for a long time, and only consumes a small amount of power when the picture changes. It can also display information in the shutdown state, which makes its battery life strong. In addition, the display content of the e-ink screen 21 is still clearly visible under strong light, and there will be no situation like a liquid crystal display where reflection causes unclear viewing, which is convenient for use in environments with complex light such as homes, commercial board game venues, and education and training venues. Moreover, when viewing the screen of the e-ink screen 21 from different angles, the display effect is basically the same, and there is no problem of limited viewing angle, which can meet the needs of multiple people watching simultaneously in game interaction. The wireless communication module 22 may include short-range wireless communication technologies such as Bluetooth, ZigBee, Wi-Fi, UWB (Ultra Wideband), Z-Wave (Z-Wave short-range wireless communication technology), etc., and may also include mobile cellular technologies such as 4G and 5G. Preferably, short-range communication technologies with lower power consumption are selected. The battery module 23 may include a rechargeable battery, specifically, it may include a lithium-ion battery, etc. In some embodiments, the card device 20 may further include a wireless charging coil to facilitate charging of the battery module 23.

[0039] In the embodiments of the present application, multiple types of card devices 20 can be set according to size and shape. Each type of card device 20 can display the card information of one or more types of cards. The card device 20 can be placed in a card storage bin, and the card device 20 is selected from the card storage bin based on the needs of the electronic device 10 to display the card information, forming a specific card set. Specifically, the electronic device 10 can select the card device 20 to display the card information according to the game content to constitute a specific number of character cards, skill cards, resource cards, event cards, task cards, reward cards, etc. Further, when the card device 20 is located in the card storage bin, the card storage bin can charge the card device 20.

[0040] Please continue to refer to Figure 1 As shown, the electronic device 10 may include a game interaction implementation device, and the game interaction implementation device includes a game generation module 101, a game initialization module 102, and a game control module 103. The game generation module 101 is used to receive game generation inputs, call a multimodal large model based on the game generation inputs, so as to generate game content through the multimodal large model. The game content includes game rules, a card set, board display information, and interactive audio. The card set includes multiple types of cards. As Figure 1 As shown, the multimodal large model may include a local multimodal large model 40a and / or a cloud multimodal large model 40b.

[0041] In some implementations, the game generation input may include information in one or more dimensions. In a specific implementation, the game generation input may include at least one of a player name, the number of players, the game type, the story background, the main characters of the game, and the main plot of the game. In some implementations, the types of cards in the card set include at least one of character cards, skill cards, resource cards, event cards, task cards, and reward cards. In some implementations, the board display information includes at least one of the following game elements: the spatial layout of the game, area division, boundaries and restrictions, terrain, resources, obstacles, special areas, and events. In some implementations, the game rules include the interaction rules between characters and the task completion conditions.

[0042] In some embodiments, the game generation module 101 may also use prompt words to call a multimodal large model to convert the game content into a format that the game control module 102 can process. In some implementation manners, the game generation module 101 may specifically be used for: storing the text file organized from the game rules in a specific format for display and / or voice broadcast; extracting the information of the cards in the card set and converting it into an instruction set that can be transmitted to the card device 20 for display, including the encoding format of the display content of each card device 20; and converting the board display information into coordinates and instructions for display, where the instructions may include image resource call instructions, etc.

[0043] The game initialization module 102 is used to initialize the display content of the card device 20 and the board device 20 and the game state information based on the game content, and display and / or voice broadcast the game rules. The game state information may include the game stage, the tasks completed by the players, the quantity and distribution of the players' resources, and the usage history of the cards, etc. In a specific implementation, the game initialization module 102 may select the card device 20 according to the game content to display the card information, so as to form a specific number of character cards, skill cards, resource cards, event cards, task cards, reward cards, etc. The game initialization module 102 may send instructions to the card device 20 and the board device 20 to display the corresponding content on the card device 20 and the board device 20.

[0044] The game control module 103 is used for game process control. Specifically, the game control module 103 can, during the game process: real-time identify the card content and game elements in the board display area and their positions in the board display area to obtain a first recognition result; determine game events based on the first recognition result, game status information, and game rules to generate an event determination result and an interaction result; and control the game process based on the interaction result. In the embodiments of the present application, controlling the game process based on the interaction result may include at least one of: updating the game status information, playing interaction audio, updating the display content of the card device, and updating the display content of the board device. The interaction audio includes at least one of music, sound effects, and voice conversations.

[0045] In some embodiments, the game control module 103 can also be used for: during the game process, real-time identify the player's voice input to obtain a second recognition result. The game control module 103 can also determine game events based on the first recognition result, the second recognition result, game status information, and game rules.

[0046] In some embodiments, the game control module 103 can also be used for: real-time identify the player's voice input; if a question intention is recognized, generate a reply message from the game knowledge base or according to the current game status information and feedback it to the player.

[0047] In some embodiments, as Figure 3 shown, the game interaction implementation device can be implemented as a computer program. The electronic device 10 can include a processor and a memory for storing the program. Among them, the program includes instructions that, when executed by the processor, cause the processor to execute the game interaction implementation method.

[0048] The embodiments of the present application also provide a game interaction implementation method, which can be implemented by the system 100 as Figure 1 shown. As Figure 4 shown, the game interaction implementation method includes the following steps.

[0049] Step S401, receive game generation input, and call a multimodal large model based on the game generation input to generate game content through the multimodal large model.

[0050] In the embodiments of the present application, in the game interaction implementation system 100 as Figure 1 shown, the electronic device 10 can call the local multimodal large model 40a and / or the cloud multimodal large model 40b based on the game generation input to generate game content through the multimodal large model. In some implementations, the game generation input can include information in one or more dimensions. In a specific implementation, the game generation input can include at least one of the player name, the number of players, the game type, the story background, the main game characters, and the main game plot.

[0051] In the embodiments of the present application, the game content includes game rules, a card set, board display information, and interactive audio. In a specific implementation, the interactive audio includes at least one of music, sound effects, and voice conversations. The card set includes multiple types of cards. In some implementations, the types of cards in the card set include at least one of character cards, skill cards, resource cards, event cards, task cards, and reward cards. In some implementations, the board display information includes at least one game element among the spatial layout of the game, area division, boundaries and restrictions, terrain, resources, obstacles, special areas, and events. In some implementations, the game rules include the interaction rules between characters and the task completion conditions.

[0052] In some embodiments, the method further includes: using a prompt to invoke the multimodal large model to convert the game content into a format that can be processed by the game control module through the multimodal large model. In Figure 1 In the game interaction implementation system 100 shown, the electronic device 10 can invoke the local multimodal large model 40a and / or the cloud multimodal large model 40b to convert the game content into a format that can be processed by the game control module through the multimodal large model.

[0053] In a typical implementation, various types of information input by players are first collected and sorted out, such as player names, the number of players, game types, story backgrounds, main characters, and plotlines. This information will serve as the basis for subsequent generation of game content. Data preprocessing operations are performed on the collected information. This includes cleaning the text information, removing redundant characters, normalizing punctuation, etc.; then performing text encoding to convert the text into a digital form that can be processed by a computer. For example, using the word vector representation method to map each word to a low-dimensional vector space so that words with similar semantics are closer in the vector space. The preprocessed information is input into a multi-modal large model in the cloud or locally. The multi-modal large model is based on a neural network architecture internally, such as the Transformer architecture, and its core principle is the multi-head attention mechanism. The model has already acquired the ability to understand game-related semantics and logic through learning a large amount of data on the rules, plotlines, character settings, etc. of different types of games. When the player-customized information is input, the model performs inference and generation operations in its huge parameter space. For example, according to the story background and main characters set by the player, the model will refer to the learned similar story structures and character relationships to generate matching game rules, such as the interaction rules between characters, task completion conditions, etc.; at the same time, it generates the corresponding card composition, including the number, attributes, functions, etc. of different types of cards, and the chessboard layout, such as the basis for area division, the association between special terrains and the story background, etc. Finally, post-processing is performed on the game content generated by the multi-modal large model. This step mainly converts the relatively abstract game information output by the model into a specific and operable format. For example, organizing the generated game rules into a clear text file and storing them in a specific format for subsequent display and voice broadcast on the host device; extracting the card composition information and converting it into an instruction set that can be directly transmitted to the game cards for display, including the encoding format of the electronic ink screen display content of each card; for the chessboard layout information, converting it into instructions for controlling projection or large-screen display, such as coordinates, image resource calls, etc.

[0054] Its algorithm description is as follows: Let the set of player input information be A = {a1, a2,..., a k ..., a n}, where a k represents a single input information element (such as the number of players a1, game type a2, etc.). The data preprocessing process can be represented as a function F, then the preprocessed information set A f = F(A). The multi-modal large model is represented as a function M game , its input is A f , and the output is the game content set G = {g1, g2,..., g m}, where g lRepresents game content in different aspects (such as rule g1, card set g2, board layout g3, etc.), that is, G = M game (A f ). The post - processing process is represented as function Q, and through post - processing, the final information set G that can be directly applied to the game is obtained m = Q(G).

[0055] Step S402, initialize the display content of the card device and the board device and the game state information based on the game content, and display and / or voice - broadcast the game rules

[0056] The game state information may include the game stage, tasks completed by the player, the quantity and distribution of the player's resources, and the usage history of the cards, etc. In a specific implementation, in Figure 1 the electronic device 10 shown, the game initialization module 102 can select the card device 20 according to the game content to display card information, so as to form a specific number of character cards, skill cards, resource cards, event cards, task cards, reward cards, etc. The game initialization module 102 can send instructions to the card device 20 and the board device 20 to display corresponding content on the card device 20 and the board device 20

[0057] In a specific implementation, multiple types of card devices 20 can be set according to size and shape, and each type of card device 20 can display information of one or more types of cards. The card device 20 can be placed in the card storage bin, and the electronic device 10 selects the card device 20 from the card storage bin according to the game content requirement to display card information, forming a specific card set. The electronic device 10 can select the card device 20 according to the game content to display card information, so as to form a specific number of character cards, skill cards, resource cards, event cards, task cards, reward cards, etc

[0058] Combined with Figure 1 and Figure 2 shown, after the electronic device 10 completes game generation, it transmits the card display information to multiple card devices 20 through wireless communication technologies (such as Bluetooth, Zigbee, etc.), controls the electronic ink screen 21 of the card device 20 to display the initial card face content, including character initial attributes, initial resource quantity, etc. At the same time, it transmits the board display information to the board device 30, so that the projection or large screen presents a complete initial game scene, such as initial terrain, building distribution, etc. Display and voice - broadcast the main game rules on the display interface of the electronic device 10 itself, so that players can clearly understand the basic gameplay and objectives of the game

[0059] During the game process, the following steps are executed

[0060] Step S403: Real-time recognize the card content and game elements in the chessboard display area and their positions in the chessboard display area to obtain a first recognition result.

[0061] In the embodiments of the present application, an image of the chessboard display area can be collected by an image acquisition device such as a camera, and the collected image can be subjected to image recognition to recognize the card content and game elements in the chessboard display area and their positions in the chessboard display area.

[0062] Specifically, the camera continuously collects image data of the chessboard display area to obtain an image sequence at a fixed frame rate (for example, 30 frames per second). Each frame of the collected image is preprocessed, including image grayscale processing, converting the color image into a grayscale image to reduce the amount of data and highlight the contour and texture information of the image; performing an image filtering operation, such as Gaussian filtering, to remove noise interference in the image and make the image smoother and clearer for subsequent feature extraction. The preprocessed image is analyzed using an object detection algorithm based on a convolutional neural network (CNN). The CNN can consist of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer slides a convolutional kernel over the image for feature extraction. For example, convolutional kernels of different sizes and weights can extract features such as edges, textures, and shapes in the image; the pooling layer performs dimensionality reduction on the features extracted by the convolutional layer. For example, the max pooling operation can retain the local maximum values in the feature map, reducing the amount of data while retaining the main feature information; the fully connected layer integrates and classifies the features after multiple convolutions and poolings, and finally outputs the recognized target categories (such as various types of cards, game elements on the chessboard, etc.) in the image and the position information of the target in the image (usually represented in the form of the coordinates of a bounding box). For example, when a card is detected, the CNN model can determine whether it is a character card or a skill card and determine the coordinates of its upper left corner and lower right corner in the image, thereby determining the relative position of the card on the chessboard.

[0063] The recognition algorithm can be described as follows: Let the image collected by the image acquisition device at time t be I t , and the image preprocessing process is represented by the function P t , then the preprocessed image I p,t = P t (I t ). The object detection model based on the convolutional neural network is represented by M visual , its input is I p,t , and the output is the recognition result set R t = {(c1, p1), (c2, p2),..., (c i , p i )}, where c i represents the recognized target category, and p iIndicates the position information of the target in the image (such as bounding box coordinates), i.e., R t = M visual (I p,t )

[0064] Step S404: Determine game events based on the first recognition result, game status information, and game rules to generate an event determination result and an interaction result

[0065] Step S405: Control the game process based on the interaction result

[0066] In the embodiments of the present application, controlling the game process based on the interaction result may include at least one of updating game status information, playing interaction audio, updating the display content of the card device, and updating the display content of the chessboard device. The interaction audio includes at least one of music, sound effects, and voice conversations

[0067] In some embodiments, as Figure 4 shown, the method for implementing game interaction further includes: Step S406: During the game, real-time identify the player's voice input to obtain a second recognition result. In step S404, determine game events based on the first recognition result, the second recognition result, game status information, and game rules. In some embodiments, the method for implementing game interaction further includes: Step S407: If a question intention is recognized, generate a reply message from the game knowledge base or according to the current game status information and feedback it to the player

[0068] In specific implementation, the player's voice conversation can be collected through the microphone of the electronic device 10 or the chessboard device 30, and the voice is converted into text information using speech recognition technology (such as calling a multi-modal large model) for understanding and analysis. Specifically, the player has a real-time conversation during the game, and natural language processing technology is used to understand the player's question intention, and an accurate reply message is generated from the game knowledge base or according to the current game status and feedback it to the player. For example, if the player asks about the detailed effect of a certain skill or the goal of the current game task, a clear and accurate answer can be given

[0069] In Figure 1In the system 100 shown, the game control module 103 continuously receives the image recognition results (the first recognition results) and the speech recognition results (the second recognition results), and makes a comprehensive judgment in combination with the current game state information. The game state information includes the current stage of the game (such as the preparation stage, the battle stage, the plot advancement stage, etc.), the list of tasks completed by the player, the quantity and distribution of the player's resources, the usage history of the cards, etc. According to the card position and type information in the image recognition results, and the player instruction information in the speech recognition results, a matching judgment is made with the game rules. For example, if the image recognizes that the player places an attack skill card in a specific area of the chessboard, and at the same time the speech recognition detects that the player says "launch an attack on the enemy player", the game control module 103 can judge whether this operation is legal according to the current game rules. If it is legal, determine the effect of the attack according to the rules, such as the damage value caused to the health of the enemy player, whether to trigger the effects of other cards or chessboard elements, etc., and then generate the interactive result information. The game control module 103 controls the subsequent process of the game according to the interactive result information, including sending display content update instructions to the card device 20 and the chessboard device 30, for example, displaying the information of the reduced health on the player's card under attack, and displaying the change of the attack special effect or the trigger area on the chessboard device 30; at the same time, select and play the corresponding music and sound effects according to the interactive result, such as playing the battle sound effect during an attack, and playing the specific plot sound effect when a special event is triggered, etc. In addition, the game control module 103 can also process the player's real-time conversations. When receiving the player's consultation or information query request, it uses natural language processing technology to understand the player's intention, and generates an accurate reply message from the game knowledge base or according to the current game state and feedbacks it to the player.

[0070] The algorithm can be described as: Let the set of state information at the current moment of the game be S = {s1, s2,..., s x ...}, the set of image recognition results be R t = {(c1, p1), (c2, p2),..., (c i , p i )...}, and the text of the speech recognition result be W. The set of game rules is represented as G r = {(r1, e1), (r2, e2),..., (r j , e j )...}, where r j represents a single game rule, and e j represents the effect or event corresponding to the rule. The game state judgment function is J, whose input is S, R, and W, and the output is the judgment result D (legal or illegal) and the set of interactive result information E = {(e i1 , v i1 ), (e i2 , v i2),...,(e iu ,v iu )...}, where e iu represents an interaction event, and v iu represents the parameters corresponding to the event (such as damage value, resource change amount, etc.), that is, (D, E) = J(S, R, W). The function for controlling the game process according to the interaction result is O, whose input is E, and is responsible for updating the display of game elements, playing audio, and processing player conversations, etc., that is, O(E).

[0071] One or more embodiments of the present application can highly personalize game generation: utilize a multimodal large model to generate unique tabletop games based on the rich and diverse input information of players, break the fixed mode of traditional tabletop games, and meet the personalized needs of players; achieve intelligent game monitoring and refereeing: through vision and speech recognition technologies, comprehensively and real-time monitor the game process, accurately judge the game state and the compliance of operations, and automatically act as the game referee role to enhance the fairness and fluency of the game; create a dynamic game experience: during the game process, dynamically update the display content of game cards and the game board based on game events, and cooperate with matching music, sound effects, and real-time dialogue interactions between players and the game system to create a highly immersive, interactive, and constantly changing game experience.

[0072] An exemplary embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method according to the exemplary embodiments of the application.

[0073] An exemplary embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method according to the exemplary embodiments of the application.

[0074] Refer to Figure 5 , and now the structural block diagram of an electronic device that can be the electronic device 10 of the present application will be described. It is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0075] As Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0076] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device capable of inputting information into the electronic device 500. The input unit 506 can receive input digital or character information and generate key signal inputs related to the user settings and / or function controls of the electronic device. The output unit 507 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include, but is not limited to, magnetic disks and optical disks. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0077] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 executes the various methods and processes described above. For example, in some embodiments, the game interaction implementation method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 can be configured to execute Figure 4 the game interaction implementation method shown.

[0078] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when executed by the processor or controller, the program codes cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0079] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0080] As used in the present application, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., a disk, an optical disc, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal for providing machine instructions and / or data to a programmable processor.

[0081] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0082] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0083] A computer system can include clients and servers. The clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs that run on the respective computers and have a client-server relationship with each other.

[0084] The above are only the preferred embodiments of the present application and do not impose any formal restrictions on the present application. Although the present application has been disclosed above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the equivalent embodiments by using the technical content disclosed above within the scope of the technical solution of the present application. However, as long as the technical solution content of the present application is not departed from, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application still fall within the scope of the technical solution of the present application.

Claims

1. An electronic device, characterized in that: include: A game generation module, configured to receive a game generation input, and call a multimodal macromodel based on the game generation input to generate game content through the multimodal macromodel; Using the prompt word to call the multimodal big model, so as to convert the game content into a format that can be processed by the game control module through the multimodal big model; the game content includes game rules, a card set, board display information and interactive audio, and the card set includes multiple types of cards; A game initialization module, used to initialize the display content and game status information of the card device and the board device based on the game content, and to display and / or voice broadcast the game rules; the electronic device is wirelessly connected to the card device, and the card device is used to receive and display card information; Game control module, used during the game: Collecting image data of a chessboard display area of ​​the chessboard device through a camera, and identifying card content and game elements in the chessboard display area and their positions in the chessboard display area in real time based on the image data to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

2. The device according to claim 1, characterized in that The game control module is also used to: during the game, recognize the player's voice input in real time to obtain a second recognition result; wherein the game control module performs the game event determination based on the first recognition result, the second recognition result, the game status information and the game rules.

3. The device according to claim 1, characterized in that The game generation module is specifically used for: The game rules are organized into a text file and stored in a specific format for display and / or voice broadcast; Extracting information about cards in the card set and converting it into an instruction set that can be transmitted to the card device for display, including the encoding format of the display content of each card device; The chessboard display information is converted into coordinates and instructions for display, wherein the instructions include image resource calling instructions.

4. The device according to claim 1, characterized in that The game control module is also used for: Recognize player voice input in real time; If the question intent is recognized, response information is generated from the game knowledge base or based on the current game status information and fed back to the player.

5. The device according to claim 1, characterized in that The game generation input includes at least one of a player name, a number of players, a game type, a story background, a main character of the game, and a main plot of the game; and / or The types of cards in the card set include at least one of role cards, skill cards, resource cards, event cards, task cards, and reward cards; and / or The board display information includes: at least one game element selected from the game's spatial layout, area divisions, boundaries and restrictions, terrain, resources, obstacles, special areas, and events; and / or The game rules include interaction rules between characters and task completion conditions; and / or The game status information includes the game stage, the tasks completed by the player, the player's resource quantity and distribution, and the card usage history.

6. An electronic device, characterized in that: include: processor; as well as Memory for storing programs, The program includes instructions, and when the instructions are executed by the processor, the processor executes a method for implementing game interaction, including: Receive game generation input, call the multimodal big model based on the game generation input, so as to generate game content through the multimodal big model; call the multimodal big model using a prompt word, so as to convert the game content into a format that can be processed by the instruction through the multimodal big model; the game content includes game rules, a card set, board display information and interactive audio, and the card set includes multiple types of cards; Initialize the display content and game status information of the card device and the board device based on the game content, and display and / or voice broadcast the game rules; the electronic device is wirelessly connected to the card device, and the card device is used to receive and display card information; During the game: Collecting image data of a chessboard display area of ​​the chessboard device through a camera, and identifying card content and game elements in the chessboard display area and their positions in the chessboard display area in real time based on the image data to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

7. The electronic device according to claim 6, characterized in that: The method for implementing game interaction also includes: during the game, real-time recognition of player voice input to obtain a second recognition result; wherein the game event determination is performed based on the first recognition result, the second recognition result, the game status information and the game rules.

8. A game interaction implementation system, characterized in that: include: The electronic device according to any one of claims 1 to 7; A plurality of card devices, the card devices are wirelessly connected to the electronic device and are used to receive and display card information; A chessboard device, used for displaying chessboard display information; The camera is used to collect image data of the chessboard display area of ​​the chessboard device.

9. The system according to claim 8, characterized in that The card device comprises: An electronic ink screen, used to display the card information; A wireless communication module, used for wireless communication with the electronic device; A battery module is used to supply power to the electronic ink screen and the wireless communication module.

10. A method for realizing game interaction, applied to an electronic device, characterized in that: include: receiving a game generation input, and calling a multimodal macromodel based on the game generation input to generate game content through the multimodal macromodel; Using the prompt word to call the multimodal big model, so as to convert the game content into a format that can be processed by the game control module through the multimodal big model; the game content includes game rules, a card set, board display information and interactive audio, and the card set includes multiple types of cards; Initialize the display content and game status information of the card device and the board device based on the game content, and display and / or voice broadcast the game rules; the electronic device is wirelessly connected to the card device, and the card device is used to receive and display card information; During the game: Collecting image data of a chessboard display area of ​​the chessboard device through a camera, and identifying card content and game elements in the chessboard display area and their positions in the chessboard display area in real time based on the image data to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause a computer to execute the method according to claim 10 .

Images