Game interaction implementation method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a game interaction implementation method and system, electronic equipment and a storage medium, and the electronic equipment part comprises a game generation module which is used for receiving game generation input and calling a multi-mode large model based on the game generation input to generate game content; the game initialization module is used for initializing display contents and game state information of the card device and the chessboard device based on game contents, and displaying and / or voice broadcasting game rules; the game control module is used for identifying card contents and game elements in the chessboard display area and positions of the card contents and the game elements in the chessboard display area in real time in the game process, and judging game events based on an identification result, game state information and game rules so as to generate an event judgment result and an interaction result; and controlling the game process based on the interaction result. The system part comprises the electronic equipment, a card device and a chessboard device. And brand-new, highly-customized and intelligent interactive game experience can be brought to players.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a method, system, electronic device and storage medium for implementing game interaction based on a multimodal large model. Background Art

[0002] Traditional table games often have fixed rules, gameplay and content, lacking flexibility and customization. Players tend to get bored after playing many times, and the game preparations, such as card making and board drawing, are cumbersome. In addition, the game process either relies on the player's memory and control, resulting in relatively simple game rules; or requires other physical props and recording tools, resulting in cumbersome game play. Summary of the invention

[0003] With the rapid development of artificial intelligence technology, especially the powerful ability of multimodal large models in content generation, applying it to the field of tabletop games is expected to solve many pain points of traditional tabletop games and bring players a new, highly customized and intelligent interactive gaming experience.

[0004] In view of this, the embodiments of the present application provide a method, system, electronic device and storage medium for realizing game interaction to solve the above technical problems. The embodiments of the present application can be applied to: family entertainment, which can provide family members with rich and varied family board game time that can be customized at any time, and enhance family interaction and joyful atmosphere; commercial board game venues, board game bars, cafes and other venues can use the present invention to attract more customers, provide unique game services, and enhance competitiveness; education and training, educational board games can be designed, and knowledge questions and answers, task challenges, etc. that are adapted to the game scene can be generated through artificial intelligence, making the learning process more interesting, and applied to school education, corporate training and other scenarios.

[0005] In a first aspect, an embodiment of the present application provides an electronic device, including: a game generation module, which is used to receive a game generation input, and call a multimodal large model based on the game generation input to generate game content through the multimodal large model, wherein the game content includes game rules, a card set, chessboard display information and interactive audio, and the card set includes multiple types of cards; a game initialization module, which is used to initialize the display content and game status information of a card device and a chessboard device based on the game content, and display and / or voice broadcast the game rules; a game control module, which is used to: in real time identify the card content and game elements in the chessboard display area and their positions in the chessboard display area during the game, and obtain a first recognition result; perform game event determination based on the first recognition result, the game status information and the game rules to generate an event determination result and an interactive result; and control the game process based on the interactive result, including: updating the game status information, playing interactive audio, updating the display content of the card device, and updating the display content of the chessboard device, wherein the interactive audio includes at least one of music, sound effects and voice dialogue.

[0006] In some embodiments, the game control module is also used to: during the game, recognize the player's voice input in real time to obtain a second recognition result; wherein the game control module performs the game event determination based on the first recognition result, the second recognition result, the game status information and the game rules.

[0007] In some embodiments, the game generation module is further used to call the multimodal big model using a prompt word, so as to convert the game content into a format that can be processed by the game control module through the multimodal big model.

[0008] In some embodiments, the game generation module is specifically used to: organize the game rules into a text file and store it in a specific format for display and / or voice broadcast; extract the information of the cards in the card set and convert it into an instruction set that can be transmitted to the card device for display, including the encoding format of the display content of each card device; convert the chessboard display information into coordinates and instructions for display, and the instructions include image resource call instructions.

[0009] In some embodiments, the game control module is further used to: recognize the player's voice input in real time; if the question intention is recognized, generate reply information from the game knowledge base or based on the current game status information and feedback it to the player.

[0010] In some embodiments, game generation input includes at least one of player name, number of players, game type, story background, main game character and main game plot; and / or the type of cards in the card set includes at least one of role cards, skill cards, resource cards, event cards, task cards and reward cards; and / or the board display information includes: at least one game element of the game's spatial layout, area division, boundaries and restrictions, terrain, resources, obstacles, special areas and events; and / or the game rules include interaction rules between characters and task achievement conditions; and / or the game status information includes the game stage, tasks completed by the player, the player's resource quantity and distribution, and the card usage history.

[0011] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a processor; and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes a game interaction implementation method, comprising: receiving a game generation input, calling a multimodal large model based on the game generation input, so as to generate game content through the multimodal large model, wherein the game content comprises game rules, a card set, chessboard display information and interactive audio, and the card set comprises multiple types of cards; initializing the display content and game status information of a card device and a chessboard device based on the game content, and displaying and / or voice broadcasting the game rules; during the game process: real-time identification of card content and game elements in a chessboard display area and their positions in the chessboard display area to obtain a first recognition result; performing game event determination based on the first recognition result, the game status information and the game rules to generate an event determination result and an interactive result; controlling the game process based on the interactive result, comprising: updating at least one of the game status information, playing interactive audio, updating the display content of a card device, and updating the display content of a chessboard device, and the interactive audio comprises at least one of music, sound effects and voice dialogue.

[0012] In some embodiments, executing the game interaction implementation method also includes: during the game, real-time recognition of player voice input to obtain a second recognition result; wherein the game event determination is performed based on the first recognition result, the second recognition result, the game status information and the game rules.

[0013] In some embodiments, before initializing the display content and game status information of the card device and the chessboard device based on the game content, it also includes: using a prompt word to call the multimodal big model to convert the game content into a format that can be processed by the game control module through the multimodal big model.

[0014] In a third aspect, an embodiment of the present application provides a game interaction implementation system, comprising: the electronic device mentioned above; a plurality of card devices, the card devices being wirelessly connected to the electronic device for receiving and displaying card information; and a chessboard device for displaying chessboard display information.

[0015] In some embodiments, the card device includes: an electronic ink screen for displaying the card information; a wireless communication module for wirelessly communicating with the electronic device; and a battery module for supplying power to the electronic ink screen and the wireless communication module.

[0016] In a fourth aspect, an embodiment of the present application provides a method for realizing game interaction, comprising: receiving a game generation input, calling a multimodal large model based on the game generation input, so as to generate game content through the multimodal large model, wherein the game content includes game rules, a card set, chessboard display information and interactive audio, and the card set includes multiple types of cards; initializing the display content and game status information of a card device and a chessboard device based on the game content, and displaying and / or voice broadcasting the game rules; during the game: real-time identification of card content and game elements in a chessboard display area and their positions in the chessboard display area to obtain a first recognition result; performing game event determination based on the first recognition result, the game status information and the game rules to generate event determination results and interactive results; controlling the game process based on the interactive result, including: updating at least one of the game status information, playing interactive audio, updating the display content of the card device, and updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogues.

[0017] In a fifth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned game interaction implementation method.

[0018] The game interaction implementation method, system, electronic device and storage medium provided in the embodiments of the present application generate game content based on game-generated inputs with the help of a multimodal large model, monitor the game process using visual and voice recognition technology, act as a referee and provide interactive support, and dynamically adjust the display of game elements and audio playback according to the game process. This can bring players a new, highly customized and intelligently interactive game experience.

[0019] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic block diagram of a game interaction implementation system provided in an embodiment of the present application is shown.

[0022] Figure 2 A schematic block diagram of a card device provided in an embodiment of the present application is shown.

[0023] Figure 3 A schematic block diagram of an electronic device provided in an embodiment of the present application is shown.

[0024] Figure 4 A flow chart of a method for implementing game interaction provided in an embodiment of the present application is shown.

[0025] Figure 5 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present application is shown. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0027] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0028] In the embodiments of the present application, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0029] Moreover, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0030] In the description of the embodiments of the present application, words such as "example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "example" or "for example" in the embodiments of the present application is not to be interpreted as being more preferred or having more advantages than another embodiment or design. The use of words such as "example" or "for example" is intended to present relative concepts in a clear manner.

[0031] In addition, the "plurality" in the embodiments of the present application refers to two or more than two. In view of this, in the embodiments of the present application, "plurality" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and there is no limit on which ones are included. For example, including at least one of A, B and C, then A, B, C, A and B, A and C, B and C, or A, B and C can be included.

[0032] It should be noted that, in the embodiments of the present application, "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0033] It should be noted that in the embodiments of the present application, "connection" can be understood as electrical connection, and the connection between two electrical components can be a direct or indirect connection between the two electrical components. For example, the connection between A and B can be either a direct connection between A and B or an indirect connection between A and B through one or more other electrical components.

[0034] The present application embodiment provides a game interaction implementation system, such as Figure 1 As shown, the game interaction implementation system 100 includes an electronic device 10, a plurality of card devices 20 and a chessboard device 30.

[0035] In the embodiment of the present application, the electronic device 10 is used as a game console device, and the electronic device 10 may include various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0036] In the embodiment of the present application, the chessboard device 30 may include a device or apparatus having a display function such as a projector and a display screen. Figure 1 The chessboard device 30 and the electronic device 10 shown in the figure are separate boxes. In actual implementation, the chessboard device 30 and the electronic device 10 can be integrated together, that is, the electronic device 10 includes the chessboard device 30.

[0037] In the embodiments of the present application, Figure 1 As shown, the plurality of card devices 20 are respectively compared to card devices 201 to 20 n The card device 20 is wirelessly connected to the electronic device 10. The wireless connection method may include short-range wireless communication technologies such as Bluetooth, ZigBee, Wi-Fi, UWB (ultra-wideband), Z-Wave (Z-wave short-range wireless communication technology), and the wireless connection method may also include mobile cellular technologies such as 4G and 5G, preferably short-range communication technologies. The card device 20 can display visual information such as graphics and text. In the embodiment of the present application, the card device 20 is used to display the card information sent by the electronic device 10. Card devices 201 to 20 n When displaying information of different types of cards, a card set including multiple types of cards can be formed, and the types of cards may include but are not limited to at least one of role cards, skill cards, resource cards, event cards, task cards, and reward cards. In other words, by making the card device 20 display different card information, a card set consisting of specific cards can be formed.

[0038] See also Figure 2As shown, a card device 20 of a typical implementation may include an electronic ink screen 21, a wireless communication module 22 and a battery module 23. The electronic ink screen 21 has low power consumption. After the display content is refreshed, it will stay on the screen for a long time. Only when the screen changes, a small amount of power is consumed. Information can also be displayed in the off state, which makes it have strong endurance. In addition, the display content of the electronic ink screen 21 is still clearly visible under strong light, and will not reflect light like an LCD screen, which makes it difficult to see clearly. It is convenient to use in complex light environments such as homes, commercial board game venues, and educational training venues. In addition, the electronic ink screen 21 has basically the same display effect when viewing the screen from different angles, and there is no problem of limited viewing angle, which can meet the needs of multiple people watching at the same time in game interaction. The wireless communication module 22 may include short-distance wireless communication technologies such as Bluetooth, ZigBee, Wi-Fi, UWB (ultra-wideband), Z-Wave (Z-wave short-distance wireless communication technology), and may also include mobile cellular technologies such as 4G and 5G, preferably short-distance communication technologies with low power consumption. The battery module 23 may include a rechargeable battery, specifically a lithium-ion battery, etc. In some implementations, the card device 20 may further include a wireless charging coil to facilitate charging the battery module 23 .

[0039] In the embodiment of the present application, multiple types of card devices 20 can be set according to size and shape, and each type of card device 20 can display card information of one or more types of cards. The card device 20 can be placed in a card storage bin, and a card device 20 is selected from the card storage bin to display card information based on the needs of the electronic device 10 to form a specific card set. The specific electronic device 10 can select a card device 20 to display card information according to the game content to form a specific number of role cards, skill cards, resource cards, event cards, task cards, reward cards, etc. Further, when the card device 20 is located in the card storage bin, the card storage bin can charge the card device 20.

[0040] Please continue reading Figure 1 As shown, the electronic device 10 may include a game interaction implementation device, which includes a game generation module 101, a game initialization module 102, and a game control module 103. The game generation module 101 is used to receive game generation input, and call the multimodal big model based on the game generation input to generate game content through the multimodal big model. The game content includes game rules, card sets, board display information, and interactive audio. The card sets include multiple types of cards. Figure 1 As shown, the multimodal big model may include a local multimodal big model 40a and / or a multimodal big model 40b in the cloud.

[0041] In some implementations, the game generation input may include information of one or more dimensions. In a specific implementation, the game generation input may include at least one of the player name, the number of players, the game type, the story background, the main role of the game, and the main plot of the game. In some implementations, the types of cards in the card set include at least one of the role cards, skill cards, resource cards, event cards, task cards, and reward cards. In some implementations, the chessboard display information includes: at least one game element of the spatial layout, regional division, boundaries and restrictions, terrain, resources, obstacles, special areas, and events of the game. In some implementations, the game rules include interaction rules between characters and task achievement conditions.

[0042] In some embodiments, the game generation module 101 can also use the prompt word to call the multimodal large model, so as to convert the game content into a format that can be processed by the game control module 102 through the multimodal large model. In some embodiments, the game generation module 101 can be specifically used to: organize the game rules into a text file, store it in a specific format for display and / or voice broadcast; extract the information of the cards in the card set, and convert it into an instruction set that can be transmitted to the card device 20 for display, including the encoding format of the display content of each card device 20; convert the chessboard display information into coordinates and instructions for display, and the instructions may include image resource call instructions, etc.

[0043] The game initialization module 102 is used to initialize the display content and game status information of the card device 20 and the chessboard device 20 based on the game content, and display and / or voice broadcast the game rules. The game status information may include the game stage, the tasks completed by the player, the number and distribution of the player's resources, and the use history of the cards, etc. In a specific implementation, the game initialization module 102 can select the card device 20 to display the card information according to the game content to form a specific number of role cards, skill cards, resource cards, event cards, task cards, reward cards, etc. The game initialization module 102 can send instructions to the card device 20 and the chessboard device 20 to display the corresponding content on the card device 20 and the chessboard device 20.

[0044] The game control module 103 is used for controlling the game process. Specifically, during the game process, the game control module 103 can: identify the card content and game elements in the chessboard display area in real time and their positions in the chessboard display area to obtain a first recognition result; determine the game event based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; and control the game process based on the interaction result. In an embodiment of the present application, controlling the game process based on the interaction result may include: updating the game state information, playing the interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and at least one of the interactive audio includes at least one of music, sound effects and voice dialogue.

[0045] In some embodiments, the game control module 103 can also be used to: during the game, recognize the player's voice input in real time to obtain a second recognition result. The game control module 103 can also determine the game event based on the first recognition result, the second recognition result, the game state information and the game rules.

[0046] In some implementations, the game control module 103 may also be used to: recognize player voice input in real time; if the question intent is recognized, generate reply information from the game knowledge base or based on current game status information and feed it back to the player.

[0047] In some embodiments, Figure 3 As shown, the game interaction implementation device can be implemented as a computer program, and the electronic device 10 may include a processor and a memory for storing the program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes the game interaction implementation method.

[0048] The present application also provides a method for implementing game interaction. Figure 1 The system 100 shown is implemented as Figure 4 As shown, the game interaction implementation method includes the following steps.

[0049] Step S401, receiving game generation input, calling the multimodal big model based on the game generation input, so as to generate game content through the multimodal big model.

[0050] In the embodiment of the present application, Figure 1 In the game interaction implementation system 100 shown, the electronic device 10 can call the local multimodal large model 40a and / or the multimodal large model 40b in the cloud based on the game generation input to generate game content through the multimodal large model. In some implementations, the game generation input can include information of one or more dimensions. In a specific implementation, the game generation input can include at least one of the player name, the number of players, the game type, the story background, the main role of the game, and the main plot of the game.

[0051] In an embodiment of the present application, the game content includes game rules, a card set, board display information, and interactive audio. In a specific implementation, the interactive audio includes at least one of music, sound effects, and voice dialogue. The card set includes multiple types of cards. In some implementations, the types of cards in the card set include at least one of role cards, skill cards, resource cards, event cards, task cards, and reward cards. In some implementations, the board display information includes: at least one game element of the game's spatial layout, regional division, boundaries and restrictions, terrain, resources, obstacles, special areas, and events. In some implementations, the game rules include interaction rules between characters and task achievement conditions.

[0052] In some embodiments, the method further includes: using a prompt word to call the multimodal large model, so as to convert the game content into a format that can be processed by the game control module through the multimodal large model. Figure 1 In the game interaction implementation system 100 shown, the electronic device 10 can call the local multimodal big model 40a and / or the multimodal big model 40b in the cloud to convert the game content into a format that can be processed by the game control module through the multimodal big model.

[0053] In a typical implementation, firstly, various information input by players is collected and sorted, such as player name, number of players, game type, story background, main characters and plot, etc. This information will serve as the basis for the subsequent generation of game content. Data preprocessing operations are performed on the collected information. This includes cleaning text information, removing redundant characters, normalizing punctuation marks, etc.; then text encoding is performed to convert the text into a digital form that can be processed by the computer, such as using word vector representation to map each word to a low-dimensional vector space, so that words with similar semantics are close in the vector space. The preprocessed information is input into a multimodal large model in the cloud or locally. The multimodal large model is based on a neural network architecture, such as the Transformer architecture, and its core principle is the multi-head attention mechanism. The model has the ability to understand game-related semantics and logic by learning data such as rules, plots, and character settings of a large number of different types of games. After inputting player-customized information, the model performs reasoning and generation operations in its huge parameter space. For example, according to the story background and main characters set by the player, the model will refer to the learned similar story structure and role relationship to generate matching game rules, such as interaction rules between characters, task achievement conditions, etc.; at the same time, it will generate the corresponding card composition, including the number, attributes, functions, etc. of different types of cards, as well as the chessboard layout, such as the basis for regional division, the association between special terrain and story background, etc. Finally, the game content generated by the multimodal large model is post-processed. This step is mainly to convert the relatively abstract game information output by the model into a specific and operable format. For example, the generated game rules are organized into a clear text file and stored in a specific format for subsequent display and voice broadcast on the host device; the card composition information is extracted and converted into an instruction set that can be directly transmitted to the game card for display, including the encoding format of the electronic ink screen display content of each card; for the chessboard layout information, it is converted into coordinates that can be used to control projection or large-screen display, image resource call and other instructions.

[0054] The algorithm is described as follows: Let the player input information set be A = {a1, a2, ..., a k ...,a n}, where a k Represents a single input information element (such as the number of players a1, the game type a2, etc.). The data preprocessing process can be expressed as a function F, then the preprocessed information set A f =F(A). The multimodal large model is represented by the function M game , whose input is A f , the output is the game content set G = {g1,g2,...,g m}, where g lRepresents different aspects of the game content (such as rules g1, card set g2, board layout g3, etc.), that is, G = M game (A f The post-processing process is represented by function Q, and the final information set G that can be directly applied to the game is obtained after post-processing. m =Q(G).

[0055] Step S402, initializing the display content and game status information of the card device and the board device based on the game content, and displaying and / or voice broadcasting the game rules.

[0056] Game status information may include the game stage, the tasks completed by the player, the number and distribution of the player's resources, and the usage history of the cards. Figure 1 In the electronic device 10 shown, the game initialization module 102 can select the card device 20 to display card information according to the game content to form a specific number of role cards, skill cards, resource cards, event cards, task cards, reward cards, etc. The game initialization module 102 can send instructions to the card device 20 and the chessboard device 20 to display corresponding content on the card device 20 and the chessboard device 20.

[0057] In a specific implementation, multiple types of card devices 20 can be set according to size and shape, and each type of card device 20 can display information of one or more types of cards. The card device 20 can be placed in a card storage bin, and the electronic device 10 can select a card device 20 from the card storage bin to display card information based on the game content to form a specific card set. The electronic device 10 can select a card device 20 to display card information based on the game content to form a specific number of role cards, skill cards, resource cards, event cards, task cards, reward cards, etc.

[0058] Combination Figure 1 and Figure 2 As shown, after the electronic device 10 completes the game generation, it transmits the card display information to multiple card devices 20 through wireless communication technology (such as Bluetooth, Zigbee, etc.), and controls the electronic ink screen 21 of the card device 20 to display the initial card content, including the initial attributes of the character, the initial number of resources, etc. At the same time, the chessboard display information is transmitted to the chessboard device 30, so that the projection or large screen presents a complete initial game scene, such as the initial terrain, building distribution, etc. The main game rules are displayed and voice broadcast on the display interface of the electronic device 10 itself, so that players can clearly understand the basic gameplay and goals of the game.

[0059] Perform the following steps during gameplay:

[0060] Step S403, identifying the card content and game elements in the chessboard display area and their positions in the chessboard display area in real time to obtain a first recognition result.

[0061] In an embodiment of the present application, images of the chessboard display area can be captured by an image capture device such as a camera, and image recognition can be performed on the captured images to identify the card content and game elements in the chessboard display area and their positions in the chessboard display area.

[0062] Specifically, the camera continuously collects image data of the chessboard display area and obtains an image sequence at a fixed frame rate (for example, 30 frames per second). Each frame of the collected image is preprocessed, including image grayscale processing, converting the color image into a grayscale image, reducing the amount of data and highlighting the contour and texture information of the image; performing image filtering operations, such as Gaussian filtering, to remove noise interference in the image, making the image smoother and clearer, and facilitating subsequent feature extraction. The preprocessed image is analyzed using a target detection algorithm based on a convolutional neural network (CNN). CNN can be composed of multiple convolutional layers, pooling layers, and fully connected layers. The convolution layer extracts features by sliding convolution kernels on the image. For example, convolution kernels of different sizes and weights can extract features such as edges, textures, and shapes in the image. The pooling layer performs dimensionality reduction processing on the features extracted by the convolution layer. For example, the maximum pooling operation can retain the local maximum value in the feature map, reducing the amount of data while retaining the main feature information. The fully connected layer integrates and classifies the features after multiple convolutions and pooling, and finally outputs the target category identified in the image (such as various types of cards, game elements on the board, etc.) and the location information of the target in the image (usually expressed in the form of coordinates of the bounding box). For example, when a card is detected, the CNN model can determine whether it is a character card or a skill card, and determine its upper left and lower right coordinates in the image, thereby determining the relative position of the card on the board.

[0063] The recognition algorithm can be described as follows: Let the image captured by the image acquisition device at time t be I t , the image preprocessing process is represented by function P t , then the preprocessed image I p,t =P t (I t ). The object detection model based on convolutional neural network is represented as M visual , whose input is I p,t , the output is the recognition result set R t ={(c1,p1),(c2,p2),...,(c i ,p i )}, where c i represents the recognized target category, p iRepresents the location information of the target in the image (such as bounding box coordinates), that is, R t =M visual (I p,t ).

[0064] Step S404, performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result.

[0065] Step S405, controlling the game process based on the interaction result.

[0066] In an embodiment of the present application, controlling the game process based on the interactive results may include: updating game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and at least one of the interactive audio includes at least one of music, sound effects and voice dialogue.

[0067] In some embodiments, Figure 4 As shown, the method for implementing game interaction also includes: step S406, in the game process, real-time recognition of the player's voice input to obtain a second recognition result. In step S404, the game event is determined based on the first recognition result, the second recognition result, the game state information and the game rules. In some embodiments, the method for implementing game interaction also includes: step S407, if the question intention is recognized, a reply information is generated from the game knowledge base or based on the current game state information and fed back to the player.

[0068] In a specific implementation, the player's voice conversation can be collected through the microphone of the electronic device 10 or the chessboard device 30, and the voice can be converted into text information using voice recognition technology (such as calling a multimodal large model) for understanding and analysis. Specifically, the player conducts a real-time conversation during the game, uses natural language processing technology to understand the player's question intention, and generates accurate reply information from the game knowledge base or based on the current game status to feed back to the player. For example, if a player asks about the detailed effect of a skill or the goal of the current game task, a clear and accurate answer can be given.

[0069] exist Figure 1In the system 100 shown, the game control module 103 continuously receives the image recognition result (first recognition result) and the voice recognition result (second recognition result), and makes a comprehensive judgment in combination with the current game state information. The game state information includes the stage of the current game (such as the preparation stage, the battle stage, the plot advancement stage, etc.), the list of tasks completed by the player, the number and distribution of the player's resources, the use history of the card, etc. According to the card position and type information in the image recognition result, and the player instruction information in the voice recognition result, the game rules are matched and judged. For example, if the image recognizes that the player places an attack skill card in a specific area of ​​the chessboard, and the voice recognizes that the player says "launch an attack on the enemy player", the game control module 103 can judge whether the operation is legal according to the current game rules. If it is legal, the effect of the attack is determined according to the rules, such as the damage value caused to the enemy player's health value, whether to trigger the effects of other cards or chessboard elements, etc., and then generate interactive result information. The game control module 103 controls the subsequent process of the game according to the interactive result information, including sending display content update instructions to the card device 20 and the chessboard device 30, such as displaying information about the reduction of life value on the attacked player card, and displaying the change of attack special effects or triggering area on the chessboard device 30; at the same time, according to the interactive results, the corresponding music and sound effects are selected and played, such as playing combat sound effects when attacking, playing specific plot sound effects when triggering special events, etc. In addition, the game control module 103 can also process the real-time dialogue of the players. When receiving the consultation or information query request from the players, the natural language processing technology is used to understand the intention of the players, and accurate reply information is generated from the game knowledge base or according to the current game status to feedback to the players.

[0070] The algorithm can be described as follows: Let the current state information set of the game be S = {s1, s2, ..., s x ...}, the image recognition result set is R t ={(c1,p1),(c2,p2),...,(c i ,p i )...}, the speech recognition result text is W. The game rule set is represented by G r ={(r1,e1),(r2,e2),...,(r j ,e j )...}, where r j represents a single game rule, e j Indicates the effect or event corresponding to the rule. The game state judgment function is J, whose input is S, R and W, and the output is the judgment result D (legal or illegal) and the interaction result information set E = {(e i1 ,v i1 ),(e i2 ,v i2),...,(e iu ,v iu )...}, where e iu Indicates an interactive event, v iu Indicates the parameters corresponding to the event (such as damage value, resource change, etc.), that is, (D, E) = J(S, R, W). The function that controls the game process according to the interaction result is O, whose input is E, which is responsible for updating the display of game elements, playing audio, and processing player dialogues, that is, O(E).

[0071] One or more embodiments of the present application can generate highly personalized games: use a multimodal large model to generate a unique tabletop game based on the rich and diverse input information of the players, break the fixed pattern of traditional tabletop games, and meet the personalized needs of players; realize intelligent game monitoring and refereeing: through visual and voice recognition technology, realize comprehensive and real-time monitoring of the game process, accurately judge the game status and operation compliance, automatically play the role of game referee, and improve the fairness and smoothness of the game; create a dynamic game experience: dynamically update the game cards and chessboard display content based on game events during the game, and cooperate with matching music, sound effects, and real-time dialogue interaction between players and the game system to create a highly immersive, interactive and constantly changing game experience.

[0072] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to the exemplary embodiments of the present application.

[0073] The exemplary embodiments of the present application further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to enable the computer to perform the method according to the exemplary embodiments of the present application.

[0074] refer to Figure 5 , the structural block diagram of the electronic device that can be used as the electronic device 10 of the present application will now be described, which is an example of hardware devices that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0075] like Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0076] A plurality of components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information to the electronic device 500, and the input unit 506 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 507 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 may include but is not limited to a disk, an optical disk. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0077] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the game interaction implementation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 may be configured to execute by any other appropriate means (e.g., by means of firmware). Figure 4 The game interaction implementation method shown.

[0078] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0079] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0080] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0082] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0083] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0084] The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technical personnel in the field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present application. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An electronic device, characterized in that: include: A game generation module, configured to receive a game generation input, and call a multimodal macromodel based on the game generation input to generate game content through the multimodal macromodel, wherein the game content includes game rules, a card set, board display information, and interactive audio, and the card set includes multiple types of cards; A game initialization module, used to initialize the display content and game status information of the card device and the chessboard device based on the game content, and to display and / or voice broadcast the game rules; Game control module, used during the game: Recognize card content and game elements in the chessboard display area and their positions in the chessboard display area in real time to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

2. The device according to claim 1, characterized in that The game control module is also used to: during the game, recognize the player's voice input in real time to obtain a second recognition result; wherein the game control module performs the game event determination based on the first recognition result, the second recognition result, the game status information and the game rules.

3. The device according to claim 1, characterized in that The game generation module is further used to call the multimodal large model using a prompt word, so as to convert the game content into a format that can be processed by the game control module through the multimodal large model.

4. The device according to claim 3, characterized in that The game generation module is specifically used for: The game rules are organized into a text file and stored in a specific format for display and / or voice broadcast; Extracting information about cards in the card set and converting it into an instruction set that can be transmitted to the card device for display, including the encoding format of the display content of each card device; The chessboard display information is converted into coordinates and instructions for display, wherein the instructions include image resource calling instructions.

5. The device according to claim 1, characterized in that The game control module is also used for: Recognize player voice input in real time; If the question intent is recognized, response information is generated from the game knowledge base or based on the current game status information and fed back to the player.

6. The device according to claim 1, characterized in that The game generation input includes at least one of a player name, a number of players, a game type, a story background, a main character of the game, and a main plot of the game; and / or The types of cards in the card set include at least one of role cards, skill cards, resource cards, event cards, task cards, and reward cards; and / or The board display information includes: at least one game element selected from the game's spatial layout, area divisions, boundaries and restrictions, terrain, resources, obstacles, special areas, and events; and / or The game rules include interaction rules between characters and task completion conditions; and / or The game status information includes the game stage, the tasks completed by the player, the player's resource quantity and distribution, and the card usage history.

7. An electronic device, characterized in that: include: processor; as well as Memory for storing programs, The program includes instructions, and when the instructions are executed by the processor, the processor executes a method for implementing game interaction, including: receiving a game generation input, and calling a multimodal macromodel based on the game generation input to generate game content through the multimodal macromodel, wherein the game content includes game rules, a card set, board display information, and interactive audio, and the card set includes multiple types of cards; Initialize the display content and game status information of the card device and the chessboard device based on the game content, and display and / or voice broadcast the game rules; During the game: Recognize card content and game elements in the chessboard display area and their positions in the chessboard display area in real time to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

8. The electronic device according to claim 7, characterized in that: The method for implementing game interaction also includes: during the game, real-time recognition of player voice input to obtain a second recognition result; wherein the game event determination is performed based on the first recognition result, the second recognition result, the game status information and the game rules.

9. The electronic device according to claim 7, characterized in that: Before initializing the display content and game status information of the card device and the chessboard device based on the game content, it also includes: using prompt words to call the multimodal large model to convert the game content into a format that can be processed by the game control module through the multimodal large model.

10. A game interaction implementation system, characterized in that: include: The electronic device according to any one of claims 1 to 9; A plurality of card devices, the card devices are wirelessly connected to the electronic device and are used to receive and display card information; The chessboard device is used to display chessboard display information.

11. The system according to claim 10, characterized in that The card device comprises: An electronic ink screen, used to display the card information; A wireless communication module, used for wireless communication with the electronic device; A battery module is used to supply power to the electronic ink screen and the wireless communication module.

12. A method for realizing game interaction, characterized in that: include: receiving a game generation input, and calling a multimodal macromodel based on the game generation input to generate game content through the multimodal macromodel, wherein the game content includes game rules, a card set, board display information, and interactive audio, and the card set includes multiple types of cards; Initialize the display content and game status information of the card device and the chessboard device based on the game content, and display and / or voice broadcast the game rules; During the game: Recognize card content and game elements in the chessboard display area and their positions in the chessboard display area in real time to obtain a first recognition result; Performing game event determination based on the first recognition result, the game state information and the game rules to generate an event determination result and an interaction result; Controlling the game process based on the interactive result includes: updating the game status information, playing interactive audio, updating the display content of the card device, updating the display content of the chessboard device, and the interactive audio includes at least one of music, sound effects and voice dialogue.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause a computer to execute the method according to claim 12 .

Images