Intelligent game assisting method, system and equipment based on multi-modal understanding and medium

Abstract

The invention is applicable to the technical field of games, and provides an intelligent game assisting method based on multi-modal understanding, which comprises the following steps: acquiring a game state data stream and multi-modal input information of a player, analyzing the game state data stream to generate a game state signal, and analyzing the multi-modal input information to generate a player intention signal; constructing a scene dynamic graph based on the game state signal and the player intention signal; retrieving in a game knowledge base based on the scene dynamic graph to generate an enhanced knowledge set; and based on the scene dynamic graph and the enhanced knowledge set, generating a game guidance scheme, and marking an expected game state change after execution for key steps of the game guidance scheme, thereby realizing multi-modal input understanding to situational problem modeling and knowledge retrieval. And then, an end-to-end automatic process of generating an atomization operation instruction and an expected effect is provided, so that smooth intelligent auxiliary experience is provided.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of games, and particularly relates to an intelligent game assistance method, system, device and medium based on multi-modal understanding. BACKGROUND

[0002] With the rapid development of the game industry, the content of network games is becoming increasingly complex, such as various types of MMORPG games with different world views emerging in an endless stream. In order to help novice players master the rules and skills, the industry generally introduces various novice guidance methods for guidance, mainly including: preset static guidance tutorials in games, official FAQ documents, player community forums, and real-time mutual assistance between players. Such methods constitute the current basic framework of game assistance support, aiming to provide information reference for players.

[0003] However, the above-mentioned prior art solutions have a series of significant limitations. First, the content of static tutorials and FAQ documents is fixed and generalized, and cannot provide personalized guidance suggestions according to the real-time state, operation habits or specific card points of players; second, the forum retrieval and player assistance mode rely on the active help of players and the online response of others, and there is a risk of delayed response, uneven or even incorrect guidance, which cannot guarantee the immediacy and accuracy of problem solving; in addition, some games also try to introduce intelligent customer service systems based on keyword matching or fixed rule libraries, which can achieve automatic response, but their understanding of the deep semantics of player natural language questions and the understanding of complex game context are seriously insufficient. The above-mentioned intelligent customer service systems are often fragmented in function, lack coherent analysis of player historical behavior, long-term goals and current game situation, and are difficult to provide consistent personalized support.

[0004] In summary, the prior art cannot meet the urgent needs of players for real-time, accurate and personalized game guidance, and also brings continuous customer service cost pressure to game operators, therefore, there is an urgent need for an intelligent assistance solution that can deeply understand multi-modal input of players and combine with real-time game context to provide personalized guidance. SUMMARY

[0005] The embodiments of the present application provide an intelligent game assistance method, system, device and medium based on multi-modal understanding, which can solve one of the above-mentioned prior art problems.

[0006] In a first aspect, the embodiments of the present application provide an intelligent game assistance method based on multi-modal understanding, comprising: obtaining game state data stream of a player and multi-modal input information, and analyzing the game state data stream to generate game state signals and analyzing the multi-modal input information to generate player intention signals; constructing a scenario dynamic graph based on the game state signal and the player intention signal; performing retrieval in a game knowledge base based on the scenario dynamic graph to generate an enhanced knowledge set; generating a game guidance scheme based on the scenario dynamic graph and the enhanced knowledge set, and labeling an expected game state change after execution for key steps of the game guidance scheme.

[0007] Further, the parsing of the game state data stream to generate a game state signal comprises: acquiring a game state data stream in real time through a game application program interface or memory reading technology, and generating structured data; encoding the structured data to generate the game state signal, the game state signal being a real-time state graph with a player character as a core node and attributes, relationships, state effects as edges and attributes.

[0008] Further, the multi-modal input information comprises text information and image information; The parsing of the multi-modal input information to generate a player intention signal comprises: using a pre-trained language model to encode the text information to extract question entities and expected targets, forming an intention text vector; using a visual understanding model to process the image information to identify player visual focal points, interface abnormal prompts and environmental characteristics, forming an intention visual vector; fusing the intention text vector and the intention visual vector to generate a player intention signal.

[0009] Further, the constructing of a scenario dynamic graph based on the game state signal and the player intention signal comprises: comparing a player subjective question represented in the player intention signal with a game objective fact reflected in the game state signal to identify a core contradiction pair; based on the core contradiction pair, extracting associated entities, attributes and states from the game state signal as extension nodes, and constructing a scenario dynamic graph with preset causal relationships and conditional relationships as edges.

[0010] Further, the performing of retrieval in a game knowledge base based on the scenario dynamic graph to generate an enhanced knowledge set comprises: encoding the scenario dynamic graph through a graph neural network and converting it into descriptive text to generate a contextualized query vector; performing similarity retrieval in a game knowledge base based on the contextualized query vector to recall candidate knowledge items; analyzing each of the candidate knowledge items to extract specific preconditions required for executing the candidate knowledge items, and generating a precondition list; performing item-by-item logical comparison between the precondition list and the real-time game state, and checking whether the current game state satisfies all preconditions; comprehensively matching the context degree and the precondition satisfaction degree, sorting and screening each of the candidate knowledge items, and forming an enhanced knowledge set.

[0011] Further, based on the scenario dynamic graph and the enhanced knowledge set, a game guidance scheme is generated, and the expected game state changes after execution of key steps of the game guidance scheme are labeled, including: Based on the scenario dynamic graph and the enhanced knowledge set, a dynamic operation sequence composed of multiple atomized operation steps is generated by combining a large language model, wherein each atomized operation step corresponds to a player operation instruction that can be independently executed in the game; Based on the game operation logic and the causal relationship in the scenario dynamic graph, the atomized operation steps are logically sorted to form the game guidance scheme; determining key steps in the game guidance scheme, and analyzing the game effect generated after executing the key steps according to the game rules and the game knowledge base; mapping the game effect into specific data items in the game state signal, and combining the real-time game state to predict the expected game state after execution, and labeling the expected game state in a structured form to the corresponding key step.

[0012] Further, the method further includes: based on the real-time game state data stream, analyzing the time sequence characteristics of the player's behavior in real time; matching the time sequence characteristics with a predefined difficulty mode library, the difficulty mode library including multiple difficulty modes; when the matching is successful, based on the matched difficulty mode and the current game state signal, constructing an agent intention signal, and based on the game state signal and the agent intention signal, constructing a simplified scenario dynamic graph; based on the simplified scenario dynamic graph, searching in the game knowledge base to generate simple prompt information, and actively pushing to the player.

[0013] In a second aspect, the embodiments of the present application provide an intelligent game auxiliary system based on multi-modal understanding, including: a first processing module: configured to acquire game state data stream of a player and multi-modal input information, analyze the game state data stream to generate game state signal, and analyze the multi-modal input information to generate player intention signal; The second processing module is configured to construct a scenario dynamic graph based on the game state signal and the player intention signal. The third processing module is configured to search in a game knowledge base based on the scenario dynamic graph to generate an enhanced knowledge set. The fourth processing module is configured to generate a game guidance scheme based on the scenario dynamic graph and the enhanced knowledge set, and label an expected game state change after execution for a key step of the game guidance scheme.

[0014] In a third aspect, an embodiment of the present application provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the intelligent game assistance method based on multi-modal understanding when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which comprises a computer program stored in the computer readable storage medium, and the computer program is executable on a processor to implement the intelligent game assistance method based on multi-modal understanding.

[0016] Compared with the prior art, the embodiment of the present application has the following beneficial effects: The present application discloses an intelligent game assistance method based on multi-modal understanding, which realizes an end-to-end automatic process from multi-modal input understanding, to situational problem modeling and knowledge retrieval, to atomized operation instruction and expected effect generation. A player can directly obtain a specific and executable operation sequence and effect expectation from a fuzzy question or a difficult behavior, and the whole process is seamlessly connected to provide a smooth intelligent assistance experience. Specifically, the player's subjective "intention signal" and the game's objective "state signal" are decoupled and processed in parallel, and a "scenario dynamic graph" is constructed by comparing the core contradictions between the two. Thus, the core contradiction between the player's subjective cognition and the game's objective reality can be accurately identified. Further, the scenario dynamic graph is used to search in a game knowledge base, and the retrieved knowledge items are compared with the real-time game state signal to filter out all the currently unexecutable knowledge, so as to ensure that the generated guidance scheme is immediately executable for the player, greatly improving the practicability of the suggestion and the player experience. In addition, the game state signal flow is analyzed in real time to actively identify the player's potential difficult mode, and a lightweight and timely prompt is provided before the player asks for help, which can effectively reduce the player's frustration and significantly improve the friendliness of the game experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0018] Figure 1 is a flowchart of an intelligent game auxiliary method based on multi-modal understanding provided by an embodiment of the present application; Figure 2 is a structural schematic diagram of an intelligent game auxiliary system based on multi-modal understanding provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary details.

[0020] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0021] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0022] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to a determination" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [a described condition or event]" or "in response to detecting [a described condition or event]" depending on the context.

[0023] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0024] In the present application, the reference to "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0025] Please refer to Figure 1 As shown in the drawings, the present application is an intelligent game auxiliary method based on multi-modal understanding, comprising the following steps: S100, obtaining the game state data stream of the player and the multi-modal input information, and analyzing the game state data stream to generate a game state signal and analyzing the multi-modal input information to generate a player intention signal; In some embodiments, the analysis of the game state data stream to generate a game state signal comprises: Through the game application program interface or memory reading technology, the game state data stream of the player in the game is obtained in real time, and structured data is generated; The structured data is encoded to generate the game state signal, which is a real-time state graph with the player character as the core node, and the attributes, relationships, state effects as edges and attributes.

[0026] In this embodiment, the structured game state data stream is obtained in real time through the official application program interface (API) provided by the game client, the authorized memory data reading technology, or the secure data channel established with the game server. It can be understood that the obtained game state data stream is high-frequency, low-latency streaming data, which ensures the real-time nature of the state. At the same time, the game state data stream is multi-source and disordered, so it is necessary to extract structured data containing context from the game state data stream for subsequent generation of game guidance schemes.

[0027] The structured data extracted from the game state data stream includes, for example, an MMORPG game, a full state of a player character, an environment and entity state, a task and progress state, and the like. The full state of the player character includes, for example, a state of the player character in the game, such as life value, magic value, attack power, defense power, and the like, a current cooling state of all skills, a resource consumption state, a detailed list of all gains and losses of the player, a source, a remaining time, and a specific numerical value, an absolute coordinate of the player character, a direction, and a type, a level, an accurate blood volume, a position coordinate, and a key state effect on each entity of all hostile entities and non-hostile entities in a field of view, and a completion state of each task in the game, such as an accurate target description, a completion progress percentage of all active tasks, a current stage, a remaining time, and a triggered mechanism flag of a copy challenge.

[0028] Further, the structured data is cleaned, standardized, and uniformly encoded, and the data is organized into a dynamic and interconnected real-time state graph with the player character as the core. Specifically, in the real-time state graph, the player character is the central node, and each attribute of the player character, the equipment list, the backpack item list, and the skill list are attributes or child nodes of the central node. Each active gain or loss effect of the player character is connected to the central node as an effect edge with attributes. For example, in one embodiment, the player receives a 'broken armor' effect, which reduces physical defense by 40% for 1.5 seconds, and is encoded as an effect edge associated with the central node of the player character, with a timestamp and specific parameters. The remaining time and the real-time adjustment of the physical defense in the effect edge. In addition, other entities in the game environment, such as teammates, NPCs, monsters, and the like, are added to the real-time state graph as nodes to form connection edges with the spatial relationship or hostile relationship between the player character, thereby converting the structured game state data stream of the player in the game into a dynamic, structured, and contextually complete real-time state graph. The real-time state graph as a game state signal can be directly used for subsequent logical reasoning.

[0029] In some embodiments, the multi-modal input information includes text information and image information; The analyzing the multi-modal input information to generate a player intent signal includes: encoding the text information using a pre-trained language model to extract a question entity and an expected target, and forming an intent text vector; processing the image information using a visual understanding model to identify a player visual focus, an interface abnormal prompt, and an environment feature, and forming an intent visual vector; Fusing the intention text vector and the intention visual vector generates a player intention signal.

[0030] In this embodiment, the multi-modal input information is a subjective question in the game process of the player, and a visual platform is provided for the player to input a natural language question and input a game scene screenshot or short video, etc. for the question. Thus, the multi-modal input information includes text information and image information, and the subjective world of the player is deeply understood by analyzing the text information and the image information.

[0031] Specifically, a pre-trained language model based on a Transformer architecture is used for intention classification, for example, BERT is fine-tuned on a specific game corpus to distinguish whether the text information of the player is consultation, help-seeking or complaint, etc. Then, the game entities involved in the text information are extracted through named entity recognition. Further, the semantic relationship between entities is understood through dependency syntax analysis, and finally an intention text vector is output.

[0032] Further specifically, the image information is processed through a pre-trained visual understanding model. Specifically, the focus information that the player tries to convey through the image is identified, such as identifying the marking information of the player's circle, arrow pointing, etc. in the image information, and the abnormal state prompt of the game interface, such as the flashing warning icon, the gray unusable skill icon, the abnormal health / energy bar, etc. At the same time, the overall environmental characteristics shown in the image information also need to be identified, specifically the game scene where the player character is located in the game screenshot uploaded by the player, such as "team copy BOSS battle scene", "arena 1V1 duel", etc. Thus, the visual elements in the above image information are converted into symbolic state descriptions to generate an intention visual vector.

[0033] Specifically, in one embodiment, the player issues a question: "Why does my mage always die quickly in the dragon copy?" while uploading a screenshot of the current game screen, thereby, the intent recognition is performed on the text question to determine that the player belongs to the "help" intent, and then the game entities "mage" and "dragon copy" are identified in the text question, further, through the dependency syntax analysis, it can be determined that among the above game entities, "mage" is the agent and "dragon copy" is the patient, that is, "mage" is the subject of performing the action of "fighting", and "dragon copy" is the object of the action of "fighting", and "die quickly" is a description of the state of "mage" in the process of "fighting the dragon copy", finally, a structured intent text vector is output, which contains entity information and encodes the action, attribute and target relationship between entities, which is conducive to subsequent fusion analysis with image information. Further, for the game screen screenshot in the image information, the pre-trained visual understanding model is used to process the game screen screenshot uploaded by the player, first, the focus information that the player tries to convey through the image is identified. For example, it is detected whether there are annotations manually added by the player in the image, such as circles, arrows or highlight boxes, etc. These annotations often point to the parts that the player considers to be key or have problems, such as abnormal damage sources received by the character, abnormal positions of skill release, etc. The visual understanding model can accurately identify these annotation information, thereby focusing on the key points that the player pays attention to. At the same time, the image recognition model also identifies abnormal state prompts in the game interface, such as flashing warning icons that may indicate that the character is in danger or affected by special effects; gray skill icons indicate that the current character cannot use certain skills; abnormal health / energy bars directly reflect the current state of the player character, such as low health value that may cause the character to die quickly, thereby identifying these abnormal state prompts helps to understand the specific performance of the player character in the game when encountering problems. In addition, the visual understanding model also needs to identify the overall environmental features exhibited by the image information, that is, the game scene in which the player character is located, for example, by analyzing the scene elements and monster layout in the image, it is determined that it is a "team copy BOSS battle scene", in this scene, the player character may face the attack of multiple powerful monsters, and the battle difficulty is high. Understandably, the game scene is different, the game strategy and game difficulty are different, and the game scene is crucial to understanding the background of the player's problem and analyzing the reasons for the problem. Finally, the various visual elements identified by the visual understanding model, that is, the above player visual focus, interface abnormal prompt and overall environmental feature, are converted into symbolic state description, and finally an intent visual vector is generated.

[0034] Further, the above-mentioned intention text vector and intention visual vector are input into a multi-modal fusion module, which usually adopts a cross-modal attention mechanism, so that the text information can focus on the image area features related thereto, for example, the "skill" mentioned in the text information can be associated with the corresponding skill icon in the image, and at the same time the image information can also assist in understanding the reference of the text. Thus, through fusion, a unified player intention signal can be generated. Taking the above-mentioned player question as an example, the player intention signal can be structured as {“subject”:“player character-mage”,“target entity”:“dragon dungeon”,“perception problem”:“high frequency of death”,“emotion label”:“confusion”,“question intention”:“help”,“expected target”:“survive and defeat”}. It can be understood that, according to different multi-modal input information of the player, the generated player intention signal is different, and provides a clear subjective perspective for subsequent analysis.

[0035] S200, constructing a scenario dynamic graph based on the game state signal and the player intention signal; The application decouples and processes in parallel the player's subjective "intention signal" and the game's objective "state signal", and constructs a "scenario dynamic graph" by comparing the core contradictions between the two, so as to accurately identify the core contradictions between the player's subjective cognition and the game's objective reality.

[0036] In some embodiments, the above-mentioned step S200 includes: Comparing the player's subjective problem represented in the player intention signal with the game's objective fact reflected in the game state signal to identify a core contradiction pair; Based on the core contradiction pair, extracting the associated entities, attributes and states from the game state signal as extension nodes, and constructing a scenario dynamic graph with preset causal relationships and conditional relationships as edges.

[0037] In this embodiment, the difference between the player's subjective problem represented in the player intention signal and the game's objective fact reflected in the game state signal is converted into an understandable and inferable structure graph.

[0038] In this embodiment, the player intention signal generated in the receiving step S100 is received, and the player's target and perception problem in the player intention signal are extracted. Then, through a rule engine, an attribution analysis is performed on the game's objective fact reflected in the game state signal to determine the objective constraint factor. Thus, the objective constraint factor is compared with the player's subjective problem represented in the player intention signal to generate a core contradiction pair.

[0039] Specifically, it is determined that there is a conflict between the player's goal and the perceived problem, and then the real-time state graph constituted by the game state signals in the above step S100 is obtained, and attribution analysis is performed from various aspects such as character attributes, skill use, equipment and resource reserves, and environmental team cooperation to determine objective constraints. Taking the above player question as an example, the player's goal is "survive and defeat", and the perceived problem is "high frequency of death". Therefore, it is confirmed that there is a conflict between the player's goal and the objective state perceived by the player. The player expects to successfully survive and complete the instance challenge, but the actual game state shows that the character is constantly dying and cannot achieve the goal. Obviously, there is a contradiction between the goal and the state. Therefore, in the objective fact network constituted by the game state signals, that is, the real-time state graph, attribution analysis is performed along the preset causal relationship. For the problem of "high frequency of death", in the aspect of character attributes, the analysis can be performed along the life value number → defense value number → magic resistance value, such as checking whether the life value is too low, leading to easy death under the attack of the dragon; whether the defense and magic resistance are insufficient to effectively resist the physical and magical damage of the dragon, in the aspect of skill use, the analysis can be performed along the skill damage coefficient → skill cooling time → skill release timing → skill range and target selection, such as analyzing whether the skill damage is sufficient to cause effective attack on the dragon; whether the skill cooling time is too long, leading to the inability to use critical skills at critical moments; whether the skill release timing is appropriate, whether there is a situation of releasing the skill too early or too late; whether the skill range is appropriate, whether the dragon can be accurately hit, and whether the multiple attack modes of the dragon can be simultaneously coped with due to the skill range problem, in the aspect of equipment and resources, the analysis can be performed along the equipment quality and attribute addition → potion use → resource reserve, by checking the equipment quality and attributes to determine whether sufficient survival and output capabilities can be provided; and checking whether the potions are used in time to restore life value and magic value; whether the resource reserve is sufficient, and whether some critical skills or props cannot be used due to insufficient resources, in the aspect of environment and team, the analysis can be performed along the team cooperation situation (if there is a team) → monster (dragon) attack mode and intensity → spatial position relationship, to determine whether it is a team instance, check whether the cooperation between the team members is tacit, whether there is a situation of each fighting for himself; whether the attack mode and intensity of the dragon exceed the bearing range of the character; whether the spatial position relationship between the character and the dragon is reasonable, whether the character is frequently attacked due to the stationing problem, through the above attribution analysis, the determined objective constraints are "the skill A range is too small, when the dragon releases the attack skill, the character cannot dodge in time and cannot effectively counterattack, at the same time, due to the stationing problem, the character cannot obtain effective support from the team, leading to high frequency of death" Further, the objective constraint factor inferred after the above attribution analysis is compared with the subjective problem in the player intention signal, i.e., "high frequency of death", and in the comparison process, if the player does not mention the specific reason, the problem description is deepened, the core contradiction pair is generated with the above attribution analysis objective constraint factor, specifically, { "player subjective problem: high frequency of death", "objective constraint: survival ability", "contradiction attribution: skill A range is too small, when attacking skill in the dragon release range, it cannot be avoided in time and cannot effectively counterattack, at the same time, due to the positioning problem, it is difficult to obtain effective support from teammates, resulting in high frequency of death"}, thereby pointing out that the core reason for the above "high frequency of death" is "skill range and positioning"; and if the player attribution is wrong, the cognitive bias is identified, and the core contradiction pair is output, such as in the above example of high frequency of death, the player uploads the text information that "attack is not enough to quickly defeat the dragon to die", which has cognitive bias with the objective constraint factor determined by the attribution analysis, therefore, in the core contradiction pair, the cognitive bias also needs to be identified to guide the player to correct, and the corresponding core contradiction pair output is: { "player subjective problem: high frequency of death", "player attribution error: output ability", "objective constraint: survival ability", "contradiction attribution: skill A range is too small, when attacking skill in the dragon release range, it cannot be avoided in time and cannot effectively counterattack, at the same time, due to the positioning problem, it is difficult to obtain effective support from teammates, resulting in high frequency of death"}.

[0040] Further, based on the above core contradiction pair, all nodes and edges directly related to it are automatically extracted from the real-time state graph of the game state signal to construct a more focused situational dynamic graph. Specifically, the situational dynamic graph takes the above core contradiction pair as the center node, takes the entities and attributes in the core contradiction pair as the expansion nodes, takes the causal relationship, conditional relationship or inhibition relationship as the edge, and extracts attributes for each node to construct. For example, around the objective constraint of "survival ability", the nodes of "player character", "skill A", "dragon", "teammate character" are extracted, and the effect edges of "player character" and "gain effect" and "decrease effect" are connected, and the connection edge between "player character" representing "player character has skill A" and "skill A" is connected, and the node attribute of "skill A action range" is identified for skill A, and the "dragon" node and the "skill A action range" node are connected, indicating that at a certain moment, the dragon is outside the action range of skill A, so that the player character cannot use skill A to effectively counterattack or avoid attack, it can be understood that "skill A action range" node is a virtual range node representing the effective action area of skill A, and the connection edge representing the positional relationship between "teammate" and "player character" is connected, and so on, thereby constructing the situational dynamic graph.

[0041] Specifically, the construction of the scenario dynamic graph is based on a real-time state graph of game state signals. Entities and corresponding attribute values in the game state signals are filled into nodes such as "player-skill-enemy-environment" in the core contradiction pair and relationship edges such as "has-imposes-located-affects", thereby forming a core contradiction path, which is a dynamic subgraph of the real-time state graph and focuses on the player's problem.

[0042] S300, retrieving in a game knowledge base based on the scenario dynamic graph, generating an enhanced knowledge set; The application retrieves in a game knowledge base through a scenario dynamic graph, and compares the retrieved knowledge items with real-time game state signals in terms of preconditions, filters out all currently unexecutable knowledge, and ensures that the finally generated guidance scheme is immediately operable for the player, greatly improving the practicality of the suggestion and the player experience.

[0043] In some embodiments, the above step S300 includes: The scenario dynamic graph is encoded through a graph neural network and converted into descriptive text to generate a contextual query vector; Similarity retrieval is performed in a game knowledge base based on the contextual query vector to recall candidate knowledge items; Each of the candidate knowledge items is parsed to extract specific preconditions required to execute the candidate knowledge items, and a precondition list is generated; The precondition list is compared with real-time game states item by item in terms of logic to check whether the current game state meets all preconditions; The candidate knowledge items are sorted and screened based on the scenario matching degree and the precondition satisfaction degree to form an enhanced knowledge set.

[0044] In this embodiment, the scenario dynamic graph is encoded through a graph neural network and converted into specific text description to generate a contextual query vector. The contextual query vector not only contains the original question, but also contains the constraint conditions for the original question. The contextual query vector is used for similarity retrieval in a game knowledge base to obtain candidate knowledge items.

[0045] In this embodiment, the context dynamic graph is encoded by a graph neural network model to generate a first query vector, the nodes and relationships in the context dynamic graph are converted into descriptive text conforming to a preset template, and a language model is used to encode the descriptive text to generate a second query vector. The first query vector and the second query vector are fused to form a contextual query vector. Specifically, the context dynamic graph is input into a graph attention network, and after standardizing the attributes of each node and splicing them into vectors as node feature vectors of the corresponding nodes, the attributes of each edge in the context dynamic graph are standardized to obtain edge feature vectors. For edges without attributes, a vector of all 1s can be assigned as the default feature. Then, through a multi-head attention mechanism, each node aggregates the information of its neighbor nodes, and the attention weight is dynamically calculated, so that the nodes on the path associated more closely with the core contradiction pair can obtain higher weights. After multi-layer GAT encoding, global attention pooling or graph reading operation is used to aggregate the information of the entire graph into a fixed-length first query vector, which deeply encodes the structural features of the core contradiction pair. For the second query vector, a pre-defined question description template is used to fill in the key information in the context dynamic graph. For example, there is a template “In the [target entity] context, the current main constraint factor is [constraint factor], which leads to [specific problem]”. The above “high frequency death” embodiment can be instantiated as “In the dragon instance context, the current main constraint factor is that the skill A range is too small, which leads to high frequency death”. Then, the generated descriptive text is further input into a pre-trained language model to obtain its sentence vector representation as the second query vector, which deeply encodes the natural language semantics of the question. Thus, the first query vector and the second query vector are fused to generate a final contextual query vector, which simultaneously contains the structural logic and semantic connotation of the question, forming a comprehensive and accurate vectorization of the current predicament.

[0046] Further specifically, each knowledge item in the game knowledge base is a knowledge vector generated by a similar multi-modal encoding method as the above contextual query vector. Therefore, based on the above contextual query vector, an approximate nearest neighbor search can be performed in the vector index of the game knowledge base to preliminarily recall candidate knowledge items with a relevance higher than a first threshold.

[0047] Further, for each candidate knowledge item retrieved, based on its structured label, the specific preconditions required for executing the suggestion are parsed. For example, there are candidate knowledge items: "use 'range expansion rune' to enhance the range of skill A", "switch to 'position adjustment skill' to change the character's position to avoid dragon attacks", "learn 'defense posture' skill to reduce damage received", "buy'magic shield generator' to provide additional defense shield", "use 'life recovery potion' to quickly restore life value", etc. Each candidate knowledge item is set with the specific preconditions required for execution as its corresponding structured label. For example, in "use 'range expansion rune' to enhance the range of skill A", the preconditions are { "player has item": "range expansion rune", "item status": "unused", "skill A level >= 1"}. Specifically, the player must have the "range expansion rune" item, which has not been used, and skill A must be at least level 1, in order to use the rune to enhance the range of skill A to improve the survivability problem caused by insufficient skill range.

[0048] Thus, the preconditions of each candidate knowledge item are generated into a precondition list, which is compared with the real-time game state obtained in step S100 item by item, to check whether the current game state meets all the preconditions of the corresponding candidate knowledge item. Specifically, the comparison process is based on the preconditions of the corresponding candidate knowledge item, and checks in the game state signal one by one, such as checking the player's skill list, equipment bar, items and quantity in the backpack, checking whether the character's attribute value meets the requirements, checking whether the environment state allows, checking whether the specific game state flag meets the requirements, etc. The candidate knowledge item that passes the condition check is retained, and then the context matching degree of each retained candidate knowledge item is obtained, i.e. the matching similarity of the corresponding candidate knowledge item and the contextualized query vector, and sorted by the matching similarity to form an enhanced knowledge set. It can be understood that the precondition satisfaction degree of the candidate knowledge item is higher than the context matching degree. If there is a condition in the candidate knowledge item that does not meet the real-time game state, no matter how high the relevance of the knowledge item is, it will be filtered out.

[0049] S400, based on the scenario dynamic graph and the enhanced knowledge set, generating a game guidance scheme, and labeling the expected game state change after execution for the key steps of the game guidance scheme.

[0050] In some embodiments, the above step S400 includes: Based on the scenario dynamic graph and the enhanced knowledge set, a dynamic operation sequence composed of multiple atomized operation steps is generated in combination with a large language model, where each atomized operation step corresponds to a player operation instruction that can be independently executed in the game. Based on the game operation logic, the atomic operation steps are logically sorted in combination with the causal relationship in the situation dynamic graph, forming the game guidance scheme; The key steps in the game guidance scheme are determined, and the game effect generated after executing the key steps is analyzed according to the game rules and the game knowledge base; The game effect is mapped to a specific data item in the game state signal, and the expected game state after execution is predicted in combination with the real-time game state, and the expected game state is labeled to the corresponding key step in a structured form.

[0051] In this embodiment, the situation dynamic graph and the enhanced knowledge set are provided as inputs to a large language model that has been specially trained or prompted for engineering optimization. The action operation sequence is generated by the large language model. The prompt words corresponding to the large model are as follows: {“You are a game coach, guiding a player who has encountered a specific problem.”; “The core problem diagnosis is as follows: [Insert the structured description of the situation dynamic graph or the key contradiction pair here].”; “The known effective operations available for selection are as follows: [Insert the knowledge entries in the enhanced knowledge set here].”; “Please select and combine from the feasible operations to generate a direct, specific, and clear operation guide for the player based on the problem diagnosis. The guide should list the steps in logical order, and each step should be an atomic, immediately executable specific action.”} Therefore, the large model generates action operation instructions containing multiple steps based on the above prompt word input, for example: “1. Open the backpack (shortcut key B). 2. Find the “range expansion rune” in the backpack and right-click to use. 3. Close the backpack. 4. Observe the dragon's position and confirm that the dragon is within the observable attack range. 5. Check the “stance adjustment skill” status, and if it is not in the cooling state, press the corresponding skill shortcut key to switch to the skill. 6. Use the direction keys to adjust the character's stance according to the dragon's attack direction and range to avoid the dragon's attack.”. Each of the above steps is a player operation instruction that the player can independently execute in the game. The large language model will logically sort the steps according to the game operation logic and the causal relationship in the situation dynamic graph to form the game guidance scheme corresponding to the above dynamic operation sequence.

[0052] Further specifically, the key steps in the game guidance scheme are obtained, and then for each key step, the game effect that should be generated in the game after the key step is executed is inferred according to the game rules and the game knowledge base, wherein the key step is an operation that can possibly produce a significant change to the game state, such as using a skill, consuming a prop, interacting with a specific object, and the like, and as in the game guidance scheme described above, the game effect of step 2 is finding the “range expansion rune” in the backpack, specifically that the “range expansion rune” disappears from the backpack and enters a consumption state. The range of skill A is enhanced, and the relevant range value in the skill description changes. When the skill is actually released in the game, the skill influence area is expanded to cover more targets or a larger range. Further, each game effect is mapped to a specific data item in the game state signal, a predefined effect-state mapping table is used for mapping, and the expected value after the operation is performed is calculated according to the real-time value of the specific data item in the current game state signal, and finally the expected game state after the execution of each key step is generated. Taking the game effect of step 2 in the game guidance scheme generated above as an example, the corresponding game effect mapping result is specifically “game state signal-backpack item list-range expansion rune-delete” and “game state signal-skill list-skill A corresponding-range value-increase”, and for the “range” value, the enhancement effect of the “range expansion rune” is increased, assuming that the “range expansion rune” can expand the range of skill A by 20%, if the original skill A range value is 10 (unit: game distance unit), then after expansion, it becomes 10 x (1+20%) = 12. Then, formalized expected annotations are added to the key steps, such as “expected change: after execution, the range value of skill A should be increased from

[10] to about

[12] ”. Through these annotations, the in-game feedback that should be directly observed after the operation is successful is clearly defined, which is beneficial to the player to clearly understand the game effect after executing the game guidance scheme.

[0053] In some embodiments, the above-mentioned intelligent game assistance method based on multi-modal understanding further comprises: Based on the real-time game state data stream, the time sequence characteristics of the player's behavior are analyzed in real time; The time sequence characteristics are matched with a pre-defined difficulty pattern library, and the difficulty pattern library includes a plurality of difficulty patterns; When the matching is successful, an agent intention signal is constructed based on the matched difficulty pattern and the current game state signal, and a simplified situation dynamic graph is constructed based on the game state signal and the agent intention signal; Based on the simplified situation dynamic graph, retrieval is performed in the game knowledge base to generate simple prompt information, and the player is actively pushed.

[0054] In the embodiment, the game state data stream of the player is monitored in real time during the game process of the player, and the possible difficulties encountered by the player are actively predicted, and light guidance is provided in a non-intrusive or low-intrusive manner at an appropriate time, so that intervention is performed before the player feels frustrated, and the smoothness and friendliness of the game experience are significantly improved. Specifically, the intelligent game assistance method of the present application no longer passively waits for the player to ask for help, that is, simple prompt information can be generated and pushed to the player without the player inputting multi-modal input information, so as to improve the game experience of the player In the embodiment, an extensible behavior pattern library is preset, and each "difficulty pattern" in the behavior pattern library is a set of detectable rules or feature combinations based on the time sequence of the game state data stream. For example, a "continuous death pattern": within the coordinate range of a specific scene (such as the same instance entrance or the same BOSS battle area), a "death" state event (player character. health == 0 event from the game state signal) of the character occurs continuously within a short time (such as 3 minutes) more than a threshold number of times (such as 3 times), and the associated death causes are the same. A "task stagnation pattern": for a certain active task, the task progress percentage does not change for a long time (such as 15 minutes) after far exceeding the expected completion time, and the game behavior entropy of the player (such as the randomness of the movement path and the dispersion of the attack target) is high, indicating that the player may be wandering aimlessly. A "resource abnormal depletion pattern": the backpack holding quantity of a specific key resource (such as high-level healing potion or special ammunition) shows a monotonous rapid downward trend within a short time, and the downward speed deviates significantly from the normal consumption model of the resource in the current scene, accompanied by a sharp fluctuation in the health of the player character or a stagnation of the task progress. A "inefficient behavior cycle pattern": a repeated invalid cycle of the player's operation sequence is detected within a short time, for example, repeatedly using a specific damage type skill on an enemy immune to the type of skill (derived from the skill usage sequence and the target entity damage immunity state signal). It can be understood that each difficulty pattern in the behavior pattern library is calculated and labeled with a pattern feature vector corresponding to the difficulty pattern, which is used to calculate the matching degree with the time sequence features of the real-time game state data stream of the player to determine whether the player triggers the corresponding difficulty pattern.

[0055] Further, the game state data stream of the player is analyzed in real time to obtain time sequence features, which are matched with the behavior features of each difficulty pattern in the behavior pattern library to identify a signal pattern that the player may have fallen into a difficult situation or is about to fall into a difficult situation. Specifically, the game state data stream is windowed and calculated, and the features required by each difficulty pattern in the difficulty pattern library are extracted in real time, such as event count, trend slope, and sequence repeatability, to calculate the corresponding time sequence features. When the time sequence features and the pattern feature vector of any difficulty pattern in the behavior pattern library reach a matching threshold, it is determined that the difficulty pattern is activated.

[0056] Further specifically, when the difficulty mode is identified, based on the difficulty mode and the current game state signal, the agent intention signal is constructed, for example, for the "continuous death mode", the agent intention signal can be constructed as { "perception problem": "repeated death in [coordinate range R]", "suspected cause": "[the cause of the last death]"}, and the coordinate range and the last death cause in the agent intention signal are specific signals obtained through the game state signal.

[0057] Further, a game state signal snapshot at the moment when the difficulty mode is activated is captured, and based on this, combined with the agent intention signal, a simplified situation dynamic graph focusing on the difficulty mode is constructed, which only contains core nodes directly related to the difficulty mode, such as fatal skills, player characters, etc., and the simplified situation dynamic graph is constructed in a similar way to the situation dynamic graph in step S200.

[0058] Further, based on the simplified situation dynamic graph, retrieval is performed in the game knowledge base. It is worth noting that in this step, the retrieval process of the simplified situation dynamic graph in the game knowledge base is similar to the retrieval process in step S300, both of which are to encode and convert the simplified situation dynamic graph into a descriptive text, generate a situational query vector, recall candidate knowledge items, determine the final knowledge item, and finally output the knowledge item combined with the agent intention signal as a natural language prompt that is concise, direct, and centered on reminders or key operations. For example: "It is detected that you repeatedly die in [coordinate range R] due to [suspected cause], you can try [knowledge item]". As can be understood, [coordinate range R], [suspected cause], and [knowledge item] in the above natural language prompt are filled in through the agent intention signal and the retrieval result in the game knowledge base, and finally generate a simple prompt information such as "It is detected that you repeatedly die in [BOSS_A combat area] due to [BOSS_A skill A], you can try [purchase'magic shield generator' to provide additional defense shield]" and actively push to the player.

[0059] It is worth noting that in the pushing process, the same difficulty mode is not triggered repeatedly in a short period of time; when the player is in a tense combat state, the prompt is delayed or in a more simplified UI form (such as icon flashing); when a certain mode is triggered for the first time, a more detailed prompt is provided, and only a keyword reminder can be provided subsequently to avoid information bombardment. In addition, if the guidance triggered by a certain difficulty mode is repeatedly verified to be effective (the player's behavior is improved accordingly), the trigger threshold of the difficulty mode can be appropriately reduced to make it more sensitive; on the contrary, if the guidance is often ignored or ineffective, the threshold is increased to reduce ineffective disturbance.

[0060] Thus, the application realizes an end-to-end automated process from multi-modal input understanding, to situational problem modeling and knowledge retrieval, to atomized operation instructions and expected effect generation. From a vague question or a difficult behavior, the player can directly obtain a specific and executable operation sequence and effect expectation. The whole process is seamlessly connected, providing a smooth intelligent assistance experience. At the same time, through real-time analysis of the game state signal stream, the player's potential difficult mode is actively identified, and light and timely prompts are provided before the player asks for help, which can effectively reduce the player's frustration and significantly improve the friendliness of the game experience.

[0061] Referring to Figure 2 The application also provides an intelligent game assistance system based on multi-modal understanding, which comprises: A first processing module 201 is configured to acquire game state data stream of a player and multi-modal input information, analyze the game state data stream to generate game state signals, and analyze the multi-modal input information to generate player intention signals; A second processing module 202 is configured to construct a situational dynamic graph based on the game state signals and the player intention signals; A third processing module 203 is configured to retrieve in a game knowledge base based on the situational dynamic graph to generate an enhanced knowledge set; A fourth processing module 204 is configured to generate a game guidance scheme based on the situational dynamic graph and the enhanced knowledge set, and label expected game state changes after execution for key steps of the game guidance scheme.

[0062] It can be understood that the contents in the intelligent game assistance method embodiment based on multi-modal understanding as Figure 1 shown are all applicable to the intelligent game assistance system embodiment based on multi-modal understanding. The intelligent game assistance system embodiment based on multi-modal understanding specifically realizes the same functions as the intelligent game assistance method embodiment based on multi-modal understanding as Figure 1 shown, and achieves the same beneficial effects as the intelligent game assistance method embodiment based on multi-modal understanding as Figure 1 shown.

[0063] It should be noted that the information interaction and execution process between the above systems, because of the same concept as the method embodiment, the specific functions and the technical effects brought about can be referred to the method embodiment part, and will not be repeated here.

[0064] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or software. In addition, the specific names of the functional units and modules are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0065] Please refer to Figure 3 The embodiment of the present application also provides a computer device 3, comprising a memory 302 and a processor 301 and a computer program 303 stored in the memory 302, when the computer program 303 is executed on the processor 301, the intelligent game assisting method based on multi-modal understanding is realized as any one of the above methods.

[0066] The computer device 3 can be a desktop computer, a notebook computer, a palm computer and a cloud server and the like. The computer device 3 can include, but is not limited to, a processor 301, a memory 302. Those skilled in the art can understand that, Figure 3 The computer device 3 is only an example and does not constitute a limitation on the computer device 3, and can include more or fewer components than shown, or combine certain components, or different components, for example, it can also include input and output devices, network access devices and the like.

[0067] The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0068] The memory 302 may, in some embodiments, be an internal storage unit of the computer device 3, such as a hard disk or a memory of the computer device 3. The memory 302 may, in other embodiments, also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, and the like equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store an operating system, an application program, a Boot Loader, data, and other programs, such as program codes of the computer program, and the like. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0069] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the intelligent game assisting method based on multi-modal understanding is realized.

[0070] In the embodiment, the integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of the above-mentioned method embodiments when executed by a processor. The computer program includes computer program codes, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program codes to the photographing device / computer device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk, and the like. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunications signal.

[0071] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An intelligent game assistance method based on multi-modal understanding, characterized in that, The method comprises the following steps: acquiring a game state data stream of a player and multi-modal input information, and analyzing the game state data stream to generate a game state signal and analyzing the multi-modal input information to generate a player intention signal; constructing a scenario dynamic graph based on the game state signal and the player intention signal; performing retrieval in a game knowledge base based on the scenario dynamic graph to generate an enhanced knowledge set; generating a game guidance scheme based on the scenario dynamic graph and the enhanced knowledge set, and labeling an expected game state change after execution of a key step of the game guidance scheme.

2. The method of claim 1, wherein, The analysis of the game state data stream to generate a game state signal comprises the following steps: acquiring a game state data stream of a player in real time through a game application program interface or memory reading technology, and generating structured data; encoding the structured data to generate the game state signal, which is a real-time state graph with a player character as a core node and attributes, relationships, state effects as edges and attributes.

3. The method of claim 1, wherein, The multi-modal input information comprises text information and image information. The analysis of the multi-modal input information to generate a player intention signal comprises the following steps: encoding the text information using a pre-trained language model to extract question entities and expected targets, forming an intention text vector; processing the image information using a visual understanding model to identify player visual focus, interface abnormal prompts and environmental characteristics, forming an intention visual vector; fusing the intention text vector and the intention visual vector to generate a player intention signal.

4. The method of claim 1, wherein, The construction of a scenario dynamic graph based on the game state signal and the player intention signal comprises the following steps: comparing a player subjective question represented in the player intention signal with a game objective fact reflected in the game state signal to identify a core contradiction pair; extracting associated entities, attributes and states from the game state signal as extension nodes based on the core contradiction pair, and constructing a scenario dynamic graph with preset causal relationships and conditional relationships as edges.

5. The method of claim 1, wherein, The retrieval in a game knowledge base based on the scenario dynamic graph to generate an enhanced knowledge set comprises the following steps: encoding the scenario dynamic graph through a graph neural network and converting it into a descriptive text to generate a contextualized query vector; performing similarity retrieval in the game knowledge base based on the contextualized query vector to recall candidate knowledge items; analyzing each of the candidate knowledge items to extract specific prerequisites for executing the candidate knowledge items, and generating a prerequisite list; performing item-by-item logical comparison between the prerequisite list and the real-time game state to check whether the current game state meets all the prerequisites; sorting and filtering each of the candidate knowledge items based on the contextual matching degree and the prerequisite satisfaction degree to form an enhanced knowledge set.

6. The method of claim 1, wherein, The generation of a game guidance scheme based on the scenario dynamic graph and the enhanced knowledge set, and the labeling of an expected game state change after execution of a key step of the game guidance scheme comprises the following steps: Based on the situation dynamic graph and the enhanced knowledge set, a dynamic operation sequence composed of a plurality of atomized operation steps is generated in combination with a large language model, wherein each atomized operation step corresponds to a player operation instruction that can be independently executed in the game; Based on the game operation logic and the causal relationship in the situation dynamic graph, the atomized operation steps are logically sorted to form the game guidance scheme; Determine the key steps in the game guidance scheme, and analyze the game effect generated after executing the key steps according to the game rules and the game knowledge base; Map the game effect to specific data items in the game state signal, and combine the real-time game state to predict the expected game state after execution, and mark the expected game state to the corresponding key step in a structured form.

7. The method of claim 1, wherein, The method further comprises: Based on the real-time game state data stream, the time sequence characteristics of the player's behavior are analyzed in real time; Match the time sequence characteristics with a pre-defined difficulty pattern library, which includes a plurality of difficulty patterns; When a match is successful, an agent intent signal is constructed based on the matched difficulty pattern and the current game state signal, and a simplified situation dynamic graph is constructed based on the game state signal and the agent intent signal; Based on the simplified situation dynamic graph, search in the game knowledge base to generate simple prompt information, and actively push to the player.

8. An intelligent game assistance system based on multi-modal understanding, characterized in that, It includes: The first processing module is used for obtaining the game state data stream of the player and the multi-modal input information, and analyzing the game state data stream to generate the game state signal and analyzing the multi-modal input information to generate the player intent signal; The second processing module is used for constructing a situation dynamic graph based on the game state signal and the player intent signal; The third processing module is used for searching in the game knowledge base based on the situation dynamic graph to generate an enhanced knowledge set; The fourth processing module is used for generating a game guidance scheme based on the situation dynamic graph and the enhanced knowledge set, and marking the expected game state change after execution for the key steps of the game guidance scheme.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to realize the method of any one of claims 1 to 7.

Images