Information interaction method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses an information interaction method and device, electronic equipment and a readable storage medium. According to the embodiment of the invention, the game picture data of the user is obtained in the game process of the user, and the key information in the game picture data is determined, so that the strategy distribution and the reward function of the object controlled by the user in the current state and the state value function of the object controlled by the user in the next state are determined according to the key information in the game picture data; and determining a target action of the object controlled by the user in the current state according to the strategy distribution of the object controlled by the user, the state value function and the reward function, thereby interacting with the user according to the target action. In the embodiment of the invention, the game state of the user can be recognized in real time according to the game picture data of the user, and then the action of the operation object is accurately predicted and interacts with the user, so that the timeliness and accuracy of game assistance can be improved, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to an information interaction method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] With the continuous development of computer technology, artificial intelligence technology has been widely applied in many fields. The common application of artificial intelligence technology in the field of human-computer interaction includes a game assistance system, which is an artificial intelligence system that provides operation suggestions for users during the game process. However, the game assistance provided by the existing game assistance system usually lacks timeliness and accuracy, so the user's experience is usually not good. SUMMARY

[0003] Therefore, the embodiments of the present application provide an information interaction method and device, electronic equipment and readable storage medium to identify the game state of the user in real time according to the game picture data of the user, and interact with the user according to the action of the predicted operation object, so as to improve the timeliness and accuracy of game assistance and improve the user's experience.

[0004] In a first aspect, the embodiments of the present application provide an information interaction method, which comprises:

[0005] Obtaining game picture data of a target user;

[0006] Determining key information in the game picture data, the key information including at least one of state data of a control object, task state of a current task, task target of the current task, and environment data of an environment where the control object is located;

[0007] According to the key information, determining a policy distribution of the control object in the current state, a reward function, and a state value function of the next state;

[0008] According to the policy distribution, the state value function and the reward function, determining a target action of the control object in the current state;

[0009] According to the target action, interacting with the target user.

[0010] Optionally, the determination of the key information in the game picture data comprises:

[0011] According to the game picture data, determining the key information based on a pre-trained multi-modal model, the multi-modal model being obtained by training based on a first sample set, the first sample set including historical game picture data and historical key information of multiple users.

[0012] Optionally, the determining the policy distribution, the reward function and the state value function of the next state of the control object in the current state according to the key information comprises:

[0013] According to the key information, the policy distribution, the state value function and the reward function are determined based on a pre-trained data prediction model, the data prediction model being obtained by training based on a second sample set, the second sample set comprising historical key information, historical policy distribution, historical reward function and historical state value function of a next state of a historical state of a historical control object of a plurality of users in the historical state.

[0014] Optionally, the determining the target action of the control object in the current state according to the policy distribution, the state value function and the reward function comprises:

[0015] According to the preset search algorithm, the priority of each executable action is determined according to the policy distribution, the state value function and the reward function.

[0016] The target action is determined according to each priority.

[0017] Optionally, the interacting with the target user according to the target action comprises:

[0018] Detecting actual behavior information of the control object, the actual behavior information comprising at least one of an actual action and an actual task target of the control object.

[0019] In response to the actual behavior information being inconsistent with predicted behavior information of the control object, an abnormal prompt is output to the target user, the predicted behavior information comprising at least one of the target action and the task target.

[0020] Optionally, the determining the policy distribution, the reward function and the state value function of the next state of the control object in the current state according to the key information comprises:

[0021] Determining a plurality of sub-tasks corresponding to the current task;

[0022] Determining a sub-task policy distribution, a sub-task state value function and a sub-reward function corresponding to each sub-task.

[0023] In a second aspect, an embodiment of the present application provides an information interaction device, the device comprising:

[0024] A data acquisition unit is configured to acquire game screen data of a target user.

[0025] information determining unit, configured to determine key information in the game picture data, the key information comprising at least one of state data of a controlled object, a task state of a current task, a task target of the current task, and environment data of an environment in which the controlled object is located;

[0026] function determining unit, configured to determine, according to the key information, a policy distribution of the controlled object in a current state, a reward function, and a state value function in a next state;

[0027] action determining unit, configured to determine, according to the policy distribution, the state value function, and the reward function, a target action of the controlled object in the current state;

[0028] interaction unit, configured to interact with the target user according to the target action.

[0029] In a third aspect, an electronic device is provided, including a memory and a processor, the memory being configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of the first aspect.

[0030] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method according to any one of the first aspect.

[0031] In a fifth aspect, a computer program product is provided, the computer program product including computer program / instructions, the computer program / instructions being executed by a processor to implement the method according to any one of the first aspect.

[0032] In the embodiments of the present application, game picture data of a user is acquired in a game process of the user, key information in the game picture data is determined, then a policy distribution of an object controlled by the user in a current state, a reward function, and a state value function in a next state are determined according to the key information in the game picture data, then a target action of the object controlled by the user in the current state is determined according to the policy distribution of the object controlled by the user, the state value function, and the reward function, and interaction is performed with the user according to the target action. In the embodiments of the present application, the game state of the user can be recognized in real time according to the game picture data of the user, then the action of the operation object can be accurately predicted and interaction is performed with the user, so that the timeliness and accuracy of game assistance can be improved, and the use experience of the user can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0034] Figure 1 is a flowchart of an information interaction method according to an embodiment of the present application;

[0035] Figure 2 is a flowchart of an information interaction method according to an embodiment of the present application;

[0036] Figure 3 is a flowchart of an information interaction method according to an embodiment of the present application;

[0037] Figure 4 is a schematic diagram of an information interaction device according to an embodiment of the present application;

[0038] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The present application is described below based on embodiments, but the present application is not limited to only these embodiments. In the following detailed description of the present application, some specific details are described in detail. The present application can also be fully understood without the description of these details by those skilled in the art. In order to avoid confusion of the essence of the present application, well-known methods, processes, flows, elements and circuits are not described in detail.

[0040] In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0041] Unless the context clearly requires otherwise, throughout the description and the claims of this application, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to".

[0042] In the description of the present application, it should be understood that the terms "first", "second", and the like are used only for the purpose of description and should not be construed as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise stated, the meaning of "multiple" is two or more.

[0043] The solutions described in the specification and embodiments, if related to personal information processing, will be processed on the premise of legality (for example, obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and only within the prescribed or agreed range. Users refuse to process personal information other than the necessary information required for basic functions, which will not affect the user's use of basic functions.

[0044] The game auxiliary system can provide operation suggestions for the user during the user's game process, can effectively reduce the game difficulty of the user, and thus can improve the user's use experience. Existing game auxiliary systems generally include three categories. One type of game auxiliary system obtains the game state of the user by reading key data in the game memory or network communication, and modifies the game data to achieve game assistance, that is, game hacking. This type of game auxiliary system is usually illegal, and if the game data is modified too much, the account may be permanently banned in severe cases. One type of game auxiliary system can obtain the game state of the user through the interface provided by the game official, but this type of game auxiliary system usually does not have the ability of situational understanding and strategy suggestion, so it is difficult to intelligently interact with the user and real-time correct based on the user's behavior. Therefore, the assistance capability provided for the user is usually limited. One type of game auxiliary system identifies the game state of the user through a visual model, but this type of game auxiliary system usually cannot provide real-time assistance for the user, nor can it adjust the operation suggestions according to the user's real-time operation.

[0045] To solve the above problems, an information interaction method, device, electronic equipment and readable storage medium are provided in embodiments of the present application to identify the game state of the user in real time according to the game picture data of the user, and interact with the user according to the predicted motion of the operation object, thereby improving the timeliness and accuracy of game assistance and improving the user's use experience.

[0046] It is easy to understand that in the present embodiment, the execution subject of the information interaction method can be a game auxiliary system. Specifically, the game auxiliary system can implement the information interaction method in the present embodiment by calling and executing corresponding program codes. The game auxiliary system can be installed in a corresponding terminal device such as a smart phone, a tablet computer, a smart wearable device, a notebook computer, a desktop computer or a smart speaker. Alternatively, in some embodiments, the game auxiliary system can also be installed in a corresponding data processing terminal of a machine device such as a vehicle, a robot or an aircraft, and the present application does not limit this.

[0047] In the present embodiment, the interaction between the game auxiliary system and the user can be initiated by the user. Specifically, the user can initiate a dialogue with the game auxiliary system. After the user initiates the dialogue, the game auxiliary system can give the user a corresponding reply to the user's real-time game picture.

[0048] Alternatively, the voice interaction between the game auxiliary system and the user can also be initiated by the game auxiliary system. Specifically, the game auxiliary system can actively provide the user with interaction content matching the current satisfied preset interaction condition when detecting that any preset interaction condition is satisfied.

[0049] In an alternative implementation, the preset interaction condition can be detecting that the user makes a corresponding operation, and when the preset interaction condition is met, the interaction content provided by the game assistance system can be content related to the operation currently made by the user. For example, when it is detected that the user invokes an application, the game assistance system can respond to the application invocation operation currently made by the user.

[0050] It should be understood that the above given preset interaction conditions and the interaction content provided by the game assistance system when the preset interaction conditions are met are merely illustrative. In actual application, they can be pre-set by a program developer or a user.

[0051] Further, in order not to affect the normal work and study of the user, in the embodiment, before providing the active interaction content to the user, the game assistance system can also detect whether the user is currently in a busy state, and if it is confirmed that the user is in the busy state, the game assistance system can temporarily stop active interaction, thereby avoiding disturbing the user.

[0052] Alternatively, the game assistance system can determine by detecting the number of operations of the terminal device or the machine device by the user within a current preset time period. For example, the game assistance system can confirm that the user is currently in the busy state when it is detected that the number of operations of the terminal device or the machine device by the user within the current preset time period is less than or equal to a preset number threshold, and correspondingly, the game assistance system can confirm that the user is not currently in the busy state when it is detected that the number of operations of the terminal device or the machine device by the user within the current preset time period is greater than the preset number threshold.

[0053] The following is described by means of a method embodiment. Figure 1 is a flowchart of the information interaction method of the embodiment of the application. As shown in Figure 1 the method of the embodiment includes the following steps:

[0054] Step S100, game screen data of a target user is acquired.

[0055] After the target user starts the game assistance system, the target user can perform human-computer interaction with the game assistance system. Therefore, in this step, the game assistance system can acquire game screen data of the target user in real time through a terminal device or an image acquisition device such as a camera after authorization by the target user.

[0056] Step S200, key information in the game screen data is determined.

[0057] In the embodiments of the present application, the key information refers to visual elements that play a key role in user operation and decision-making, and can include at least one of a control object, state data of the control object, a task state of a current task of the control object, a task target of the current task, and environment data of an environment in which the control object is located.

[0058] The state data of the control object can include the number of control objects, position data, action data, posture data, life value, score, etc.; the task state of the current task can include not started, in progress, completed, etc.; and the environment data can be the topography, building distribution, weather, etc. of the environment in which the object is located. The task state of the current task can further include the task progress of the current task. For example, if the current task is house building, and a control object needs to pick up a preset number of building materials, the task state of the current task can include the pickup progress of the building materials.

[0059] In order to determine the key information from a large amount of information contained in the game picture data, in this step, the game auxiliary system can identify the key information in the game picture data based on the game picture data of the target user obtained and the pre-trained image processing model. Optionally, the embodiments can process the game picture data based on one or more image processing models, thereby improving the recognition ability of the game picture data.

[0060] In an optional implementation, the image processing model can be a multi-modal model, which can be a visual language model (VLM) for realizing visual question answering (VQA), such as a ViT (Vision Transformer)-transformer model, a M6 (Multi-modal MegaModel), a ViLBERT (Vision-and-Language Bidirectional Encoder Representations from Transformers), etc.; or a multi-modal model based on contrastive learning, such as a CLIP (Contrastive Language-Image Pre-training), an ALBEF (Align Before Fuse), a FILIP (Fusion-in-Transformer for Image-Text Pre-training), etc.

[0061] The multi-modal model can be obtained by training according to a first sample set. The first sample set includes a plurality of first samples, and each first sample includes historical game screen data and historical key information of a corresponding user. When the first sample set is used to train the visual language model, the corresponding question text and answer text can be determined according to each historical key information. The question text of the historical key information is used to instruct the visual language model to identify the corresponding historical key information, and the answer text is used to instruct the visual language model to generate the corresponding answer information according to the historical key information. For example, if the historical key information is a control object, the question text of the historical key information can be "what game character is included in the current screen", and the answer text can be "character B in game A is included in the current screen".

[0062] Taking Vit-transformers as an example, the Vit-transformers model is a model with ViT module and transformer module as the backbone network, wherein the ViT module is used to process image data, and the transformer module is used to process text data. Therefore, when the Vit-transformers model is trained according to the first sample set, the historical game screen data of the user in each first sample can be used as the input of the ViT module, and the question text of the historical key information of the user in each first sample can be used as the input of the transformer module. At the same time, the answer text of the historical key information of the user in each first sample is used as the training target to train the Vit-transformers model until the Vit-transformers reaches the training termination condition, such as loss function convergence, accuracy of the Vit-transformers reaching a preset threshold, etc.

[0063] Therefore, when the image processing model is a visual language model, the game assistance system can use the game screen data of the target user and the pre-set question text as the input of the visual language model, so that the visual language model can directly output the key information in the game screen data of the target user.

[0064] Contrastive learning is a method of optimizing feature representations of a model by maximizing similarities between similar samples or minimizing distances between dissimilar samples. The similarities between each sample image and each sample description text are calculated by a contrastive learning loss function, such as an InfoNCE (Information Noise Contrastive Estimation) loss function. The model parameters are adjusted by optimizing these similarities, so that matching images and description texts are closer in the feature space, and dissimilar images and description texts are more dispersed in the feature space. Therefore, when the first sample set is used to train the multi-modal model based on contrastive learning, the corresponding description text can be determined according to the historical key information.

[0065] Taking CLIP as an example, CLIP is a model with a text encoding module, an image encoding module, and a contrastive learning module as the backbone network. The image encoding module is used to process image data, the text encoding module is used to process text data, and the contrastive learning module is used to determine the similarity calculation result of the simultaneously input image data and text data. Therefore, when training CLIP according to the first sample set, the historical game screen data of the user in each first sample can be input to the image encoding module, and the description text of the historical key information of the user in the randomly selected first sample can be input to the text encoding module. Then, the text features output by the text encoding module and the image features output by the image encoding module are input to the contrastive learning module. At the same time, CLIP is trained to maximize the similarity of the image features and the text features corresponding to the same first sample, and to minimize the similarity of the image features and the text features corresponding to different first samples, until CLIP reaches the training termination condition.

[0066] Therefore, when the image processing model is a multi-modal model based on contrastive learning, the game assistance system can input the game screen data of the target user and the pre-set description text to the multi-modal model based on contrastive learning, so that the multi-modal model based on contrastive learning can output the similarity between the game screen data of the target user and the description text. If the similarity is higher than the pre-set similarity value, it can be determined that the game screen data of the target user includes the key information included in the description text.

[0067] In another optional implementation, the image processing model can be a semantic segmentation model, such as a FCN (Fully Convolutional Network), a U-Net, a DeepLab, or the like. When the first sample set is used to train the semantic segmentation model, the class labels of each pixel in the historical game screen data in the corresponding historical game screen can be determined according to the historical key information in each first sample, that is, a historical segmentation mask. The size of the historical segmentation mask is usually consistent with the size of the corresponding historical game screen.

[0068] When the semantic segmentation model is trained according to the first sample set, the historical game screen data of the user in each first sample can be used as the input of the semantic segmentation model, and the semantic segmentation model can be trained with the historical segmentation mask corresponding to each historical game screen data as the target until the semantic segmentation model reaches a training termination condition.

[0069] Therefore, when the image processing model is a semantic segmentation model, the game auxiliary system can use the game screen data of the target user as the input of the semantic segmentation model, so that the semantic segmentation model can output the segmentation mask corresponding to the game screen data of the target user, and determine the number of pixels corresponding to each class label. If the number of pixels corresponding to any class label is greater than a preset number threshold, it can be determined that the game screen data of the target user includes the key information corresponding to the class label.

[0070] For example, if the number of pixels corresponding to the class label of character C in the segmentation mask is 12,000, which is greater than the preset number threshold of 10,000, it can be determined that the game screen data of the target user includes character C.

[0071] In step S300, the strategy distribution of the control object in the current state, the reward function, and the state value function in the next state are determined according to the key information.

[0072] In this step, the game auxiliary system can determine the strategy distribution of the control object, the value state function, and the reward function of the action executable by the control object in the current state (that is, the executable action) in the strategy distribution according to the key information in the game screen data. Alternatively, if the current task is complex, the game auxiliary system can decompose the current task to determine a plurality of subtasks corresponding to the current task, and further determine the subtask strategy distribution, the subtask state value function, and the historical reward function of the historical control object in the historical state in the subtask strategy distribution corresponding to each subtask.

[0073] The strategy distribution includes executable actions and prior probabilities of each executable action. For example, if the current task of the target user is a rock-paper-scissors game, the actions executable by the control object can include rock action, scissors action, and cloth action, and the initial prior probabilities of the three executable actions are all 1 / 3.

[0074] In an optional implementation, the game auxiliary system can determine the strategy distribution of the control object in the current state, the reward function, and the state value function in the next state according to the key information in the game picture data and based on a pre-trained data prediction model.

[0075] The data prediction model of the embodiment can be a neural network, such as a convolutional neural network and a model based on a convolutional neural network, a recurrent neural network and a model based on a recurrent neural network, etc., which is not limited in the embodiment.

[0076] The data prediction model can be obtained by training based on a second sample set. The second sample set can include a plurality of second samples, and each second sample includes historical key information of a corresponding user, historical strategy distribution of a historical control object, a historical state value function, and a historical reward function.

[0077] Optionally, the data prediction model can be a multi-task learning model. The multi-task learning model can output the prediction results of multiple tasks at the same time, specifically, the strategy distribution of the control object, the state value function, and the historical reward function. Therefore, when training the data prediction model based on the second sample, the historical key information of each user can be used as the input of the data prediction model, and the corresponding historical strategy distribution, historical state value function, and historical reward function can be used as the target to train the data prediction model until the data prediction model reaches the training termination condition.

[0078] In step S400, a target action of the control object in the current state is determined according to the strategy distribution and the state value function.

[0079] In this step, the game auxiliary system can predict the action suitable for execution by the control object in the current state through multi-player game reinforcement learning.

[0080] Figure 2 FIG. 1 is a flowchart of an information interaction method according to an embodiment of the present application. As shown in FIG. 1, in an optional implementation of the embodiment, step S400 can include the following steps: Figure 2

[0081] In step S410, the priority of each executable action is determined based on a preset search algorithm according to the strategy distribution, the state value function, and the reward function.

[0082] ​In this step, the game auxiliary system can determine the priority of each action in the policy distribution based on a preset search algorithm according to the policy distribution and the state value function of the control object in the current state. Optionally, the preset search algorithm can be Monte Carlo Tree Search (MCTS). When the priority of each action is determined based on the Monte Carlo Tree Search algorithm, the priority U(s, a) of the control object performing action a in the current state s can be represented by the following formula:

[0083]

[0084] wherein Q(s, a) represents the action value function of the state-action pair (s, a), that is, the average cumulative reward after performing action a from s; c is a pre-set constant; P(s, a) represents the prior probability of action a; ∑ b N(s, b) represents the total number of times that all actions b of the control object are selected to be performed in the current state s; N(s, a) represents the number of times that action a of the control object is selected to be performed in the current state s.

[0085] wherein the average value Q(s, a) of the state-action pair (s, a) can be represented by the following formula:

[0086] Q(s, a) = r(s, a) + γ · V(s ′ );

[0087] wherein r(s, a) represents the reward value of the control object performing action a in the current state s, which is determined according to the reward function R(s, a) of the control object performing action a in the current state s; γ represents a discount factor; V(s') represents the state value function of the next state of the control object in the current state s.

[0088] Step S420, determining the target action according to the priorities.

[0089] After determining the priority of each executable action of the control object in the current state, the game auxiliary system can determine the executable action with the highest priority as the target action of the control object in the current state.

[0090] Step S500, interacting with the target user according to the target action.

[0091] After determining the target action of the control object in the current state, the game auxiliary system can generate corresponding interaction text according to the target action, and output the interaction text to the user through the terminal device, or determine the interaction voice according to the interaction text, and output the interaction voice to the target user through the terminal device or the voice playing device.

[0092] Figure 3is a flowchart of the information interaction method of an embodiment of the present application. As shown in Figure 3 In an optional implementation of the present embodiment, step S500 can include the following steps:

[0093] Step S510: detecting actual behavior information of the control object.

[0094] In the present embodiment, the interaction text can be used to prompt the target user to control the control object to perform the target action. Therefore, in an optional implementation of the present embodiment, the game assistance system can detect actual behavior information of the control object. The actual behavior information of the control object can include at least one of actual action and actual task target of the control object.

[0095] Optionally, the actual behavior information of the control object can be determined by image recognition on the game screen data of the target user that is acquired again, that is, the game assistance system can return to step S100-step S200.

[0096] Step S520: in response to the actual behavior information being inconsistent with the predicted behavior information of the control object, outputting an abnormal prompt to the target user.

[0097] In the present step, the game assistance system can match the actual behavior information of the control object with the predicted behavior information of the control object. If it is determined that the actual behavior information of the control object is inconsistent with the predicted behavior information of the control object, the game assistance system can generate an abnormal prompt and output the interaction content of the abnormal prompt to the target user through a terminal device or a voice playing device. The predicted behavior information of the control object can include target action of the control object and task target of the current task.

[0098] For example, if the game assistance system predicts that the task target of the control object is to build a bridge, but the control object is building a house, the game assistance system can determine that the actual behavior of the control object is inconsistent with the predicted behavior information; or, if the game assistance system predicts that the travel route of the control object is inconsistent with the actual travel route of the control object, the game assistance system can determine that the actual behavior of the control object is inconsistent with the predicted behavior information.

[0099] In the present embodiment, the game state of the user can be determined by image recognition, and the action of the control object can be predicted based on reinforcement learning and the game state of the user to provide game assistance for the user. Therefore, the legality of the game assistance can be improved while the timeliness and accuracy of the game assistance are improved, so that the use experience of the user can be improved.

[0100] In the embodiment of the present application, the game screen data of the user is acquired during the game process of the user, and key information in the game screen data is determined, and then the strategy distribution of the object controlled by the user in the current state, the reward function and the state value function in the next state are determined according to the key information in the game screen data, and then the target action of the object controlled by the user in the current state is determined according to the strategy distribution of the object controlled by the user, the state value function and the reward function, and interaction is performed with the user according to the target action. In the embodiment of the present application, the game state of the user can be recognized in real time according to the game screen data of the user, and then the action of the operation object is accurately predicted and interaction is performed with the user, so that the timeliness and accuracy of game assistance can be improved, and the use experience of the user is improved.

[0101] Figure 4 is a schematic diagram of an information interaction device of the embodiment of the present application. As shown in Figure 4 , the information interaction device of the embodiment of the present application comprises a data acquisition unit 401, an information determination unit 402, a function determination unit 403, an action determination unit 404 and an interaction unit 405.

[0102] The data acquisition unit 401 is configured to acquire game screen data of a target user. The information determination unit 402 is configured to determine key information in the game screen data, the key information comprising at least one of state data of a control object, task state of a current task, task target of the current task and environment data of an environment in which the control object is located. The function determination unit 403 is configured to determine, according to the key information, a strategy distribution of the control object in a current state, a reward function and a state value function in a next state. The action determination unit 404 is configured to determine, according to the strategy distribution, the state value function and the reward function, a target action of the control object in the current state. The interaction unit 405 is configured to interact with the target user according to the target action.

[0103] Further, the information determination unit 402 is further configured to determine the key information based on a pre-trained multi-modal model according to the game screen data, the multi-modal model being obtained by training based on a first sample set, the first sample set comprising historical game screen data and historical key information of a plurality of users.

[0104] Further, the function determination unit 403 comprises a first function determination sub-unit.

[0105] The first function determination subunit is configured to determine the strategy distribution, the state value function and the reward function based on a pre-trained data prediction model according to the key information, wherein the data prediction model is obtained by training based on a second sample set, and the second sample set includes historical key information, historical strategy distribution, historical reward function and historical state value function in a next state of a historical state of a historical control object of a plurality of users in the historical state.

[0106] Further, the action determination unit 404 includes a priority determination subunit and an action determination subunit.

[0107] The priority determination subunit is configured to determine the priority of each executable action based on a preset search algorithm according to the strategy distribution, the state value function and the reward function; and the action determination subunit is configured to determine the target action according to the priority.

[0108] Further, the interaction unit 405 includes an information determination subunit and a prompt output subunit.

[0109] The information determination subunit is configured to detect actual behavior information of the control object, wherein the actual behavior information includes at least one of an actual action and an actual task target of the control object; and the prompt output subunit is configured to output an abnormal prompt to the target user in response to the actual behavior information being inconsistent with predicted behavior information of the control object, wherein the predicted behavior information includes at least one of the target action and the task target.

[0110] Further, the function determination unit 403 includes a task determination subunit and a second function determination subunit.

[0111] The task determination subunit is configured to determine a plurality of subtasks corresponding to the current task; and the second function determination subunit is configured to determine a subtask strategy distribution, a subtask state value function and a sub-reward function corresponding to each subtask.

[0112] In the embodiments of the present application, game screen data of a user is acquired in a game process of the user, and key information in the game screen data is determined, and then a strategy distribution, a reward function and a state value function in a next state of a control object controlled by the user in a current state are determined according to the key information in the game screen data, and then a target action of the control object controlled by the user in the current state is determined according to the strategy distribution, the state value function and the reward function of the control object controlled by the user, so as to interact with the user according to the target action. In the embodiments of the present application, the game state of the user can be recognized in real time according to the game screen data of the user, and then the action of the control object is accurately predicted and interacted with the user, so that the timeliness and accuracy of game assistance can be improved, and thus the use experience of the user can be improved.

[0113] Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. (For example...) Figure 5 As shown, electronic device 5 is a general-purpose data processing device, which includes a general-purpose computer hardware structure, including at least a processor 501 and a memory 502. The processor 501 and memory 502 are connected via a bus 503. The memory 502 is adapted to store instructions or programs executable by the processor 501. The processor 501 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 501 executes the instructions stored in the memory 502 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 503 connects the aforementioned components together, and also connects the aforementioned components to a display controller 504, a display device, and an input / output (I / O) device 505. The input / output (I / O) device 505 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 505 is connected to the system via an input / output (I / O) controller 506.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0116] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0117] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0118] Another embodiment of the present application relates to a non-volatile storage medium for storing a computer readable program for causing a computer to execute some or all of the above-mentioned method embodiments.

[0119] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned method embodiments can be completed by a program stored in a storage medium, including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage media that can store program codes.

[0120] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An information exchange method, characterized in that, The method includes: Obtain game screen data from the target user; Determine key information in the game screen data, the key information including at least one of the following: the state data of the controlled object, the task status of the current task, the task objective of the current task, and the environmental data of the environment in which the controlled object is located; Based on the key information, determine the policy distribution, reward function, and state value function of the manipulated object in the current state and the next state. The target action of the manipulated object in the current state is determined based on the policy distribution, the state value function, and the reward function; Interact with the target user based on the target action.

2. The method according to claim 1, characterized in that, The key information determined from the game screen data includes: Based on the game screen data, the key information is determined using a pre-trained multimodal model. The multimodal model is trained on a first sample set, which includes historical game screen data and historical key information from multiple users.

3. The method according to claim 1, characterized in that, The step of determining the policy distribution, reward function, and state value function of the manipulated object in the current state and the next state based on the key information includes: Based on the key information, the policy distribution, the state value function, and the reward function are determined using a pre-trained data prediction model. The data prediction model is trained on a second sample set, which includes historical key information, historical policy distribution, historical reward function, and historical state value function of the next state of the historical manipulation objects of multiple users in historical states.

4. The method according to claim 1, characterized in that, Determining the target action of the manipulated object in the current state based on the policy distribution, the state value function, and the reward function includes: Based on a preset search algorithm, the priority of each executable action is determined according to the policy distribution, the state value function, and the reward function; The target action is determined based on the respective priorities.

5. The method according to claim 1, characterized in that, The interaction with the target user based on the target action includes: Detect the actual behavior information of the manipulated object, the actual behavior information including at least one of the actual actions of the manipulated object and the actual task objective; In response to a discrepancy between the actual behavior information and the predicted behavior information of the manipulated object, an abnormal prompt is output to the target user, wherein the predicted behavior information includes at least one of the target action and the task objective.

6. The method according to claim 1, characterized in that, The step of determining the policy distribution, reward function, and state value function of the manipulated object in the current state and the next state based on the key information includes: Identify multiple subtasks corresponding to the current task; Determine the subtask policy distribution, subtask state value function, and sub-reward function corresponding to each subtask.

7. An information interaction device, characterized in that, The device includes: The data acquisition unit is used to acquire game screen data of the target user. An information determination unit is used to determine key information in the game screen data. The key information includes at least one of the following: the status data of the controlled object, the task status of the current task, the task objective of the current task, and the environmental data of the environment in which the controlled object is located. The function determination unit is used to determine the policy distribution, reward function, and state value function of the manipulated object in the current state and the next state based on the key information. An action determination unit is used to determine the target action of the manipulated object in the current state based on the policy distribution, the state value function, and the reward function; An interaction unit is used to interact with the target user based on the target action.

8. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the method as described in any one of claims 1-6.

Images