Intelligent agent reinforcement learning method and interaction method

Abstract

The invention provides an agent reinforcement learning method and an interaction method, and relates to the technical field of artificial intelligence. The specific implementation scheme comprises the steps of obtaining a current environment state of an external environment based on a candidate moment; determining an estimated object intention of the to-be-interacted object according to the current environment state and the historical action sequence of the to-be-interacted object; based on a role behavior style of a to-be-trained agent, determining an agent action matched with the pre-estimated object intention; the agent action is executed, an interaction response result based on the agent action is obtained, and the interaction response result comprises a state transition result of the external environment and / or an actual interaction action of the to-be-interacted object; and according to the interaction response result, updating the pre-estimated object intention and the agent action so as to realize reinforcement learning of the to-be-trained agent.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an intelligent agent reinforcement learning method and an interaction method. Background Art

[0002] In the process of interaction between the agent and the environment, reinforcement learning can be used to achieve specific goals. Intention recognition and action decision-making are one of the important contents of agent reinforcement learning.

[0003] However, in some intelligent agent reinforcement learning practices, there are problems with poor accuracy in intention recognition and poor action decision-making. Summary of the invention

[0004] The present disclosure provides an intelligent agent reinforcement learning method, an interaction method and device, an electronic device and a medium.

[0005] According to one aspect of the present disclosure, there is provided an agent reinforcement learning method, which is applied to an agent to be trained, the method comprising: obtaining a current environment state of an external environment based on a candidate moment, the external environment comprising a preset rule library for obtaining output results based on input data; determining an estimated object intention of the object to be interacted with based on the current environment state and a historical action sequence of the object to be interacted with, the historical action sequence comprising at least one historical action of the object to be interacted with based on the candidate moment; determining an agent action matching the estimated object intention based on the role behavior style of the agent to be trained, the role behavior style indicating the strategic tendency of the agent to be trained when executing an action decision; executing the agent action, and obtaining an interactive response result based on the agent action, the interactive response result comprising a state transition result of the external environment and / or an actual interactive action of the object to be interacted with; and updating the estimated object intention and the agent action based on the interactive response result to achieve reinforcement learning of the agent to be trained.

[0006] Through agent reinforcement learning, the agent can accurately identify the actual interaction intentions of the object to be interacted with and decide on targeted agent actions. It can also help the agent to more accurately simulate the complex decision-making environment of humans, gradually improve the accuracy of agent interaction, and effectively improve the interaction effect of the agent.

[0007] In some embodiments, determining the estimated object intention of the object to be interacted with according to the current environment state and the historical action sequence of the object to be interacted with includes: determining at least one target object intention of the object to be interacted with according to the historical action sequence and the trigger action sequence matching each of the candidate object intentions based on multiple candidate object intentions in a preset intention pool; determining the intention evaluation value of each target object intention according to the current environment state, the subject behavior style of the object to be interacted with, and at least one of the preset knowledge bases of the intelligent agent to be trained; and taking the target object intention corresponding to the highest intention evaluation value as the estimated object intention of the object to be interacted with, wherein the estimated object intention includes a task and a key-value pair for describing the task.

[0008] In some embodiments, determining at least one target object intention of the object to be interacted with based on the historical action sequence and the trigger action sequence that matches each of the candidate object intentions includes: based on the historical action sequence and the trigger action sequence that matches each of the candidate object intentions, taking the candidate object intention whose action sequence similarity is higher than a preset threshold as the target object intention of the object to be interacted with.

[0009] In some embodiments, determining the agent action that matches the intention of the estimated object based on the character behavior style of the agent to be trained includes: determining the reward evaluation value of each candidate execution action based on a first action reward weight based on the character behavior style and at least one preset candidate execution action that matches the intention of the estimated object; and taking the candidate execution action corresponding to the highest reward evaluation value as the agent action that matches the intention of the estimated object.

[0010] In some embodiments, the character's behavior style is indicated by multiple behavior style parameters, and the first action reward weight includes a reward weight value that matches each of the behavior style parameters; the reward evaluation value of each of the candidate execution actions is determined based on the first action reward weight based on the character's behavior style and at least one preset candidate execution action that matches the estimated object intention, including: for any candidate execution action, according to the reward weight value that matches each of the behavior style parameters, calculating the weighted evaluation value of the candidate execution action based on the multiple behavior style parameters, as the reward evaluation value of the candidate execution action.

[0011] In some embodiments, the method also includes: determining a reward evaluation value for each of the candidate execution actions based on a second action reward weight based on the subject's behavior style and at least one preset candidate execution action that matches the intention of the estimated object; and taking the candidate execution action corresponding to the highest reward evaluation value as the intelligent agent action that matches the intention of the estimated object.

[0012] In some embodiments, executing the intelligent agent action and obtaining the interactive response result based on the intelligent agent action includes: sending the intelligent agent action to the external environment so that the external environment determines the interactive response result according to the actual interactive action of the intelligent agent action and the object to be interacted; and obtaining the interactive response result fed back by the external environment.

[0013] In some embodiments, updating the estimated object intention and the intelligent agent action according to the interaction response result includes: determining a quantifiable scalar feedback signal generated by the actions of the interacting parties according to the state transition result of the external environment and / or the actual interaction action of the object to be interacted; determining whether the actual object intention of the object to be interacted is satisfied according to the scalar feedback signal; and returning to determining the estimated object intention and the intelligent agent action if the actual object intention of the object to be interacted is not satisfied.

[0014] In some embodiments, the method further includes: when the actual object intention of the object to be interacted with is satisfied, updating the weight of at least one candidate execution action that matches the estimated object intention; wherein the weight value corresponding to the intelligent agent action in the at least one candidate execution action is increased.

[0015] In some embodiments, the method further includes: generating an intention inquiry request when the intention evaluation values ​​of each of the target objects' intentions are lower than a preset threshold; and sending the intention inquiry request to the object to be interacted with, so that the object to be interacted with informs the intelligent agent to be trained of the actual object intention.

[0016] In some embodiments, the agent to be trained is embedded with a neural network model, and the neural network model includes an intention assessment network and an action strategy network.

[0017] In some embodiments, determining the estimated object intention of the object to be interacted with based on the current environment state and the historical action sequence of the object to be interacted with includes: through the intention evaluation network, selecting the target object intention from a preset intention pool according to the current environment state and the historical action sequence of the object to be interacted with, as the estimated object intention of the object to be interacted with.

[0018] In some embodiments, determining the agent action that matches the estimated object intention based on the role behavior style of the agent to be trained includes: determining the agent action to be executed through the action strategy network according to the role behavior style of the agent to be trained and at least one preset candidate execution action that matches the estimated object intention.

[0019] According to another aspect of the present disclosure, there is provided an agent interaction method, the method comprising: obtaining a current environmental state of an external environment and a historical action sequence of an object to be interacted with, the historical action sequence comprising at least one historical action of the object to be interacted with based on the current moment; determining an estimated object intention of the object to be interacted with based on the current environmental state and the historical action sequence of the object to be interacted with; determining an agent action that matches the estimated object intention based on the role behavior style of the agent, the role behavior style indicating the strategic tendency of the agent when executing action decisions; and executing the agent action to realize the interaction process with the object to be interacted with.

[0020] According to another aspect of the present disclosure, there is provided an agent reinforcement learning device, comprising: an acquisition module, for acquiring a current environment state of an external environment based on a candidate moment, wherein the external environment includes a preset rule library for obtaining an output result based on input data; a first processing module, for determining an estimated object intention of the object to be interacted with based on the current environment state and a historical action sequence of the object to be interacted with, wherein the historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment; a second processing module, for determining an agent action that matches the estimated object intention based on the role behavior style of the agent to be trained, wherein the role behavior style indicates a strategic tendency of the agent to be trained when executing an action decision; a third processing module, for executing the agent action and acquiring an interactive response result based on the agent action, wherein the interactive response result includes a state transition result of the external environment and / or an actual interactive action of the object to be interacted with; and a fourth processing module, for updating the estimated object intention and the agent action based on the interactive response result to achieve reinforcement learning of the agent to be trained.

[0021] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned agent reinforcement learning method.

[0022] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned intelligent agent reinforcement learning method.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the above-mentioned agent reinforcement learning method when executed by a processor.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. Figure 1 A schematic diagram of an application environment involved in the agent reinforcement learning method according to an embodiment of the present disclosure is schematically shown; Figure 2 A flowchart of an intelligent agent reinforcement learning method according to an embodiment of the present disclosure is schematically shown; Figure 3 A schematic diagram schematically shows a process of intelligent agent reinforcement learning according to an embodiment of the present disclosure; Figure 4 A flowchart of an intelligent agent interaction method according to an embodiment of the present disclosure is schematically shown; Figure 5 A block diagram of an intelligent agent reinforcement learning device according to an embodiment of the present disclosure is schematically shown; Figure 6 A block diagram of an electronic device for executing an agent reinforcement learning method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0030] In the process of interaction between the agent and the environment, reinforcement learning can be used to achieve specific goals. Intention recognition and action decision-making are one of the important contents of agent reinforcement learning. However, in some practices of agent reinforcement learning, there are phenomena of poor accuracy in intention recognition and poor action decision-making.

[0031] The disclosed embodiments provide an intelligent agent reinforcement learning method, device and computer-readable storage medium. The intelligent agent reinforcement learning device can be integrated into an electronic device, which can be a terminal device or a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal device can be a mobile phone, a tablet, a computer, a smart wearable device, a smart home appliance, a robot, a driving assistance device, etc.

[0032] According to an agent reinforcement learning method of an embodiment of the present disclosure, the current environment state of the external environment based on the candidate moment can be obtained, and the external environment includes a preset rule library for obtaining output results based on input data. According to the current environment state and the historical action sequence of the object to be interacted, the estimated object intention of the object to be interacted is determined, and the historical action sequence includes at least one historical action of the object to be interacted based on the candidate moment. Then, based on the role behavior style of the agent to be trained, the agent action that matches the estimated object intention is determined, and the role behavior style indicates the strategic tendency of the agent to be trained when executing action decisions. Next, the agent action is executed, and the interactive response result based on the agent action is obtained. The interactive response result includes the state transition result of the external environment and / or the actual interactive action of the object to be interacted. And, according to the interactive response result, the estimated object intention and the agent action are updated to realize reinforcement learning of the agent to be trained.

[0033] Figure 1 The following schematically shows an application environment diagram of the agent reinforcement learning method according to an embodiment of the present disclosure. Figure 1 As shown, agents 101, 102 and object 103 can interact with each other based on the external environment. Agents 101 and 102 can enhance their ability to recognize intentions and make action decisions based on the agent reinforcement learning method of the embodiment of the present disclosure. For example, during the agent interaction process, the agent can recognize the evaluation object intention of the object to be interacted with and determine the agent action that matches the evaluation object intention.

[0034] Taking agent 101 as an example, and taking object 103 as the object to be interacted, agent 101 can obtain the current environment state of the external environment and the historical action sequence of object 103, and the historical action sequence can include at least one historical action of object 103 based on the current moment. Agent 101 determines the estimated object intention of object 103 based on the current environment state and the historical action sequence of object 103. Agent 101 determines the agent action that matches the estimated object intention based on the preset role behavior style, and the role behavior style indicates the strategic tendency of agent 101 when executing action decision. Agent 101 executes the agent action to realize the interaction process with object 103.

[0035] Figure 1 The following schematically shows an application environment diagram of the agent reinforcement learning method according to an embodiment of the present disclosure. It should be noted that: Figure 1 The examples shown are merely examples of application environments in which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0036] It should be understood that Figure 1 The number of agents and objects in the example is only illustrative. Any number of agents and objects may be provided as required.

[0037] The present disclosure provides an intelligent agent reinforcement learning method. Figure 1 For an example of an application environment, refer to Figure 2~Figure 6 To describe the agent reinforcement learning method according to an exemplary embodiment of the present disclosure.

[0038] Figure 2 The flowchart of the intelligent agent reinforcement learning method according to one embodiment of the present disclosure is schematically shown.

[0039] like Figure 2 As shown, the agent reinforcement learning method 200 of the embodiment of the present disclosure may, for example, include operations S210 to S250.

[0040] Operation S210 is to obtain a current environment state of an external environment based on a candidate time, where the external environment includes a preset rule base for obtaining output results according to input data.

[0041] Operation S220 , determining an estimated object intention of the object to be interacted with according to the current environment state and a historical action sequence of the object to be interacted with, where the historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment.

[0042] Operation S230, based on the role behavior style of the intelligent agent to be trained, determine the agent action that matches the estimated object intention, and the role behavior style indicates the strategic tendency of the intelligent agent to be trained when executing action decisions.

[0043] Operation S240, executing the agent action, and obtaining an interaction response result based on the agent action, wherein the interaction response result includes a state transition result of the external environment and / or an actual interaction action of the object to be interacted with.

[0044] Operation S250, updating the estimated object intention and agent action according to the interaction response result to achieve reinforcement learning for the training agent.

[0045] The implementation process of each operation is schematically illustrated below by way of example.

[0046] Operation S210 is to obtain a current environment state of an external environment based on a candidate time, where the external environment includes a preset rule base for obtaining output results according to input data.

[0047] Exemplarily, the current state of the external environment based on the candidate time can be obtained, and the external environment includes a preset rule base for obtaining output results based on input data. For example, the external environment can receive the actions of both parties in the interaction, and convert the actions of both parties in the interaction into quantifiable rewards and feed them back to the intelligent agent. The quantifiable reward includes the current state after the interaction, and the current state after the interaction can include the state transition result of the external environment, and can also include the action selection information of the object to be interacted.

[0048] Operation S220 , determining an estimated object intention of the object to be interacted with according to the current environment state and a historical action sequence of the object to be interacted with, where the historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment.

[0049] Exemplarily, based on multiple candidate object intentions in a preset intention pool, at least one target object intention of the object to be interacted with can be determined according to historical action sequences and trigger action sequences matching each candidate object intention.

[0050] For example, the action sequence similarity between the historical action sequence and the trigger action sequence matching each candidate object intention may be calculated, and the candidate object intention with the action sequence similarity higher than a preset threshold may be used as the target object intention of the object to be interacted with.

[0051] The intention evaluation value of each target object intention can be determined according to at least one of the current environment state, the subject behavior style of the object to be interacted with, and the preset knowledge base of the intelligent agent to be trained. And the target object intention corresponding to the highest intention evaluation value is used as the estimated object intention of the object to be interacted with. The estimated object intention includes a task and a key-value pair for describing the task.

[0052] The preset knowledge base of the intelligent agent to be trained may include mapping data of the physical world, and may also include the memory of common sense of the intelligent agent to be trained. The mapping data and common sense memory in the preset knowledge base may be correct data or incorrect data. The preset knowledge base can be updated and replaced online.

[0053] The subject behavior style can indicate the strategic tendency of the object to be interacted with when making action decisions. The subject behavior style can be indicated by the personality characteristics, behavioral habits, user portrait and other data of the object to be interacted with. For example, if the object to be interacted with has a socially proactive personality, the object to be interacted with may want the agent to communicate with it and help it complete the task positively. If the object to be interacted with has a socially avoidant personality, the object to be interacted with may want the agent not to communicate with it and complete the task by itself.

[0054] When there is a target object intention whose intention evaluation value is greater than a preset threshold, the target object intention corresponding to the highest intention evaluation value is used as the estimated object intention of the object to be interacted with.

[0055] When the intention evaluation values ​​of the intentions of each target object are lower than the preset threshold, an intention inquiry request is generated, and the intention inquiry request is sent to the object to be interacted with, so that the object to be interacted with informs the intelligent agent to be trained of the actual object intention.

[0056] Intent recognition may be ambiguous. By sending an intent query request to the object to be interacted with, it is helpful to actively eliminate ambiguity through intelligent agent interaction, improve the accuracy of intent recognition, and further improve the effectiveness of the intelligent agent's action decision-making.

[0057] When the intention evaluation values ​​of each target object's intention are lower than the preset threshold, there may be no object intention in the representation intention pool that matches the historical action sequence of the object to be interacted with. In this case, the trained agent can directly output the formatted intention recognition result as the evaluation object intention of the object to be interacted with.

[0058] As an optional method, object input data of the object to be interacted with based on the candidate time can also be obtained, and the estimated object intention of the object to be interacted with can be obtained by encoding the object input data. The object input data includes, for example, text, image, audio, video and other types of data.

[0059] Operation S230, based on the role behavior style of the intelligent agent to be trained, determine the agent action that matches the estimated object intention, and the role behavior style indicates the strategic tendency of the intelligent agent to be trained when executing action decisions.

[0060] Exemplarily, the reward evaluation value of each candidate execution action is determined according to the first action reward weight based on the character behavior style and at least one preset candidate execution action that matches the estimated object intention. The candidate execution action corresponding to the highest reward evaluation value is used as the agent action that matches the estimated object intention.

[0061] The character behavior style of the intelligent agent to be trained may have an impact on the action decision of the intelligent agent. The character behavior style can be indicated by behavior style parameters such as the current state of the intelligent agent to be trained and preset social value. The character behavior style can be indicated by multiple behavior style parameters, and the first reward weight value includes a reward weight value matching each behavior style parameter.

[0062] For example, the reward weights corresponding to the current state and preset social value of the agent to be trained are 0.6 and 0.4 respectively. The current state of the agent to be trained is busy, and the preset social value is willing to help others.

[0063] Suppose that the candidate execution action that matches the estimated object intention is "actively fetching food for others". Because the current state of the agent to be trained is busy, the agent to be trained may not be able to actively fetch food for others at present. The reward evaluation value corresponding to the candidate execution action can be 1*0+1*0.4=0.4.

[0064] Assume that the candidate action that matches the estimated object intention is "tell others the location of food", the reward evaluation value corresponding to the candidate action can be 1*0.6+1*0.4=1.

[0065] According to the first action reward weight based on the character's behavior style and at least one preset candidate execution action that matches the estimated object's intention, the reward evaluation value of each candidate execution action is determined. Specifically, for any candidate execution action, according to the reward weight value that matches each behavior style parameter, the weighted evaluation value of the candidate execution action based on multiple behavior style parameters is calculated as the reward evaluation value of the candidate execution action. The candidate execution action corresponding to the highest reward evaluation value is used as the intelligent agent action that matches the estimated object's intention.

[0066] As an optional method, the reward evaluation value of each candidate execution action can be determined based on a second reward weight value based on the subject's behavior style and at least one preset candidate execution action that matches the intention of the estimated object, and the candidate execution action corresponding to the highest reward evaluation value can be used as the intelligent agent action that matches the intention of the estimated object.

[0067] The subject behavior style of the object to be interacted with may also have an impact on the action decision of the intelligent agent. The subject behavior style can be indicated by behavior style parameters such as the current state and personality profile characteristics of the object to be interacted with. The subject behavior style can be indicated by multiple behavior style parameters, and the second reward weight value includes a reward weight value matching each behavior style parameter.

[0068] For example, the reward weight values ​​corresponding to the current state and personality profile characteristics of the object to be interacted with are 0.5 and 0.5 respectively. The current state of the object to be interacted with is idle, and the personality profile characteristics are social avoidance.

[0069] Suppose that the candidate execution action that matches the estimated object's intention is "actively fetching food for others". Because the object to be interacted with has a socially avoidant personality, the object to be interacted with may not want the agent to actively help fetch food. The reward evaluation value corresponding to the candidate execution action can be 1*0.5+1*0=0.5.

[0070] Suppose that the candidate action that matches the estimated object's intention is "tell others the location of the food". This candidate action may be more in line with the personality profile of the object to be interacted with, and the reward evaluation value corresponding to the candidate action can be 1*0.5+1*0.5=1.

[0071] According to the second action reward weight based on the subject's behavior style and at least one preset candidate execution action that matches the estimated object intention, the reward evaluation value of each candidate execution action is determined. Specifically, for any candidate execution action, according to the reward weight value matching each behavior style parameter, the weighted evaluation value of the candidate execution action based on multiple behavior style parameters is calculated as the reward evaluation value of the candidate execution action.

[0072] In addition, the reward evaluation value of each candidate execution action can also be calculated based on the first action reward weight of the character's behavior style and the second action reward weight based on the subject's behavior style. And the candidate execution action corresponding to the highest reward evaluation value is used as the intelligent agent action that matches the estimated object intention. Among them, the action reward weight corresponding to each behavior style parameter can be set according to the actual situation, and this embodiment does not limit this.

[0073] By setting the role behavior style of the intelligent agent, it is helpful for the intelligent agent to simulate human personality preference characteristics. By simulating human-like behavior, it is helpful to improve the flexibility of the intelligent agent's action decision-making and the accuracy of interactive response.

[0074] Operation S240, executing the agent action, and obtaining an interaction response result based on the agent action, wherein the interaction response result includes a state transition result of the external environment and / or an actual interaction action of the object to be interacted with.

[0075] Exemplarily, the agent action is sent to the external environment so that the external environment determines the interaction response result according to the agent action and the actual interaction action of the object to be interacted. The agent to be trained obtains the interaction response result fed back by the external environment, and the interaction response result can be a quantifiable scalar feedback signal.

[0076] The external environment can be a preset rule base for obtaining output results based on input data. The external environment can transfer from the current environment state to the post-interaction state based on the received actions of the two interacting parties. The external environment feeds back the interaction response result to the intelligent agent to be trained. The interaction response result can include the state transfer result of the external environment and the action selection information of the object to be interacted. The action selection information can indicate the actual interaction action triggered by the object to be interacted.

[0077] Operation S250, updating the estimated object intention and agent action according to the interaction response result to achieve reinforcement learning for the training agent.

[0078] The agent is the core of the reinforcement learning system. The agent's actions affect not only the current moment's reward, but also the next moment and even the future reward. The basic principle of the agent to be trained in the reinforcement learning process is that if an agent's action brings positive rewards to the environment, the agent's action will be strengthened, otherwise it will be gradually weakened.

[0079] Exemplarily, according to the state transition result of the external environment and / or the actual interaction action of the object to be interacted, a quantifiable scalar feedback signal generated by the actions of the two interacting parties is determined. For example, the scalar feedback signal can be 1, 0, or -1, which respectively indicates that the actual object intention of the object to be interacted is satisfied, the action of the agent is irrelevant to the actual object intention of the object to be interacted, and the actual object intention of the object to be interacted is blocked.

[0080] According to the scalar feedback signal, determine whether the actual object intention of the object to be interacted with is satisfied. If the actual object intention of the object to be interacted with is not satisfied, return to the operation of determining the estimated object intention and the agent action.

[0081] When the actual object intention of the object to be interacted with is satisfied, the actual object intention is regarded as a completed intention and its life cycle ends. In addition, the action sequence matching the estimated object intention is recounted. Exemplarily, the weight of at least one candidate execution action matching the estimated object intention is updated. Among them, the weight value of the candidate execution action corresponding to the executed intelligent agent action is increased.

[0082] As an optional approach, the agent to be trained is embedded with a neural network model, which includes an intention evaluation network and an action strategy network.

[0083] The intention evaluation network can be used to select the target object intention from the preset intention pool according to the current environment state and the historical action sequence of the object to be interacted with, as the estimated object intention of the object to be interacted with.

[0084] The action strategy network can be used to determine the action of the agent to be executed based on the role behavior style of the agent to be trained and at least one preset candidate execution action that matches the estimated object intention.

[0085] The neural network model may be, for example, a large language model, a Transformer model, etc., which is not limited in this embodiment.

[0086] By using a neural network model for intention assessment, the actual intention of the object to be interacted with can be predicted based on the current state of the environment and the state data of the object to be interacted with. By establishing an action decision network and an action value network, the agent's actions can be adjusted during the reinforcement learning process to achieve the learning goal of maximizing the reward value.

[0087] Through agent reinforcement learning, the agent can accurately identify the actual interaction intentions of the object to be interacted with and decide on targeted agent actions. It can also help the agent to more accurately simulate the complex decision-making environment of humans, gradually improve the accuracy of agent interaction, and effectively improve the interaction effect of the agent.

[0088] By pre-configuring multiple candidate object intentions, multiple candidate execution actions, multiple role behavior styles and other configuration information, it is beneficial to train intelligent agents with different behavior styles based on multiple configuration information, which is beneficial to improving the efficiency of intelligent agent reinforcement learning, reducing the cost of intelligent agent reinforcement learning, and facilitating the realization of an automated intelligent agent reinforcement learning process.

[0089] Figure 3 A schematic diagram of an agent reinforcement learning process according to an embodiment of the present disclosure is schematically shown.

[0090] like Figure 3 As shown, based on the historical action sequence of the object to be interacted with and the trigger action sequence in the preset intention pool that matches the intention of each candidate object, the action sequence similarity between the historical action sequence and each trigger action sequence is calculated.

[0091] The candidate object intentions whose action sequence similarity is higher than a preset threshold are taken as the target object intentions of the object to be interacted with.

[0092] According to the current state of the external environment, the main behavior style of the object to be interacted with, and the preset knowledge base of the intelligent agent to be trained, the intention evaluation value of each target object intention is determined. And the target object intention corresponding to the highest intention evaluation value is used as the estimated object intention of the object to be interacted with. The estimated object intention corresponds to at least one preset candidate execution action.

[0093] According to the first action reward weight based on the character behavior style of the agent to be trained and at least one preset candidate execution action matching the estimated object intention, the reward evaluation value of each candidate execution action is determined. And the candidate execution action corresponding to the highest reward evaluation value is used as the agent action matching the estimated object intention.

[0094] Execute the agent action to obtain an interactive response result based on the agent action. Exemplarily, the agent action is sent to the external environment so that the external environment determines the interactive response result according to the agent action and the actual interactive action of the object to be interacted. The interactive response result includes the state transition result of the external environment and / or the actual interactive action of the object to be interacted.

[0095] According to the state transition results of the external environment and / or the actual interaction actions of the objects to be interacted, determine the quantifiable scalar feedback signal generated by the actions of the two interacting parties. According to the scalar feedback signal, determine whether the actual object intention of the object to be interacted is satisfied. If the actual object intention of the object to be interacted is not satisfied, return to the operation of determining the estimated object intention and the action of the agent. If the actual object intention of the object to be interacted is satisfied, end this round of learning.

[0096] Figure 4 The flowchart of the intelligent agent interaction method according to one embodiment of the present disclosure is schematically shown.

[0097] like Figure 4 As shown, the intelligent agent interaction method 400 of the embodiment of the present disclosure may, for example, include operations S410 to S440.

[0098] Operation S410: acquiring a current environment state of an external environment and a historical action sequence of an object to be interacted with, where the historical action sequence includes at least one historical action of the object to be interacted with based on a current moment.

[0099] Operation S420: determining an estimated object intention of the object to be interacted with according to the current environment state and the historical action sequence of the object to be interacted with.

[0100] Operation S430, based on the agent's role behavior style, determine the agent's action that matches the estimated object intention, and the role behavior style indicates the agent's strategic tendency when executing action decisions.

[0101] Operation S440: executing agent actions to implement an interaction process with the object to be interacted with.

[0102] The implementation process of each operation is schematically illustrated below by way of example.

[0103] The agent can obtain the current state of the external environment and the historical action sequence of the object to be interacted with based on the current moment, and determine the estimated object intention of the object to be interacted with based on the current state of the environment and the historical action sequence.

[0104] Exemplarily, the intelligent agent may acquire the intention recognition capability based on the reinforcement learning process of the above embodiment. Based on the acquired intention recognition capability, the intelligent agent may recognize the estimated object intention of the object to be interacted with.

[0105] The agent can determine the agent action that matches the estimated object intention based on the preset character behavior style. The character behavior style indicates the strategic tendency of the agent when executing action decisions.

[0106] Exemplarily, the intelligent agent can acquire action decision-making capability based on the reinforcement learning process of the above embodiment. The intelligent agent can decide the intelligent agent action to be executed based on the acquired action decision-making capability.

[0107] Next, the agent executes the determined agent action to realize the interaction process with the object to be interacted.

[0108] This helps the intelligent agent to accurately identify the actual interaction intention of the object to be interacted with and decide on targeted intelligent agent actions, which is conducive to improving the accuracy of intelligent agent interaction and can effectively improve the intelligent agent interaction effect.

[0109] Figure 5 A block diagram of an intelligent agent reinforcement learning device according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 5 As shown, the intelligent agent reinforcement learning device 500 of the embodiment of the present disclosure includes, for example, an acquisition module 510, a first processing module 520, a second processing module 530, a third processing module 540 and a fourth processing module 550.

[0111] The acquisition module 510 is used to obtain the current environment state of the external environment based on the candidate moment, and the external environment includes a preset rule library for obtaining output results based on input data. The first processing module 520 is used to determine the estimated object intention of the object to be interacted with based on the current environment state and the historical action sequence of the object to be interacted with. The historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment. The second processing module 530 is used to determine the agent action that matches the estimated object intention based on the role behavior style of the agent to be trained. The role behavior style indicates the strategic tendency of the agent to be trained when executing action decisions. The third processing module 540 is used to execute the agent action and obtain the interactive response result based on the agent action. The interactive response result includes the state transition result of the external environment and / or the actual interactive action of the object to be interacted with. The fourth processing module 550 is used to update the estimated object intention and agent action according to the interactive response result to realize reinforcement learning of the agent to be trained.

[0112] In some embodiments, the first processing module includes: a first processing sub-module, used to determine at least one target object intention of the object to be interacted with based on multiple candidate object intentions in a preset intention pool, according to historical action sequences and trigger action sequences matching each candidate object intention; a second processing sub-module, used to determine the intention evaluation value of each target object intention based on the current environment state, the subject behavior style of the object to be interacted with, and at least one of the preset knowledge bases of the intelligent body to be trained; and a third processing sub-module, used to use the target object intention corresponding to the highest intention evaluation value as the estimated object intention of the object to be interacted with, wherein the estimated object intention includes a task and a key-value pair for describing the task.

[0113] In some embodiments, the first processing submodule includes: a first processing unit, which is used to take the candidate object intention whose action sequence similarity is higher than a preset threshold as the target object intention of the object to be interacted with based on the historical action sequence and the trigger action sequence matching each candidate object intention.

[0114] In some embodiments, the second processing module includes: a fourth processing sub-module, used to determine the reward evaluation value of each candidate execution action based on the first action reward weight based on the character's behavior style and at least one preset candidate execution action that matches the intention of the estimated object; and a fifth processing sub-module, used to take the candidate execution action corresponding to the highest reward evaluation value as the intelligent agent action that matches the intention of the estimated object.

[0115] In some embodiments, the character's behavior style is indicated by multiple behavior style parameters, and the first action reward weight includes a reward weight value that matches each behavior style parameter; the fourth processing sub-module includes: a second processing unit, which is used to calculate, for any candidate execution action, a weighted evaluation value of the candidate execution action based on multiple behavior style parameters according to the reward weight value that matches each behavior style parameter, as the reward evaluation value of the candidate execution action.

[0116] In some embodiments, the second processing module also includes: a sixth processing sub-module, used to determine the reward evaluation value of each candidate execution action based on the second action reward weight based on the subject's behavior style and at least one preset candidate execution action that matches the estimated object intention; and a seventh processing sub-module, used to take the candidate execution action corresponding to the highest reward evaluation value as the intelligent agent action that matches the estimated object intention.

[0117] In some embodiments, the third processing module includes: an eighth processing sub-module, used to send the intelligent agent action to the external environment, so that the external environment determines the interaction response result based on the intelligent agent action and the actual interaction action of the object to be interacted with; and a ninth processing sub-module, used to obtain the interaction response result fed back by the external environment.

[0118] In some embodiments, the fourth processing module includes: a tenth processing sub-module, which is used to determine the quantifiable scalar feedback signal generated by the actions of the interacting parties based on the state transition results of the external environment and / or the actual interaction actions of the objects to be interacted; an eleventh processing sub-module, which is used to determine whether the actual object intention of the object to be interacted is satisfied based on the scalar feedback signal; and a twelfth processing sub-module, which is used to return to the operation of determining the estimated object intention and intelligent agent action when the actual object intention of the object to be interacted is not satisfied.

[0119] In some embodiments, the fourth processing module also includes: a thirteenth processing sub-module, which is used to update the weight of at least one candidate execution action that matches the estimated object intention when the actual object intention of the object to be interacted with is satisfied; wherein the weight value corresponding to the intelligent agent action in at least one candidate execution action is increased.

[0120] In some embodiments, the device also includes: a fifth processing module, which is used to generate an intention inquiry request when the intention evaluation values ​​of each target object's intention are lower than a preset threshold; and send the intention inquiry request to the object to be interacted with, so that the object to be interacted with informs the intelligent agent to be trained of the actual object's intention.

[0121] In some embodiments, the agent to be trained is embedded with a neural network model, which includes an intention assessment network and an action strategy network.

[0122] In some embodiments, the first processing module also includes: a fourteenth processing sub-module, which is used to select the target object intention from the preset intention pool through the intention evaluation network according to the current environment state and the historical action sequence of the object to be interacted with, as the estimated object intention of the object to be interacted with.

[0123] In some embodiments, the second processing module also includes: a fifteenth processing submodule, which is used to determine the intelligent agent action to be executed through an action strategy network according to the role behavior style of the intelligent agent to be trained and at least one preset candidate execution action that matches the estimated object intention.

[0124] It should be noted that the information collection, storage, use, processing, transmission, provision and disclosure involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0125] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0126] Figure 6 A block diagram of an electronic device for executing an agent reinforcement learning method according to an embodiment of the present disclosure is schematically shown.

[0127] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0128] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0129] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the agent reinforcement learning method. For example, in some embodiments, the agent reinforcement learning method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the agent reinforcement learning method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the agent reinforcement learning method in any other appropriate manner (e.g., by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable agent reinforcement learning device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with an object, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the object; and a keyboard and pointing device (e.g., a mouse or trackball) through which the object can provide input to the computer. Other types of devices can also be used to provide interaction with an object; for example, the feedback provided to the object can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the object can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., an object computer having a graphical object interface or a web browser through which the object can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a terminal and a server. The terminal and the server are generally remote from each other and usually interact through a communication network. The relationship between the terminal and the server is generated by computer programs running on the corresponding computers and having a terminal-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0138] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An agent reinforcement learning method, characterized in that: Applied to an intelligent agent to be trained, the method comprises: Acquiring a current state of an external environment based on a candidate time, wherein the external environment includes a preset rule base for obtaining output results according to input data; Determining an estimated object intention of the object to be interacted with according to the current environment state and a historical action sequence of the object to be interacted with, wherein the historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment; Determining an agent action that matches the estimated object intention based on the role behavior style of the agent to be trained, wherein the role behavior style indicates a strategic tendency of the agent to be trained when executing an action decision; Executing the agent action and obtaining an interactive response result based on the agent action, wherein the interactive response result includes a state transition result of the external environment and / or an actual interactive action of the object to be interacted with; and According to the interactive response result, the estimated object intention and the agent action are updated to achieve reinforcement learning of the agent to be trained.

2. The method according to claim 1, characterized in that The step of determining the estimated object intention of the object to be interacted with according to the current environment state and the historical action sequence of the object to be interacted with comprises: Based on a plurality of candidate object intentions in a preset intention pool, determining at least one target object intention of the object to be interacted with according to the historical action sequence and a trigger action sequence matching each of the candidate object intentions; Determine the intention evaluation value of each of the target objects' intentions according to at least one of the current environment state, the subject behavior style of the object to be interacted with, and the preset knowledge base of the intelligent agent to be trained; and The target object intention corresponding to the highest intention evaluation value is used as the estimated object intention of the object to be interacted with, The estimated object is intended to include a task and a key-value pair for describing the task.

3. The method according to claim 2, characterized in that The determining, according to the historical action sequence and the trigger action sequence matching the intentions of each candidate object, at least one target object intention of the object to be interacted with comprises: According to the historical action sequence and the trigger action sequence matching each of the candidate object intentions, the candidate object intention with an action sequence similarity higher than a preset threshold is taken as the target object intention of the object to be interacted with.

4. The method according to claim 2, characterized in that The step of determining an agent action that matches the estimated object intention based on the role behavior style of the agent to be trained includes: Determining a reward evaluation value of each candidate execution action according to a first action reward weight based on the character's behavior style and at least one preset candidate execution action that matches the estimated object intention; and The candidate execution action corresponding to the highest reward evaluation value is taken as the agent action that matches the estimated object intention.

5. The method according to claim 4, characterized in that The character behavior style is indicated by a plurality of behavior style parameters, and the first action reward weight includes a reward weight value matching each of the behavior style parameters; The step of determining the reward evaluation value of each candidate execution action according to the first action reward weight based on the character behavior style and at least one preset candidate execution action matching the estimated object intention includes: For any candidate execution action, according to the reward weight value matched with each of the behavior style parameters, a weighted evaluation value of the candidate execution action based on the multiple behavior style parameters is calculated as the reward evaluation value of the candidate execution action.

6. The method according to claim 2, characterized in that The method further comprises: Determining a reward evaluation value of each candidate execution action according to a second action reward weight based on the subject's behavior style and at least one preset candidate execution action that matches the estimated object intention; and The candidate execution action corresponding to the highest reward evaluation value is taken as the agent action that matches the estimated object intention.

7. The method according to claim 1, characterized in that The executing the agent action and obtaining an interactive response result based on the agent action includes: Sending the agent action to the external environment so that the external environment determines the interaction response result according to the agent action and the actual interaction action of the object to be interacted with; and The interactive response result fed back by the external environment is obtained.

8. The method according to claim 1, characterized in that: The updating of the estimated object intention and the agent action according to the interactive response result includes: Determine, according to the state transition result of the external environment and / or the actual interaction action of the object to be interacted, a quantifiable scalar feedback signal generated by the actions of both interacting parties; determining, according to the scalar feedback signal, whether the actual object intention of the object to be interacted with is satisfied; and In the case where the actual object intention of the object to be interacted with is not satisfied, the operation of determining the estimated object intention and the agent action is returned.

9. The method according to claim 8, characterized in that The method further comprises: When the actual object intention of the to-be-interacted object is satisfied, updating the weight of the at least one candidate execution action that matches the estimated object intention; Among them, the weight value corresponding to the agent action in at least one candidate execution action is increased.

10. The method according to claim 3, characterized in that The method further comprises: When the intention evaluation values ​​of the intentions of the target objects are all lower than a preset threshold, generating an intention query request; and The intention inquiry request is sent to the object to be interacted with, so that the object to be interacted with informs the intelligent agent to be trained of the actual object intention.

11. The method according to claim 1, characterized in that: The intelligent agent to be trained is embedded with a neural network model, and the neural network model includes an intention evaluation network and an action strategy network.

12. The method according to claim 11, characterized in that The step of determining the estimated object intention of the object to be interacted with according to the current environment state and the historical action sequence of the object to be interacted with comprises: Through the intention evaluation network, according to the current environment state and the historical action sequence of the object to be interacted with, the target object intention is selected from the preset intention pool as the estimated object intention of the object to be interacted with.

13. The method according to claim 11, characterized in that The step of determining an agent action that matches the estimated object intention based on the role behavior style of the agent to be trained includes: The action strategy network is used to determine the action of the agent to be executed according to the role behavior style of the agent to be trained and at least one preset candidate execution action that matches the estimated object intention.

14. An agent interaction method, characterized in that: The method comprises: Acquire the current environment state of the external environment and a historical action sequence of the object to be interacted with, wherein the historical action sequence includes at least one historical action of the object to be interacted with based on the current moment; Determining an estimated object intention of the object to be interacted with according to the current environment state and the historical action sequence of the object to be interacted with; Determining an agent action that matches the estimated object intention based on the agent's role behavior style, wherein the role behavior style indicates a strategic tendency of the agent when executing an action decision; Execute the agent action to realize the interaction process with the object to be interacted with.

15. An intelligent agent reinforcement learning device, characterized in that: include: An acquisition module, used to acquire the current state of the external environment based on the candidate time, wherein the external environment includes a preset rule base for obtaining output results according to input data; A first processing module, configured to determine an estimated object intention of the object to be interacted with according to the current environment state and a historical action sequence of the object to be interacted with, wherein the historical action sequence includes at least one historical action of the object to be interacted with based on the candidate moment; A second processing module is used to determine an agent action that matches the estimated object intention based on the role behavior style of the agent to be trained, wherein the role behavior style indicates the strategy tendency of the agent to be trained when executing action decision; A third processing module is used to execute the agent action and obtain an interactive response result based on the agent action, wherein the interactive response result includes a state transition result of the external environment and / or an actual interactive action of the object to be interacted with; as well as The fourth processing module is used to update the estimated object intention and the agent action according to the interaction response result to achieve reinforcement learning of the agent to be trained.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent agent reinforcement learning method described in any one of claims 1 to 13.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the intelligent agent reinforcement learning method described in any one of claims 1 to 13.

Images