Generation method and device of scene question and answer model of intelligent agent, equipment and medium

Abstract

The embodiment of the invention provides a method for generating a scene question and answer model of an agent. The method can be applied to the technical field of artificial intelligence. The method for generating the scene question and answer model of the intelligent agent comprises the steps of generating a scene question set related to a scene where the intelligent agent is located according to full scene information; acquiring scene recall information related to each scene problem in the scene problem set; based on the scene recall information, generating a matched scene answer for each scene question in the scene question set, and forming a scene answer set; and according to the scene question set and the matched scene answer set, training a preset question and answer model, and generating a scene question and answer model of the intelligent agent. The embodiment of the invention further provides a device and equipment for generating the scene question and answer model of the intelligent agent, a storage medium and a program product.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, specifically to the field of image processing technology, and more specifically to a method, device, equipment, medium and product for generating a scene question-answering model of an intelligent agent. Background Art

[0002] Artificial Intelligence (AI) is an important driving force for the new round of scientific and technological revolution and industrial transformation. It is a new key technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. As an important part of intelligent science, artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine (i.e., intelligent agent) that can respond in a similar way to human intelligence.

[0003] Artificial General Intelligence (AGI) refers to a general artificial intelligence entity that has efficient learning and generalization capabilities, can autonomously generate and complete tasks according to the complex dynamic environment in which it is located, has autonomous perception, cognition, decision-making, learning, execution and social collaboration capabilities, and conforms to human emotions, ethics and moral concepts. Therefore, one of the most important capabilities of general artificial intelligence is the ability to provide scenario-related question and answer needs for the scenario in which it is located.

[0004] However, the existing technology does not yet have the ability to realize scenario-based question-answering capabilities of intelligent agents. It is usually aimed at question-answering scenarios such as chatting or knowledge graphs. Moreover, the question-answering interaction capabilities of intelligent agents in these question-answering scenarios are also greatly limited, resulting in a low level of intelligence. Summary of the invention

[0005] In view of at least one of the above problems, the embodiments of the present invention aim to provide a method, apparatus, device, medium and product for generating a scenario question and answer model of an intelligent agent that can realize the generation of a scenario question and answer model, thereby significantly improving the intelligent agent's ability to interact in a timely and accurate manner with respect to scenario questions, enabling the intelligent agent to give more fluent and natural answers to scenario questions, ensuring that they are more in line with real human reactions, thereby achieving a higher level of intelligence.

[0006] One aspect of an embodiment of the present invention provides a method for generating a scenario question and answer model of an intelligent agent, which includes: generating a scenario question set related to the scenario in which the intelligent agent is located based on full scenario information; obtaining scenario recall information related to each scenario question in the scenario question set; based on the scenario recall information, generating a matching scenario answer for each scenario question in the scenario question set to form a scenario answer set; and training a preset question and answer model based on the scenario question set and the matching scenario answer set to generate a scenario question and answer model for the intelligent agent.

[0007] According to one embodiment of the present invention, before generating a scene problem set related to the scene in which the agent is located based on the full scene information, it also includes: obtaining the full scene information related to the scene in which the agent is located in response to the received model generation task instructions.

[0008] According to one embodiment of the present invention, in generating a scene problem set related to the scene in which the intelligent agent is located based on the full scene information, it includes: generating local scene information of each object with each object in the full scene information as the center; generating a local scene problem set of the local scene information of each object based on preset scene problem samples and preset problem generation rules, which are used to constitute the scene problem set.

[0009] According to one embodiment of the present invention, obtaining scene recall information related to each scene problem in a scene problem set includes: obtaining scene visual information related to each scene problem in the scene problem set; performing information recall on the full scene information based on the scene visual information to obtain scene recall information.

[0010] According to one embodiment of the present invention, a matching scenario answer is generated for each scenario question in a scenario question set based on scenario recall information to form a scenario answer set, including: extracting problem scenario information in the scenario recall information that matches each scenario question in the scenario question set; and reorganizing the problem scenario information according to preset answer generation rules to generate corresponding scenario answers.

[0011] According to one embodiment of the present invention, in training a preset question and answer model according to a scenario question set and a matching scenario answer set to generate a scenario question and answer model of an intelligent agent, it includes: generating a training data set that conforms to a preset data format according to the scenario question set and the matching scenario answer set; and inputting the training data set into the preset question and answer model according to preset training rules to generate a scenario question and answer model.

[0012] Another aspect of an embodiment of the present invention provides a device for generating a scenario question-answering model of an intelligent agent, which includes a question generation module, an information acquisition module, an answer generation module, and a model training module. The question generation module is used to generate a scenario question set related to the scenario in which the intelligent agent is located based on the full scenario information; the information acquisition module is used to obtain scenario recall information related to each scenario question in the scenario question set; the answer generation module is used to generate a matching scenario answer for each scenario question in the scenario question set based on the scenario recall information to form a scenario answer set; the model training module is used to perform training on a preset question-answering model based on the scenario question set and the matching scenario answer set to generate a scenario question-answering model of the intelligent agent.

[0013] Another aspect of an embodiment of the present invention provides an electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method for generating a scenario question and answer model of the above-mentioned intelligent agent.

[0014] Another aspect of an embodiment of the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method for generating a scenario question-answering model of the above-mentioned intelligent agent.

[0015] Another aspect of an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the method for generating a scenario question-answering model of the above-mentioned intelligent agent.

[0016] The method for generating a scene question-answering model of an intelligent agent provided by an embodiment of the present invention can at least partially solve the problem in the related art that the intelligent agent cannot well implement scene question-answering, resulting in a low level of intelligence, and thus can achieve at least one of the following technical effects: Compared with the prior art, in which the intelligent agent can only realize the question-answering capabilities of the chat and knowledge graph types, the scene question-answering model provided by the embodiment of the present invention can enable the intelligent agent to accurately and timely answer the questions about the scene content according to the scene in which it is located, so that the intelligent agent can achieve the scene question-answering capabilities that general artificial intelligence should have. Secondly, based on the scene question-answering model, the intelligent agent can be given the ability to understand and answer questions about scene information in a real world environment or a virtual world environment, including identifying and describing relevant information about visual objects and their relative relationship information, showing a wide range of applicability and flexibility, and helping to realize the cross-domain application development of the intelligent agent. In addition, only through the construction of the above-mentioned low-cost training data set, combined with the fine-tuning of the preset question-answering model, the capabilities of up to 90% of the GPT-4o model can be achieved under limited resources, that is, without relying on large-scale data and computing resources, it has achieved performance close to the current top language model. Moreover, based on the generation of the above-mentioned scene question-answering model, the intelligent agent can give a more fluent and natural answer to the scene question, ensuring that it is more in line with the real human reaction, thereby achieving a higher level of intelligence.

[0017] Therefore, the embodiments of the present invention provide an overall framework based on advanced training data construction and model fine-tuning strategies, and provide an efficient and scalable solution, thereby providing a new solution for improving scenario problem capabilities. Practical verification is sufficient to show that this technical solution has demonstrated extremely high practicality and innovation in promoting the development of general artificial intelligence, and has important commercial application value and scientific research value.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and illustrative and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which: Figure 1 A diagram schematically showing an application scenario of a method, apparatus, device, medium, and program product for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention; Figure 2 A flowchart schematically shows a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention; Figure 3 A flowchart of an application scenario of training data construction of a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown; Figure 4A flowchart of an application scenario generated by model training of a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown; Figure 5 A structural block diagram schematically shows a device for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention; and Figure 6 A block diagram of an electronic device suitable for implementing a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0020] The above-mentioned drawings are part of the specification of the embodiments of the present invention, which illustrate exemplary embodiments of the present invention. The attached drawings and the description of the specification are used together to illustrate the principles of the embodiments of the present invention. It should be understood that the above general description of the drawings and the following specific implementations are only exemplary and illustrative, and they cannot limit the scope of the present invention. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0022] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0023] The terms “first”, “second”, etc. used in the present invention do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0024] The directional terms used in the present invention, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0025] The words “include,” “including,” “have,” “contain,” etc. used in the present invention are open-ended terms, meaning including but not limited to.

[0026] The term "and / or" used in the present invention includes any or all combinations of the items mentioned.

[0027] Regarding the present invention, "plurality" includes "two" and "more than two"; regarding the present invention, "plurality of groups" includes "two groups" and "more than two groups".

[0028] The terms "substantially" and "approximately" used in the present invention are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0031] There are solutions in the prior art that can obtain dialogue / question-answering training data sets based on large language models (LLMs). For example, patent CN117033608B provides a knowledge graph generative question-answering method based on a large language model. It first generates a training set based on a large language model with stronger capabilities, and then fine-tunes the weaker / open source base model through various fine-tuning methods. After that, the data distillation method is used to allow the originally weaker base model to reach an approximate level of the stronger large language model in some aspects.

[0032] However, the existing technology does not involve the training scheme of large language models in the field of scenario question answering, but mostly involves question answering scenarios such as small talk or domain knowledge graphs. Moreover, such question answering scenarios also have the problem that the intelligence level is not high enough, resulting in poor user experience, and it is impossible to achieve relatively timely and accurate question answering capabilities. Moreover, the required language model usually requires high-cost data training and model fine-tuning, which also limits the application and development of this technology on intelligent agents.

[0033] In view of at least one of the above problems, the embodiments of the present invention are intended to enable the generation of a scenario question-answering model, thereby significantly improving the timely and accurate interaction capabilities of the intelligent agent for scenario questions and answers, and to enable the intelligent agent to give more fluent and natural answers to scenario questions, ensuring that they are more in line with real human reactions, thereby achieving a higher level of intelligence. To this end, the embodiments of the present invention propose for the first time a solution on how to build a scenario question-answering model with scenario question-answering capabilities, design a specific prompt template, and demonstrate the effect comparison of the final fine-tuning model.

[0034] One aspect of an embodiment of the present invention provides a method for generating a scenario question and answer model of an intelligent agent, which includes: generating a scenario question set related to the scenario in which the intelligent agent is located based on full scenario information; obtaining scenario recall information related to each scenario question in the scenario question set; based on the scenario recall information, generating a matching scenario answer for each scenario question in the scenario question set to form a scenario answer set; and training a preset question and answer model based on the scenario question set and the matching scenario answer set to generate a scenario question and answer model for the intelligent agent.

[0035] Figure 1 An application scenario diagram schematically illustrates a method, apparatus, device, medium, and program product for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention.

[0036] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0037] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0038] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0039] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0040] It should be noted that the method for generating the scene question and answer model of the intelligent agent provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the device for generating the scene question and answer model of the intelligent agent provided in the embodiment of the present invention can generally be set in the server 105. The method for generating the scene question and answer model of the intelligent agent provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the device for generating the scene question and answer model of the intelligent agent provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0041] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0042] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 A method for generating a scenario question-answering model of an intelligent agent in a disclosed embodiment is described in detail.

[0043] Figure 2 The flowchart of the method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0044] like Figure 2 As shown, one aspect of an embodiment of the present invention provides a method for generating a scenario question-answering model of an intelligent agent, which includes operations S201 to S204.

[0045] In operation S201, a scene problem set related to the scene in which the agent is located is generated according to the full scene information; In operation S202, scenario recall information related to each scenario question in the scenario question set is obtained; in operation S203, a matching scenario answer is generated for each scenario question in the scenario question set based on the scenario recall information to form a scenario answer set; and In operation S204, the preset question-answering model is trained according to the scenario question set and the matching scenario answer set to generate a scenario question-answering model of the intelligent agent.

[0046] The intelligent agent may be the execution subject of the method for generating the scenario question-answering model in the embodiment of the present invention, or may be an execution party controlled by the method for generating the scenario question-answering model, and may specifically be a humanoid intelligent robot or other AI device, which may usually have an actuator capable of completing specific action tasks. For example, a humanoid robot may use a mechanical manipulator to complete the action task of picking up an object.

[0047] In the embodiments of the present invention, the scene can be the real world where an agent is located, such as the agent is in the bedroom or living room of the user; in addition, the scene can also be the virtual world "imagined" or "located" by the agent, such as the agent implanting itself into the game environment world, or building a virtual world based on the received environment information. Among them, the objects and people existing in these real worlds or virtual worlds can constitute the components of the scene.

[0048] In addition, scene question and answer is that users ask questions based on the content of the above scene, and the intelligent agent can give reasonable responses to the user's questions by observing the scene information. Specifically, users can ask about the type, quantity, action, position of visual objects in the scene, and the relative relationship between multiple visual objects, etc., and the intelligent agent can give targeted responses based on the obtained scene content. Scene question and answer is one of the capabilities that general artificial intelligence must have, which requires the intelligent agent to be able to answer questions about the scene content of the scene it is in.

[0049] The full scene information is the information about all objects and the relationships between objects in the environment obtained by the intelligent agent based on the detection of the real environment or virtual environment, and also includes the intelligent agent's own state information.

[0050] Among them, all object information includes information such as objects, people, number of objects, types of objects, size of objects, functions of objects, spatial positions, etc. in the environment; information on the relationship between objects can include the spatial relationship between objects in the environment (such as a cup on a table) and the interaction between objects and people (such as a person standing in front of a table with a cup and a kettle pouring water), and can also include the interaction between people (such as a conversation between character A and task B), etc.; the agent's own state information can be the agent's own action execution state information detected at the current moment, such as the walking action of the agent in the motion state, the action information of the hand-waving action (such as the torque of the wrist, the angle of rotation, the estimated execution time, and the spatial position of each key point of the fingers, etc.) or the static state information (such as the spatial position, the current static posture, etc.).

[0051] The scene question set is a data set of questions related to the scene content generated by the agent based on the above-mentioned full scene information. These questions can reflect the agent's cognition and understanding of the current scene.

[0052] Each scenario question in the scenario question set can reflect part of the information in the scenario corresponding to the question, and the information related to the scenario question can be used as the above-mentioned scenario recall information. The scenario recall information can be the scenario information related to the scenario question obtained after extracting the full scene information according to the scenario question, and can be the information of part of the scene.

[0053] The scenario answer is the responsive content that matches the scenario question after extracting the scenario recall information based on the scenario question, and the scenario answer set is the data set composed of these scenario answers.

[0054] Each scenario question in the scenario question set can have a matching scenario answer in the scenario answer set, and the constituent matching relationship between these scenario questions and scenario answers can form training data based on the scenario question set and the scenario answer set. The preset question and answer model can be a common large language model (such as the Qwen model) provided in an embodiment of the present invention, which is used as a base model for generating the scenario question and answer model and does not need to rely on large-scale data and a large amount of computing resources. Therefore, by inputting the training data of the above-mentioned scenario question set and scenario answer set into the preset question and answer model, the generation of the scenario question and answer model of an embodiment of the present invention can be realized.

[0055] Therefore, compared with the prior art in which the intelligent agent can only realize the question-answering capabilities of the chat and knowledge graph types, the scene question-answering model provided by the embodiment of the present invention can enable the intelligent agent to accurately and timely answer questions about the scene content according to the scene in which it is located, so that the intelligent agent achieves the scene question-answering capabilities that general artificial intelligence should have. Secondly, based on the scene question-answering model, the intelligent agent can be given the ability to understand and answer questions about scene information in a real world environment or a virtual world environment, including identifying and describing relevant information about visual objects and their relative relationship information, showing a wide range of applicability and flexibility, and helping to realize the cross-domain application development of the intelligent agent. In addition, only through the construction of the above-mentioned low-cost training data set, combined with the fine-tuning of the preset question-answering model, without relying on large-scale data and computing resources, the performance close to the current top language model is achieved. Moreover, based on the generation of the above-mentioned scene question-answering model, the intelligent agent can give a more fluent and natural answer to the scene question, ensuring that it is more in line with the real human reaction, thereby achieving a higher level of intelligence.

[0056] In order to enable those skilled in the art to have a clearer understanding of the method for generating the scenario question-answering model of the above-mentioned intelligent agent in the embodiment of the present invention, the following is further provided: Figure 3-Figure 4 Description.

[0057] Figure 3 An application scenario flow chart of a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0058] In an embodiment of the present invention, the method for generating the above-mentioned scenario question-answering model may include two parts: construction of training data and training of a base model.

[0059] Among them, in the training data construction in the embodiment of the present invention, candidate questions (i.e., Query) can be generated for a given scenario first, and then based on the given scenario and the candidate questions, relevant information in the scenario is recalled, and then a matching answer (i.e., Answer) is generated.

[0060] Figure 3 An application scenario flow chart of a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0061] like Figure 2-Figure 3 As shown, according to one embodiment of the present invention, before operation S201 generates a scene problem set related to the scene where the agent is located according to the full scene information, it also includes: In response to the received model generation task instruction, full scene information related to the scene in which the agent is located is obtained.

[0062] The model generation task instruction may be a task execution instruction parameter generated in response to a user input action that requires model generation, and includes parameter content for controlling the agent to execute the generation of the above-mentioned scenario question-answering model, so that after receiving the instruction, the agent can perform environmental detection on the scene in which it is located to obtain the above-mentioned full scene information. In addition, the model generation task instruction may also be a task execution instruction parameter generated autonomously by the agent based on the current environment or the interaction scene with the user, etc. After generating the instruction, the agent can respond to the instruction and perform environmental detection on the scene in which it is located to obtain the above-mentioned full scene information.

[0063] This enables the agent to detect the environment more intelligently and efficiently and obtain the required full-scene information, such as Figure 3 Operation S301 is shown.

[0064] like Figure 2-Figure 3 As shown, according to one embodiment of the present invention, in operation S201, a scene problem set related to the scene where the agent is located is generated according to the full scene information, including: Generate local scene information of each object with each object in the full scene information as the center; According to the preset scene problem samples and preset problem generation rules, a local scene problem set of the local scene information of each object is generated to form a scene problem set.

[0065] The local scene information is the local scene data obtained by disassembling and extracting the local scene content of the full scene information, wherein the local scene data is related to the corresponding object. Specifically, each object mentioned in the full scene information can be confirmed, and then the corresponding local scene content can be extracted with each object as the center. For example, with the table as the center, all objects related to the table in front, back, top, bottom, left and right of the space where the table is located and the information on the relationship between the objects are extracted as the local scene content of the table; and then with another object other than the table as the center, the objects in the space around the object and the information on the relationship between the objects are obtained and extracted as another corresponding local scene content. These local scene contents can constitute the local scene information corresponding to the object.

[0066] The preset scenario question samples are pre-set scenario description information related to scenario questions and answers and question information designed based on the scene description information, which can be understood as sample data of "scene description (Scene) - generated question (Query)".

[0067] For example, the sample 1 of the preset scenario question can be: the input scenario description is "You are on the floor, and there is an open door and a red pen on your right". The corresponding output generation questions can include "What is on your right? What is on your right? Is there a door on your right? Is there a pen on your right? What is to the right of your right? Is the door on your right open or closed? What color is the pen on your right? How many pens are there on your right? Where is the pen? Where is the door? Is the door open? How many pens are there? Is there a red pen in the scene? Is there anything the same color as a tomato? Where is the door?" etc.

[0068] Sample 2 of the preset scenario question can be: the input scenario description is "The door is on your right, on the floor, and there are 1 plant, 2 blue toys, and 1 cupboard on your right". The corresponding output generation questions can include "Where is the door? Where is the door? What is on the right side of the door? How many plants are there on the right side of the door? How many toys are there on the right side of the door? What color are the toys? On which side of the door are the toys? Where is the cupboard on the door? How many toys are there?" etc.

[0069] Sample 3 of the preset scenario question can be: the input scenario description is "The storage box is in front of you, on the floor, and to its right there is a baby room and a blue cylindrical coffee table". The corresponding output generation questions can include "What color is the coffee table? What shape is the coffee table? What's on the coffee table? Where is the locker? Where is the storage box? Is there anything the same color as the tomato? What's to the right of the storage box? What's to the right of the storage box? Where is the baby room? What's in the baby room?" etc.

[0070] Therefore, the scene description input in the above-mentioned preset scene problem samples can be information such as the object name, color, position, etc. about the local scene information in the detected environment world, as well as other object information in front, behind, left, and right of the corresponding object. Therefore, the intelligent agent can generate matching scene questions for the local scene information corresponding to each input object based on the preset question generation rules by imitating the above-mentioned preset scene problem samples, thereby forming a scene problem set. Among them, the preset scene problem samples can be preset in the form of few-shot learning.

[0071] The preset question generation rules may be preset rules for generating matching scenario questions based on scenario description information. In an embodiment of the present invention, the preset question generation rules may include question generation requirements for question objects, questioning methods, question subjects, questioning scopes, and rule combinations. Among them, these question generation requirements may include the following: (1) Question object: The question object can be the name, color, shape, number, size, relative position, absolute position, function, etc. of the objects in the local scene.

[0072] (2) Questioning methods: There are at least two types of questioning methods. One is what / who / how, which is a sentence structure of what it is; the other is is / has, which is a sentence structure of “is it?”

[0073] (3) Question subject 1: The subject of a question can have a multi-hop relationship. For example, “What shape is the cup?” is a 1-hop question, “What can the object on the right side of the cup be used for?” is a 2-hop question, and “What is behind the object on the right side of the cup?” is a 3-hop question. Each question has a mutual relationship.

[0074] (4) Question subject 2: The subject of a question can be one, two or more. For example, “Which is bigger, the cup or the mouse on the table?” has two subjects, while “What shape is the cup on the table?” has one subject.

[0075] (5) Question scope: Questions need to be asked about all objects in the input scene description, and the generated questions must conform to Chinese expressions.

[0076] (6) Rule combination: For the generated questions, you can arbitrarily combine the rules (1) to (5) above to generate questions with different sentence structures as much as possible and cover all sentence structures as much as possible.

[0077] Therefore, according to the above-mentioned preset question generation rules, the scenario question and answer model can have more complex question-asking capabilities, which is more in line with the interactive habits of human scenario question and answering, making each scenario question in the generated scenario question set more natural and in line with the scenario needs.

[0078] Furthermore, the preset question generation model is fine-tuned according to the preset question generation rules, and the preset scene question samples are input into the preset question generation model as reference samples for scene question generation, so that a local scene question set corresponding to the local scene information of each object can be generated. These local scene question sets may include multiple questions (Queries) generated by the scene description of the local scene information of a single object, such as Figure 3 Operation S302 is shown. The above-mentioned preset question generation model can be implemented using a large language model such as GPT-4o.

[0079] Therefore, through the preset question generation model, each local scene in the whole scene can be traversed, so as to obtain the local scene question sets corresponding to all the local scenes of the whole scene, and these local scene question sets can be used as the components of the scene question set. Furthermore, the scene question answering model provided by the embodiment of the present invention can enable the intelligent agent to identify and describe the relevant information of the visual objects and their relative relationship information according to the scene in which it is located, and provide accurate and timely questionable questions about the scene content.

[0080] like Figure 2-Figure 3 As shown, according to an embodiment of the present invention, in operation S202, the scene recall information related to each scene problem in the scene problem set is obtained, including: Obtain scene visual information related to each scene problem in the scene problem set; Information recall is performed on the full scene information according to the scene visual information to obtain scene recall information.

[0081] The scene visual information may be the visual information within the scene that is related to the corresponding scene problem and described in natural language, including object description information related to the scene problem, such as the color, shape, position of the object and the natural language description content of the adjacent objects.

[0082] The scene recall information is the information related to the corresponding scene problem obtained by recalling the information related to the scene visual information in the full scene information based on the above scene visual information. It can cover more scene information related to the objects described by the scene visual information and the relationship between objects, such as Figure 3 Operation S303 is shown.

[0083] For example, if a scene question (Query) of the corresponding scene question set is "Which is greater, the number of sofas or the number of green plants in the scene?", then the corresponding scene visual information may include "the color, shape, number, position of the sofas and related visual information of nearby objects", "the color, shape, number, position of the green plants and related visual information of nearby objects". Accordingly, the scene recall information obtained by performing recall in the full scene information according to the above scene visual information and scene questions may include "the item information in the room now includes 2 sofas. sofa_1, on your left, is located in the living room. Its color is white and its volume is 3373529. In front of it are 1 green plant, 1 green storage box, 1 green closed trash can, 1 hanging painting, 1 paper ball, and 3 yellow building blocks. On its right are 3 green plants, 1 green open door, 1 green closed trash can, and 1 hanging painting. On its left are 2 green plants, and behind it are 1 hanging painting. Its surroundings are divided into There is 1 pink clothing, 1 white sofa, 1 yellow bookshelf, 1 yellow carpet, sofa_0, on your left, located on the ground, its color is white, its volume is 1768717, there is 1 pink pillow inside it, there are 3 green plants on its right, 1 green open door, 1 green closed trash can, there are 3 green plants on its left, 1 green storage box, 1 paper ball, there is 1 green closed trash can, 2 paintings, 1 yellow painting, 1 mirror in front of it, there are 1 living room, 1 white sofa, 1 dining room around it, there is 1 painting behind it", etc.

[0084] Therefore, by summarizing the scene visual information related to the scene question with the help of the above-mentioned scene recall information, the finally generated scene answer can be guaranteed to match the scene question, while also providing a more complex scene answer that conforms to human response habits.

[0085] like Figure 2-Figure 3 As shown, according to an embodiment of the present invention, in operation S203, based on the scenario recall information, a matching scenario answer is generated for each scenario question in the scenario question set to form a scenario answer set, including: Extracting problem scenario information matching each scenario question in the scenario question set from the scenario recall information; The question scenario information is reorganized according to the preset answer generation rules to generate the corresponding scenario answer.

[0086] like Figure 3 Operation S304 is shown. In the embodiment of the present invention, for each generated scenario question, a corresponding scenario answer (Answer) is matched based on the scenario recall information corresponding to the scenario question.

[0087] Specifically, the problem scene information is descriptive information directly related to the problem content of a scene problem extracted from the matching scene recall information. For example, if the problem content of the scene problem involves location and objects (such as what color is the potted plant on the ground?), the problem scene information may include the ground, the potted plant, and the location and color of the potted plant.

[0088] Specifically, the process of extracting scene information for this question can be understood as the thinking and statistical process of the intelligent agent. According to the scene question to be answered and the natural language description of the scene recall information corresponding to the scene question, the required information is extracted by relying on some preset extraction rules. The preset extraction rules can limit the requirements or scope of information extraction, such as "an object is in front of, behind, left or right of the table, then the object will not be on the table", "the concept of shape is not a type, it should be a cuboid, cylinder, etc." The constraints of information extraction reflected by this can reflect the "thinking" process (i.e. Prompt_Thought) in the process of generating the answer to the scene.

[0089] The preset answer generation rule may be a preset rule for generating a scenario answer matching the scenario question according to the above-mentioned question scenario information. In an embodiment of the present invention, the preset answer generation rule may include answer generation requirements such as answering person, answering method, answering restrictions, etc. These answer generation requirements may include the following: (1) Answering person: Questions should be answered in the first person, using a child's tone.

[0090] (2) Answering method: Answers should be brief, fluent, and natural, and should conform to the logic of normal human speech and expression.

[0091] (3) Answer restriction 1: The answer should be generated based on the thinking process of the input, without adding other information or making additional explanations.

[0092] (4) Answer restriction 2: Do not include the specific item number in the answer, but you can include the item name. For example, do not include toy_01, but you can include "toy car".

[0093] Therefore, according to the above-mentioned preset answer generation rules, the ability to think about answers generated based on understanding can be demonstrated, so that the scenario question and answer model has a more complex answer effect, and at the same time is more in line with the interactive habits of human scenario question and answer, making each scenario answer in the generated scenario answer set more natural and in line with the scenario needs.

[0094] Furthermore, the preset answer generation model is fine-tuned according to the preset answer generation rules, and the problem scenario information is used as input data of the preset answer generation model, so that the preset answer generation model can reorganize the problem scenario information according to the preset answer generation rules to generate a scenario answer (Answer) that matches the corresponding scenario question.

[0095] Among them, the above-mentioned preset answer generation model can be the aforementioned preset question generation model, such as a large language model such as GPT-4o, which can provide a two-stage answer generation method based on COT (Chain of Thought), first extracting problem scenario information related to the scenario question from the relevant scenario recall information, and then generating the final matching scenario answer based on the "thinking" process for the above-mentioned problem scenario information, namely Prompt_Final.

[0096] Specifically, the process of generating answers to the above scenarios can be further illustrated by the following examples.

[0097] First, set a scene judgment message System_Msg, such as "SYS_MSG = "You are an AI visual assistant, and you are in a home scene. The surrounding environment information you see is described by some sentences. You will see multiple objects, and you can see the location, color, size of each object, as well as what objects are in front of, behind, left, right, top, bottom, and inside the object. When you answer questions, assume that you are in a real environment. " Therefore, the corresponding scenario questions (queries) can be generated as "What color is the potted plant on the ground? Where is the sofa?" According to the above scenario question, the corresponding scenario recall information (Scene) can be obtained as "The current item information in the room includes 2 sofas. Sofa_1 is in front of you, located in the living room, its volume is 1995324, behind it there are 1 pink plant, 6 green plants, 1 pink rag, 1 blue rag, 1 pink plate, around it there are 1 sofa, 1 paper ball, in front of it there are 1 green photo, 1 green painting, 1 pink painting, 1 green building block, 3 yellow paintings, 1 blue building block, 1 silver flower, and 1 green picture. "Sprinkle, 1 green chair, 1 pink toy, sofa_0, in front of you, on the ground, its volume is 1344120, it has 1 toy, 1 yellow cylindrical coffee table, behind it there are 6 green plants, 1 pink rag, 1 pink plate, to the right of it there are 1 pink plant, 1 pink painting, around it there are 1 sofa, 1 yellow carpet, 1 toy, in front of it there are 1 green photo, 1 green painting, 1 green building block, 1 blue building block, 1 silver shower head, 1 green chair" Furthermore, the relevant problem scenario information can be extracted (i.e., thought) from the above scenario recall information as follows: 1. **Sofa location**: - `sofa_0` On the ground, in front there are green photos, green paintings, green building blocks, blue building blocks, silver shower heads, and green chairs.

[0098] - `sofa_1` In the living room, there are green photos, green paintings, pink paintings, green building blocks, yellow paintings, blue building blocks, silver shower heads, green chairs, and pink toys in front.

[0099] 2. **Color of potted plants**: - There is a pink plant to the right of `sofa_0`.

[0100] - Behind `sofa_1` there is 1 pink plant and 6 green plants.

[0101] Based on the above question scenario information, the preset answer generation model can be used to fine-tune the preset answer generation rules to reorganize the information, and the scenario answers corresponding to the above scenario questions are obtained as follows: - Color of potted plants on the ground: pink.

[0102] - Location of sofas: `sofa_0` is on the floor, `sofa_1` is in the living room.

[0103] Therefore, the intelligent agent can accurately and promptly answer questions about the scene content according to the scene it is in, so that the intelligent agent has the scene question and answer capability that general artificial intelligence should have. These scene answers constitute the scene answer set, which can enable the intelligent agent to give more fluent and natural answers to scene questions, ensuring that they are more in line with real human reactions, thereby achieving a higher level of intelligence.

[0104] Figure 4 A flowchart of another application scenario of the method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0105] like Figure 2-Figure 4 As shown, according to one embodiment of the present invention, in operation S204, the preset question-answering model is trained according to the scenario question set and the matching scenario answer set to generate the scenario question-answering model of the agent, including: Generate a training data set that complies with a preset data format based on a scenario question set and a matching scenario answer set; The training data set is input into the preset question-answering model according to the preset training rules to generate a scenario question-answering model.

[0106] like Figure 4 As shown, the training data 401 may include training data sets corresponding to the scenario question set and the scenario answer set. The preset data format is a data training format for arranging the scenario question set and the scenario answer set, which can be designed according to the training framework of the preset question-answering model. For example, the input data of the large language model Qwen model can correspond to the alpaca format, and the data definition of the scenario question set and the scenario answer set can be performed in the instruct-output structure to generate the corresponding training data set.

[0107] The training data set may be a scenario question-answering data set defined in the above-mentioned preset data format, which may be used as the designated training data of the preset question-answering model to implement data training of the preset question-answering model to generate the required scenario question-answering model. Specifically, the training data set may be represented by the following data example 1 and data example 2: Data Example 1: { "system": "Receive input from others, combine [common sense knowledge], [scenario information], and [memory] to generate a fluent response to the user's question.", "instruction": "[Common sense knowledge]: \tThe foreign name of computer is computer\tThe Chinese name of computer is computer\tThe alias of computer is computer\tThe components of computer belong to computer\n[Scene information]: The information of items in the room now includes: There is no corresponding object in the scene.\n[Memory]:\nQuestion: What is a computer?\n", "input": "", "output": "A computer is an electronic device that receives information in digital form and performs a series of operations based on predetermined instructions to produce output results." } Data Example 2: { "system": "Receive input from others, combine [common sense knowledge], [scenario information], and [memory] to generate a fluent response to the user's question.", "instruction": "[Common sense]: The explanation of bedside cupboard is: bedside cupboard is a small cabinet set on both sides of the bed in modern furniture, which can be used to store sundries. The shape is similar to the common bedside cabinets in modern times. Foreign name: Bedside cupboard, pronunciation: chuang'tou'gui\tThe bedside book is suitable for the bedside\tThere is a white-faced mother-in-law at the head of the bed. The opening is 16 opening\n[Scene information]: The information of items in the room now includes: 10 books: book_3, on your left, on the desk, its volume is 1343, behind it there are 2 bedside tables, 4 books, 1 cabinet, and on its right there is 1 cupboard, \tbook_1, behind you, in the bookshelf, its volume is 1445, there are 2 books around it, there is 1 book behind it, there are 2 bedside tables on its left, and there are 3 books in front of it, \tbook_4, behind you, in the bookshelf, its color is blue, its volume is 731, there are 3 books around it, and behind it there are There are 2 books and 2 bedside tables on its left. \tbook_5, behind you, in the bookshelf, its color is blue, its volume is 724, there are 3 books around it, there are 2 books behind it, and there are 2 bedside tables on its left. \tbook_6, behind you, in the bookshelf, its color is blue, its volume is 737, there are 3 books around it, there are 2 books behind it, and there are 2 bedside tables on its left. \tbook_7, behind you, in the bookshelf, its color is blue, its volume is 7 36, it has 3 books around it, 2 books behind it, and 2 bedside tables on its left, \tbook_8, behind you, in the bookshelf, its color is blue, its volume is 716, it has 3 books around it, 2 books behind it, and 2 bedside tables on its left, \tbook_9, behind you, in the bookshelf, its volume is 2857, it has 2 books behind it, 2 bedside tables on its left, and 2 books in front of it, \tbook_0, behind you, on the desk, it The volume of is 875, there are 3 books behind it, and 2 bedside tables on its left, \tbook_2, behind you, in the bookshelf, its volume is 1447, there are 2 bedside tables on its left, and 4 books in front of it, \n[Memory]: 4 days 7 hours 17 minutes ago, the book was not on the bedside table in the bedroom, the book was in the yellow TV cabinet\t4 days 7 hours 17 minutes ago, the book was not on the bedside table in the bedroom, the book was in the yellow TV cabinet\t4 days 7 hours 17 minutes ago, the book was not on the bedside table in the bedroom, the book was in the bedroom,The book is in the yellow TV cabinet\t4 days, 7 hours, and 17 minutes ago, the book was not on the bedside table in the bedroom. The book was in the yellow TV cabinet\t4 days, 7 hours, and 17 minutes ago, the book was not on the bedside table in the bedroom. The book was in the yellow TV cabinet\t4 days, 7 hours, and 17 minutes ago, the book was not on the bedside table in the bedroom. The book was in the yellow TV cabinet\t4 days, 7 hours, and 17 minutes ago, the book was not on the bedside table in the bedroom. The book was in the bedroom. The book was in the yellow TV cabinet\ t4 days, 7 hours, and 17 minutes ago, the book is no longer on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t4 days, 6 hours, and 42 minutes ago, the book is no longer on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t4 days, 6 hours, and 42 minutes ago, the book is no longer on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t4 days, 6 hours, and 42 minutes ago, the book is no longer on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t4 days, 6 hours, and 42 minutes ago, The book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t2 days 6 hours and 30 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t2 days 6 hours and 30 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t2 days 6 hours and 30 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t2 days 6 hours and 30 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet.\t2 days 26 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet\t2 days and 26 minutes ago, the book is not on the bedside table in the bedroom. The book is in the yellow TV cabinet\t2 days and 26 minutes ago, the book is not on the bedside table in the bedroom. The book is in the bedroom. The book is in the yellow TV cabinet\t2 days and 26 minutes ago, the book is not on the bedside table in the bedroom. The book is in the bedroom. The book is in the yellow TV cabinet\t1 hour and 52 minutes ago, the book is not in the bedroom. The book is on the bedside table in the bedroom\nQuestion: How many books are there on the right side of the bedside cabinet? \n", "input": "", "output": "There are 9 books on the right side of the bedside table." } The preset training rules may be model fine-tuning rules that constrain the data training of the preset question-answering model, such as Lora training rules, wherein the preset training rules may also be set according to the training framework of the preset question-answering model. Figure 4The model 402 shown in the figure, the preset question-answering model can be a common large language model (such as the Qwen model) provided in the embodiment of the present invention, which is used as the base model for generating the scenario question-answering model, and does not need to rely on large-scale data and a large amount of computing resources. For example, the Qwen2.5 14b model can be trained with the above training data set based on the preset training rules of the Lora training rules to generate the scenario question-answering model of Lora-qwen2.5-14b (such as Figure 4 Model 403 is shown), which involves training hyperparameters lr=5e-4 and epoch=3.

[0108] Specifically, a training data set that conforms to a preset data format can be generated based on the aforementioned scenario question set and scenario answer set, and the training data set may include more scenario questions and data samples that match scenario answers. Taking the 4700 data samples of the above training data set as an example, on the same evaluation set, using the same evaluation criteria, after evaluating GPT4o-few-shot model 1, GPT4o-cot model 2, and Lora-qwen2.5-14b model 3, the reasonable rate of the following model question-answering evaluation can be obtained as shown in Table 1:

[0109] Among them, model 1 is a method of generating answers by prompting GPT-4o once in a few-shot manner, model 2 is a method of generating training data in an embodiment of the present invention, and model 3 is a model effect after fine-tuning based on the training data lora. According to the content shown in Table 1, it can be seen that the effect of the fine-tuned model is about 90% of the ability of model 2, and is better than the effect of model 1.

[0110] In other words, the method for generating the above-mentioned scenario question-answering model in an embodiment of the present invention can achieve more than 90% of the interactive capabilities of the GPT-4o model by only constructing and fine-tuning low-cost training data.

[0111] In summary, the method for generating a scene question-answering model of an intelligent agent provided by an embodiment of the present invention can at least partially solve the problem of low intelligence level caused by the inability of intelligent agents to well implement scene question-answering in related technologies, and thus can achieve at least one of the following technical effects: (1) Higher scene question-answering capabilities: The method for generating a scene question-answering model of an intelligent agent provided by the above embodiments of the present invention can give the intelligent agent the ability to understand and answer questions about scene information in a real or virtual environment. This includes identifying and describing the type, quantity, action, position of visual objects, and the relative relationship between objects. This ability is an important component of developing general artificial intelligence.

[0112] (2) Efficient training method: By constructing low-cost training data and combining it with fine-tuning of large language models, the processing capacity of the GPT-4o model is up to 90% under limited resources. This means that the technology can achieve performance close to that of the current top language models without relying on large-scale data and computing resources.

[0113] (3) Universality: It is applicable not only to real-world scenarios but also to virtual world environments, showing wide applicability and flexibility, which is especially important for developing intelligent agents for cross-domain applications.

[0114] (4) Innovative framework design: The overall framework adopts advanced training data construction and model fine-tuning strategies, combined with the advantages of large language models, to provide an efficient and scalable solution, providing new ideas for improving scenario question-answering capabilities. These highlights show that this technical solution has important commercial value and scientific research value in promoting the development of general artificial intelligence.

[0115] Therefore, the above-mentioned scenario question and answer model trained by the method for generating the scenario question and answer model of the intelligent agent provided by the embodiment of the present invention can give the embodied intelligent agent the ability to understand and answer questions about scenario information in a real or virtual environment, so that it has a more fluent and natural question and answer interaction level, conforms to the interaction habits of humans in normal question and answer scenarios, and has a higher level of intelligence that is more in line with human reality.

[0116] Based on the above-mentioned method for generating a scene question-answering model of an intelligent agent, the present invention also provides a device for generating a scene question-answering model of an intelligent agent. Figure 5 The device is described in detail.

[0117] Figure 5 The structure block diagram of the device for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0118] like Figure 5 As shown, the generation device 500 of the scene question and answer model of the intelligent agent in this embodiment includes a question generation module 510, an information acquisition module 520, an answer generation module 530 and a model training module 540.

[0119] The question generation module 510 is used to generate a scene question set related to the scene the agent is in according to the full scene information. In one embodiment, the question generation module 510 can be used to perform the operation S201 described above, which will not be described in detail here.

[0120] The information acquisition module 520 is used to acquire the scene recall information related to each scene problem in the scene problem set. In one embodiment, the information acquisition module 520 can be used to perform the operation S202 described above, which will not be repeated here.

[0121] The answer generation module 530 is used to generate a matching scenario answer for each scenario question in the scenario question set based on the scenario recall information to form a scenario answer set. In one embodiment, the answer generation module 530 can be used to perform the operation S203 described above, which will not be repeated here.

[0122] The model training module 540 is used to perform training on the preset question-answering model according to the scenario question set and the matching scenario answer set to generate a scenario question-answering model of the agent. In one embodiment, the model training module 540 can be used to perform the operation S202 described above, which will not be repeated here.

[0123] According to an embodiment of the present invention, any multiple modules of the question generation module 510, the information acquisition module 520, the answer generation module 530 and the model training module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the question generation module 510, the information acquisition module 520, the answer generation module 530 and the model training module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the question generation module 510, the information acquisition module 520, the answer generation module 530 and the model training module 540 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.

[0124] Figure 6 A block diagram of an electronic device suitable for implementing a method for generating a scenario question-answering model of an intelligent agent according to an embodiment of the present invention is schematically shown.

[0125] The above-mentioned electronic device provided by an embodiment of the present invention includes one or more processors and a memory, and the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method for generating the scenario question and answer model of the above-mentioned intelligent agent.

[0126] like Figure 6As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0127] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0128] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0129] The present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method for generating the scenario question-answering model of the above-mentioned intelligent agent.

[0130] The computer-readable storage medium may be included in the device / apparatus / system described in the above embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0131] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0132] An embodiment of the present invention also includes a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the method for generating the scenario question and answer model of the above-mentioned intelligent agent.

[0133] The computer program includes program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0134] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0135] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0136] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0137] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0138] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.

[0140] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for generating a scenario question-answering model of an intelligent agent, characterized in that: include: Generate a scenario problem set related to the scenario in which the agent is located based on the full scenario information; Obtaining scenario recall information related to each scenario question in the scenario question set; Based on the scenario recall information, generating a matching scenario answer for each scenario question in the scenario question set to form a scenario answer set; as well as The preset question-answering model is trained according to the scenario question set and the matching scenario answer set to generate the scenario question-answering model of the agent.

2. The method according to claim 1, characterized in that Before generating a scene problem set related to the scene where the agent is located according to the full scene information, the method further includes: In response to the received model generation task instruction, the full scene information related to the scene where the agent is located is obtained.

3. The method according to claim 1, characterized in that In said generating a scene problem set related to the scene where the agent is located according to the full scene information, it includes: Generate local scene information of each object with each object in the global scene information as the center; According to the preset scene problem samples and the preset problem generation rules, a local scene problem set of the local scene information of each object is generated to constitute the scene problem set.

4. The method according to claim 1, characterized in that: The step of obtaining the scene recall information related to each scene problem in the scene problem set includes: Acquire scene visual information related to each scene problem in the scene problem set; Information recall is performed on the full scene information according to the scene visual information to obtain the scene recall information.

5. The method according to claim 1, characterized in that In the step of generating a matching scenario answer for each scenario question in the scenario question set based on the scenario recall information to form a scenario answer set, the step includes: Extracting problem scenario information matching each scenario problem in the scenario problem set from the scenario recall information; The question scenario information is reorganized according to preset answer generation rules to generate corresponding scenario answers.

6. The method according to claim 1, characterized in that In the step of training a preset question-answering model according to the scenario question set and the matching scenario answer set to generate the scenario question-answering model of the agent, the step includes: Generate a training data set in a preset data format according to the scenario question set and the matching scenario answer set; The training data set is input into the preset question-answering model according to preset training rules to generate the scenario question-answering model.

7. A device for generating a scene question-answering model of an intelligent agent, characterized in that: include: A question generation module, used to generate a scenario question set related to the scenario in which the agent is located based on the full scenario information; An information acquisition module, used to acquire scene recall information related to each scene problem in the scene problem set; An answer generation module, configured to generate a matching scenario answer for each scenario question in the scenario question set based on the scenario recall information to form a scenario answer set; as well as The model training module is used to perform training on a preset question-answering model according to the scenario question set and the matching scenario answer set to generate a scenario question-answering model of the agent.

8. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images