Robot training scene generation method, device, equipment, medium and program

Abstract

The invention relates to the technical field of scene generation, and discloses a robot training scene generation method and device, equipment, a medium and a program, and the method comprises the steps: obtaining a robot training scene demand, and converting the robot training scene demand into text information; performing information extraction on the text information to obtain structured information conforming to a preset text format and attribute information containing text keywords in the text information; constructing a scene generation program according to the structured information and the attribute information; and performing simulation based on the scene generation program to generate a robot training scene. According to the method, simulation is carried out by utilizing the scene generation program, the robot training scene is automatically generated, the software development workload can be reduced, the working efficiency is improved, manual understanding and execution deviations do not exist, and the consistency standard of scene construction can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of scene generation, and in particular to a method, device, equipment, medium and program for generating a robot training scene. Background Art

[0002] With the continuous development of science and technology, robotics technology has become an indispensable part of modern high-tech industries and daily life, and is widely used in manufacturing, medical industry, agriculture, service industry and other fields.

[0003] Tasks such as autonomous obstacle avoidance, human-machine collaboration, and anthropomorphic operations place higher demands on robot technology. This requires the establishment of a large number of scenarios for training and testing robots, but complex environments and conditions such as extreme weather, different terrains, and traffic flow are difficult to achieve or replicate in the real world, making the construction of virtual training scenarios the main way to train and test robots.

[0004] At present, engineers still mainly build virtual training scenes through programming. However, the construction of large-scale three-dimensional virtual training scenes for robots requires a lot of software development work. The above traditional methods easily lead to low efficiency in the construction of three-dimensional virtual scenes. In addition, the manually constructed training scenes may have human understanding and execution deviations, and consistency standards cannot be guaranteed. Summary of the invention

[0005] In view of this, the present invention provides a robot training scene generation method, device, equipment, medium and program to solve the problem that traditional manual construction methods easily lead to low efficiency in constructing three-dimensional virtual scenes and fail to meet scene consistency standards.

[0006] In a first aspect, the present invention provides a method for generating a robot training scene, the method comprising:

[0007] Obtain robot training scenario requirements and convert them into text information;

[0008] Extracting information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information;

[0009] Construct a scene generation program based on structured information and attribute information;

[0010] Simulate based on the scenario generation program to generate robot training scenarios.

[0011] Beneficial effects: The robot training scene requirements input by the user are converted into text information, and then the text information is extracted to obtain structured information that conforms to the preset text format and attribute information containing text keywords in the text information to build a scene generation program, so as to use the scene generation program for simulation and automatically generate robot training scenes, which can reduce the workload of software development and improve work efficiency. There is no human understanding and execution deviation, and the consistency standard of scene construction can be guaranteed.

[0012] In an optional implementation, a scene generation program is constructed according to the structured information and the attribute information, including:

[0013] According to the structured information and attribute information, the target task information is obtained;

[0014] and / or, replacing text keywords in the attribute information to obtain replacement information, and converting the replacement information into exploration task information based on the structured information;

[0015] Based on the target task information and / or the exploration task information, a scenario generation program is constructed.

[0016] Beneficial effects: According to the structured information and attribute information corresponding to the text information, the scenario generation program is automatically written to build the robot training scenario, reducing the workload of software development and improving work efficiency. Through similar retrieval, the same attribute words or phrases in the user's text information are replaced to facilitate the generation of a large number of other similar robot training scenarios, which can cover a wide range of training conditions and possibilities.

[0017] In an optional implementation, replacing the text keywords in the attribute information to obtain replacement information includes:

[0018] Convert attribute information into vectorized attribute information;

[0019] Perform similarity search based on vectorized attribute information to obtain similar keywords;

[0020] The text keywords in the attribute information are replaced with similar keywords to obtain replacement information; wherein the text keywords include at least one of a scene entity, an interactive scene entity, and scene configuration information.

[0021] Beneficial effect: By performing similarity search on attribute information, the obtained replacement information can be used to replace words or phrases with the same attributes in text information, thereby expanding and enriching the text information.

[0022] In an optional implementation, a scenario generation program is constructed based on the target task information and / or the exploration task information, including:

[0023] According to the target task information and / or the exploration task information, a plurality of sub-target tasks and / or a plurality of sub-exploration tasks are obtained;

[0024] Identify the scene type corresponding to each sub-target task and / or each sub-exploration task; wherein the scene type includes at least one of an indoor environment, an outdoor environment, an industrial environment, a static interaction scene, and a dynamic interaction scene;

[0025] Based on the scenario type, each sub-goal task and / or each sub-exploration task is sent to a corresponding preset scenario construction program to obtain a scenario code output by each preset scenario construction program; wherein the type of the scenario construction program corresponds to the scenario type one by one;

[0026] The scene codes output by each preset scene construction program are integrated to construct a scene generation program.

[0027] Beneficial effects: Identify the scenario types of subtasks contained in the target task information and / or exploration task information, use the corresponding preset scenario construction program to generate scenario code, thereby improving program generation efficiency; and use the exploration task information to replace and expand the scene, thereby improving the richness of the scene.

[0028] In an optional implementation, the simulation is performed based on the scenario generation program to generate a robot training scenario, including:

[0029] Convert scenario generation procedures into simulation information;

[0030] According to the preset simulation model and simulation information, the robot training scene is simulated and generated.

[0031] Beneficial effects: By converting the scene generation program into simulation information and using the preset simulation model to simulate and generate the robot training scene, the scene construction is automatically completed without human understanding and execution deviation, which can ensure consistency standards and strong applicability.

[0032] In an optional implementation, the robot training scenario requirement includes voice requirement information and / or text requirement information; converting the robot training scenario requirement into text information includes:

[0033] Perform voice-to-text conversion on the voice demand information to obtain text information;

[0034] And / or, intercept the text demand information to obtain text information.

[0035] Beneficial effects: The user's intention is obtained through a variety of ways to obtain the robot training scenario requirements with greater flexibility, and it is converted into text information to facilitate the construction of the scenario generation program.

[0036] In an optional embodiment, the method further includes:

[0037] Displays the robot training scene.

[0038] Beneficial effect: By displaying the robot training scene and other scene information, it is convenient for users to view, modify and use it.

[0039] In a second aspect, the present invention provides a robot training scene generation system, the system comprising:

[0040] An input-output unit, used to obtain robot training scenario requirements and convert the robot training scenario requirements into text information;

[0041] The intelligent unit is used to extract information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information; and to construct a scene generation program based on the structured information and the attribute information;

[0042] The simulation unit is used to perform simulation based on the scenario generation program to generate robot training scenarios.

[0043] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the robot training scene generation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the robot training scene generation method of the first aspect or any corresponding embodiment thereof.

[0045] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the robot training scene generation method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a structural block diagram of a robot training scenario generation system according to an embodiment of the present invention;

[0048] Figure 2 is a structural block diagram of another robot training scene generation system according to an embodiment of the present invention;

[0049] Figure 3 is a workflow diagram of a robot training scenario generation system according to an embodiment of the present invention;

[0050] Figure 4 is a schematic diagram of the structure of an intelligent unit according to an embodiment of the present invention;

[0051] Figure 5 is a schematic flow chart of a method for generating a robot training scene according to an embodiment of the present invention;

[0052] Figure 6 is a flowchart of another method for generating a robot training scene according to an embodiment of the present invention;

[0053] Figure 7 is a flowchart of another method for generating a robot training scene according to an embodiment of the present invention;

[0054] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] With the continuous development of science and technology, robotics technology has become an indispensable part of modern high-tech industries and daily life, and is widely used in manufacturing, medical industry, agriculture, service industry and other fields. However, tasks such as autonomous obstacle avoidance, human-machine collaboration, and anthropomorphic operations have put forward higher requirements for robotics technology. This requires the establishment of a large number of scenarios for training and testing robots, but complex environments and conditions such as extreme weather, different terrains, and traffic flow are difficult to achieve or replicate in the real world. This makes the construction of virtual training scenarios the main way to train and test robots.

[0057] However, the traditional method of engineers building virtual training scenes through programming generally has the following problems: 1) The construction of large-scale robot three-dimensional virtual training scenes requires a lot of software development work, resulting in low efficiency in the construction of three-dimensional virtual scenes; 2) Manually built training scenes are relatively simple and cannot cover a wide range of training conditions and possibilities; 3) Manually built training scenes may have deviations in human understanding and execution, and consistency standards cannot be guaranteed.

[0058] Therefore, the embodiment of the present invention provides a robot training scene generation solution, which automatically writes a scene generation program according to the user's intention, completes the robot training scene construction, reduces the workload of software development, improves work efficiency, and does not have human understanding and execution deviations, and can ensure consistency standards. At the same time, the same attribute words or phrases in the text information are replaced to generate a large number of other similar robot training scenes, which can cover a wide range of training conditions and possibilities.

[0059] According to an embodiment of the present invention, a robot training scene generation system 100 is provided for automatically generating a robot training scene, such as Figure 1 As shown, the system mainly includes: an intelligent unit 110 , an input-output unit 120 and a simulation unit 140 .

[0060] Specifically, the input-output unit 120 is used to obtain the robot training scenario requirements and convert the robot training scenario requirements into text information;

[0061] The intelligent unit 110 is used to extract information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information; and to construct a scene generation program based on the structured information and the attribute information;

[0062] The simulation unit 140 is used to perform simulation based on the scenario generation program to generate a robot training scenario.

[0063] In some optional embodiments, see again Figure 1 The above system also includes a retrieval unit 130, which is used to replace the text keywords in the attribute information to obtain replacement information.

[0064] Specifically, the intelligent unit 110 is used to receive text information sent by the input and output unit 120, send attribute information to the search unit 130 and receive replacement information output by the search unit, and send a scene generation program to the simulation unit 140. The text information is information composed of words, numbers and punctuation marks formed by the user through the input and output unit 120 according to his own intention.

[0065] Specifically, the input-output unit 120 is used to send text information to the intelligent unit 110 , and is also used to receive scene information sent by the simulation unit 140 .

[0066] Specifically, the retrieval unit 130 is used to receive the attribute information sent by the intelligent unit 110, perform similarity retrieval on the attribute information and the internal information of the knowledge base, retrieve words or phrases similar to the attribute information, convert them into replacement information, and send them to the intelligent unit 110. The attribute information is words or phrases such as time, scene entity, dynamic interaction entity, static interaction entity, etc. that can be replaced in the text information.

[0067] Specifically, the simulation unit 140 is used to receive the scene generation program sent by the smart unit 110 and generate simulation information and scene information, and is used to send the scene information to the input and output unit 120. The simulation information refers to all data and programs in scene generation and use, and the scene information refers to all data and programs that can be displayed through the input and output unit 120.

[0068] In some optional embodiments, such as Figure 2 As shown, the input-output unit 120 includes an input module 121 and an output module 122. Figure 3 As shown, the input module 121 is used to convert the user's intention into text information, and the output module 122 is used to display the scene information sent by the simulation unit 140.

[0069] Exemplarily, the input module 121 is a tool such as a keyboard or a voice recognition module that can convert the user's intention into text information, and the output module 122 is a tool such as a display screen or a three-dimensional projection that can allow the user to monitor and view scene information.

[0070] In some optional embodiments, such as Figure 4 As shown, the intelligent unit 110 includes a large language model 111, a task allocation agent 112, an indoor environment agent 113, an outdoor environment agent 114, an industrial environment agent 115, a static interaction agent 116, a dynamic interaction agent 117, an information replacement agent 118, and a code integration agent 119. Specifically, the large language model 111 is used to convert the text information sent by the input and output unit 120 into structured information and attribute information and send it to the task allocation agent 112.

[0071] Specifically, see again Figure 3The task allocation agent 112 is used to receive the structured information and attribute information sent by the large language model and convert them into target task information, to receive the replacement information sent by the retrieval unit 130 and convert it into exploration task information, and to distribute the target task information and exploration task information to one or more of the indoor environment agent 113, the outdoor environment agent 114, the industrial environment agent 115, the static interaction agent 116, the dynamic interaction agent 117, the information replacement agent 118 and the code integration agent 119. It should be noted that the task allocation agent 112 will extract and identify the structured information and attribute information, and then transmit it to the corresponding agent. For example, if an outdoor training environment is to be created, it will be sent to the outdoor environment agent. In practical applications, the types of agents such as indoor environment, outdoor environment, industrial environment, static interaction and dynamic interaction can be divided according to specific scenarios.

[0072] Specifically, see again Figure 3 The indoor environment agent 113 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and create an indoor environment generation program; the outdoor environment agent 114 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and create an outdoor environment generation program; the industrial environment agent 115 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and create an industrial environment generation program; the static interaction agent 116 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and create a static interaction generation program; the dynamic interaction agent 117 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and create a dynamic interaction generation program.

[0073] Specifically, see again Figure 3 The information replacement agent 118 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and send the attribute information to the retrieval unit 130; the code integration agent 119 is used to receive the target task information and exploration task information sent by the task assignment agent 112, and integrate the programs generated by each agent into a scene generation program.

[0074] It should be noted that the above-mentioned intelligent agents can be some preset scene construction programs, models, etc.

[0075] In some optional embodiments, see again Figure 2 , the retrieval unit 130 includes a word embedding model 131, a vector library module 132, a knowledge base module 133 and a similarity retrieval module 134. Figure 3The word embedding model 131 is used to convert the attribute information sent by the intelligent unit 110 into vectorized attribute information, and to convert the scene configuration information into vectorized scene configuration information; the vector library module 132 is used to store the vectorized attribute information and the vectorized scene configuration information; the knowledge base module 133 is used to store scene configuration information of different attributes; the similarity retrieval module 134 is used to perform similarity retrieval on the vectorized attribute information and the vectorized scene configuration information and send replacement information to the intelligent unit 110.

[0076] In some optional embodiments, see again Figure 2 The simulation unit 140 includes a simulation generation module 141 and a simulation display module 142. Figure 3 The simulation generation module 141 is used to receive the scene generation program sent by the intelligent unit 110 and convert it into simulation information. The simulation display module 142 is used to convert the simulation information into scene information and send it to the input and output unit 120.

[0077] Exemplarily, the simulation generation module 141 may be a simulation tool such as Gazebo, Webots, or CoppeliaSim, but the present embodiment is not limited thereto.

[0078] The robot training scenario generation system of the embodiment of the present invention utilizes a large language model and completes the construction of robot training scenarios according to the user's intention, thereby reducing the workload of software development and improving work efficiency; through similar retrieval of the knowledge base, it replaces the same-attribute words or phrases in the user's text information to generate a large number of other similar robot training scenarios, which can cover a wide range of training conditions and possibilities; it provides multiple simulation tool interfaces including Gazebo, Webots, and CoppeliaSim, which are highly applicable; it uses a large model to automatically complete the construction of robot training scenarios, without human understanding and execution deviation, and can ensure consistency standards.

[0079] According to an embodiment of the present invention, an embodiment of a method for generating a robot training scenario is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0080] In this embodiment, a method for generating a robot training scene is provided, which can be used for a computer device or electronic device, such as a computer, a tablet computer, etc., for scene generation. Figure 5 is a flowchart of a method for generating a robot training scene according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0081] Step S501, obtaining robot training scenario requirements, and converting the robot training scenario requirements into text information.

[0082] Specifically, the robot training scenario requirements include voice requirement information and / or text requirement information, the voice requirement information is converted into text to obtain text information, and / or the text requirement information is intercepted to obtain text information. The text information is information composed of words, numbers and punctuation marks formed according to the user's intention.

[0083] Exemplarily, the text demand information input by the user may be received through a keyboard, a touch screen, etc., and the voice demand information of the user may be received through a language recognition module.

[0084] The embodiment of the present invention obtains the user's intention through multiple ways to obtain the robot training scene requirements, which is more flexible and converts it into text information to facilitate the construction of a scene generation program.

[0085] Step S502: extract information from the text information to obtain structured information that complies with a preset text format and attribute information containing text keywords in the text information.

[0086] Specifically, a prompt word project can be constructed in advance, in which the text format of structured information, the type of text keywords and other information can be defined, and the project can be continuously adjusted and optimized to extract structured information and attribute information.

[0087] Exemplarily, the structured information is a structured text that organizes scene entities (indoor environment, outdoor environment, industrial environment), dynamic interaction entities, and static interaction entities in the text information in a specific format according to the order in which the entities appear in the text information and the simulation creation logic, such as "create an outdoor training scene", "include static interaction objects at intersections", "include dynamic interaction objects on sunny days", "include dynamic interaction objects on cars", "the object under test is a humanoid robot", etc.

[0088] Exemplarily, the attribute information is words or phrases such as time, dynamic interactive entities, static interactive entities, etc. that can be replaced in the text information, such as "crossroads", "sunny day", "car", etc.

[0089] Step S503, constructing a scene generation program according to the structured information and attribute information.

[0090] Specifically, the structured information and attribute information are transformed to determine the specific scene to be constructed, and the corresponding scene construction program is used to automatically generate the corresponding scene code, and the scene generation program is integrated.

[0091] Step S504, performing simulation based on the scenario generation program to generate a robot training scenario.

[0092] Specifically, the scenario generation program is transformed using a simulation tool or a simulation model, thereby simulating a robot training scenario.

[0093] The robot training scene generation method provided in this embodiment converts the robot training scene requirements input by the user into text information, and then extracts information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information, to construct a scene generation program, and then use the scene generation program to perform simulation and automatically generate robot training scenes, which can reduce the workload of software development, improve work efficiency, and eliminate human understanding and execution deviations, thereby ensuring the consistency standard of scene construction.

[0094] In this embodiment, a method for generating a robot training scene is provided, which can be used for a computer device or electronic device, such as a computer, a tablet computer, etc., for scene generation. Figure 6 is a flowchart of a method for generating a robot training scene according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0095] Step S601, obtain robot training scenario requirements, and convert the robot training scenario requirements into text information. For details, please refer to Figure 5 Step S501 of the illustrated embodiment will not be described in detail here.

[0096] Step S602: extract information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information. Figure 5 Step S502 of the illustrated embodiment will not be described in detail here.

[0097] Step S603, constructing a scene generation program according to the structured information and attribute information.

[0098] Specifically, the above step S603 includes:

[0099] Step S6031, obtaining target task information according to structured information and attribute information.

[0100] Specifically, the structured information and attribute information are combined to obtain the target task information. For example, the structured information is "including static interaction objects at intersections", "including dynamic interaction objects at sunny days", and "including dynamic interaction objects at cars". The attribute information is "intersection", "sunny day", and "car". Then the target task information can include multiple sub-target tasks, namely, "create an intersection generation program", "create a sunny day generation program", and "create a car generation program".

[0101] Step S6032: replace the text keywords in the attribute information to obtain replacement information, and convert the replacement information into exploration task information based on the structured information.

[0102] In some optional implementations, the above step S6032 includes:

[0103] Step a1, converting attribute information into vectorized attribute information.

[0104] Exemplarily, a word embedding model can be trained, and the attribute information can be input into the trained word embedding model. The word embedding model can automatically convert it into N-dimensional vectorized attribute information. For example, when "crossroads" is input, the output is [0.15246, 0.89898, 0.251654, 0.23546, ... ].

[0105] Step a2: perform similarity search based on the vectorized attribute information to obtain similar keywords.

[0106] Exemplarily, similarity search is performed on the vectorized attribute information in a preset vector library, and the retrieved similar vectors are converted into similar keywords with the same attribute. For example, similar keywords for "crossroads" may be "T-junction", "roundabout", etc., similar keywords for "sunny day" may be "cloudy", "heavy rain", "light rain", etc., and similar keywords for "car" may be "bus", "truck", "truck", etc., and this embodiment is not limited thereto.

[0107] Step a3: Replace the text keywords in the attribute information with similar keywords to obtain replacement information, wherein the text keywords include at least one of the scene entity, the interactive scene entity and the scene configuration information.

[0108] Specifically, the original text keywords in the attribute information are replaced with similar keywords, for example, "crossroads" is replaced with "T-junction", to obtain replacement information, and then the structured information and the replacement information are used to generate corresponding exploration task information.

[0109] The embodiment of the present invention performs similarity retrieval on attribute information so as to use the obtained replacement information to replace words or phrases with the same attributes in text information, thereby expanding and enriching the text information.

[0110] Step S6033, construct a scenario generation program based on the target task information and / or exploration task information.

[0111] In some optional implementations, the above step S6033 includes:

[0112] Step b1, obtaining multiple sub-target tasks and / or multiple sub-exploration tasks according to the target task information and / or the exploration task information.

[0113] Exemplarily, the target task information is split into multiple sub-target tasks, such as "create an intersection generation program", "create a sunny day generation program", "create a car generation program"; the exploration task information is split into multiple sub-exploration tasks, such as "create a T-junction generation program", "create a heavy rain generation program", "create a truck generation program", etc.

[0114] Step b2, identifying the scene type corresponding to each sub-target task and / or each sub-exploration task; wherein the scene type includes at least one of an indoor environment, an outdoor environment, an industrial environment, a static interaction scene, and a dynamic interaction scene.

[0115] Specifically, the scene type can be identified based on the keywords in the task, and the specific scene type and classification can be set in combination with the actual scene.

[0116] Step b3, based on the scenario type, each sub-target task and / or each sub-exploration task is sent to the corresponding preset scenario construction program to obtain the scenario code output by each preset scenario construction program; wherein the type of the scenario construction program corresponds one-to-one to the scenario type.

[0117] Specifically, each scene type corresponds to a preset scene construction program, which is used to generate corresponding scene codes. For example, the scene construction program corresponding to the static interactive scene can generate intersection codes, T-junction codes, etc.

[0118] Step b4, integrating the scene codes output by each preset scene construction program to construct a scene generation program.

[0119] Specifically, parameter transfer rules, code nesting rules, etc. are set to integrate various scene codes to obtain a scene generation program.

[0120] The embodiments of the present invention identify the scenario types of subtasks contained in the target task information and / or exploration task information, and use the corresponding preset scenario construction program to generate scenario code, thereby improving program generation efficiency; and, use the exploration task information to replace and expand the scene, thereby improving the richness of the scene.

[0121] In the present invention, according to the structured information and attribute information corresponding to the text information, the scenario generation program is automatically written to build the robot training scenario, reduce the workload of software development, and improve work efficiency. Through similar retrieval, the same attribute words or phrases in the user's text information are replaced, which is convenient for generating a large number of other similar robot training scenarios, which can cover a wide range of training conditions and possibilities.

[0122] Step S604, performing simulation based on the scenario generation program to generate a robot training scenario.

[0123] Specifically, the above step S604 includes:

[0124] Step S6041, converting the scene generation program into simulation information.

[0125] Specifically, simulation information refers to all data and programs in scene generation and use, which can be obtained by inputting the scene generation program into a simulation tool, wherein the simulation tool can be Gazebo, Webots, CoppeliaSim, but the embodiments of the present invention are not limited thereto.

[0126] Step S6042, simulating and generating a robot training scene according to a preset simulation model and simulation information.

[0127] Specifically, it can provide multiple simulation tool interfaces including Gazebo, Webots, and CoppeliaSim, which are highly applicable. Then, simulation tools and simulation information are used for simulation to obtain robot training scenarios and a series of scenario information.

[0128] The embodiment of the present invention converts the scene generation program into simulation information, and uses a preset simulation model to simulate and generate a robot training scene, thereby automatically completing the scene construction without human understanding and execution deviation, ensuring consistency standards, and having strong applicability.

[0129] Step S605, displaying the robot training scene.

[0130] Exemplarily, a display screen or other device is used to display the robot training scene and other scene information, so that users can view, modify and use it easily.

[0131] The robot training scenario generation method provided in this embodiment automatically writes a scenario generation program according to the user's intention through a large language model to complete the robot training scenario construction, reduce software development workload, and improve work efficiency; through similar retrieval of the knowledge base, replace the same attribute words or phrases in the user's text information, and generate a large number of other similar robot training scenarios, which can cover a wide range of training conditions and possibilities; provide multiple simulation tool interfaces including Gazebo, Webots, and CoppeliaSim, which are highly applicable; in addition, the robot training scenario construction can be automatically completed without human understanding and execution deviation, and consistency standards can be guaranteed.

[0132] The following is a detailed description of the robot training scene generation method of the present invention in conjunction with a specific application example. The application example can be applied to Figure 2 The robot training scene generation system shown in Figure 7 As shown, the application example includes the following steps:

[0133] Step S701: the intelligent unit receives text information sent by the input and output unit, and converts the text information into structured information and attribute information.

[0134] Step S702, the task allocation agent converts the structured information and attribute information into target task information and sends it to at least one of the indoor environment agent, outdoor environment agent, industrial environment agent, static interaction agent, dynamic interaction agent, information replacement agent and code integration agent.

[0135] Step S703: the information replacement agent sends the attribute information to the retrieval unit according to the user's exploratory needs, and the task allocation agent converts the replacement information output by the retrieval unit into exploratory task information.

[0136] Step S704, the indoor environment agent, outdoor environment agent, industrial environment agent, static interaction agent, and dynamic interaction agent generate corresponding programs according to the target task information and exploration task information, integrate the agents into a scene generation program through code, and send it to the simulation unit.

[0137] Step S705: The simulation unit generates simulation information and scene information according to the scene generation program, and sends the scene information to the input and output unit.

[0138] Step S706: the input and output unit receives the scene information for display.

[0139] This application example uses a large language model to automatically write a scenario generation program based on the user's intention, complete the construction of the robot training scenario, reduce the workload of software development, and improve work efficiency; through similar retrieval of the knowledge base, replace the same attribute words or phrases in the user's text information, and generate a large number of other similar robot training scenarios, which can cover a wide range of training conditions and possibilities; in addition, the robot training scenario construction can be automatically completed without human understanding and execution deviation, and consistency standards can be guaranteed.

[0140] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0141] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0142] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0143] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0145] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0146] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0147] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0148] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0149] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for generating a robot training scene, characterized in that: The method comprises: Obtain robot training scenario requirements, and convert the robot training scenario requirements into text information; Extracting information from the text information to obtain structured information that complies with a preset text format and attribute information containing text keywords in the text information; Constructing a scene generation program according to the structured information and the attribute information; A simulation is performed based on the scenario generation program to generate a robot training scenario.

2. The method according to claim 1, characterized in that The step of constructing a scene generation program according to the structured information and the attribute information comprises: Obtaining target task information according to the structured information and the attribute information; and / or, replacing text keywords in the attribute information to obtain replacement information, and converting the replacement information into exploration task information based on the structured information; Based on the target task information and / or the exploration task information, a scenario generation program is constructed.

3. The method according to claim 2, characterized in that The replacing of the text keywords in the attribute information to obtain the replacement information includes: Converting the attribute information into vectorized attribute information; Perform similarity search based on the vectorized attribute information to obtain similar keywords; The text keywords in the attribute information are replaced with the similar keywords to obtain replacement information; wherein the text keywords include at least one of a scene entity, an interactive scene entity, and scene configuration information.

4. The method according to claim 2, characterized in that: The step of constructing a scenario generation program based on the target task information and / or the exploration task information includes: According to the target task information and / or the exploration task information, a plurality of sub-target tasks and / or a plurality of sub-exploration tasks are obtained; Identify the scene type corresponding to each of the sub-target tasks and / or each of the sub-exploration tasks; wherein the scene type includes at least one of an indoor environment, an outdoor environment, an industrial environment, a static interaction scene, and a dynamic interaction scene; Based on the scenario type, each of the sub-target tasks and / or each of the sub-exploration tasks is sent to a corresponding preset scenario construction program to obtain a scenario code output by each preset scenario construction program; wherein the type of the scenario construction program corresponds to the scenario type one by one; The scene codes output by each preset scene construction program are integrated to construct a scene generation program.

5. The method according to any one of claims 1 to 4, characterized in that The step of simulating based on the scenario generation program to generate a robot training scenario includes: converting the scenario generation program into simulation information; According to the preset simulation model and the simulation information, a robot training scene is simulated and generated.

6. The method according to any one of claims 1 to 4, characterized in that The robot training scenario requirement includes voice requirement information and / or text requirement information; and converting the robot training scenario requirement into text information includes: Performing voice-to-text conversion on the voice demand information to obtain the text information; And / or, intercepting the text demand information to obtain the text information.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The robot training scene is displayed.

8. A robot training scene generation system, characterized in that: The system comprises: An input-output unit, used to obtain robot training scenario requirements and convert the robot training scenario requirements into text information; An intelligent unit is used to extract information from the text information to obtain structured information that conforms to a preset text format and attribute information containing text keywords in the text information; and to construct a scene generation program based on the structured information and the attribute information; The simulation unit is used to perform simulation based on the scenario generation program to generate a robot training scenario.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the robot training scene generation method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the robot training scene generation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to enable a computer to execute the robot training scene generation method according to any one of claims 1 to 7.

Images