A model training method and device

Abstract

The application provides a model training method and device. The method comprises: acquiring first images of a work scene collected by each camera device, and extracting first semantic information of each first image; constructing a work scene model according to each first image and corresponding first semantic information, and pose information of the camera device; acquiring second images of multiple specified visual angles and corresponding second semantic information from the work scene model; determining, for each second semantic information, a similarity between the second semantic information and a text feature of a target object to obtain first similarity information; training an action prediction model using each second image, each second semantic information and each first similarity information until a difference value between a predicted action for an instruction output by the action prediction model and an expert action is less than a preset threshold value, and obtaining a trained action prediction model. The application can improve the generalization ability of the model, and the model can be applied to different tasks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agents, and particularly relates to a model training method and device. BACKGROUND

[0002] Generally, a model is trained by using deep learning to control an agent, so that the agent can complete a task. Usually, a model needs to be specially trained for a task to obtain a better task completion effect.

[0003] This characteristic leads to poor generalization of the trained model, which cannot be generalized to other tasks, so that the agent cannot complete multiple tasks. SUMMARY

[0004] Embodiments of the application provide a model training method and device, an agent control method and device, an electronic device, a readable storage medium, and a computer program product, which can solve the problem of poor model generalization of an agent.

[0005] In a first aspect, embodiments of the application provide a model training method, comprising:

[0006] obtaining first images of a work scene collected by each camera device, and extracting first semantic information of each first image, the first image being an image collected when an agent performs an expert action for an instruction, the first image including an imaging area corresponding to a target object, the installation positions of the camera devices being different, the instruction being used to instruct the agent to operate the target object, and the first semantic information including semantic features of each pixel point in the first image;

[0007] constructing a work scene model according to each first image, the corresponding first semantic information, and pose information of the camera device;

[0008] obtaining second images of multiple specified viewing angles and corresponding second semantic information from the work scene model, the second semantic information including semantic features of each pixel point in the second image;

[0009] determining a similarity between the second semantic information and a text feature of the target object for each second semantic information, to obtain first similarity information, the first similarity information including similarities between the text feature of the target object and the semantic features of each pixel point in the second image;

[0010] training an action prediction model by using each second image, each second semantic information, and each first similarity information, until a difference value between a predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, to obtain a trained action prediction model.

[0011] In an embodiment, the constructing the work scene model according to each of the first images, the corresponding first semantic information, and the pose information of the camera device comprises:

[0012] constructing a NERF model according to each of the first images and the corresponding pose information of the camera device;

[0013] fusing each of the first semantic information and the NERF model to obtain the work scene model.

[0014] In an embodiment, the obtaining, from the work scene model, a plurality of second images of specified perspectives and corresponding second semantic information comprises:

[0015] rendering points in a plurality of intervals in the work scene model to obtain a plurality of the second images and the corresponding second semantic information, the intervals being determined according to the specified perspectives.

[0016] In an embodiment, the determining, for each of the second semantic information, a similarity between the second semantic information and the text feature of the target object to obtain first similarity information comprises:

[0017] calculating, for each of the second semantic information, a cosine similarity between the text feature of the target object and a semantic feature of each of the pixel points in the second semantic information to obtain the first similarity information.

[0018] In an embodiment, the specified perspective is determined according to a work space of the agent, the instruction, and the expert action.

[0019] In a second aspect, an embodiment of the present application provides a model training apparatus, comprising:

[0020] an obtaining module configured to obtain first images collected by a plurality of camera devices and extract first semantic information of each of the first images, the first images being images collected when an agent performs an expert action for an instruction, the first images including imaging regions of corresponding target objects, the installation positions of the camera devices being different, the instruction being used to instruct the agent to operate the target objects, and the first semantic information including semantic features of each of the pixel points in the first images;

[0021] and further configured to obtain, from the task scene model, a plurality of second images of specified perspectives and corresponding second semantic information, the second semantic information including semantic features of each of the pixel points in the second images;

[0022] a constructing module configured to construct the task scene model according to each of the first images, the corresponding first semantic information, and the pose information of the camera devices;

[0023] determining, for each second semantic information, a similarity between the second semantic information and a text feature of the target object, to obtain first similarity information, the first similarity information including similarities between the text feature of the target object and semantic features of each pixel point in the second image;

[0024] training, by using each second image, each second semantic information, and each first similarity information, an action prediction model, until a difference value between a predicted action output by the action prediction model for the instruction and the expert action is less than a preset threshold, to obtain a trained action prediction model. In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method in any one of the above first aspect when executing the computer program.

[0025] In a third aspect, an embodiment of the present application provides an agent control method, including:

[0026] in response to a task, repeatedly performing the following steps until the task ends;

[0027] after the agent performs the predicted action at the last time, obtaining third images of a task scene at a current time collected by each camera device, and extracting third semantic information of each third image, the installation positions of the camera devices are different, the third image includes an imaging area of a to-be-operated object corresponding to the current time, and the third semantic information includes semantic features of each pixel point in the third image;

[0028] constructing a task scene model according to each third image, corresponding third semantic information, and pose information of the camera device;

[0029] obtaining, from the task scene model, fourth images of multiple specified view angles and corresponding fourth semantic information, the fourth semantic information including semantic features of each pixel point in the fourth image;

[0030] determining, for each fourth semantic information, a second similarity between the fourth semantic information and a text feature of the to-be-operated object, to obtain second similarity information, the second similarity information including similarities between the text feature of the to-be-operated object and semantic features of each pixel point in the fourth image;

[0031] inputting each fourth image, each fourth semantic information, and each second similarity information into the trained action prediction model, obtaining a predicted action of the current time output by the trained action prediction model, so as to control the agent to perform the predicted action of the current time, the trained action prediction model being obtained by training the method in any one of the first aspects.

[0032] In a fourth aspect, the embodiments of the present application provide an agent control device, comprising:

[0033] a prediction module, configured to repeatedly perform the following steps in response to a task until the task ends:

[0034] obtaining third images of a task scene of a current time collected by each camera device after the agent performs the predicted action of the last time, and extracting third semantic information of each third image, the installation positions of the camera devices being different, the third image including an imaging area of a to-be-operated object corresponding to the current time, and the third semantic information including semantic features of each pixel point in the third image;

[0035] the prediction module is further configured to construct a task scene model according to each third image and corresponding third semantic information and pose information of the camera device;

[0036] the prediction module is further configured to obtain fourth images of multiple specified visual angles and corresponding fourth semantic information from the task scene model, the fourth semantic information including semantic features of each pixel point in the fourth image;

[0037] the prediction module is further configured to determine a second similarity between the fourth semantic information and a text feature of the to-be-operated object for each fourth semantic information, and obtain second similarity information, the second similarity information including similarities between the text feature of the to-be-operated object and the semantic features of each pixel point in the fourth image;

[0038] inputting each fourth image, each fourth semantic information, and each second similarity information into the trained action prediction model, obtaining a predicted action of the current time output by the trained action prediction model, so as to control the agent to perform the predicted action of the current time, the trained action prediction model being obtained by training the method in any one of the first aspects.

[0039] In a fifth aspect, the embodiments of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor implementing the method in any one of the first aspect or the second aspect when executing the computer program.

[0040] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method in any one of the first aspect or the second aspect.

[0041] In a seventh aspect, an embodiment of the present application provides a computer program product, when the computer program product runs on an electronic device, causing the electronic device to execute the method in any one of the first aspect or the second aspect.

[0042] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0043] In the embodiment of the present application, the working scene model is constructed according to each first image, corresponding first semantic information and pose information of the camera device; the second image of each specified view and corresponding second semantic information are obtained from the working scene model, the second semantic information includes the semantic features of each pixel point in the second image; for each second semantic information, the similarity between the second semantic information and the text features of the target object is determined to obtain first similarity information, the first similarity information includes the similarity between the text features of the target object and the semantic features of each pixel point in the first image; the action prediction model is trained by using each second image, each second semantic information and each first similarity information, until the difference value between the predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, and the trained action prediction model is obtained, which can give the model certain prior information, and the model can use these prior information to pay more attention to the target object and determine the position of the target object, that is, the position of the target object can be accurately determined in different scenes, and then the model can be applied to different tasks to improve the generalization ability of the model. Meanwhile, the second image of the specified view and the corresponding second semantic information are obtained by using the working scene model, the training data is enhanced, the utilization rate of the training data is improved, the model is better adapted to different scenes, and the generalization ability of the model is improved.

[0044] It can be understood that the beneficial effects of the above-mentioned second aspect to seventh aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating any creative labor.

[0046] Figure 1 is a flowchart of the model training method provided by an embodiment of the present application;

[0047] Figure 2 is a structural schematic diagram of a model training apparatus provided by an embodiment of the present application;

[0048] Figure 3 is a flowchart of an agent control method provided by an embodiment of the present application;

[0049] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0051] It should be understood that the term “includes” when used in the specification and the appended claims herein, specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0052] It should also be understood that the term “and / or” when used in the specification and the appended claims herein means any one or more of the associated listed items and includes all possible combinations of the associated listed items.

[0053] As used in the description of the application and the appended claims herein, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [the described condition or event] is detected” can be interpreted as meaning “upon determining” or “in response to determining” or “upon detecting [the described condition or event]” or “in response to detecting [the described condition or event],” depending on the context.

[0054] In addition, in the description of the application and the appended claims herein, the terms “first,” “second,” “third,” and the like are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0055] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified descriptions are not necessarily all referring to the same embodiment, however, but can refer to one or more but not all embodiments. The terms "including," "comprising," "featuring," and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless otherwise noted, the terms "including" and "comprising" are used in an inclusive sense, and should be interpreted as meaning "including, but not limited to."

[0056] Generally, a model is trained by using deep learning to control an agent to complete a task. In order to obtain a better task completion effect, the model is usually specially trained for the task. This feature leads to poor generalization of the trained model, which cannot be generalized to other tasks, so that the trained model cannot be applied to multi-task, resulting in the design and training of the model corresponding to different tasks. This way is time-consuming. In addition, the existing way to improve the generalization of the model is to train the model by using imitation learning, but this way needs a large amount of expert data for training, and the expert data of some tasks is difficult or impossible to collect.

[0057] Based on this, the embodiments of the application provide a model training method and device, which train a model by using images, semantic information and similarity information, so that the model can accurately determine the position of an object to be operated and predict actions in different scenes, and improve the generalization of the model.

[0058] Figure 1 is a flowchart of a model training method provided by an embodiment of the application. As shown in Figure 1 The model training method comprises the following steps.

[0059] S11: Obtain first images of a working scene collected by each camera device, and extract first semantic information of each first image.

[0060] The working scene includes an agent and a target object. The installation positions of the camera devices are different. The first images are images collected when the agent performs an expert action according to an instruction.

[0061] The instruction is used to instruct the agent to operate the target object. The instruction is an instruction input by a user, and the instruction contains text information of the target object.

[0062] In application, expert data is collected for instructions in a working scenario. The expert data includes actions of an intelligent agent in the working scenario that can complete instructions according to environmental perception. In a simulation environment, the expert actions are obtained by the intelligent agent performing actions through a manually written script. In a real environment, the expert actions are obtained by the intelligent agent performing actions through human demonstration.

[0063] A first image is obtained by setting a camera device at different viewing angles. The first image includes an imaging area corresponding to a target object. The first image can be an RGBD image. Semantic features of each pixel point in the first image are extracted by a vision model and a vision language large model, and the first semantic information includes the semantic features of each pixel point in the first image. The semantic features of these pixel points are aligned with the text features of the target object.

[0064] S12: Construct a working scenario model according to each first image and the corresponding first semantic information, and the pose information of the camera device.

[0065] In a possible implementation, step S12 includes:

[0066] S121: Construct a NERF model according to each first image and the pose information of the corresponding camera device.

[0067] In application, a NERF (Neural Radiance Field) three-dimensional reconstruction technology is used to construct a NERF model according to each first image and the pose information of the corresponding camera device. The NERF model covers the range of all shooting viewing angles.

[0068] S122: Fuse each first semantic information with the NERF model to obtain a working scenario model.

[0069] In application, each first semantic information is fused with the NERF model to obtain a working scenario model, and the working scenario model can render images at any viewing angle and corresponding semantic features.

[0070] S13: Obtain second images at multiple specified viewing angles and corresponding second semantic information from the working scenario model.

[0071] The second semantic information includes semantic features of each pixel point in the second image, and the second image includes an imaging area corresponding to the target object.

[0072] In application, step S13 includes:

[0073] The points in multiple intervals in the working scenario model are rendered to obtain multiple second images and corresponding second semantic information, and the intervals are determined according to the specified viewing angles.

[0074] The specified view angle is determined according to the workspace of the agent, the instruction, and the expert action. The workspace of the agent is the workspace in which the agent completes the instruction, and includes the agent, the target object, and the surrounding environment. Specifically, the representative view angle is determined according to the workspace of the agent, and the view angle reflects the instruction and the action performed by the agent, to obtain the specified view angle. These specified view angles are not limited to the shooting view angle of the camera device.

[0075] It can be understood that the second image with the representative view angle is generated by the work scene model that can generate any view angle, the training data is enhanced, and the utilization rate of the training data is improved. In addition, when the number of second images is greater than the number of first images, the number of training data is increased, the training data is further enhanced, and the utilization rate of the training data is improved.

[0076] In application, the boundary is determined according to the specified view angle, and the corresponding interval is obtained. The points in the multiple intervals in the work scene model are rendered by using the NERF volume rendering formula, to obtain multiple second images and corresponding second semantic information. The NERF volume rendering formula is as follows: wherein, t n is the near boundary, t f is the far boundary, r(t) = o + td represents a three-dimensional point along the camera ray direction, o represents the camera optical center, d represents the camera ray direction, σ(r(t)) represents the volume rendering density, which is used to represent the scene density of the camera light passing through the point r(t), and c(r(t), d) represents the color of the current camera light direction r(t) point.

[0077] S14: For each second semantic information, the similarity between the second semantic information and the text feature of the target object is determined, to obtain first similarity information.

[0078] The first similarity information includes the similarity between the text feature of the target object and the semantic feature of each pixel point in the first image.

[0079] In application, after the text information of the target object is extracted from the instruction, the text feature of the target object is obtained by encoding. Because the semantic feature of each pixel point has been aligned with the text feature of the target object, the similarity between them can be calculated. The obtained first similarity information can be represented as a text similarity feature field.

[0080] In application, step S14 includes:

[0081] For each second semantic information, the cosine similarity between the text feature of the target object and the semantic feature of each pixel point in the second semantic information is calculated, to obtain the first similarity information.

[0082] Specifically, because the semantic features of each pixel point have been aligned to the text feature space, the cosine similarity of the semantic features of the region where the target object is located and the text features is high, and the cosine similarity of other regions is low. Further, the region with high cosine similarity in the first similarity information represents the target object. This makes the region with high cosine similarity more likely to be focused on by the model, and thus the model can accurately determine the location of the target object.

[0083] S15: training the action prediction model using each second image, each second semantic information and each first similarity information until the difference value between the predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, and obtaining the trained action prediction model.

[0084] In application, the behavior cloning method of imitation learning is used to minimize the predicted action and the expert action, and obtain the difference value between the predicted action and the expert action. Specifically, the function is argmin||Apre-Agt||, where Apre represents the predicted action, and Agt represents the expert action. The function is continuously iteratively optimized by gradient descent. When the difference value of the function is less than the preset threshold, the trained action prediction model is obtained. 2

[0085] It can be understood that the second image and the corresponding second semantic information of the specified perspective are obtained by using the work scene model, the training data is enhanced, the utilization rate of the training data is improved, the model can be trained well using a small amount of expert data, and thus the model has strong generalization ability and even has zero-shot ability (zero learning). At the same time, the problem of needing a large amount of expert data to train the model and the problem of low utilization efficiency of the expert data are solved.

[0086] ​The embodiment constructs a work scene model according to each first image, corresponding first semantic information and pose information of the camera device; obtains a plurality of second images of specified perspectives and corresponding second semantic information from the work scene model, the second semantic information including semantic features of each pixel point in the second image; determines, for each second semantic information, a similarity between the second semantic information and a text feature of the target object, to obtain first similarity information including a similarity between the text feature of the target object and the semantic features of each pixel point in the first image; and trains an action prediction model using each second image, each second semantic information and each first similarity information, until a difference value between a predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, to obtain a trained action prediction model. The model can be given certain prior information, and the model can use the prior information to pay more attention to the target object and determine the position of the target object, that is, the position of the target object can be accurately determined in different scenes, and the model can be applied to different tasks to improve the generalization ability of the model. The work scene model is used to obtain the second images of the specified perspectives and the corresponding second semantic information, to enhance the training data, improve the utilization rate of the training data, make the model better adapt to different scenes and improve the generalization ability of the model.

[0087] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0088] Corresponding to the method described in the above embodiment, only the part related to the embodiments of the present application is shown for convenience of description.

[0089] Figure 2 is a structural schematic diagram of a model training device provided by an embodiment of the present application. As shown in Figure 2 , the model training device comprises:

[0090] The acquisition module 10 is configured to acquire first images collected by each camera device and extract first semantic information of each first image, the first image being an image collected when an agent performs an expert action for an instruction, the first image including an imaging area corresponding to a target object, the installation positions of the camera devices being different, the instruction being used to instruct the agent to operate the target object, and the first semantic information including semantic features of each pixel point in the first image.

[0091] The model training device is further configured to obtain a plurality of second images of specified perspectives and corresponding second semantic information from the work scene model, the second semantic information including semantic features of each pixel point in the second image.

[0092] The constructing module 11 is configured to construct a work scene model according to each first image, corresponding first semantic information, and pose information of the camera device.

[0093] The determining module 12 is configured to determine, for each second semantic information, a similarity between the second semantic information and the text feature of the target object, to obtain first similarity information including a similarity between the text feature of the target object and a semantic feature of each pixel point in the second image.

[0094] The training module 13 is configured to train the action prediction model by using each second image, each second semantic information, and each first similarity information, until a difference value between a predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, to obtain a trained action prediction model.

[0095] In one embodiment, the constructing module is specifically configured to construct a NERF model according to each first image and corresponding pose information of the camera device, and fuse each first semantic information with the NERF model to obtain the work scene model.

[0096] In one embodiment, the obtaining module is specifically configured to render points in multiple intervals in the work scene model to obtain multiple second images and corresponding second semantic information, where the intervals are determined according to a specified view angle.

[0097] In one embodiment, the determining module is specifically configured to calculate, for each second semantic information, a cosine similarity between the text feature of the target object and a semantic feature of each pixel point in the second semantic information, to obtain the first similarity information.

[0098] Figure 3 is a flowchart of an embodiment of the agent control method provided in the present application. As shown in Figure 3 , the agent control method comprises the following steps.

[0099] S21: in response to a task, repeatedly performing the following steps until the task ends.

[0100] S22: after the agent performs the predicted action at the previous time, obtaining third images of the task scene at a current time captured by each camera device, and extracting third semantic information of each third image.

[0101] wherein the installation positions of the camera devices are different, the third image includes an imaging area of the to-be-operated object corresponding to the current time, and the third semantic information includes a semantic feature of each pixel point in the third image.

[0102] In application, the task is subdivided in response to the task. Since the NERF is static, after the agent executes the predicted action of the previous time, the state of the workspace of the agent changes, and the object to be operated at the current time also changes, which is different from the object to be operated at the previous time, and the NERF model needs to be changed accordingly.

[0103] After the agent executes the predicted action of the previous time, the task scene is photographed by the camera device arranged at different angles to obtain a third image.

[0104] S23: Construct a task scene model according to each third image and corresponding third semantic information, and pose information of the camera device.

[0105] S24: Obtain fourth images of multiple specified angles and corresponding fourth semantic information from the task scene model.

[0106] Among them, the fourth image includes an imaging area corresponding to the object to be operated at the current time, and the fourth semantic information includes semantic features of each pixel point in the fourth image.

[0107] S25: For each fourth semantic information, determine a second similarity between the fourth semantic information and the text features of the object to be operated, and obtain second similarity information.

[0108] Among them, the second similarity information includes the similarity between the text features of the object to be operated and the semantic features of each pixel point in the fourth image.

[0109] S26: Input each fourth image, each fourth semantic information and each second similarity information into the trained action prediction model to obtain the predicted action of the current time output by the trained action prediction model, so as to control the agent to execute the predicted action of the current time.

[0110] Among them, the trained action prediction model is trained by the model training method described in the above embodiment.

[0111] In application, after the agent executes the predicted action of the current time, the predicted action of the current time is equivalent to the predicted action of the previous time when entering the action prediction of the next time, and the next time is the current time. Repeat steps S22-S26 to obtain the predicted action.

[0112] In this embodiment, by inputting each fourth image, each fourth semantic information and each second similarity information into the trained action prediction model, the predicted action of the current time output by the trained action prediction model is obtained, so that the model can use the prior knowledge of the fourth image, the fourth semantic information and the second similarity information to pay more attention to the object to be operated and accurately determine the position of the object to be operated in different task scenes, and then accurately predict the action.

[0113] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0114] For ease of explanation, only the parts related to the embodiments of this application are shown in the methods described in the above embodiments.

[0115] In one embodiment, the intelligent agent control device includes:

[0116] The prediction module is used to repeatedly execute the following steps in response to a task until the task is completed.

[0117] It is also used to acquire the third image of the task scene at the current moment captured by each camera device after the agent performs the predicted action of the previous moment, and to extract the third semantic information of each third image. The installation positions of each camera device are different. The third image includes the imaging area of ​​the object to be operated corresponding to the current moment. The third semantic information includes the semantic features of each pixel in the third image.

[0118] It is also used to construct a task scene model based on each of the third images and the corresponding third semantic information, as well as the pose information of the camera device.

[0119] It is also used to obtain multiple fourth images from specified perspectives and corresponding fourth semantic information from the task scene model, wherein the fourth semantic information includes the semantic features of each pixel in the fourth image.

[0120] It is also used to determine, for each fourth semantic information, a second similarity between the fourth semantic information and the text features of the object to be operated on, and obtain second similarity information, the second similarity information including the similarity between the text features of the object to be operated on and the semantic features of each pixel in the fourth image.

[0121] It is also used to input each fourth image, each fourth semantic information and each second similarity information into the trained action prediction model to obtain the predicted action at the current moment output by the trained action prediction model, so as to control the agent to execute the predicted action at the current moment.

[0122] The trained action prediction model was obtained by training the model training method described in the above embodiments.

[0123] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 2 of this embodiment includes: at least one processor 20 ( Figure 4(Only one is shown in the diagram), memory 21, and computer program 22 stored in said memory 21 and executable on said at least one processor 20, wherein said processor 20 executes said computer program 22 to implement the steps in any of the above method embodiments.

[0124] The electronic device 2 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 2 and does not constitute a limitation on electronic device 2. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The processor 20 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or memory of the electronic device 2. In other embodiments, the memory 21 may be an external storage device of the electronic device 2, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 2. Furthermore, the memory 21 may include both internal and external storage units of the electronic device 2. The memory 21 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0127] It should be noted that the information interaction, execution process and the like between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and the specific functions and the technical effects brought by the same can be referred to the method embodiments part, and will not be repeated here.

[0128] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for convenient distinction, and does not limit the protection scope of the present application. The specific first process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0129] The embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in each of the above method embodiments.

[0130] The embodiments of the present application provide a computer program product, when the computer program product runs on an electronic device, the electronic device executes the steps in each of the above method embodiments.

[0131] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods through a computer program to instruct relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some cases, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0132] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0133] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0134] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the above-described apparatus / network device embodiments are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0136] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A model training method, characterized in that, The method comprises the following steps: acquiring first images of a working scene collected by each camera device, and extracting first semantic information of each first image, the first image being an image collected when an agent performs an expert action for an instruction, the first image comprising an imaging area corresponding to a target object, the installation positions of each camera device being different, the instruction being used to instruct the agent to operate the target object, the first semantic information comprising semantic features of each pixel point in the first image; constructing a working scene model according to each first image, corresponding first semantic information and pose information of the camera device; acquiring second images of multiple specified view angles and corresponding second semantic information from the working scene model, the second semantic information comprising semantic features of each pixel point in the second image, the specified view angle being determined according to a working space of the agent, the instruction and the expert action; for each second semantic information, determining a similarity between the second semantic information and a text feature of the target object to obtain first similarity information, the first similarity information comprising similarities between the text feature of the target object and the semantic features of each pixel point in the second image; training an action prediction model by using each second image, each second semantic information and each first similarity information until a difference value between a predicted action for the instruction output by the action prediction model and the expert action is less than a preset threshold, and obtaining a trained action prediction model; wherein the constructing a working scene model according to each first image, corresponding first semantic information and pose information of the camera device comprises: constructing a NERF model according to each first image and corresponding pose information of the camera device; fusing each first semantic information with the NERF model to obtain the working scene model.

2. The method of claim 1, wherein, The acquiring second images of multiple specified view angles and corresponding second semantic information from the working scene model comprises: rendering points in multiple intervals in the working scene model to obtain multiple second images and corresponding second semantic information, the intervals being determined according to the specified view angle.

3. The method of claim 1, wherein, The determining a similarity between the second semantic information and a text feature of the target object for each second semantic information to obtain first similarity information comprises: calculating a cosine similarity between the text feature of the target object and the semantic features of each pixel point in the second semantic information for each second semantic information to obtain the first similarity information.

4. An agent control method characterized by, The method comprises the following steps: in response to a task, repeatedly performing the following steps until the task ends; after the agent performs a predicted action at a previous time, acquiring third images of a task scene at a current time collected by each camera device, and extracting third semantic information of each third image, the installation positions of each camera device being different, the third image comprising an imaging area corresponding to an object to be operated at the current time, the third semantic information comprising semantic features of each pixel point in the third image; construct a task scene model according to each of the third images, the corresponding third semantic information, and pose information of the camera device; obtain fourth images and corresponding fourth semantic information of a plurality of specified viewpoints from the task scene model, the fourth semantic information including semantic features of each pixel point in the fourth images; determine, for each fourth semantic information, a second similarity between the fourth semantic information and a text feature of the object to be operated, to obtain second similarity information, the second similarity information including similarities between the text feature of the object to be operated and the semantic features of each pixel point in the fourth images; input each fourth image, each fourth semantic information, and each second similarity information into the trained action prediction model to obtain a predicted action of the current time output by the trained action prediction model, to control the agent to perform the predicted action of the current time, the trained action prediction model being obtained by training the method of any one of claims 1 to 3.

5. A model training apparatus characterized by comprising: comprising: an acquisition module configured to acquire first images captured by each camera device and extract first semantic information of each first image, the first images being images captured when an agent performs an expert action for an instruction, the first images including imaging regions of corresponding target objects, the installation positions of the camera devices being different, the instruction being used to instruct the agent to operate the target objects, and the first semantic information including semantic features of each pixel point in the first images; and further configured to obtain second images and corresponding second semantic information of a plurality of specified viewpoints from a task scene model, the second semantic information including semantic features of each pixel point in the second images, the specified viewpoints being determined according to a working space of the agent, the instruction, and the expert action; a construction module configured to construct the task scene model according to each of the first images, the corresponding first semantic information, and pose information of the camera device; a determination module configured to determine, for each second semantic information, a similarity between the second semantic information and a text feature of the target object, to obtain first similarity information, the first similarity information including similarities between the text feature of the target object and the semantic features of each pixel point in the second images; a training module configured to train an action prediction model using each second image, each second semantic information, and each first similarity information, until a difference value between a predicted action of the instruction output by the action prediction model and the expert action is less than a preset threshold, to obtain a trained action prediction model; wherein the construction module is specifically configured to construct a NERF model according to each first image and the corresponding pose information of the camera device, and fuse each first semantic information with the NERF model to obtain the working scene model.

6. An agent control apparatus characterized by comprising: comprising: a prediction module configured to repeatedly perform the following steps in response to a task until the task ends; The method further comprises: obtaining third images of the task scene collected by each camera device at the current time after the agent performs the predicted action at the previous time, and extracting third semantic information of each third image, wherein the installation positions of the camera devices are different, the third image comprises an imaging area of the object to be operated corresponding to the current time, and the third semantic information comprises semantic features of each pixel point in the third image; The method further comprises: constructing a task scene model according to each third image, the corresponding third semantic information, and the pose information of the camera device; The method further comprises: obtaining fourth images of multiple specified visual angles and corresponding fourth semantic information from the task scene model, wherein the fourth semantic information comprises semantic features of each pixel point in the fourth image; The method further comprises: determining a second similarity between the fourth semantic information and the text features of the object to be operated, obtaining second similarity information, and the second similarity information comprises similarities between the text features of the object to be operated and the semantic features of each pixel point in the fourth image; The method further comprises: inputting each fourth image, each fourth semantic information, and each second similarity information into the trained action prediction model to obtain the predicted action at the current time output by the trained action prediction model, so as to control the agent to perform the predicted action at the current time, and the trained action prediction model is obtained by the method in any one of claims 1 to 3.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method in any one of claims 1 to 4 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program implements the method in any one of claims 1 to 4 when executed by the processor.

Images