Object-attribute-relation graph-based model training method, device and equipment

Abstract

The application provides a model training method and device based on an object-attribute-relation graph. The method comprises: obtaining an object-level relation graph of a sample image, and obtaining K slot vectors and K segmentation masks through a pre-trained slot encoder; converting the object-level relation graph into a slot-level relation tensor according to the K segmentation masks and the true value instance masks of each object; injecting the slot-level relation tensor into a to-be-trained slot dynamics model to perform a first-step prediction, outputting a first-step prediction relation and K predicted slot vectors; injecting an h-1 step prediction relation into the model to perform an h step prediction, and sequentially obtaining H step prediction trajectories; training a model based on at least the H step prediction trajectories and an object-attribute-relation graph sequence of a sample image sequence; and performing H step prediction on a target image through the trained model, and outputting H step prediction trajectories representing the relation evolution process of each object.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and robot control technology, and in particular to a model training method, apparatus and device based on object-attribute-relationship graph. Background Technology

[0002] Model-based reinforcement learning predicts state evolution in the latent space using a world model, which allows the agent to use imagination and deduction to train control strategies. Object-centric world models further decompose the scene into several slots, each corresponding to an object.

[0003] However, existing object-centric world models generally suffer from representation drift in multi-object interaction scenarios, namely: slots lack semantic anchoring, and slots are prone to swapping when objects come into contact or occlude; relationships between objects rely entirely on implicit emergence through attention mechanisms, lacking explicit structured constraints. Since the imaginary deduction of the world model is a stepwise autoregressive process, single-step representation errors are amplified by subsequent steps, ultimately leading to the performance collapse of control strategies trained based on this imaginary trajectory.

[0004] Currently, there is a lack of effective technical means to suppress the representation drift caused by the above interactions, which makes the object-centered world model perform poorly in downstream control tasks. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a model training method, apparatus and device based on object-attribute-relationship graph, so as to overcome the above problems or at least partially solve the above problems.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: A first aspect of this application discloses a model training method based on an object-attribute-relationship graph, the method comprising: The relation extractor extracts the relation from the sample image to obtain the object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image. The sample image is slot-encoded by a pre-trained slot encoder to obtain K slot vectors and K segmentation masks for the sample image; Based on the K slot vectors and K segmentation masks of the sample image, as well as the ground truth instance mask of each object in the sample image, the object-level relation graph of the sample image is converted into the slot-level relation tensor of the sample image. The slot-level relationship tensor of the sample image is injected into the slot dynamics model to be trained and the first step of prediction is performed. The first step prediction relationship and the first step K predicted slot vectors of the sample image are output. Take h sequentially from 2 to H-1, inject the h-1 step prediction relationship output by the slot dynamics model to be trained into the slot dynamics model to be trained and perform the h-step prediction, output the h-step prediction relationship of the sample image and the h-step K predicted slot vectors; The slot dynamics model to be trained is trained based on at least H-step predicted trajectories of sample images and object-attribute-relationship graph sequences of sample image sequences to obtain trained slot dynamics models; the sample image sequence consists of H consecutive sample images starting with sample images. The target image is predicted using the trained slot dynamics model in H-step, and the predicted trajectory of the target image in H-step is output. The predicted trajectory in H-step represents the evolution of the relationship between various objects in the target image.

[0007] Optionally, it also includes: Input the K slot vectors of the sample image into the policy model and the reward model to be trained, and output the first step of the robotic arm action and the first step reward value of the sample image respectively. Input the K predicted slot vectors of the first step into the policy model and the reward model to be trained, and output the robotic arm action and the reward value of the second step of the sample image respectively; The slot-level relationship tensor of the sample image is injected into the slot dynamics model to be trained and the first step of prediction is performed. The output of the first step predicted relationship and the first step K predicted slot vectors of the sample image are included: The first step robotic arm action, the slot-level relationship tensor of the sample image, and the K slot vectors of the sample image are processed by the slot dynamics model to be trained, and the first step predicted relationship and the first step K predicted slot vectors of the sample image are output. Taking h sequentially from 2 to H-1, the (h-1)th step prediction relation output by the slot dynamics model to be trained is injected into the slot dynamics model to be trained, and the h-th step prediction is performed. The h-th step prediction relation and the h-th step K predicted slot vectors of the output sample image are included: Take h sequentially from 2 to H-1, and perform the following steps: The h-th step robotic arm action, h-1 step prediction relationship, and h-1 step K prediction slot vectors output by the slot dynamics model to be trained are processed by the slot dynamics model to be trained, and the h-th step prediction relationship and h-th step K prediction slot vectors are output from the sample image. Based at least on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, the slot dynamics model to be trained is trained to obtain the trained slot dynamics model, including: Based on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, the training strategy model, the training reward model, and the training slot dynamics model are performed to obtain the trained slot dynamics model, the trained strategy model, and the trained reward model.

[0008] Optionally, based on the K slot vectors and K segmentation masks of the sample image, and the ground truth instance mask of each object in the sample image, the object-level relation graph of the sample image is converted into a slot-level relation tensor of the sample image, including: Based on the K segmentation masks of the sample image and the ground instance masks of each object in the sample image, determine the IoU between each segmentation mask and each ground instance mask; Using the IoU between each segmentation mask and each ground truth instance mask as the edge weight, and using the slot corresponding to each segmentation mask and the object corresponding to each ground truth instance mask as nodes, a weighted bipartite graph is constructed. Based on a weighted bipartite graph, the goal is to maximize the sum of edge weights of the K slots corresponding to the K segmentation masks and bind the K slots to each object in the sample image. In the case that there is a r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 1; In the case that there is no r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 0.

[0009] Optionally, the slot dynamics model to be trained is trained based at least on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, to obtain the trained slot dynamics model, including: Based on the predicted relationship at step h of the sample image, and the relationship label in the object-attribute-relationship graph of the sample image at step h of the sample image sequence, determine the relationship loss value at step h; Based on the predicted slot vector at step h of the sample image, and the object label and attribute label in the object-attribute-relationship graph of the sample image at frame h in the sample image sequence, the slot loss value at step h is determined. Based at least H-step relationship loss and H-step slot loss, the model parameters of the slot dynamics model to be trained are updated to obtain the trained slot dynamics model.

[0010] Optionally, it also includes: Input the K predicted slot vectors of step H into the reward model to be trained, and output the reward value of the target image at step H. Based on the H-step predicted trajectory of sample images and the object-attribute-relationship graph sequence of sample image sequences, the policy model, reward model, and slot dynamics model to be trained are trained to obtain the trained slot dynamics model, the trained policy model, and the trained reward model, including: Based on the reward value from step 1 to step H, the reward value for step H is obtained; Based on the H-step reward value, H-step relation loss value, and H-step slot loss value, the model parameters of the policy model, reward model, and slot dynamics model to be trained are updated to obtain the trained slot dynamics model, trained policy model, and trained reward model.

[0011] Optionally, the target image is predicted using an H-step method based on the trained slot dynamics model, and the predicted H-step trajectory of the target image is output, including: The target image is slot-encoded using the pre-trained slot encoder to obtain K slot vectors of the target image; Input the K slot vectors of the target image into the trained policy model, and output the first step of the robotic arm action of the target image; Control the robotic arm according to the first step of the target image's robotic arm movement; The trained slot dynamics model is used to process the first step robotic arm action and the K slot vectors of the target image output by the strategy model, and outputs the first step prediction relationship and the first step K predicted slot vectors of the target image. Input the first K predicted slot vectors into the trained policy model, and output the second step robotic arm action of the target image; Control the robotic arm according to the second step of the target image's robotic arm movement; Take h sequentially from 2 to H-1, process the h-th step robotic arm action output by the strategy model, the h-1-th step prediction relationship output by the trained slot dynamics model, and the h-1-th step K predicted slot vectors, and output the h-th step prediction relationship and the h-th step K predicted slot vectors of the target image. Input the h-th step K predicted slot vectors of the target image into the trained policy model, and output the h-th step robotic arm action of the target image. The robotic arm is controlled according to the h-th step of the target image.

[0012] A second aspect of this application discloses a model training apparatus based on an object-attribute-relationship graph, the apparatus comprising: The relation extraction module is used to extract relations from the sample image through the relation extractor, and obtain the object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image. The slot encoding module is used to encode the sample image using a pre-trained slot encoder, resulting in K slot vectors and K segmentation masks for the sample image. The relation transcribing module is used to transcrib the object-level relation graph of the sample image into a slot-level relation tensor based on the K slot vectors and K segmentation masks of the sample image, as well as the ground instance mask of each object in the sample image. The relation slot prediction module is used to inject the slot-level relation tensor of the sample image into the slot dynamics model to be trained and perform the first step prediction, outputting the first step prediction relation and the first step K predicted slot vectors of the sample image; taking h from 2 to H-1 in sequence, the h-1 step prediction relation output by the slot dynamics model to be trained is injected into the slot dynamics model to be trained and the h-th step prediction is performed, outputting the h-th step prediction relation and the h-th step K predicted slot vectors of the sample image; The model training module is used to train the slot dynamics model to be trained based on at least H-step predicted trajectories of sample images and object-attribute-relationship graph sequences of sample image sequences, so as to obtain the trained slot dynamics model; the sample image sequence is a series of H frame sample images with sample images as the starting frame; The trajectory prediction module is used to perform H-step prediction on the target image using the trained slot dynamics model, and outputs the H-step predicted trajectory of the target image. The H-step predicted trajectory represents the evolution process of the relationship between various objects in the target image.

[0013] A third aspect of this application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the model training method based on object-attribute-relationship graph described in the first aspect of this application.

[0014] A fourth aspect of this application discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the object-attribute-relationship graph-based model training method described in the first aspect of this application.

[0015] A fifth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the object-attribute-relationship graph-based model training method described in the first aspect of this application.

[0016] The embodiments of this application have the following advantages: In this embodiment, an object-level relation graph of the sample image is first obtained through a relation extractor, and K segmentation masks are obtained through a slot encoder. Based on the K segmentation masks and the ground truth instance masks of each object, the object-level relation graph is rewritten into a slot-level relation tensor, accurately mapping the originally object-defined relations to unnamed slot pairs. On this basis, the slot-level relation tensor is injected into the slot dynamics model, ensuring that each prediction step has explicit typified relation constraints. This provides the model with stable structured priors at interaction moments such as contact and occlusion, reducing the probability of slot content swapping and fundamentally suppressing representation drift.

[0017] Meanwhile, during the prediction process of the tank dynamics model, the prediction relationship of step h-1 is injected into the model before the prediction of step h, realizing the autoregressive coupling between relationship prediction and tank prediction. Since the relationship injected in each step is a structured relationship predicted by the model itself, this relationship continuously provides explicit constraints at each step of the prediction trajectory, so that the single-step prediction error will not be amplified without constraints in subsequent steps, reducing the cumulative deviation of the multi-step prediction trajectory and improving the overall accuracy of the H-step prediction trajectory.

[0018] Furthermore, the slot dynamics model is trained based on at least H-step predicted trajectories and object-attribute-relationship graph sequences of sample image sequences, enabling the model to learn the ability to predict relationships from slot vectors. During the inference phase, when performing H-step predictions on the target image using the trained slot dynamics model, the predicted relationships for each step can be output without relying on external relationship ground truths, thus continuing to advance subsequent predictions. This eliminates the dependence on simulated physical states or external scene graph recognition networks, facilitating practical deployment.

[0019] In summary, due to the improved accuracy of the predicted trajectory, the downstream control strategy trained based on this predicted trajectory can obtain more reliable training signals, achieving higher control success rate and sample efficiency in multi-object manipulation tasks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the steps of a model training method based on an object-attribute-relationship graph provided in an embodiment of this application. Figure 2 This is an overall architecture diagram of a model training method based on an object-attribute-relationship graph provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a model training device based on an object-attribute-relationship graph provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] First, the shortcomings of the relevant technologies are analyzed: Model-based reinforcement learning predicts state evolution in the latent space through a world model, allowing the agent to train its control strategy through imaginative deduction. Object-centric world models further decompose the scene into slots, each corresponding to an object. However, the applicant has identified at least the following problems with existing object-centric world models: 1. Slots lack semantic anchoring, making them prone to misalignment during interaction. In existing object-centric world models, slots lack a binding relationship with specific objects. When multiple objects come into contact, collide, or occlude, the content of the slots can be swapped between different objects. For example, a slot that originally tracked object A might switch to tracking object B, resulting in inconsistent slot semantics over time.

[0024] Second, relationships between objects emerge implicitly, lacking explicit structural constraints. Existing object-centric world models implicitly delegate the modeling of relationships between objects to the attention mechanism of Transformers, with no explicit relationship representation within the model itself. At moments of abrupt relationship changes such as contact or separation, the attention weights change drastically and unstablely, making it difficult for the model to accurately capture these abrupt changes, resulting in a significant increase in prediction errors at the moment of interaction.

[0025] Third, errors in autoregressive predictions are amplified at each stage. The imagined deduction of the world model is a stepwise autoregressive process, with each step's prediction result serving as the input for the next. Under the combined effect of the two problems mentioned above, errors in single-step predictions caused by slot misalignment or relationship misjudgments are continuously amplified in subsequent steps, ultimately leading to long-term imagined trajectories deviating from the actual trajectories. Since control strategy training relies on imagined trajectories, trajectory deviations directly cause strategy degradation, preventing the object-centric world model from achieving expected performance in downstream control tasks.

[0026] To address the aforementioned issues, this application proposes a model training method, apparatus, and device based on an object-attribute-relationship graph. The technical concept is as follows: An object-level relationship graph is obtained through a relationship extractor; K segmentation masks are obtained through a pre-trained slot encoder; Hungarian matching is performed using the intersection-union ratio (IUU) of the segmentation masks and the object ground truth instance masks to transcribe the object-level relationship graph into a slot-level relationship tensor, thereby mapping the object-defined relationships to unnamed slots; the slot-level relationship tensor is injected into the slot dynamics model, enabling each step in multi-step prediction to obtain explicit typed relationship constraints and autoregressively outputting predicted relationships and predicted slot vectors; the model is trained based on H-step predicted trajectories and sample image sequences, allowing the model to self-output relationship predictions during the inference stage, eliminating dependence on external relation ground truths.

[0027] The technical solution of this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Reference Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of a model training method based on an object-attribute-relationship graph provided in an embodiment of this application. Figure 1 As shown, the method may include steps S110 to S170: Step S110: Extract relations from the sample image using a relation extractor to obtain an object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image.

[0029] In this step, the sample images are image frames used during the training phase, which can come from a simulation environment or data collected by a real robot. The object-level relationship graph is used to describe the spatial and physical relationships between the various objects in the image.

[0030] The object-level relationship graph is represented by a graph structure, where each node corresponds to an object in the sample image; an object refers to a movable body (e.g., a cube, sphere, cylinder, etc. on a table); in a scene containing a robotic arm, the robotic arm as a whole can also be considered a node. Each edge in the graph represents the type of relationship between two objects, including but not limited to contact, support, above, directional relationships (e.g., front / back / left / right), gripping, and inside a container.

[0031] The relation extractor can generate these relations based on the physical simulation state through preset judgment rules, or it can estimate them from images through a separately trained scene graph recognition network.

[0032] Step S120: Slot encoding is performed on the sample image using a pre-trained slot encoder to obtain K slot vectors and K segmentation masks for the sample image.

[0033] In this step, the slot encoder employs an autoencoder structure based on a slot attention mechanism. Its input is the original RGB (Red, Green, Blue) image, and its output consists of K fixed-dimensional slot vectors and K segmentation masks. Each slot vector is a vector representation in a low-dimensional latent space, corresponding to an object or background in the image. Each segmentation mask is a two-dimensional probability map with the same resolution as the input image, indicating which pixels in the image are claimed by the corresponding slot; that is, the value of each pixel in the mask represents the probability that the pixel belongs to the corresponding slot. The slot encoder is pre-trained in a large amount of image data under self-supervised conditions. After entering the training process of this application, its parameters are frozen and it is not used as a trainable module in subsequent updates.

[0034] Step S130: Based on the K slot vectors and K segmentation masks of the sample image, and the ground truth instance mask of each object in the sample image, the object-level relation graph of the sample image is converted into the slot-level relation tensor of the sample image.

[0035] This step addresses the alignment problem caused by unnamed slots. The relationship output by step S110 is defined by objects, such as "object A is on top of object B". However, the slots output by step S120 are unnamed, meaning it is unknown which slot corresponds to object A and which slot corresponds to object B. Therefore, it is impossible to directly assign object-level relationships to slots.

[0036] To this end, this step calculates the Intersection over Union (IoU) between the K segmentation masks and the ground truth instance masks of each object in each sample image. Then, the Hungarian algorithm is used to find the optimal match, assigning each object to the slot with the highest overlap with its segmentation mask. After matching, each edge in the object-level relationship graph (where both the source and target objects have been matched to their corresponding slots) is transcribed into a multi-label relationship indicator between slot pairs, resulting in a slot-level relationship tensor. This slot-level relationship tensor is... A three-dimensional tensor, where R is the total number of relation types, and each element indicates whether a certain relation exists between a pair of slots.

[0037] Step S140: Inject the slot-level relationship tensor of the sample image into the slot dynamics model to be trained and perform the first step prediction, outputting the first step prediction relationship and the first step K predicted slot vectors of the sample image.

[0038] In this step, the slot dynamics model employs a Transformer structure that alternates between object attention and temporal attention. Its inputs are the K slot vectors at the current time step, the currently executed action information, and the relation tensor at the current time step. The outputs are the K predicted slot vectors for the next time step and the predicted relations for the next time step.

[0039] In this context, injection refers to the structured application of the slot-level relation tensor to the object attention computation of the slot dynamics model. Specifically, the type embedding vectors corresponding to each activation relation type in the slot-level relation tensor are aggregated to obtain the edge embedding between each pair of slots. Then, a linear transformation is used to map this edge embedding to the scalar biases of multiple attention heads, and these biases are superimposed on the attention score of the object attention. In this way, the slot-level relation tensor directly influences which other slots each slot should pay attention to during the attention computation, achieving structured prior injection.

[0040] In the first step of prediction, the slot dynamics model outputs the first step predicted slot vector and the first step predicted relationship based on the slot vector of the current frame of the sample image, the current action, and the slot-level relationship tensor obtained in step S130.

[0041] Step S150: Take h sequentially from 2 to H-1, inject the h-1th step prediction relationship output by the slot dynamics model to be trained into the slot dynamics model to be trained and perform the h-th step prediction, output the h-th step prediction relationship of the sample image and the h-th step K predicted slot vectors.

[0042] Specifically, from step 2 to step H-1, the slot dynamics model no longer uses the true values ​​of external relations. Instead, it injects the predicted relations output by itself in the previous step as the relation input for the current step into the model. Each step's prediction consists of two parts: the K predicted slot vectors for the next time step and the predicted relations for the next time step. The predicted relations continue to participate in the deduction as relation inputs for the next step.

[0043] Thus, the slot dynamics model, starting from the initial frame, generates a predicted trajectory with H time steps through H-1 steps of autoregressive deduction, where each step includes a predicted slot vector and a predicted relationship.

[0044] Step S160: Train the slot dynamics model to be trained based on at least the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence to obtain the trained slot dynamics model; the sample image sequence is a series of H consecutive sample images with the sample image as the starting frame.

[0045] In this step, the object-attribute-relationship graph sequence of the sample image sequence is extracted by the relation extractor in step S110 on each frame, and is used as the supervision label in the training process.

[0046] The H-step predicted trajectories generated in steps S140 to S150 are compared with the aforementioned supervised labels, and the loss function is calculated. The loss function includes at least: the difference loss between the predicted slot vector and the actual slot encoding (measuring the accuracy of slot prediction), and the difference loss between the predicted relationship and the actual relationship (measuring the accuracy of relationship prediction). The parameters of the slot dynamics model are updated through backpropagation, enabling it to gradually learn accurate dynamic evolution laws.

[0047] Step S170: Perform H-step prediction on the target image using the trained slot dynamics model, and output the H-step predicted trajectory of the target image. The H-step predicted trajectory represents the evolution process of the relationship between various objects in the target image.

[0048] During the inference phase, the input is the target image (i.e., the current frame captured by the camera in the actual deployment scenario). The target image is encoded into K initial slot vectors and K segmentation masks by a pre-trained slot encoder. Then, following the autoregressive approach of steps S140 to S150, the trained slot dynamics model is injected with its own predicted relationships at each step, inferring H steps forward frame by frame.

[0049] The final output H-step predicted trajectory describes the evolution of the relationships between various objects in the scene over the next H time steps starting from the target image. It can be used for downstream tasks, such as the decision-making basis for control strategies or the forward-looking assessment of scene changes. The technical solution adopted in this embodiment transforms the explicit object-level relationship graph into a slot-level relationship tensor and injects it into the slot dynamics model during the training phase. This allows the model to obtain typified relationship constraints at each step of multi-step prediction. By autoregressively driving subsequent predictions with its own predicted relationships, the model is freed from dependence on the truth values ​​of external relationships during the inference phase. Through comparative training with H-step predicted trajectories and true sequences, the model learns the joint evolution law of relationships and slots. Since each prediction step is guided by the explicit relationship structure, the probability of slot cross-positioning at the moment of interaction is reduced, and the cumulative error of multi-step prediction is effectively suppressed, thereby improving the accuracy of long-term predicted trajectories.

[0050] As an optional implementation, step S130 may include steps S130-1 to S130-5: Step 130-1: Determine the IoU between each segmentation mask and each ground truth instance mask based on the K segmentation masks of the sample image and the ground truth instance mask of each object in the sample image.

[0051] Intersection over Union (IoU) is a metric that measures the degree of overlap between two masks. It is calculated by dividing the area of ​​their intersection by the area of ​​their union, and its value ranges from 0 to 1. The closer the IoU is to 1, the higher the degree of overlap between the two masks, meaning the better the segmentation result of that slot matches the true contour of the object. This step calculates the IoU between the segmentation mask of each slot and the ground truth instance mask of each object, obtaining... An IoU matrix (O is the number of objects) that reflects the matching affinity between each slot and each object.

[0052] Before calculating IoU, the segmentation mask needs to be binarized. The segmentation mask is a pixel-wise softmax output between slots, meaning the sum of the probabilities of the K slots for each pixel in the image is 1. This embodiment uses pixel-wise argmax binarization, assigning each pixel to the slot with the highest probability, forming a one-hot binary mask for each slot. This method effectively avoids the problem of a fixed threshold (e.g., 0.5) erasing the entire mask when dealing with small objects or soft boundaries, ensuring the accuracy of subsequent IoU calculations. Experimental data shows that after using argmax binarization, the binding dropout rate of the same model decreases from 42.5% using the threshold method to 11.5%, ensuring the accuracy of subsequent IoU calculations.

[0053] Step 130-2: Using the IoU between each segmentation mask and each ground truth instance mask as the edge weight, and using the slot corresponding to each segmentation mask and the object corresponding to each ground truth instance mask as nodes, construct a weighted bipartite graph.

[0054] In this step, K slots are designated as nodes on one side of the bipartite graph, and O objects are designated as nodes on the other side. For any slot k and any object o, the IoU value calculated in step S130-1 is used as the weight of the edge connecting these two nodes, constructing a complete weighted bipartite graph. This bipartite graph fully describes all possible matching relationships between slots and objects and their matching quality.

[0055] Step 130-3: Based on the weighted bipartite graph, with the goal of maximizing the sum of edge weights of the K slots corresponding to the K segmentation masks, bind the K slots to each object in the sample image.

[0056] In this step, the Hungarian algorithm is used to solve the maximum weight matching problem on the weighted bipartite graph. The one-to-one correspondence between slots and objects that maximizes the total matching weight is found. Each slot is assigned a unique object with the highest overlap with its segmentation mask, ensuring the global optimality of the matching.

[0057] After matching is completed, matches with an IoU lower than a preset threshold (e.g., 0.3) are discarded, meaning the object is considered unbound to any slot. This is because a low IoU match implies that the slot segmentation quality is insufficient to reliably represent the object; forcibly binding it might introduce incorrect relationship information into subsequent processing. Experimental data shows that in 2 / 3 / 4 object tasks, using... The criteria and argmax binarization resulted in binding dropout rates of 10.2%, 14.6%, and 15.4%, respectively, demonstrating the system's robustness to edge loss of this magnitude.

[0058] Step 130-4: If there is a r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, determine that the r-th element of the slot pair ij in the slot-level relationship tensor is 1.

[0059] Step 130-5: If there is no r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, determine that the r-th element of the slot pair ij in the slot-level relationship tensor is 0.

[0060] Specifically, after binding slots to objects, the object-level relationships can be transferred to the slot level. For each relationship edge in the object-level relationship graph, representing the r-th relationship (such as contact, support, etc.) between the i-th and j-th objects, based on the binding results of step S130-3, the slot index bound to the i-th object and the slot index bound to the j-th object are found. Then, the r-th element of the corresponding slot pair in the slot-level relationship tensor is set to 1. For non-existent relationships, they are set to 0. Through traversal, each relationship in the object-level relationship graph is transferred to the corresponding position in the slot-level relationship tensor, ultimately forming a complete relationship. Slot-level relation tensor.

[0061] The technical solution of this implementation achieves globally optimal binding between slots and objects by constructing a weighted bipartite graph and using the Hungarian algorithm to solve for optimal matching, avoiding erroneous correspondences that may result from local greedy matching. Low-quality matches are discarded using an IoU threshold, filtering out unreliable binding relationships and preventing erroneous relationship information from contaminating the slot-level relationship tensor. Binarization using argmax ensures that the segmentation mask of small objects is not completely erased during IoU calculation, preserving bindings that might otherwise be lost due to a fixed threshold and improving the binding success rate.

[0062] As an optional implementation, step S160 may include steps S160-1a to S160-3a: Step S160-1a: Based on the h-th step predicted relationship of the sample image, and the relationship label in the object-attribute-relationship graph of the h-th frame sample image in the sample image sequence, determine the h-th step relationship loss value.

[0063] In this step, for each step h in the predicted trajectory, the difference between the predicted relationship at step h output by the model and the relationship label of the sample image at frame h is the relationship loss value at step h. The relationship loss value at step h reflects the accuracy of the slot dynamics model in predicting the interaction relationship between objects at that time step. The smaller the loss value, the better the relationship predicted by the model matches the actual relationship.

[0064] Specifically, the relation loss employs a multi-label classification loss per slot pair and per relation type. In some embodiments, considering the low proportion of slot pairs with relations in real-world scenarios (most slot pairs do not have any relation between them) and the severe imbalance between positive and negative samples, Focal Loss is used as the loss function. Focal Loss, by introducing a modulation factor, reduces the contribution of a large number of easily classified negative samples to the loss, allowing the model to focus more on the sparse positive relation samples during training, thereby improving the accuracy of relation prediction.

[0065] Step S160-2a: Based on the h-th step predicted slot vector of the sample image, and the object label and attribute label in the object-attribute-relationship graph of the h-th frame sample image in the sample image sequence, determine the h-th step slot loss value.

[0066] In this step, for each step h in the predicted trajectory, the h-th step predicted slot vector output by the model describes the model's prediction of the scene state at the next moment. The slot vector obtained by encoding the h-th frame sample image in the sample image sequence through the pre-trained slot encoder is used as the true slot label, and the difference between the two is the h-th step slot loss value.

[0067] Specifically, the slot loss can be calculated using mean squared error (MSE) or smoothed L1 loss, which measures the element-wise difference between the predicted slot vector and the true slot vector. This loss measures the accuracy of the model's prediction of the states of various objects in the scene (such as position, pose, appearance features, etc.). The smaller the loss value, the more accurate the model's prediction of the scene evolution.

[0068] In some implementations, image reconstruction loss can also be introduced as an auxiliary supervision, the predicted slot vector is reconstructed into an image by the decoder, and the pixel-level reconstruction error is calculated with the real image, further constraining the slot vector to carry sufficient visual information.

[0069] Step S160-3a: Based at least on the H-step relationship loss value and the H-step slot loss value, update the model parameters of the slot dynamics model to be trained to obtain the trained slot dynamics model.

[0070] In this step, the relationship loss values ​​from step 1 to step H are summed (e.g., by summing or averaging) to obtain the H-step relationship loss value, and the slot loss values ​​from step 1 to step H are summed to obtain the H-step slot loss value. The weighted sum of these two values ​​is used as the total loss of the slot dynamics model. The gradient of the total loss with respect to each trainable parameter of the slot dynamics model is calculated through backpropagation, and an optimizer is used to update the parameters. After multiple rounds of iterative training, the trained slot dynamics model is obtained.

[0071] In some implementations, a course learning strategy can be introduced during the training process. That is, in the early stage of training (such as the first 30% of steps), a teacher-mandated approach is adopted, injecting real relationships in each step so that the model can learn to use these relationship information. In the later stage of training, a planned sampling approach is gradually adopted, in which each step randomly selects between real relationships and the model's own predicted relationships with a certain probability (such as 0.5), so that the model can gradually adapt to the self-provided relationship mode in the inference stage, alleviate the exposure bias in autoregressive prediction, and prevent the divergence of self-predicted relationships.

[0072] The technical solution of this embodiment achieves fine-grained supervision of the predictive capability of the slot dynamics model by calculating the relation loss and slot loss at each step separately. The relation loss ensures that the model learns to capture the interaction relationships between objects, while the slot loss ensures that the model learns to accurately predict the scene state. The joint optimization of the two enables the model to have dual predictive capabilities for both scene changes and relationships between objects.

[0073] As an optional implementation, the method, in addition to the steps described above, further includes steps A1 to A2: Step A1: Input the K slot vectors of the sample image into the policy model and the reward model to be trained, and output the first step robotic arm action and the first step reward value of the sample image respectively.

[0074] In this step, the strategy model determines the actions the robotic arm should perform based on the current scene state. Its input is the K slot vectors at the current moment (representing the state of each object), and its output is the robotic arm action (e.g., the pose increment of the end effector and the opening and closing amount of the gripper). The reward model evaluates the quality of the current state. Its input is also K slot vectors, and its output is a reward value, which measures how close the current state is to the task objective.

[0075] Step A2: Input the K predicted slot vectors from step 1 into the policy model and the reward model to be trained, and output the robotic arm action and reward value of step 2 of the sample image respectively.

[0076] In this step, after the slot dynamics model completes its prediction and outputs the first K predicted slot vectors in step 1, the policy model and reward model again infer based on these predicted slot vectors, outputting the robotic arm action and reward value for step 2, respectively. Similarly, in each subsequent prediction step, the policy model outputs the action for the current step based on the slot vectors predicted in the previous step, while the reward model outputs the reward value for the current step based on the predicted slot vectors. The action is used as a conditional input by the slot dynamics model to predict the next scene evolution, forming a closed-loop imaginary process of "scene prediction - action decision - re-prediction".

[0077] Based on the steps described above, steps S140 to S160 are further specified as follows: Step S140-1: Process the first step robotic arm action, the slot-level relationship tensor of the sample image, and the K slot vectors of the sample image output by the slot dynamics model to be trained, and output the first step predicted relationship and the first step K predicted slot vectors of the sample image. Specifically, in the first prediction step, the input to the slot dynamics model includes three parts: the K slot vectors at the current moment, the slot-level relationship tensor, and the first step robotic arm action output by the policy model. After fusing the above information, the slot dynamics model outputs a prediction of the scenario at the next moment, namely the first step prediction relationship and the first step K predicted slot vectors.

[0078] Step S150-1: Take h from 2 to H-1 in sequence and perform the following steps: Process the h-th step robotic arm action output by the strategy model, the h-1 step prediction relationship output by the h-1 step of the slot dynamics model, and the h-1 step K prediction slot vectors through the slot dynamics model to be trained, and output the h-th step prediction relationship and the h-th step K prediction slot vectors of the sample image.

[0079] Specifically, for each step from step 2 to step H-1, the input to the slot dynamics model also includes three parts: the robotic arm action output by the strategy model based on the predicted slot vectors from the previous step, the predicted relationship output by the slot dynamics model from the previous step, and the K predicted slot vectors output by the slot dynamics model from the previous step. Starting from step 2, the relationship input no longer comes from external truth values, but is entirely predicted by the model itself in the previous step, achieving self-sufficiency in the relationship.

[0080] Step S160-1b: Based on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, train the policy model, reward model, and slot dynamics model to be trained, and obtain the trained slot dynamics model, the trained policy model, and the trained reward model.

[0081] Specifically, the H-step predicted trajectory includes the predicted slot vector, predicted relation, robotic arm action, and reward value for each step. During training, on the one hand, the predicted slot vector and predicted relation are compared with the actual slot codes and actual relations at the corresponding time points in the sample image sequence to calculate the slot prediction loss and relation prediction loss, updating the parameters of the slot dynamics model. On the other hand, a discounted cumulative reward is calculated based on the predicted trajectory, serving as the optimization objective of the policy model, enabling the actions output by the policy model to obtain higher cumulative rewards, while simultaneously updating the reward model to more accurately evaluate state value. The three sets of models are updated alternately or jointly within a unified training framework.

[0082] The technical solution of this embodiment tightly couples scene evolution prediction and action decision-making by incorporating the robotic arm action output by the strategy model at each step of the predicted trajectory. The slot dynamics model fully considers the impact of actions on scene changes during prediction, while the strategy model makes the next action decision based on the predicted future scene, with the two mutually reinforcing each other. The reward model provides real-time evaluation at each step of the predicted trajectory, enabling the strategy model to learn the optimal strategy for long-term decision-making from imagination.

[0083] As an optional implementation, the method, in addition to the steps described above, also includes step A3: Step A3: Input the H-th predicted slot vector into the reward model to be trained, and output the H-th reward value of the target image.

[0084] After completing all H-step scenario predictions, the K predicted slot vectors of step H are input into the reward model to obtain the reward value for step H. Thus, from step 1 to step H, each time step in the predicted trajectory corresponds to a reward value, fully describing the state value of each point on the imagined trajectory and providing a complete reward sequence for subsequent long-range reward calculations.

[0085] Based on the steps described above, step S160-1b further includes steps S160-1b-1 to S160-1b-2: S160-1b-1: Based on the reward value from step 1 to step H, obtain the reward value for step H.

[0086] In this step, the reward values ​​from step 1 to step H constitute a reward sequence of length H, representing the model's assessment of the environmental state at each predicted time step. The reward value of step H can be the reward sequence itself, or it can be the reward value after discounting and accumulation.

[0087] For example, using The method calculates the cumulative discounted reward at each time step, and then weights and sums the immediate reward of the current step with the discounted rewards of subsequent steps to obtain the target value used for policy optimization. In this way, the rewards of individual steps are aggregated into an overall reward that reflects the quality of long-term decisions.

[0088] S160-1b-2: Based on the H-step reward value, H-step relation loss value, and H-step slot loss value, update the model parameters of the policy model, reward model, and slot dynamics model to be trained, and obtain the trained slot dynamics model, the trained policy model, and the trained reward model.

[0089] In this step, the training process jointly optimizes three sets of models. The total loss function includes at least H-step reward values, H-step relation loss values, and H-step slot loss values. Specifically, the policy model updates its parameters to maximize the H-step reward value, ensuring the policy's output action achieves higher cumulative rewards on the imagined trajectory; the reward model updates its parameters to more accurately predict actual rewards; and the slot dynamics model updates its parameters to minimize slot and relation losses. The three models can be updated alternately (e.g., first updating the slot dynamics and reward models, then updating the policy model based on the imagined trajectory output by the updated models), or they can be jointly backpropagated under the same loss function. Through multiple rounds of iterative training, the three models converge to a relatively optimal state.

[0090] The technical solution of this implementation accumulates the reward value of each step in the predicted trajectory into an H-step reward value. This transforms the optimization objective of the policy model from an isolated single-step reward to a multi-step cumulative return that reflects the overall effect of long-term decision-making, guiding the policy model to learn more forward-looking control decisions. Simultaneously, by incorporating slot loss, relational loss, and reward loss into a unified training framework, the slot dynamics model, reward model, and policy model work together to improve. This synergistic optimization ultimately enhances control performance in multi-object manipulation tasks.

[0091] As an optional implementation, step S170 may further include steps S170-1 to S170-9: Step S170-1: The target image is slot encoded using the pre-trained slot encoder to obtain K slot vectors of the target image.

[0092] At the start of the inference phase, the target image captured by the camera in the actual deployment scenario is input into the pre-trained slot encoder. This pre-trained slot encoder has the same structure and shares parameters as the slot encoder used in the training phase, and both are in a frozen state after pre-training. After processing the target image, the pre-trained slot encoder outputs K slot vectors. These slot vectors describe the latent space representation of each object and the background in the target image, serving as the basic input for all subsequent predictions and decisions.

[0093] Step S170-2: Input the K slot vectors of the target image into the trained policy model and output the first robotic arm action of the target image.

[0094] In this step, the strategy model makes decisions based on the current scene state (i.e., the K slot vectors of the target image) and outputs the first step of the robotic arm action. This robotic arm action is typically a control command such as the pose increment of the robotic arm's end effector and the opening and closing amount of the gripper.

[0095] Step S170-3: Control the robotic arm according to the robotic arm movement in step 1 of the target image.

[0096] In this step, the robotic arm action output from the strategy model in step 1 is converted into specific control signals and sent to the robotic arm for execution. The robotic arm completes the corresponding movement according to the action command, such as moving to the target position or adjusting the gripper opening, thereby realizing a real interaction with the environment.

[0097] Step S170-4: Process the first step robotic arm action and the K slot vectors of the target image output by the strategy model using the trained slot dynamics model, and output the first step prediction relationship and the first step K predicted slot vectors of the target image.

[0098] While executing the action, the trained slot dynamics model internally imagines and extrapolates the scene evolution. Using the K slot vectors of the target image as the current scene state and the first-step robotic arm action output by the policy model as conditional input, it outputs the first-step predicted relationship and the first-step K predicted slot vectors in the absence of external relational truth values. These first-step predicted relationship and the first-step K predicted slot vectors describe the changes in the scene and the evolution of relationships between objects under the current action.

[0099] Step S170-5: Input the K predicted slot vectors from step 1 into the trained policy model and output the robotic arm action from step 2 of the target image.

[0100] In this step, the K predicted slot vectors predicted by the slot dynamics model in the first step are input into the trained policy model. The policy model decides the robotic arm action in the second step based on the imagined state of scene changes after the robotic arm action in the first step.

[0101] Step S170-6: Control the robotic arm according to the robotic arm movement in step 2 of the target image.

[0102] In this step, the robotic arm's action from step 2 is sent to the robotic arm for execution. Thus, in the physical world, the robotic arm is progressively executing a sequence of actions; within the internal model, the slot dynamics model synchronously predicts the scene changes after each action, and the strategy model makes advance decisions about the next action based on the predicted future scene.

[0103] Step S170-7: Take h sequentially from 2 to H-1, process the h-th step robotic arm action output by the strategy model, the h-1 step prediction relationship output by the trained slot dynamics model, and the h-1 step K predicted slot vectors, and output the h-th step prediction relationship and the h-th step K predicted slot vectors of the target image.

[0104] In this step, for each step from step 2 to step H-1, the input to the slot dynamics model includes three parts: the current step robot action output by the strategy model based on the predicted slot vector of the previous step, the predicted relation output by the slot dynamics model of the previous step, and the predicted slot vector output by the slot dynamics model of the previous step. Based on the above information, the slot dynamics model predicts the next scenario and outputs the predicted relation for step h and the K predicted slot vectors for step h. Starting from step 2, the relation input comes entirely from the model's own prediction of the previous step and does not depend on any external relation truth values.

[0105] Step S170-8: Input the h-th step K predicted slot vector of the target image into the trained policy model, and output the h-th step robotic arm action of the target image.

[0106] In this step, the strategy model outputs the robotic arm action to be performed based on the predicted future scenario state (the K predicted slot vectors in step h), realizing a decision-making method that plans ahead in the imagination.

[0107] Step S170-9: Control the robotic arm according to the h-th step of the target image.

[0108] In this step, each robotic arm action output by the strategy model is sequentially sent to the robotic arm for execution. Through the iterative cycle of steps S170-1 to S170-9, a closed-loop control process of "perception-prediction-decision-execution-re-perception" is achieved.

[0109] The technical solution of this embodiment involves the collaborative deployment of a trained slot dynamics model, a strategy model, and a pre-trained slot encoder in a real robotic arm control system. The slot dynamics model provides predictions of scene evolution trends within the control loop, enabling the strategy model to anticipate the future state of the scene after the current action is executed. This allows for advance planning of subsequent actions, enabling the robotic arm to complete complex operations with higher efficiency and success rate in multi-object manipulation tasks.

[0110] like Figure 2 As shown, Figure 2 This is an overall architecture diagram of a model training method based on an object-attribute-relationship graph provided in an embodiment of this application. Specifically, the overall process of this solution is as follows: First, the sample images are input into the relation extractor and the pre-trained slot encoder, respectively. The relation extractor outputs an object-level relation graph of the sample images, describing the spatial and physical relationships between objects in the sample images in a graph structure. The pre-trained slot encoder outputs K slot vectors and K segmentation masks.

[0111] Secondly, by calculating the IoU between the segmentation mask and the truth instance mask of each object and performing Hungarian matching, the object-level relation graph is rewritten into a slot-level relation tensor, which solves the alignment problem that relations cannot be directly assigned to slots due to unnamed slots.

[0112] Next, the slot-level relation tensor is injected into the slot dynamics model to be trained. The slot dynamics model takes the current slot vector, the current relation tensor, and the current action output by the policy model as input to predict the slot vector and relation for the next time step. In the multi-step prediction process, starting from step 2, the model no longer uses the external relation ground truth, but instead uses the predicted relation output from the previous step as the relation input for the current step, achieving self-sufficiency in relation supply. The predicted slot vector for each step is simultaneously input into the policy model and the reward model, which output the robotic arm action and reward value for the current step, respectively.

[0113] Finally, the H-step predicted trajectory output by the slot dynamics model is compared with the object-attribute-relationship graph sequence of the sample image sequence (as a supervision signal) to calculate the slot loss and relationship loss. At the same time, the policy optimization objective is calculated based on the H-step reward value output by the reward model, and the parameters of the untrained slot dynamics model, policy model and reward model are updated together.

[0114] After training, deployment can be performed during the inference phase. The target image is input into the pre-trained slot encoder to obtain K slot vectors. The trained policy model outputs the robotic arm's actions and issues them for execution. Simultaneously, the trained slot dynamics model, based on the actions output by the policy model, outputs the predicted relationship and predicted slot vector for each step, forming a closed-loop control process of "perception-prediction-decision-execution". Experimental data show that the median policy reward of this application reaches 4.87 on the 2-object task, significantly better than the implicit relationship baseline of 0.74 and the unsupervised estimation baseline of 2.54. On the 3 / 4-object task, the relationship spectrum optimal model outperforms the implicit baseline in all 6 "task × out-of-distribution settings", with the control success rate showing a monotonically increasing trend along the spectrum. This application enables higher control success rate and sample efficiency in multi-object manipulation tasks.

[0115] This application also provides a model training device based on an object-attribute-relationship graph, referring to... Figure 3 As shown, Figure 3 This is a schematic diagram of a model training device based on an object-attribute-relationship graph provided in an embodiment of this application. The device includes: The relation extraction module 310 is used to extract relations from the sample image through the relation extractor to obtain the object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image. The slot encoding module 320 is used to encode the sample image using a pre-trained slot encoder to obtain K slot vectors and K segmentation masks of the sample image. The relation transcribing module 330 is used to transcrib the object-level relation graph of the sample image into the slot-level relation tensor of the sample image based on the K slot vectors and K segmentation masks of the sample image, as well as the truth instance mask of each object in the sample image. The relation slot prediction module 340 is used to inject the slot-level relation tensor of the sample image into the slot dynamics model to be trained and perform the first step prediction, outputting the first step prediction relation and the first step K predicted slot vectors of the sample image; taking h from 2 to H-1 in sequence, the h-1 step prediction relation output by the slot dynamics model to be trained is injected into the slot dynamics model to be trained and the h-th step prediction is performed, outputting the h-th step prediction relation and the h-th step K predicted slot vectors of the sample image; The model training module 350 is used to train the slot dynamics model to be trained based on at least H-step predicted trajectories of sample images and object-attribute-relationship graph sequences of sample image sequences, so as to obtain the trained slot dynamics model; the sample image sequence is a series of H-frame sample images with sample images as the starting frame. The trajectory prediction module 360 ​​is used to perform H-step prediction on the target image using the trained slot dynamics model, and outputs the H-step predicted trajectory of the target image. The H-step predicted trajectory represents the evolution process of the relationship between various objects in the target image.

[0116] In an optional embodiment, the device further includes: The action and reward output module is used to input the K slot vectors of the sample image into the policy model and the reward model to be trained, and output the first step robotic arm action and the first step reward value of the sample image, respectively; and to input the first step K predicted slot vectors into the policy model and the reward model to be trained, and output the second step robotic arm action and the second step reward value of the sample image, respectively. The relational slot prediction module 340 is also used for: The first step robotic arm action, the slot-level relationship tensor of the sample image, and the K slot vectors of the sample image are processed by the slot dynamics model to be trained, and the first step predicted relationship and the first step K predicted slot vectors of the sample image are output. Take h sequentially from 2 to H-1, and perform the following steps: process the h-th step robotic arm action output by the strategy model, the h-1-th step prediction relationship output by the h-1-th step of the slot dynamics model, and the h-1-th step K predicted slot vectors through the slot dynamics model to be trained, and output the h-th step prediction relationship and the h-th step K predicted slot vectors of the sample image. The model training module 350 is also used to train the training strategy model, the training reward model, and the training slot dynamics model based on the H-step predicted trajectory of the sample image and the object-attribute-relationship graph sequence of the sample image sequence, so as to obtain the trained slot dynamics model, the trained strategy model, and the trained reward model.

[0117] In an optional embodiment, the transcription module 330 is further configured to: Based on the K segmentation masks of the sample image and the ground instance masks of each object in the sample image, determine the IoU between each segmentation mask and each ground instance mask; Using the IoU between each segmentation mask and each ground truth instance mask as the edge weight, and using the slot corresponding to each segmentation mask and the object corresponding to each ground truth instance mask as nodes, a weighted bipartite graph is constructed. Based on a weighted bipartite graph, the goal is to maximize the sum of edge weights of the K slots corresponding to the K segmentation masks and bind the K slots to each object in the sample image. In the case that there is a r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 1; In the case that there is no r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 0.

[0118] In an optional embodiment, the model training module 350 is further configured to: Based on the predicted relationship at step h of the sample image, and the relationship label in the object-attribute-relationship graph of the sample image at step h of the sample image sequence, determine the relationship loss value at step h; Based on the predicted slot vector at step h of the sample image, and the object label and attribute label in the object-attribute-relationship graph of the sample image at frame h in the sample image sequence, the slot loss value at step h is determined. Based at least H-step relationship loss and H-step slot loss, the model parameters of the slot dynamics model to be trained are updated to obtain the trained slot dynamics model.

[0119] In an optional embodiment, the action and reward output module is further configured to input the H-th step K predicted slot vector into the reward model to be trained, and output the H-th step reward value of the target image; The model training module 350 is further configured to obtain the H-step reward value based on the reward value from the first step to the H-step reward value; and to update the model parameters of the strategy model to be trained, the reward model to be trained, and the slot dynamics model to be trained based on the H-step reward value, the H-step relationship loss value, and the H-step slot loss value, so as to obtain the trained slot dynamics model, the trained strategy model, and the trained reward model.

[0120] In an optional embodiment, the trajectory prediction module 360 ​​is further configured to: The target image is slot-encoded using the pre-trained slot encoder to obtain K slot vectors of the target image; Input the K slot vectors of the target image into the trained policy model, and output the first step of the robotic arm action of the target image; Control the robotic arm according to the first step of the target image's robotic arm movement; The trained slot dynamics model is used to process the first step robotic arm action and the K slot vectors of the target image output by the strategy model, and outputs the first step prediction relationship and the first step K predicted slot vectors of the target image. Input the first K predicted slot vectors into the trained policy model, and output the second step robotic arm action of the target image; Control the robotic arm according to the second step of the target image's robotic arm movement; Take h sequentially from 2 to H-1, process the h-th step robotic arm action output by the strategy model, the h-1-th step prediction relationship output by the trained slot dynamics model, and the h-1-th step K predicted slot vectors, and output the h-th step prediction relationship and the h-th step K predicted slot vectors of the target image. Input the h-th step K predicted slot vectors of the target image into the trained policy model, and output the h-th step robotic arm action of the target image. The robotic arm is controlled according to the h-th step of the target image.

[0121] It is understood that the object-attribute-relationship graph-based model training device in the embodiments of this application can implement the object-attribute-relationship graph-based model training method in the above embodiments. The object-attribute-relationship graph-based model training device has the same advantages as the object-attribute-relationship graph-based model training method in the prior art, and will not be repeated here.

[0122] This application also provides an electronic device, see embodiments thereof. Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 4 As shown, the electronic device 400 includes a memory 410 and a processor 420. The memory 410 and the processor 420 are connected via a bus for communication. The memory 410 stores a computer program that can run on the processor 420 to implement the steps of the model training method based on object-attribute-relationship graph described in the embodiments of this application.

[0123] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the object-attribute-relationship graph-based model training method described in this application.

[0124] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the object-attribute-relationship graph-based model training method described in this application.

[0125] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0126] This application describes embodiments of methods and apparatus according to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0129] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0130] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0131] The above provides a detailed description of a model training method, apparatus, and device based on an object-attribute-relationship graph provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A model training method based on object-attribute-relationship graph, characterized in that, include: The relation extractor extracts the relation from the sample image to obtain the object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image. The sample image is slot-encoded by a pre-trained slot encoder to obtain K slot vectors and K segmentation masks for the sample image; Based on the K slot vectors and K segmentation masks of the sample image, as well as the ground truth instance mask of each object in the sample image, the object-level relation graph of the sample image is converted into the slot-level relation tensor of the sample image. The slot-level relationship tensor of the sample image is injected into the slot dynamics model to be trained and the first step of prediction is performed. The first step prediction relationship and the first step K predicted slot vectors of the sample image are output. Take h sequentially from 2 to H-1, inject the h-1 step prediction relationship output by the slot dynamics model to be trained into the slot dynamics model to be trained and perform the h-step prediction, output the h-step prediction relationship of the sample image and the h-step K predicted slot vectors; The slot dynamics model to be trained is trained based on at least H-step predicted trajectories of sample images and object-attribute-relationship graph sequences of sample image sequences to obtain trained slot dynamics models; the sample image sequence consists of H consecutive sample images starting with sample images. The target image is predicted using the trained slot dynamics model in H-step, and the predicted trajectory of the target image in H-step is output. The predicted trajectory in H-step represents the evolution of the relationship between various objects in the target image.

2. The model training method based on object-attribute-relationship graph according to claim 1, characterized in that, Also includes: Input the K slot vectors of the sample image into the policy model and the reward model to be trained, and output the first step of the robotic arm action and the first step reward value of the sample image respectively. Input the K predicted slot vectors of the first step into the policy model and the reward model to be trained, and output the robotic arm action and the reward value of the second step of the sample image respectively; The slot-level relationship tensor of the sample image is injected into the slot dynamics model to be trained and the first step of prediction is performed. The output of the first step predicted relationship and the first step K predicted slot vectors of the sample image are included: The first step robotic arm action, the slot-level relationship tensor of the sample image, and the K slot vectors of the sample image are processed by the slot dynamics model to be trained, and the first step predicted relationship and the first step K predicted slot vectors of the sample image are output. Taking h sequentially from 2 to H-1, the (h-1)th step prediction relation output by the slot dynamics model to be trained is injected into the slot dynamics model to be trained, and the h-th step prediction is performed. The h-th step prediction relation and the h-th step K predicted slot vectors of the output sample image are included: Take h sequentially from 2 to H-1, and perform the following steps: The h-th step robotic arm action, h-1 step prediction relationship, and h-1 step K prediction slot vectors output by the slot dynamics model to be trained are processed by the slot dynamics model to be trained, and the h-th step prediction relationship and h-th step K prediction slot vectors are output from the sample image. Based at least on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, the slot dynamics model to be trained is trained to obtain the trained slot dynamics model, including: Based on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, the training strategy model, the training reward model, and the training slot dynamics model are performed to obtain the trained slot dynamics model, the trained strategy model, and the trained reward model.

3. The model training method based on object-attribute-relationship graph according to claim 1, characterized in that, Based on the K slot vectors and K segmentation masks of the sample image, as well as the ground truth instance masks of each object in the sample image, the object-level relation graph of the sample image is rewritten into a slot-level relation tensor of the sample image, including: Based on the K segmentation masks of the sample image and the ground instance masks of each object in the sample image, determine the IoU between each segmentation mask and each ground instance mask; Using the IoU between each segmentation mask and each ground truth instance mask as the edge weight, and using the slot corresponding to each segmentation mask and the object corresponding to each ground truth instance mask as nodes, a weighted bipartite graph is constructed. Based on a weighted bipartite graph, the goal is to maximize the sum of edge weights of the K slots corresponding to the K segmentation masks and bind the K slots to each object in the sample image. In the case that there is a r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 1; In the case that there is no r-th relationship between the i-th object and the j-th object in the object-level relationship graph of the sample image, the r-th element of the slot pair ij in the slot-level relationship tensor is determined to be 0.

4. The model training method based on object-attribute-relationship graph according to claim 1, characterized in that, Based at least on the H-step predicted trajectory of the sample images and the object-attribute-relationship graph sequence of the sample image sequence, the slot dynamics model to be trained is trained to obtain the trained slot dynamics model, including: Based on the predicted relationship at step h of the sample image, and the relationship label in the object-attribute-relationship graph of the sample image at step h of the sample image sequence, determine the relationship loss value at step h; Based on the predicted slot vector at step h of the sample image, and the object label and attribute label in the object-attribute-relationship graph of the sample image at frame h in the sample image sequence, the slot loss value at step h is determined. Based at least H-step relationship loss and H-step slot loss, the model parameters of the slot dynamics model to be trained are updated to obtain the trained slot dynamics model.

5. The model training method based on object-attribute-relationship graph according to claim 2, characterized in that, Also includes: Input the K predicted slot vectors of step H into the reward model to be trained, and output the reward value of the target image at step H. Based on the H-step predicted trajectory of sample images and the object-attribute-relationship graph sequence of sample image sequences, the policy model, reward model, and slot dynamics model to be trained are trained to obtain the trained slot dynamics model, the trained policy model, and the trained reward model, including: Based on the reward value from step 1 to step H, the reward value for step H is obtained; Based on the H-step reward value, H-step relation loss value, and H-step slot loss value, the model parameters of the policy model, reward model, and slot dynamics model to be trained are updated to obtain the trained slot dynamics model, trained policy model, and trained reward model.

6. The model training method based on object-attribute-relationship graph according to claim 2, characterized in that, The trained slot dynamics model is used to perform H-step prediction on the target image, outputting the H-step predicted trajectory of the target image, including: The target image is slot-encoded using the pre-trained slot encoder to obtain K slot vectors of the target image; Input the K slot vectors of the target image into the trained policy model, and output the first step of the robotic arm action of the target image; Control the robotic arm according to the first step of the target image's robotic arm movement; The trained slot dynamics model is used to process the first step robotic arm action and the K slot vectors of the target image output by the strategy model, and outputs the first step prediction relationship and the first step K predicted slot vectors of the target image. Input the first K predicted slot vectors into the trained policy model, and output the second step robotic arm action of the target image; Control the robotic arm according to the second step of the target image's robotic arm movement; Take h sequentially from 2 to H-1, process the h-th step robotic arm action output by the strategy model, the h-1-th step prediction relationship output by the trained slot dynamics model, and the h-1-th step K predicted slot vectors, and output the h-th step prediction relationship and the h-th step K predicted slot vectors of the target image. Input the h-th step K predicted slot vectors of the target image into the trained policy model, and output the h-th step robotic arm action of the target image. The robotic arm is controlled according to the h-th step of the target image.

7. A model training device based on an object-attribute-relationship graph, characterized in that, include: The relation extraction module is used to extract relations from the sample image through the relation extractor, and obtain the object-level relation graph of the sample image. A node in the object relation graph of the sample image represents an object in the sample image, and an edge represents the type of relationship between two objects in the sample image. The slot encoding module is used to encode the sample image using a pre-trained slot encoder, resulting in K slot vectors and K segmentation masks for the sample image. The relation transcribing module is used to transcrib the object-level relation graph of the sample image into a slot-level relation tensor based on the K slot vectors and K segmentation masks of the sample image, as well as the ground instance mask of each object in the sample image. The relation slot prediction module is used to inject the slot-level relation tensor of the sample image into the slot dynamics model to be trained and perform the first step prediction, outputting the first step prediction relation and the first step K predicted slot vectors of the sample image; taking h from 2 to H-1 in sequence, the h-1 step prediction relation output by the slot dynamics model to be trained is injected into the slot dynamics model to be trained and the h-th step prediction is performed, outputting the h-th step prediction relation and the h-th step K predicted slot vectors of the sample image; The model training module is used to train the slot dynamics model to be trained based on at least H-step predicted trajectories of sample images and object-attribute-relationship graph sequences of sample image sequences, so as to obtain the trained slot dynamics model; the sample image sequence is a series of H frame sample images with sample images as the starting frame; The trajectory prediction module is used to perform H-step prediction on the target image using the trained slot dynamics model, and outputs the H-step predicted trajectory of the target image. The H-step predicted trajectory represents the evolution process of the relationship between various objects in the target image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the model training method based on object-attribute-relationship graph as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the model training method based on object-attribute-relationship graphs as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the model training method based on object-attribute-relationship graphs as described in any one of claims 1-6.

Images