An adaptive context modeling method and apparatus for scene graph generation
By using an adaptive context modeling method, objects are automatically serialized to construct a chain structure model, which solves the problems of noise interference and incompatibility with prior layout in existing technologies, and improves the robustness and accuracy of scene graph generation.
Patent Information
- Application Number
- CN202211008807.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-22
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2042-08-22
AI Technical Summary
Existing scene graph generation methods suffer from noise interference and incompatibility with prior layout structures when modeling context information, resulting in poor generation performance.
An adaptive context modeling approach is adopted, which automatically serializes objects to build a chain structure model through object detection, object feature concatenation and chain structure matching, and uses LSTM to update the object feature representation to generate a more accurate scene map.
It improves the scene graph generation effect, makes the context fusion process adapt to the image content, gets rid of prior constraints, and improves the robustness and generation effect of the model.
Smart Images
Figure CN115393661B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, and more particularly to an adaptive context modeling method for scene graph generation, and an adaptive context modeling apparatus for scene graph generation. Background Technology
[0002] Scene graph generation is a visual detection task. Its objective is to detect objects within a scene based on an image and to identify the relationships between objects. The result is a directed scene graph, where nodes represent objects in the scene and edges represent the relationships between objects. Scene graphs serve as a bridge between images and natural language, often used to bridge the gap between upstream tasks (such as object detection) and downstream high-level visual understanding tasks (such as image captioning and visual question answering).
[0003] Most mainstream scene graph generation methods currently recognize the positive role of contextual information in scene graph generation. Contextual information in scene graph generation refers to some or all of the information that influences the semantic understanding of objects and their relationships, i.e., the local or global features of the scene image. Related research shows that modeling visual context at the object level is effective for visual tasks. When modeling the context of objects in scene graph generation tasks, it is usually necessary to assume a prior layout structure for the objects in the scene. For example, Zellers et al. (2018) arranged objects into a chain according to their positions in the image from left to right, using a Long Short-Term Memory (LSTM) neural network to transmit contextual information. Li et al. (2018) modeled objects in the scene as a fully connected graph and then used a Graph Neural Network (GNN) to transmit contextual information between nodes. Tang et al. (2019) proposed a dynamic tree structure to model scene contextual information.
[0004] Using the three commonly used prior layout structures mentioned above to model context information has several problems: Dense node connections in fully connected graphs introduce unnecessary noise during context information propagation, and the static nature of fully connected graphs does not align with the content-driven nature of visual context; dynamic tree structures require only one object as the root node, which is unsuitable for scenes containing multiple salient objects; chain structures are widely used in scene graph generation tasks due to their simplicity, but the lack of natural sequence information in images means there is no referential sequence information between the nodes to be modeled. Therefore, when modeling context information based on chain structures, it is necessary to specify the position of objects in the sequence. Existing research shows that different object ordering methods affect the scene graph generation results when using chain structures to model context. Furthermore, existing methods for modeling context based on chain structures perform the same serialization method on all images, and a single serialization method unrelated to scene content does not align with the content-driven nature of visual context. Summary of the Invention
[0005] To overcome the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide an adaptive context modeling method for scene graph generation, which can adaptively serialize objects in the scene according to the scene content, thereby improving the effect of the generated scene graph.
[0006] The technical solution of this invention is: an adaptive context modeling method for scene graph generation, comprising the following steps:
[0007] (1) Use a pre-trained object detector to perform object detection on the input image and output a series of object proposals. Select the top 80 with high confidence from these object proposals and regard them as objects existing in the scene.
[0008] (2) The refined semantic tags obtained in step (1) The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4):
[0009]
[0010] Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then...
[0011] The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6):
[0012] S o→p =Softmax(W o→p (5) O, dim = 0)
[0013] S p→o =Softmax(W p→o O, dim = 1) (6)
[0014] Among them, W o→p and W p→o The learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7):
[0015] S = S o→p ⊙S p→o (7)
[0016] Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved.
[0017] (3) Context information fusion and relationship prediction.
[0018] The adaptive context modeling method for scene graph generation proposed in this invention can automatically serialize objects to construct a chain-structured context model based on scene content. Then, based on the chain structure, LSTM is used to update the feature representation of the objects. Finally, a more accurate scene graph is generated based on the object features that have been fused with context information. Therefore, the context fusion process can be adapted to the image content, thereby freeing the scene graph generation model from prior constraints and achieving more robust performance, thus improving the quality of the generated scene graph.
[0019] An adaptive context modeling apparatus for scene graph generation is also provided, the apparatus comprising:
[0020] The detection module uses a pre-trained object detector to detect objects in the input image and outputs a series of object proposals. The top 80 object proposals with high confidence are selected as objects present in the scene.
[0021] The modeling module takes the refined semantic tags obtained by the detection module. The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4):
[0022]
[0023] Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then...
[0024] The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6):
[0025] S o→p =Softmax(W o→p (5) O, dim = 0)
[0026] S p→o =Softmax(W p→o O, dim = 1) (6)
[0027] Among them, W o→p and W p→o The learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7):
[0028] S = S o→p ⊙S p→o (7)
[0029] Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved.
[0030] The fusion prediction module performs context information fusion and relationship prediction. Attached Figure Description
[0031] Figure 1 A general flowchart of the adaptive context modeling method for scene graph generation according to the present invention is shown.
[0032] Figure 2 A modeling framework diagram of the present invention is shown. Detailed Implementation
[0033] like Figure 1 , 2 As shown, this adaptive context modeling method for scene graph generation includes the following steps:
[0034] (1) Use a pre-trained object detector to perform object detection on the input image and output a series of object proposals. Select the top 80 with high confidence from these object proposals and regard them as objects existing in the scene.
[0035] (2) The refined semantic tags obtained in step (1) The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4):
[0036]
[0037] Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then...
[0038] The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6):
[0039] S o→p =Softmax(W o→p (5) O, dim = 0)
[0040] S p→o =Softmax(W p→o O, dim = 1) (6)
[0041] Among them, W o→p and W p→o The learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7):
[0042] S = S o→p ⊙S p→o (7)
[0043] Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved.
[0044] (3) Context information fusion and relationship prediction.
[0045] The adaptive context modeling method for scene graph generation proposed in this invention can automatically serialize objects to construct a chain-structured context model based on scene content. Then, based on the chain structure, LSTM is used to update the feature representation of the objects. Finally, a more accurate scene graph is generated based on the object features that have been fused with context information. Therefore, the context fusion process can be adapted to the image content, thereby freeing the scene graph generation model from prior constraints and achieving more robust performance, thus improving the quality of the generated scene graph.
[0046] Preferably, in step (1), the target detector provides the following three types of information for each object: visual features Semantic tags and spatial location information The spatial location information includes the coordinates of the top-left corner of the object's bounding box (x, y, y). tl y tl ), lower right corner coordinates (x br y br ), center point coordinates and the size of the bounding box (x br -x tl y br -y tl ).
[0047] Preferably, in step (1), the parameters of the target detector are fixed, and the objects initially detected need to be refined to obtain more accurate feature representations and semantic labels; firstly, the semantic labels obtained from the target detector are... and spatial location information s i Mapped to the embedding space, the corresponding embedding space representations are denoted as follows: and Then, based on equations (1), (2), and (3), the semantic labels of objects in the scene are refined:
[0048]
[0049]
[0050]
[0051] Where bi-LSTM represents a bidirectional LSTM network; Represents the context information of object i; [;] indicates the feature concatenation operation; This represents a refined semantic tag; w l Used to output h of the hidden layer i Mapped to 151 dimensions.
[0052] Preferably, in step (2), solving the assignment problem includes:
[0053] (2.1) Randomly initialize a vector
[0054] (2.2) Extract the maximum value S from the matching score matrix S. e，k The corresponding row index e represents the e-th object, and the column index k represents the k-th position in the chain structure;
[0055] (2.3) Assign object e to position k.
[0056] (2.4) Let S [e，∶] =0, S [：，k] =0 indicates that the assigned object and its position will not participate in subsequent assignments; repeat the above operation until all objects are assigned. After assignment is complete, the feature representations of the n objects in the image become
[0057] Preferably, in step (3), after completing context modeling, the context and the updated feature representation of the object are fused based on equation (8):
[0058]
[0059] Where C = [c1, c2, ..., c n ] indicates that the object features are updated using context information. For n objects in an image, n(n-1) relation proposals can be combined. The model predicts all relation proposals, and the feature calculation of a single relation proposal is as shown in equation (9):
[0060] r ij =W i c i *W j c j *f i，j (9)
[0061] Among them, f i，j Features representing the minimum bounding boxes of objects i and j; W i and W jRepresents the learnable parameter matrix;
[0062] Finally, the relationship category is predicted based on equation (10):
[0063] Pr ij =softmax(W r r ij +b i，j (10)
[0064] in, W represents the relationship between object i and object j; r b represents the learnable parameter matrix; i，j This represents the bias vector.
[0065] An adaptive context modeling apparatus for scene graph generation is also provided, the apparatus comprising:
[0066] The detection module uses a pre-trained object detector to detect objects in the input image and outputs a series of object proposals. The top 80 object proposals with high confidence are selected as objects present in the scene.
[0067] The modeling module takes the refined semantic tags obtained by the detection module. The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4):
[0068]
[0069] Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then...
[0070] The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6):
[0071] S o→p =Softmax(W o→p (5) O, dim = 0)
[0072] S p→o =Softmax(W p→o O, dim = 1) (6)
[0073] Among them, W o→p and W p→oThe learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7):
[0074] S = S o→p ⊙S p→o (7)
[0075] Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved.
[0076] The fusion prediction module performs context information fusion and relationship prediction.
[0077] Preferably, in the detection module, the target detector provides the following three types of information for each object: visual features. Semantic tags and spatial location information The spatial location information includes the coordinates of the top-left corner of the object's bounding box (x, y, y). tl y tl ), lower right corner coordinates (x br y br ), center point coordinates and the size of the bounding box (x) br -x tl y br -y tl ).
[0078] Preferably, in the detection module, the parameters of the target detector are fixed, and the objects initially detected need to be refined to obtain more accurate feature representations and semantic labels; firstly, the semantic labels obtained from the target detector are... and spatial location information s i Mapped to the embedding space, the corresponding embedding space representations are denoted as follows: and Then, based on equations (1), (2), and (3), the semantic labels of objects in the scene are refined:
[0079]
[0080]
[0081]
[0082] Where bi-LSTM represents a bidirectional LSTM network; Represents the context information of object i; [;] indicates the feature concatenation operation; This represents a refined semantic tag; w l Used to output h of the hidden layer i Mapped to 151 dimensions.
[0083] Preferably, in the modeling module, solving the assignment problem includes:
[0084] (2.1) Randomly initialize a vector
[0085] (2.2) Extract the maximum value S from the matching score matrix S. e，k The corresponding row index e represents the e-th object, and the column index k represents the k-th position in the chain structure;
[0086] (2.3) Assign object e to position k.
[0087] (2.4) Let S [e，∶] =0, S [：，k] =0 indicates that the assigned object and its position will not participate in subsequent assignments; repeat the above operation until all objects are assigned. After assignment is complete, the feature representations of the n objects in the image become
[0088] Preferably, in the fusion prediction module, after completing context modeling, the context and the updated feature representation of the object are fused based on equation (8):
[0089]
[0090] Where C = [c1, c2, ..., c n ] indicates that the object features are updated using context information. For n objects in an image, n(n-1) relation proposals can be combined. The model predicts all relation proposals, and the feature calculation of a single relation proposal is as shown in equation (9):
[0091] r ij =W i c i *W j c j *f i，j (9)
[0092] Among them, f i，jFeatures representing the minimum bounding boxes of objects i and j; W i and W j Represents the learnable parameter matrix;
[0093] Finally, the relationship category is predicted based on equation (10):
[0094] Pr ij =softmax(W r r ij +b i，j (10)
[0095] in, W represents the relationship between object i and object j; r Represents the learnable parameter matrix; b i，j This represents the bias vector.
[0096] The effectiveness of the method of the present invention is verified below.
[0097] The scene graph generation task comprises three subtasks: scene graph detection, scene graph classification, and predicate classification, with their difficulty decreasing sequentially. Specifically, the scene graph detection subtask takes the original image as input and requires the model to detect objects in the image, predict the object classification, and the relationships between objects; the scene graph classification subtask takes the original image and the ground truth bounding boxes of objects as input and requires the model to predict the object classification and the relationships between objects; the predicate classification subtask takes the original image, the ground truth bounding boxes of objects, and the true object categories as input and requires the model to predict the relationships between objects.
[0098] The Visual Genome (VG) dataset is currently the largest dataset for scene graph generation tasks, containing 108,077 images, with an average of 38 objects and 22 relation annotations per image. The dataset contains 150 object categories and 50 relation categories (excluding the background class). Following other methods for processing datasets, this invention divides the dataset into a 70% training set and a 30% test set.
[0099] To test the effectiveness of the proposed method, experiments were conducted on the public dataset (VG) for scene graph generation tasks, and the results were compared with those of current mainstream scene graph generation methods. Table 1 shows the comparison of the scene graph recall rates of the proposed method and other methods on the VG dataset. The results show that the proposed method achieves the highest performance level in all metrics of the scene graph detection subtask and the R@20 metric of the scene graph classification subtask, while achieving a near-optimal average recall rate, 0.4% lower than the best performance.
[0100] Table 1. Comparison of recall rates of the proposed method with other methods on the VG dataset (unit: %)
[0101]
[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.
Claims
1. An adaptive context modeling method for scene graph generation, characterized in that: The method includes the following steps: (1) Use a pre-trained object detector to perform object detection on the input image and output a series of object proposals. Select the top 80 with high confidence from these object proposals and regard them as objects existing in the scene. (2) The refined semantic tags obtained in step (1) The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4): Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then... The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6): S o→p =Softmax(W o→p O,dim=0) (5) S p→o =Soffmax(W p→o O,dim=1) (6) Among them, W o→p and W p→o The learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7): S=S o→p ⊙S p→o (7) Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved. (3) Context information fusion and relationship prediction.
2. The adaptive context modeling method for scene graph generation according to claim 1, characterized in that: In step (1), the target detector provides the following three types of information for each object: visual features Semantic tags and spatial location information The spatial location information includes the coordinates of the top-left corner of the object's bounding box (x, y, y). tl y tl ), lower right corner coordinates (x br y br ), center point coordinates and the size of the bounding box (x br -x tl y br -y tl ).
3. The adaptive context modeling method for scene graph generation according to claim 2, characterized in that: In step (1), the parameters of the target detector are fixed, and the objects initially detected need to be refined to obtain more accurate feature representations and semantic labels; firstly, the semantic labels obtained from the target detector are... and spatial location information s i Mapped to the embedding space, the corresponding embedding space representations are denoted as follows: and Then, based on equations (1), (2), and (3), the semantic labels of objects in the scene are refined: Where bi-LSTM represents a bidirectional LSTM network; Represents the context information of object i; [;] indicates the feature concatenation operation; This represents a refined semantic tag; w l Used to output h of the hidden layer i Mapped to 151 dimensions.
4. The adaptive context modeling method for scene graph generation according to claim 3, characterized in that: In step (2), solving the assignment problem includes: (2.1) Randomly initialize a vector (2.2) Extract the maximum value S from the matching score matrix S. e，k The corresponding row index e represents the e-th object, and the column index k represents the k-th position in the chain structure; (2.3) Assign object e to position k. (2.4) Let S [e，∶] =0, S [：，k] =0 indicates that the assigned object and its position will not participate in subsequent assignments; repeat the above operation until all objects are assigned. After assignment is complete, the feature representations of the n objects in the image become 5. The adaptive context modeling method for scene graph generation according to claim 4, characterized in that: In step (3), after completing the context modeling, the context and the feature representation of the object are fused based on equation (8): Where C = [c1, c2, ..., c n ] indicates that the object features are updated using context information. For n objects in an image, n(n-1) relation proposals can be combined. The model predicts all relation proposals, and the feature calculation of a single relation proposal is as shown in equation (9): r ij =W i c i *W j c j *f i，j (9) Among them, f i，j Features representing the minimum bounding boxes of objects i and j; W i and W j Represents the learnable parameter matrix; Finally, the relationship category is predicted based on equation (10): Pr ij =softmax(W r r ij +b i，j ) (10) in, W represents the relationship between object i and object j; r Represents the learnable parameter matrix; b i，j This represents the bias vector.
6. An adaptive context modeling apparatus for scene graph generation, characterized in that: It includes: The detection module uses a pre-trained object detector to detect objects in the input image and outputs a series of object proposals. The top 80 object proposals with high confidence are selected as objects present in the scene. The modeling module takes the refined semantic tags obtained by the detection module. The vector is mapped to a 200-dimensional vector representation, and then concatenated with the object's visual features and contextual features to form the object's complete feature representation, as shown in Equation (4): Among them, w r This is a parameter matrix used to map the semantic labels of the refined objects. If the features of n objects in the image are denoted as O, then... The features O of n objects in the image are input into two learning branches: the object selection location branch and the location selection object branch, respectively, to measure the degree of matching between the object and its position in the chain structure, as shown in equations (5) and (6): S o→p =Softmax(W o→p O,dim=0) (5) S p→o =Softmax(W p→o O,dim=1) (6) Among them, W o→p and W p→o The learnable parameter matrix is obtained by performing Softmax on the learned matrix row by row in the object selection position branch. This is the adaptive score matrix learned from this branch; the position selection object branch performs Softmax on the learned matrix column-wise to obtain... By combining the score matrices learned from the two branches, the matching score matrix between the object and its position is calculated based on formula (7): S=S o→p ⊙S p→o (7) Where ⊙ represents the Hadamard product; the element S in S i，j Let represent the matching score between object i and position j. The matching score represents the benefit of context information fusion when object i is assigned to position j. After obtaining the matching score matrix between objects and positions, the object serialization problem is treated as an assignment problem to be solved. The fusion prediction module performs context information fusion and relationship prediction.
7. The adaptive context modeling apparatus for scene graph generation according to claim 6, characterized in that: In the detection module, the target detector provides the following three types of information for each object: visual features. Semantic tags and spatial location information The spatial location information includes the coordinates of the top-left corner of the object's bounding box (x, y, y). tl y tl ), lower right corner coordinates (x br y br ), center point coordinates and the size of the bounding box (x br -x tl y br -y tl ).
8. The adaptive context modeling apparatus for scene graph generation according to claim 7, characterized in that: In the detection module, the parameters of the target detector are fixed, and the objects initially detected need to be refined to obtain more accurate feature representations and semantic labels. First, the semantic labels obtained from the target detector are... and spatial location information s i Mapped to the embedding space, the corresponding embedding space representations are denoted as follows: and Then, based on equations (1), (2), and (3), the semantic labels of objects in the scene are refined: Where bi-LSTM represents a bidirectional LSTM network; Represents the context information of object i; [;] indicates the feature concatenation operation; This represents a refined semantic tag; w l Used to output h of the hidden layer i Mapped to 151 dimensions.
9. The adaptive context modeling apparatus for scene graph generation according to claim 8, characterized in that: The modeling module includes solving the assignment problem, which includes: (2.1) Randomly initialize a vector (2.2) Extract the maximum value S from the matching score matrix S. e，k The corresponding row index e represents the e-th object, and the column index k represents the k-th position in the chain structure; (2.3) Assign object e to position k. (2.4) Let S [e，∶] =0, S [：，k] =0 indicates that the assigned object and its position will not participate in subsequent assignments; repeat the above operation until all objects are assigned. After assignment is complete, the feature representations of the n objects in the image become 10. The adaptive context modeling apparatus for scene graph generation according to claim 9, characterized in that: In the fusion prediction module, after completing context modeling, the context is fused and the feature representation of the object is updated based on equation (8): Where C = [c1, c2, ..., c n ] indicates that the object features are updated using context information. For n objects in an image, n(n-1) relation proposals can be combined. The model predicts all relation proposals, and the feature calculation of a single relation proposal is as shown in equation (9): r ij =W i c i *W j c j *f i，j (9) Among them, f i，j Features representing the minimum bounding boxes of objects i and j; W i and W j Represents the learnable parameter matrix; Finally, the relationship category is predicted based on equation (10): Pr ij =softmax(W r r ij +b i，j ) (10) in, W represents the relationship between object i and object j; r Represents the learnable parameter matrix; b i，j This represents the bias vector.