Zero-sample training method, device, storage medium and electronic device

Through the zero-sample training method based on scene graph, the interactive information of graph convolution neural network and scene graph is used to solve the problem of difficulty in obtaining training samples in computer vision technology, and the accurate distinction and efficient identification of new tasks are achieved, which improves zero Accuracy and efficiency of sample learning.

CN117893876BActive Publication Date: 2024-11-08INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410027733.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-08
Publication Date
2024-11-08
Estimated Expiration
2044-01-08

AI Technical Summary

Technical Problem

In the application of existing computer vision technology in unmanned platforms, there are difficulties in distinguishing new tasks due to the difficulty in obtaining training samples and the cost of labor and material resources.

Method used

Using a zero-sample training method based on scene graphs, we can obtain knowledge graphs related to the detection samples, build category information and connection information, use graph convolutional neural network to train the detection model, and use scene graphs and classifiers to train the model to achieve new Positioning and distinguishing tasks.

Benefits of technology

Effectively utilize object interaction information, reduce background interference, improve the recognition accuracy of new tasks, solve the problem of lack of visual information, and achieve high accuracy and high efficiency zero-sample learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117893876B_ABST
    Figure CN117893876B_ABST
Patent Text Reader

Abstract

The present invention proposes a zero-sample training method, device, storage medium and electronic device based on scene graph, which relates to the field of computer vision. Zero-sample training includes: obtaining a first knowledge graph related to a detection sample; obtaining category information and connection information related to the detection sample in the first knowledge graph, and constructing a second knowledge graph related to the detection sample; training the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; obtaining a classifier according to the first training detection model; constructing a scene graph according to the category information and connection information; training the first training detection model according to the scene graph and the classifier to obtain a second training detection model. The present invention constructs a scene graph structure and utilizes the interactive information between foregrounds to form an association between new tasks and old tasks, thereby relying on the interactive information between the model and the old tasks to achieve the identification of new tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a zero-sample training method, device, storage medium and electronic device. Background Art

[0002] In recent years, with the widespread application of computer neural networks, the field of computer vision has achieved unprecedented development, injecting new vitality into more and more application fields. However, the existing computer vision technology based on computer neural networks relies on a large number of training samples for end-to-end training. This training method will have the following two effects in practical applications, especially for unmanned platforms: First, in some mission scenarios, a large number of training samples are difficult to obtain, such as rare wild animals and plants, which are difficult to obtain due to the small number of individuals. Second, collecting samples will consume a lot of manpower and material resources.

[0003] The scene graph is a form of graph structure that can display and reflect the interaction between different objects in the current scene. The construction of this interaction relationship allows the model to locate new categories based on old categories. Traditional neural networks can only solve data problems in Euclidean space, while graph convolutional neural networks can realize the propagation of information on graph structures. The information propagation process and information aggregation process between nodes in graph convolutional neural networks effectively extract the interaction information between data, and establish the channel and basis for reasoning in reasoning tasks.

[0004] This paper applies zero-shot learning to the field of object detection. While obtaining a classifier through semantic information reasoning, it also uses the interactive information of the scene graph to achieve the localization of new tasks. Unlike zero-shot classification tasks, zero-shot training tasks not only need to label the correct category for the object, but also need to further achieve the localization of new category objects. Since there are no labeled supervised samples for the new task and the background, how to distinguish the new task from the background is an important issue faced by zero-shot training tasks.

[0005] Therefore, how to distinguish new tasks from the background has become a technical problem that needs to be solved urgently in zero-sample training tasks. Summary of the invention

[0006] In view of this, the present invention aims to solve the problem in the prior art that the number of training samples obtained is small or the sample collection is difficult.

[0007] Specifically, the present invention is achieved through the following technical solutions:

[0008] According to a first aspect of the present invention, a zero-sample training method based on a scene graph is provided, comprising: obtaining a first knowledge graph related to a detection sample; obtaining category information and connection information related to the detection sample in the first knowledge graph, and constructing a second knowledge graph related to the detection sample; training the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; obtaining a classifier according to the first training detection model; constructing a scene graph according to the category information and the connection information; training the first training detection model according to the scene graph and the classifier to obtain a second training detection model.

[0009] In some technical schemes, the steps of training a first training detection model according to a scene graph and a classifier to obtain a second training detection model specifically include: obtaining all anchor boxes in the scene graph; determining the co-occurrence relationship between all anchor boxes according to the classifier; determining the foreground of multiple anchor boxes according to the propagation and fusion of the co-occurrence relationship by a graph convolutional neural network, and judging the object category based on the foreground of the anchor box; transferring the classifier to the first training detection model to obtain the second training detection model.

[0010] In some technical solutions, the expression of graph convolutional neural network is:

[0011]

[0012] Among them, H represents the output information. For the first layer H 0 =X, and use the Leaky ReLu function as the nonlinear activation function of the graph neural network. Represents the degree matrix of the corresponding graph node, where E∈R N×N represents the adjacency matrix of the corresponding graph, and I represents the identity matrix. ii =∑ j E ij represents regularization with the row vector of the adjacency matrix. K l Refers to the parameters of the l-th layer graph convolutional neural network.

[0013] In some technical solutions, the step of inferring a classifier according to the first training detection model specifically includes: determining the difference between the minimized category target value and the predicted value according to the category information and the connection information, and inferring the classifier; wherein the expression for determining the difference between the minimized category target value and the predicted value is:

[0014]

[0015] Among them, L represents minimizing the difference between the category target value and the predicted value, represents the predicted value of the classification weight of the category in the second knowledge graph, W represents the true value of the classification weight of the category obtained by the category classifier C(·|ω), M represents the number of categories, P represents the dimension of the classification weight vector, and W i,j represents the true value of the classification weights of category i and category j, Represents the predicted value of the classification weight of category i and category j.

[0016] In some technical solutions, the step of determining the co-occurrence relationship between all anchor frames according to a classifier specifically includes: determining the nodes in the scene graph, the anchor frames in the scene graph, and the association relationship between the nodes and the anchor frames according to the classifier.

[0017] In some technical solutions, the nodes of the scene graph are represented as:

[0018] x i =classifier(anchor i );

[0019] Among them, x i Indicates the class that anchori is judged to be. i represents the anchor box obtained from the scene graph, and classifier(·) represents the classifier inferred based on the second knowledge graph.

[0020] In some technical solutions, the anchor box in the scene graph is represented as:

[0021]

[0022] Among them, e i,j represents the weight of the edge connecting nodes i and j, occurrence(x i ,x j ) is used to determine the co-occurrence relationship between two objects, s is the threshold of the co-occurrence relationship, x i represents category i, x j represents category j.

[0023] According to a second aspect of the present invention, a zero-sample training device based on a scene graph is provided, comprising: an acquisition module, used to acquire a first knowledge graph related to a detection sample; a construction module, used to acquire category information and connection information related to the detection sample in the first knowledge graph, and construct a second knowledge graph related to the detection sample; a training module, used to train the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; an inference module, used to infer a classifier according to the first training detection model; the construction module, also used to construct a scene graph according to the category information and the connection information; the training module, also used to train the first training detection model according to the scene graph and the classifier to obtain a second training detection model.

[0024] According to a third aspect of the present invention, there is provided a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the zero-sample training method based on a scene graph in the first aspect or any possible implementation of the first aspect.

[0025] According to a fourth aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the zero-sample training method based on a scene graph in the first aspect or any possible implementation of the first aspect are implemented.

[0026] The technical solution provided by the present invention brings at least the following beneficial effects:

[0027] The present invention forms an association between new tasks and old tasks by constructing a scene graph structure and utilizing the interactive information between foregrounds, thereby relying on the interactive information between the model and the old tasks to achieve the identification of new tasks, thereby avoiding the interference of background information on foreground information. Constructing a scene graph effectively utilizes the interactive information between objects, further enriches the source of information, and thus weakens the disadvantages caused by the lack of visual information in the zero-shot learning problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 A flowchart of a zero-shot training method based on a scene graph provided by an embodiment of the present invention;

[0031] Figure 2 A flowchart of another zero-shot training method based on a scene graph provided by an embodiment of the present invention;

[0032] Figure 3 A block diagram of a zero-sample training device based on a scene graph provided by an embodiment of the present invention;

[0033] Figure 4 A block diagram of an electronic device provided in an embodiment of the present invention.

[0034] in, Figure 3 and Figure 4 The corresponding relationship between the reference numerals and component names is as follows:

[0035] 10 A zero-sample training device based on a scene graph, 11 an acquisition module, 12 a construction module, 13 a training module, 14 an inference module, 20 an electronic device, 21 a memory, and 22 a processor. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a zero-sample training method based on a scene graph, comprising:

[0038] S101: Obtain a first knowledge graph related to the detection sample;

[0039] S102: Obtain category information and connection information related to the detection sample in the first knowledge graph, and construct a second knowledge graph related to the detection sample;

[0040] S103: Training the information of the second knowledge graph according to the graph convolutional neural network to obtain a first training detection model;

[0041] S104: Obtaining a classifier by inference based on the first training detection model;

[0042] S105: constructing a scene graph according to the category information and the connection information;

[0043] S106: Train the first training detection model according to the scene graph and the classifier to obtain a second training detection model.

[0044] In this embodiment, firstly, a first knowledge graph related to the detection sample is obtained, and the first knowledge graph is pre-stored; then, the category information and connection information related to the detection sample are extracted to form a task-related knowledge graph, that is, a second knowledge graph is constructed; then, the information of the second knowledge graph is trained according to the graph convolutional neural network to obtain a first training detection model, and a classifier is obtained by inference according to the first training detection model. Specifically, the second knowledge graph is subjected to the graph convolutional neural network to obtain the output result of each category, and the probability value is obtained by the softmax function, and the cross entropy loss is performed with the label of each category, so as to obtain the classifier through back propagation training; at the same time, a scene graph is constructed according to the category information and connection information, wherein the scene graph can reflect the interactive relationship between the new and old tasks, for example: there is a spatial position relationship between people and horses, and there is a semantically subordinate relationship between riding a bicycle and wheels, and the distinction between new categories and backgrounds is realized through this innate and available auxiliary information. When the sample data of the old category is insufficient, this interactive relationship is particularly important as the background division of the new category. Finally, the universal classification weights generated by the scene graph and the classifier are assigned to the classifier branch of the detection model, thereby realizing the visual development of the detection model for the new task, that is, the detection sample. That is, the present invention forms an association between the new task and the old task by constructing a scene graph structure and utilizing the interactive information between the foregrounds, thereby relying on the interactive information between the model and the old tasks to realize the identification of the new task and avoid the interference of background information on the foreground information. By constructing a scene graph, the interactive information between objects is effectively utilized, and the information source is further enriched, thereby weakening the disadvantages caused by the lack of visual information in the zero-sample learning problem.

[0045] In some embodiments, the step of training a first training detection model according to a scene graph and a classifier to obtain a second training detection model specifically includes: obtaining all anchor boxes in the scene graph; determining the co-occurrence relationship between all anchor boxes according to the classifier; determining the foreground of multiple anchor boxes according to the graph convolutional neural network propagating and fusing the co-occurrence relationship, and judging the object category based on the foreground of the anchor box; transferring the classifier to the first training detection model to obtain the second training detection model.

[0046] In this embodiment, firstly, the RPN module of the faster-RCNN model is used to obtain all anchor frames in the scene graph; then, the co-occurrence relationship between all anchor frames is determined according to the classifier, wherein the co-occurrence relationship is also the interaction relationship between the anchor frames; after determining the interaction relationship between all anchor frames, the foreground of multiple anchor frames is determined according to the graph convolutional neural network, and after determining the foreground of multiple anchor frames, the category of the object is judged according to the score of the foreground of the anchor frame; finally, the classifier is transferred to the first training detection model to obtain the second training detection model. That is, the present invention determines the interaction relationship between all anchor frames in the scene graph according to the classifier, and uses this interaction relationship as the basis for dividing the background of the new task category, so as to realize the identification of the foreground and background of the new task.

[0047] In some embodiments, the graph convolutional neural network is expressed as:

[0048]

[0049] Among them, H represents the output information. For the first layer H 0 =X, and use the Leaky ReLu function as the nonlinear activation function of the graph neural network. Represents the degree matrix of the corresponding graph node, where E∈R N×N represents the adjacency matrix of the corresponding graph, and I represents the identity matrix. ii =∑ j E ij represents regularization with the row vector of the adjacency matrix. K l Refers to the parameters of the lth layer of graph convolutional neural network. That is, information propagation and information update on the knowledge graph are realized through the graph convolutional neural network.

[0050] In some embodiments, the step of inferring a classifier according to the first training detection model specifically includes: determining the difference between the minimized category target value and the predicted value according to the category information and the connection information, and inferring to obtain the classifier; wherein the expression for determining the difference between the minimized category target value and the predicted value is:

[0051]

[0052] Among them, L represents minimizing the difference between the category target value and the predicted value, represents the predicted value of the classification weight for the category in the second knowledge graph, W represents the true value of the classification weight of the category obtained by the category classifier C(·|ω), C(·|ω) represents the classifier C obtained under a specific category ω, M represents the number of categories, P represents the dimension of the classification weight vector, and W i,j represents the true value of the classification weights of category i and category j, Represents the predicted value of the classification weight of category i and category j. This embodiment calculates the loss generated by the predicted value and the true value, which is used for back propagation loss, updating the gradient, and optimizing all parameters in the network.

[0053] In some embodiments, the step of determining the co-occurrence relationship between all anchor frames according to the classifier specifically includes: determining the nodes in the scene graph, the anchor frames in the scene graph, and the association relationship between the nodes and the anchor frames according to the classifier.

[0054] In this embodiment, when determining the co-occurrence relationship between all anchor frames, it is necessary to determine the nodes in the scene graph and the anchor frames in the scene graph according to the classifier. At the same time, it is also necessary to determine the association relationship between the nodes and the anchor frames according to the classifier. Only in this way can the interaction relationship of the anchor frames in the new task be determined. That is, the present invention aggregates the relevant information of the anchor frames of the old categories, so that in the subsequent foreground judgment, the division of the new categories and backgrounds without training samples can be achieved by relying on the old categories.

[0055] In some embodiments, the nodes of the scene graph are represented as:

[0056] x i =classifier(anchor i );

[0057] Among them, x i Indicates the class that anchori is judged to be. i represents the anchor box obtained from the scene graph, and classifier(·) represents the classifier inferred based on the second knowledge graph.

[0058] In some embodiments, an anchor box in a scene graph is represented as:

[0059]

[0060] Among them, e i,j represents the weight of the edge connecting nodes i and j, occurrence(x i ,x j ) is used to determine the co-occurrence relationship between two objects, s is the threshold of the co-occurrence relationship, x i represents category i, x j represents category j.

[0061] like Figure 2 As shown, the embodiment of the present invention provides another zero-shot training method based on scene graph, including:

[0062] S201: Obtain a first knowledge graph related to the detection sample;

[0063] S202: Obtain category information and connection information related to the detection sample in the first knowledge graph, and construct a second knowledge graph related to the detection sample;

[0064] S203: Training the information of the second knowledge graph according to the graph convolutional neural network to obtain a first training detection model;

[0065] S204: Obtaining a classifier by inference according to the first training detection model;

[0066] S205: constructing a scene graph according to the category information and the connection information;

[0067] S206: Obtain all anchor frames in the scene graph;

[0068] S207: Determine the co-occurrence relationship between all anchor boxes according to the classifier;

[0069] S208: Determine the foreground of multiple anchor boxes according to the graph convolutional neural network propagation and fusion co-occurrence relationship, and make judgments based on the foreground of the anchor boxes;

[0070] S209: Transfer the classifier to the first training detection model to obtain a second training detection model.

[0071] In this embodiment, the zero-shot detection algorithm based on the scene graph first extracts a knowledge graph consisting of an undirected graph, and represents the graph as G=<V,E> . Where V = {v1,v2,...,v i ,...,v n} represents the set of related nodes in the graph, and each node represents a category. i,j =(v i ,v j )} represents the set of edges, representing the connection relationship between nodes in the graph, and has two values ​​1 and 0, representing direct connection and no direct connection respectively. The set of node features in the graph is represented by X = {x1, x2, ..., x i ,...,x n The goal of graph propagation is to predict the classification weight of the new class through information exchange between the new class nodes and the old class nodes. Here we express the classification weight of the class as Among them, N represents the total number of all categories.

[0072] Extract the task-related knowledge graph from the WordNet dataset. Assuming that the total category set C contains N categories, build a graph convolutional neural network containing N nodes. According to the information of the corresponding categories in the WordNet dataset, obtain the feature descriptions of N categories. Use the Glove model to convert the semantic description into the corresponding feature vector, and use the feature vector as the feature value of the node to participate in the subsequent calculation. If the length of the semantic vector trained by Glove is S, the feature matrix based on the knowledge graph structure can be expressed as X K ∈R N×S , using the parent-child relationship in the knowledge graph as the connection mechanism in graph construction. The edge is represented as

[0073]

[0074] Among them, e i,j It represents the edge connection relationship between node i and node j, and hyponymy(i,j) indicates whether the feature vector of node i is correlated with the feature vector of node j.

[0075] Based on the same inventive concept, Figure 3 As shown, an embodiment of the present invention also provides a zero-sample training device 10 based on a scene graph, including: an acquisition module 11, used to acquire a first knowledge graph related to a detection sample; a construction module 12, used to acquire category information and connection information related to the detection sample in the first knowledge graph, and construct a second knowledge graph related to the detection sample; a training module 13, used to train the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; an inference module 14, used to infer a classifier based on the first training detection model; the construction module 12 is also used to construct a scene graph based on the category information and connection information; the training module 13 is also used to train the first training detection model based on the scene graph and the classifier to obtain a second training detection model.

[0076] In this embodiment, first, the first knowledge graph related to the detection sample is obtained by the acquisition module 11, and the first knowledge graph is pre-stored; then, the category information and connection information related to the detection sample are extracted by the construction module 12 to form a task-related knowledge graph, that is, to construct a second knowledge graph; then, the training module 13 is used to train the information of the second knowledge graph through the graph convolutional neural network to obtain a first training detection model, and the reasoning module 14 is used to infer the classifier through the first training detection model; at the same time, the construction module 12 constructs a scene graph according to the category information and connection information, wherein the scene graph can reflect the interactive relationship between the new and old tasks, for example: there is a positional relationship between people and horses in space, and there is a semantically subordinate relationship between riding a bicycle and wheels, and the distinction between new categories and backgrounds is achieved through this innate and available auxiliary information. When the sample data of the old category is insufficient, this interactive relationship is particularly important as the background division of the new category. Finally, the general classification weights generated by the scene graph and the classifier are assigned to the classifier branch of the detection model through the training module 13, so as to realize the visual development of the detection model for the new task and obtain the second training model. That is, the present invention forms an association between new tasks and old tasks by constructing a scene graph structure and utilizing the interactive information between foregrounds, thereby relying on the interactive information between the model and the old tasks to achieve the identification of new tasks and avoid the interference of background information on foreground information. By constructing a scene graph, the interactive information between objects is effectively utilized, further enriching the information source, thereby weakening the disadvantages caused by the lack of visual information in the zero-shot learning problem.

[0077] According to a third aspect of the present invention, there is provided a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the zero-sample training method based on a scene graph in the first aspect or any possible implementation of the first aspect.

[0078] like Figure 4 As shown, according to a fourth aspect of the present invention, an electronic device 20 is provided, comprising a memory 21, a processor 22, and a computer program stored in the memory 21 and executable on the processor 22, wherein the processor 22, when executing the program, implements the steps of the zero-sample training method based on a scene graph in the first aspect or any possible implementation of the first aspect.

[0079] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0080] The present invention has been experimented on the VOC dataset, and the experimental results confirm the excellent performance of the model proposed in the present invention in the zero-sample detection problem.

[0081] Table 1 shows the performance of the zero-shot detection method based on scene graph on the VOC dataset. Each value represents the detection accuracy of the model in each category. Three representative zero-shot detection models are used for comparison, namely Raham, Demirel, and SPGP. The specific results of the experiment are shown in the following table:

[0082] Table 1 Target detection experimental results

[0083] plane bike bird boat bottle bus car chair cow table horse bike person plant sheep Raham 77 55 46 51 23 53 49 44 32 26 57 57 59 36 38 Demirel 68 72 74 48 41 61 48 25 48 73 71 71 73 3 59 SPGP 82 70 51 50 24 60 53 46 49 38 59 59 66 46 32 Ours 89 81 89 73 37 48 57 86 74 51 84 84 82 53 87

[0084] Table 2 Target detection experimental results (continued)

[0085] tv car dog sofa Train map Raham 58 38 64 38 6 45 Demirel 57 44 25 18 15 52 SPGP 59 41 65 59 9 51 Ours 78 46 57 16 27 64

[0086] The above experimental results verify that the overall detection rate of this method for both new and old categories is 64%, of which the first 16 categories are old categories that provide training samples, and the last five categories are new categories that do not have training samples. The model has achieved improved detection accuracy for both new and old categories, and has achieved effective positioning and detection of 4 new categories under the condition of only 16 categories of labeled old category samples for supervision.

[0087] In summary, the present invention introduces a high-accuracy, high-efficiency, and low-complexity algorithm for solving the zero-shot problem based on scene graphs.

[0088] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0089] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A zero-shot training method based on scene graph, characterized in that: include: Obtaining a first knowledge graph related to the detection sample; Obtaining category information and connection information related to the detection sample in the first knowledge graph, and constructing a second knowledge graph related to the detection sample; Training the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; Obtaining a classifier by inferring the first training detection model; constructing a scene graph according to the category information and the connection information; Training the first training detection model according to the scene graph and the classifier to obtain a second training detection model; The step of training the first training detection model according to the scene graph and the classifier to obtain the second training detection model specifically includes: Get all anchor boxes in the scene graph; Determining the co-occurrence relationship between all the anchor frames according to the classifier; wherein, determining the co-occurrence relationship between all the anchor frames according to the classifier specifically comprises: determining the nodes in the scene graph, the anchor frames in the scene graph, and the association relationship between the nodes and the anchor frames according to the classifier; wherein, the nodes of the scene graph are represented as: ; in, express The class that was judged represents the anchor box obtained from the scene graph, (·) represents a classifier obtained based on the second knowledge graph reasoning; the anchor box in the scene graph is represented as: ; in, represents the weight of the edge connecting nodes i and j, It is used to judge the co-occurrence relationship between two objects. s is the threshold of the co-occurrence relationship. represents category i, represents category j; Propagate and fuse the co-occurrence relationship according to the graph convolutional neural network, determine the foreground of the plurality of anchor boxes, and judge the object category according to the foreground score of the anchor box; The classifier is transferred to the first training detection model to obtain the second training detection model.

2. The scene graph-based zero-shot training method according to claim 1, characterized in that: The expression of the graph convolutional neural network is: ; in, H Represents the information output by the graph convolutional neural network, represents the first layer of the graph convolutional neural network, , X represents the input of the graph convolutional neural network, ReLu ( ) indicates that the Leaky ReLu function is used as the nonlinear activation function of the graph convolutional neural network. represents the degree matrix of the second knowledge graph node, , represents the adjacency matrix of the second knowledge graph node, , represents the identity matrix, represents regularization with the row vector of the adjacency matrix, Representation Matrix The diagonal elements in Representation Matrix The elements in , It refers to the first +1) Parameters of the layer graph convolutional neural network, The value is 0 or 1.

3. The scene graph-based zero-shot training method according to claim 1, characterized in that: The step of obtaining a classifier by inferring the first training detection model specifically includes: According to the category information and the connection information, determine to minimize the difference between the category target value and the predicted value, and infer the classifier; The expression for determining the difference between the minimized category target value and the predicted value is: ; in, L represents the difference between the minimized category target value and the predicted value, represents the predicted value of the classification weight for the category in the second knowledge graph, Represents a classifier by category The true value of the classification weight of the obtained category, represents the number of categories, represents the dimension of the classification weight vector, represents the true value of the classification weights of category i and category j, Represents the predicted value of the classification weight of category i and category j.

4. A zero-shot training device based on scene graph, characterized in that: include: An acquisition module, used to acquire a first knowledge graph related to the detection sample; A construction module, used to obtain category information and connection information related to the detection sample in the first knowledge graph, and construct a second knowledge graph related to the detection sample; A training module, used to train the information of the second knowledge graph according to a graph convolutional neural network to obtain a first training detection model; An inference module, used for inferring a classifier according to the first training detection model; The construction module is further used to construct a scene graph according to the category information and the connection information; The training module is further used to train the first training detection model according to the scene graph and the classifier to obtain a second training detection model; The step of training the first training detection model according to the scene graph and the classifier to obtain the second training detection model specifically includes: Get all anchor boxes in the scene graph; Determining the co-occurrence relationship between all the anchor frames according to the classifier; wherein, determining the co-occurrence relationship between all the anchor frames according to the classifier specifically comprises: determining the nodes in the scene graph, the anchor frames in the scene graph, and the association relationship between the nodes and the anchor frames according to the classifier; wherein, the nodes of the scene graph are represented as: ; in, express The class that was judged represents the anchor box obtained from the scene graph, (·) represents a classifier obtained based on the second knowledge graph reasoning; the anchor box in the scene graph is represented as: ; in, represents the weight of the edge connecting nodes i and j, It is used to judge the co-occurrence relationship between two objects. s is the threshold of the co-occurrence relationship. represents category i, represents category j; Propagate and fuse the co-occurrence relationship according to the graph convolutional neural network, determine the foreground of the plurality of anchor boxes, and judge the object category according to the foreground score of the anchor box; The classifier is transferred to the first training detection model to obtain the second training detection model.

5. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 3 are implemented.