Multi-granularity label oriented target detection method, system, device and medium

Abstract

The present application relates to the technical field of computer vision and artificial intelligence, and specifically provides a target detection method, system, device and medium for multi-granularity labels, which comprises the following steps: S1, constructing a fine-grained semantic graph to obtain initial semantic embeddings of multiple nodes; S2, mining key nodes and anchoring the initial semantic embeddings of the key nodes; S3, performing semantic alignment and fusion on the fine-grained semantic graph and the anchored embeddings in S2 to obtain final node embeddings; and S4, performing zero-shot detection through the final node embeddings. The method can improve the multi-granularity semantic alignment capability by constructing a fine-grained semantic graph, and systematically solves the granularity-induced semantic drift problem in open-vocabulary target detection by combining a key semantic node mining mechanism and a Laplacian propagation algorithm driven by semantic correlation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a target detection method, system, device, and medium for multi-granularity labels. Background Technology

[0002] With the widespread application of visual language models, open-vocabulary object detection has become an important research direction in the field of computer vision. This technology can detect objects of any category based on natural language descriptions without the need for a predefined label set, thus having significant application value in scenarios such as autonomous driving, robot navigation, and augmented reality. However, existing methods face the challenge of semantic inconsistency when dealing with multiple granular categories. Specifically, differences in the granularity of category labels (such as "insect" and "monarch butterfly") lead to discontinuous distribution of their text embeddings in the semantic space, which in turn causes bias in visual-semantic matching, i.e., "granularity-induced semantic drift." Different granularity labels may produce inconsistent attention responses and prediction results for the same visual region, seriously affecting the model's generalization ability and stability.

[0003] Currently, some studies have attempted to mitigate the semantic drift problem by modeling category relationships through hierarchical structures. For example, the SHiNe et al. method uses a tree-like hierarchical structure to organize categories and transmit semantic information through parent-child relationships to achieve cross-granularity alignment. However, these methods have the following limitations: First, tree structures cannot effectively model the semantic similarity between categories at the same granularity level (such as "bee" and "wasp"), and the lack of lateral connections leads to insufficient semantic propagation; second, hierarchical structures are prone to introducing semantic interference between levels, causing the unique features of fine-grained categories to be diluted during propagation; in addition, existing methods do not make sufficient use of visual attributes, making it difficult to achieve accurate alignment of fine-grained semantics.

[0004] The main drawbacks of existing technologies can be summarized as follows:

[0005] 1) Insufficient semantic alignment: Traditional hierarchical structures cannot effectively capture the semantic relationships between sibling categories and cross-level categories, resulting in the inability to fundamentally solve the semantic inconsistency in the embedding space.

[0006] 2) Fine-grained semantics are easily lost: During the global semantic propagation process, the discriminative features of fine-grained categories are easily over-smoothed, which leads to a decrease in the model's ability to distinguish between zero-shot detections.

[0007] 3) Strong dependence on labeled data: Existing methods usually rely on a large amount of labeled data to train the alignment module, which limits the adaptability and scalability in scenarios with few or zero samples.

[0008] 4) Insufficient real-time performance: Some methods require online hierarchical inference or complex alignment operations, resulting in slow inference speed and difficulty in meeting the needs of real-time applications.

[0009] In summary, existing methods still have significant shortcomings in addressing the multi-granularity semantic alignment problem in open-vocabulary target detection. Therefore, this invention proposes a target detection method, system, device, and medium for multi-granularity labels to solve the technical problems existing in the prior art. Summary of the Invention

[0010] The purpose of this invention is to provide a target detection method, system, device, and medium for multi-granularity labels, in order to solve the technical problems existing in the prior art. The specific technical solution is as follows:

[0011] A target detection method for multi-granularity labels includes the following steps:

[0012] S1. Construct a fine-grained semantic graph and obtain the initial semantic embeddings of multiple nodes;

[0013] S2. Mine key nodes and anchor the initial semantic embedding of key nodes;

[0014] S3. Perform semantic alignment and fusion between the fine-grained semantic graph and the anchored embeddings in S2 to obtain the final node embeddings.

[0015] S4. Zero-shot detection is performed through the final node embedding.

[0016] Furthermore, fine-grained semantic graphs Includes a set of nodes Edge set and visual attribute set ;

[0017] The set of nodes Includes multiple category nodes, each node Each corresponds to a specific category. , It is a set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure;

[0018] The set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across different levels;

[0019] The set of visual attributes Includes the set of visual attributes for all nodes.

[0020] Furthermore, obtaining the initial semantic embeddings of multiple nodes specifically involves:

[0021] For each node Node generation using large language model LLM visual attribute set Subsequently, a pre-trained text encoder was used. Encode it to obtain the final attribute embedding vector of the node. :

[0022] ;

[0023] in: For visual attribute set The total number of attributes in the middle. Index of the attribute; For a pre-trained text encoder; Embed the final attribute vector for the node;

[0024] For each node Construct a hierarchical text description This description integrates the node's label, parent / child node information, and the node's final attribute embedding vector; this description is then input into a text encoder. The initial embedding vector of the node is obtained. That is, the initial semantic embedding of the node:

[0025] .

[0026] Furthermore, S2 specifically refers to:

[0027] For nodes Text description conduct The perturbation generates a new set of descriptions, which are then encoded into a set of embeddings. ;

[0028] Computational embedding variance :

[0029] ;

[0030] in: express The average vector of the perturbation embeddings;

[0031] The variance value is then globally normalized to obtain the final sensitivity score. :

[0032] ;

[0033] Set a threshold ,if > Then the node The initial semantic embedding of the key node. It will be preserved, resulting in an anchored embedding. .

[0034] Furthermore, S3 specifically involves constructing a weighted adjacency matrix. Its elements Calculated using the following formula:

[0035] ;

[0036] in: It is a node Attribute embedding vector; It is a node Attribute embedding vector; It is a node The neighboring nodes; These are neighbor nodes. Attribute embedding vector; Representative node The set of all neighboring nodes in the fine-grained semantic graph; It is a temperature hyperparameter;

[0037] For adjacency matrix Normalization is performed to obtain the normalized adjacency matrix. ;

[0038] Adjacency matrix based on normalization The embedding matrix of a node is updated through iterative Laplace propagation, with the following iterative formula:

[0039] ;

[0040] in: It is the first The embedding matrix of the next iteration; It is a hyperparameter that controls the propagation intensity; after After rounds of iteration, we obtain fully propagated and aligned embedding vectors. ;

[0041] Will and Weighted fusion is performed to obtain the final node embedding. :

[0042] ;

[0043] in: It is a balancing factor; It is a node The semantically aligned embedding vector obtained after the graph propagation process.

[0044] Furthermore, regarding the adjacency matrix The normalization process specifically involves:

[0045] Define an angle matrix Its diagonal elements For all nodes The sum of the weights of connected edges. ;

[0046] The normalized adjacency matrix is ​​obtained by calculating the following formula. :

[0047] .

[0048] Furthermore, in the zero-shot detection stage, for any image region feature... By calculating its relationship with the final node embedding Matching is performed using cosine similarity between the two pairs:

[0049] ;

[0050] The point corresponding to the final predicted category of this area The category with the highest similarity score is determined, i.e.:

[0051] .

[0052] Target detection systems for multi-granularity labels include:

[0053] A fine-grained semantic graph generation module is used to generate fine-grained semantic graphs; the fine-grained semantic graphs... Includes a set of nodes Edge set and visual attribute set The set of nodes Includes multiple category nodes; each node Each corresponds to a specific category. , The set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure; the set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across levels; the visual attribute set Includes the set of visual attributes of all nodes;

[0054] The key semantic node mining module is used to mine key nodes and anchor the initial semantic embedding of key nodes.

[0055] The semantic relevance-driven Laplacian propagation module guides the weighted propagation of semantic information on the fine-grained semantic graph based on the similarity of visual attributes between nodes, enabling coherent and context-aware alignment between embedding vectors of different granularities.

[0056] An electronic device includes a memory and a processor, wherein the memory stores a computer program; the processor executes the computer program to implement the target detection method for multi-granularity labels as described above.

[0057] A readable storage medium storing a computer program, wherein a processor executes the computer program to implement the target detection method for multi-granularity labels as described above.

[0058] The application of the technical solution of the present invention has the following beneficial effects:

[0059] (1) This invention proposes a fine-grained semantic graph that integrates hierarchical edges, sibling edges and cross-layer edges, breaking through the limitations of traditional tree-like hierarchies and realizing comprehensive modeling of complex semantic relationships between categories; each node in the fine-grained semantic graph is equipped with visual attributes generated by a large language model, providing rich visual contextual information for semantic alignment.

[0060] (2) The present invention designs a key semantic node mining mechanism. By quantifying the sensitivity of nodes embedded in text perturbation, it identifies and anchors fine-grained nodes that are highly sensitive in semantics, preventing them from losing discriminative features due to excessive smoothing during graph propagation, and effectively ensuring the ability to capture subtle semantic differences.

[0061] (3) This invention proposes a semantic relevance-driven Laplace propagation algorithm, which constructs a weighted adjacency matrix based on the similarity of visual attributes between nodes, guides semantic information to propagate adaptively on the graph, realizes context-aware alignment of cross-granularity embedded vectors, and improves the consistency of visual-semantic matching.

[0062] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0063] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0064] Figure 1 This is a flowchart of a target detection method for multi-granularity labels;

[0065] Figure 2 It is a fine-grained semantic graph. Detailed Implementation

[0066] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] In the description of this invention, it should be noted that the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", "front", "rear", "lateral", "longitudinal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0068] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0069] Example:

[0070] See Figure 1 This invention provides a target detection method for multi-granularity labels, comprising the following steps:

[0071] S1. Construct a fine-grained semantic graph and obtain the initial semantic embeddings of multiple nodes;

[0072] In this embodiment, the fine-grained semantic graph Includes a set of nodes Edge set and visual attribute set ;

[0073] The set of nodes Includes multiple category nodes, each node Each corresponds to a specific category. , It is a set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure;

[0074] The set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across different levels;

[0075] The set of visual attributes Includes the set of visual attributes for all nodes.

[0076] See Figure 2 In the fine-grained semantic graph (FSG), solid ellipses represent category nodes, such as "vehicles," "car," and "airplane." Unlike traditional hierarchical structures, the FSG of this invention associates each node with a set of visual attributes, such as... Figure 2 As shown by the dashed ellipse, for example, the "car" node is associated with attributes such as "boxy bodyshape" and "high chassis". These attributes greatly enrich the visual semantic connotation of the node, making its representation no longer limited to a single text label.

[0077] Obtaining the initial semantic embeddings of multiple nodes specifically involves:

[0078] For each node Node generation using large language model LLM visual attribute set Subsequently, a pre-trained text encoder was used. Encode it to obtain the final attribute embedding vector of the node. :

[0079] ;

[0080] in: For visual attribute set The total number of attributes in the middle. Index of the attribute; For a pre-trained text encoder; The final attribute embedding vector for the node is a comprehensive semantic representation that incorporates all its visual attributes;

[0081] To obtain an initial representation of a node rich in contextual information, for each node... Construct a hierarchical text description This description integrates the node's label, parent / child node information, and the node's final attribute embedding vector; this description is then input into a text encoder. The initial embedding vector of the node is obtained. That is, the initial semantic embedding of the node:

[0082] .

[0083] This invention constructs a fine-grained semantic graph (FSG). This graph structure not only includes traditional hierarchical (parent-child) relationships but also innovatively introduces sibling edges and cross-layer edges to capture richer lateral associations between categories of the same and different granularities. Furthermore, the graph is injected with visually based attributes through a large language model (LLM), greatly enriching the semantic connotation of each category node and providing a solid foundation for subsequent semantic alignment.

[0084] S2. Mine key nodes and anchor the initial semantic embedding of key nodes; specifically:

[0085] For nodes Text description conduct Subsequent perturbations (such as adjusting word order or replacing synonyms) generate a new set of descriptions, which are then encoded into a set of embeddings. ;

[0086] Computational embedding variance :

[0087] ;

[0088] in: express The average vector of the perturbed embeddings; the larger the variance value, the more sensitive the semantics of the node are to the text description;

[0089] The variance value is then globally normalized to obtain the final sensitivity score. :

[0090] ;

[0091] It provides a relative metric, with higher scores indicating that the node... It has fine-grained semantics that are highly sensitive to disturbances and should be regarded as a key node;

[0092] Set a threshold ,if > Then the node The initial semantic embedding of the key node. It will be preserved, resulting in an anchored embedding. .

[0093] S3. Semantically align and fuse the fine-grained semantic graph with the anchored embeddings in S2 to obtain the final node embeddings. In this invention, the consistency of cross-granular semantics is enhanced by a semantic relevance-driven Laplacian propagation method. The original information of the anchored nodes is fused to retain discriminability, and the final node embeddings are obtained. Specifically:

[0094] To guide information propagation based on visual similarity on the fine-grained semantic graph, this method constructs a weighted adjacency matrix. Its elements Calculated using the following formula:

[0095] ;

[0096] in: It is a node Attribute embedding vector; It is a node Attribute embedding vector; It is a node The neighboring nodes; These are neighbor nodes. Attribute embedding vector; Representative node The set of all neighboring nodes in the fine-grained semantic graph; It is a temperature hyperparameter; this weight matrix ensures stronger information interaction between nodes with more similar visual attributes.

[0097] For adjacency matrix Normalization is performed to obtain the normalized adjacency matrix. Specifically:

[0098] Define an angle matrix Its diagonal elements For all nodes The sum of the weights of connected edges. ;

[0099] The normalized adjacency matrix is ​​obtained by calculating the following formula. :

[0100] ;

[0101] Adjacency matrix based on normalization The embedding matrix of a node is updated through iterative Laplace propagation, with the following iterative formula:

[0102] ;

[0103] in: It is the first The embedding matrix of the next iteration; It is a hyperparameter that controls the propagation intensity; after After rounds of iteration, we obtain fully propagated and aligned embedding vectors. ;

[0104] To balance global semantic consistency with local fine-grained discriminability, this method will and Weighted fusion is performed to obtain the final node embedding. :

[0105] ;

[0106] in: It is a balancing factor used to adjust the contribution ratio between the original information and the aligned information; It is a node The semantically aligned embedding vector obtained after the graph propagation process.

[0107] S4. Zero-shot detection is performed through the final node embedding; in the zero-shot detection stage, for any image region feature... By calculating its relationship with the final node embedding Matching is performed using cosine similarity between the two pairs:

[0108] ;

[0109] The point corresponding to the final predicted category of this area The category with the highest similarity score is determined, i.e.:

[0110] .

[0111] To verify the feasibility of this method, the following experimental verification is conducted:

[0112] 1) Experimental Setup: Based on the iNatLoc and FSOD datasets, a zero-shot detection protocol was adopted, with mAP50 as the primary evaluation metric. Three pre-trained detectors (Detic, CoDet, VLDet) and two backbone networks (ResNet-50, Swin-B) were used for testing. The FSG (Fine-grained Semantic Graph) was constructed using GPT-3.5 to generate 5-10 visual attributes. The KSM (Key Semantic Node Mining) module had a perturbation count of K=3 and a sensitivity threshold of the top 30%. The SLP (Semantic Relevance Driven Laplacian Propagation) module had its propagation strength set to... =0.7, iteration number T=5, balance factor =0.7.

[0113] 2) Comparative experiment:

[0114] The experimental results are shown in Tables 1 and 2.

[0115] Table 1 Comparison Results

[0116]

[0117] Under the zero-sample setting (mAP50) in Table 1, with ResNet-50 and supervision combination I (LVIS) as the control baseline, our method (GraSecon) showed an average improvement of 6.5 mAP50 on iNatLoc (53.0% vs 46.5%) and an average improvement of 5.4 mAP50 on FSOD (40.4% vs 35.0%) compared to SHiNe. Within the same table, if the supervision combination was changed, the average gain was +5.6 / +4.3 / +3.5 on iNatLoc (corresponding to II / III / IV), and +4.4 / +2.9 / +2.6 on FSOD (corresponding to II / III / IV), demonstrating a stable and considerable improvement even under stronger supervision.

[0118] Table 2 Comparison Results

[0119]

[0120] Referring to Table 2, under different combinations of pre-trained detectors and additional supervision (Table 2, supervision signals are LVIS (box annotation) + CC3M (image description)), GraSecon directly applied to CoDet / VLDet also brings significant gains. Under iNatLoc / ResNet-50, compared to SHiNe, GraSecon improves by an average of 8.0 on the CoDet branch (42.5% vs 34.5%) and by an average of 3.4 on the VLDet branch (55.9% vs 52.5%). Looking at the different levels, there are representative gains of +10.7 / +10.2 / +10.0 at L1 / L5 / L2 respectively; under FSOD / Swin-B, compared to SHiNe, GraSecon has an average improvement of 4.9 on the CoDet branch (46.1% vs 41.2%) and an average improvement of 5.2 on the VLDet branch (47.5% vs 42.3%); among them, the single-point improvement at the L1 level can reach +10.7 (CoDet) and +11.1 (VLDet).

[0121] 3) Ablation experiment

[0122] Referring to Table 3, ablation studies on the iNatLoc dataset show that removing the Key Semantic Node Mining (KSM) module leads to a 2.0% decrease in average mAP50, with a particularly significant performance degradation at the fine-grained L5 level; removing the Semantic Relevance Laplacian Propagation (SLP) module leads to a 2.8% performance degradation; and removing sibling edges and cross-layer edges (i.e., FSG structures) in the fine-grained semantic graph leads to a 4.1% performance degradation. These results fully validate the crucial role of each core module in our method (GraSecon) in mitigating semantic drift and enhancing cross-granular semantic consistency, further highlighting the comprehensive advantages of this invention in semantic representation alignment and fine-grained discriminativity.

[0123] Table 3 Ablation Experiment Results

[0124]

[0125] The experimental results above fully demonstrate the effectiveness, robustness, and practicality of this method in solving the problem of granularity-induced semantic drift in open vocabulary target detection.

[0126] The present invention also provides a target detection system for multi-granularity labels, comprising:

[0127] A fine-grained semantic graph generation module is used to generate fine-grained semantic graphs; the fine-grained semantic graphs... Includes a set of nodes Edge set and visual attribute set The set of nodes Includes multiple category nodes; each node Each corresponds to a specific category. , The set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure; the set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across levels; the visual attribute set Includes the set of visual attributes of all nodes;

[0128] The key semantic node mining module is used to mine key nodes and anchor the initial semantic embedding of key nodes.

[0129] The semantic relevance-driven Laplacian propagation module guides the weighted propagation of semantic information on the fine-grained semantic graph based on the similarity of visual attributes between nodes, enabling coherent and context-aware alignment between embedding vectors of different granularities.

[0130] In this invention, the Key Semantic Node Mining (KSM) module identifies the most semantically sensitive fine-grained nodes by calculating the embedding variance of nodes under text perturbation and "anchors" their original embeddings to prevent their unique and distinctive features from being diluted by "over-smoothing" in the subsequent global alignment process. The Semantic Relevance Driven Laplacian Propagation (SLP) module is an adaptive information propagation algorithm that guides the weighted propagation of semantic information on the graph based on the similarity of visual attributes between nodes, promoting coherent and context-aware alignment between embedding vectors of different granularities. The two modules work together to achieve efficient and robust semantic alignment on the graph structure.

[0131] The present invention also provides an electronic device corresponding to the above embodiments. The electronic device may be a processing device for a client, such as a mobile phone, a laptop, a tablet computer, a desktop computer, etc., to execute the methods of the above embodiments.

[0132] The electronic device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method described in the above embodiment.

[0133] In some implementations, the memory may be high-speed random access memory (RAM), and may also include nonvolatile memory, such as at least one disk storage.

[0134] In other implementations, the processor can be any type of general-purpose processor, such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation here.

[0135] The present invention also provides a readable storage medium corresponding to the above embodiments, wherein a computer program / instructions are stored thereon. When the computer program / instructions are executed by a processor, they implement the steps of the methods described in the above embodiments.

[0136] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0137] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0138] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method for multi-granularity labels, characterized in that, Includes the following steps: S1. Construct a fine-grained semantic graph and obtain the initial semantic embeddings of multiple nodes; Fine-grained semantic graph Includes a set of nodes Edge set and visual attribute set ; The set of nodes Includes multiple category nodes, each node Each corresponds to a specific category. , It is a set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure; The set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across different levels; The set of visual attributes Includes the set of visual attributes of all nodes; S2. Mine key nodes and anchor the initial semantic embedding of key nodes; Specifically: For nodes Text description conduct The perturbation generates a new set of descriptions, which are then encoded into a set of embeddings. ; Computational embedding variance : ； in: express The average vector of the perturbation embeddings; The variance value is then globally normalized to obtain the final sensitivity score. : ； Set a threshold ,if > Then the node The initial semantic embedding of the key node. It will be preserved, resulting in an anchored embedding. ; S3. Perform semantic alignment and fusion between the fine-grained semantic graph and the anchored embeddings in S2 to obtain the final node embeddings. Specifically, it involves constructing a weighted adjacency matrix. Its elements Calculated using the following formula: ； in: It is a node Attribute embedding vector; It is a node Attribute embedding vector; It is a node The neighboring nodes; These are neighbor nodes. Attribute embedding vector; Representative node The set of all neighboring nodes in the fine-grained semantic graph; It is a temperature hyperparameter; For adjacency matrix Normalization is performed to obtain the normalized adjacency matrix. ; Adjacency matrix based on normalization The embedding matrix of a node is updated through iterative Laplace propagation, with the following iterative formula: ； in: It is the first The embedding matrix of the next iteration; It is a hyperparameter that controls the propagation intensity; after After rounds of iteration, we obtain fully propagated and aligned embedding vectors. ; Will and Weighted fusion is performed to obtain the final node embedding. : ； in: It is a balancing factor; It is a node The semantically aligned embedding vector obtained after the graph propagation process; S4. Zero-shot detection is performed through the final node embedding.

2. The target detection method for multi-granularity labels according to claim 1, characterized in that, Obtaining the initial semantic embeddings of multiple nodes specifically involves: For each node Node generation using large language model LLM visual attribute set ; Subsequently, a pre-trained text encoder was used. Encode it to obtain the final attribute embedding vector of the node. : ； in: For visual attribute set The total number of attributes in the middle. Index of the attribute; For a pre-trained text encoder; Embed the final attribute vector for the node; For each node Construct a hierarchical text description This description integrates the node's label, parent-child node information, and the node's final attribute embedding vector; this description is then input into a text encoder. The initial embedding vector of the node is obtained. That is, the initial semantic embedding of the node: 。 3. The target detection method for multi-granularity labels according to claim 2, characterized in that, For adjacency matrix The normalization process specifically involves: Define an angle matrix Its diagonal elements For all nodes The sum of the weights of connected edges. ; The normalized adjacency matrix is ​​obtained by calculating the following formula. : 。 4. The target detection method for multi-granularity labels according to claim 3, characterized in that, In the zero-shot detection phase, for any image region feature By calculating its relationship with the final node embedding Matching is performed using cosine similarity between the two pairs: ； The point corresponding to the final predicted category of this area The category with the highest similarity score is determined, i.e.: 。 5. A target detection system for multi-granularity labels, used to implement the target detection method for multi-granularity labels as described in any one of claims 1-4, characterized in that, include: A fine-grained semantic graph generation module is used to generate fine-grained semantic graphs; the fine-grained semantic graphs... Includes a set of nodes Edge set and visual attribute set The set of nodes Includes multiple category nodes; each node Each corresponds to a specific category. , The set of categories; nodes of different granularities and nodes of the same granularity are interconnected to form a multi-level network structure; the set of edges Including hierarchical edges Brothers and cross-layer edge The hierarchical edge The sibling edge is used to connect parent and child nodes. The cross-layer edge is used to connect sibling nodes that share a common parent node. Used to connect other related nodes across levels; the visual attribute set Includes the set of visual attributes of all nodes; The key semantic node mining module is used to mine key nodes and anchor the initial semantic embedding of key nodes. The semantic relevance-driven Laplacian propagation module guides the weighted propagation of semantic information on the fine-grained semantic graph based on the similarity of visual attributes between nodes, enabling coherent and context-aware alignment between embedding vectors of different granularities.

6. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program; the processor executes the computer program to implement the target detection method for multi-granularity labels as described in any one of claims 1-4.

7. A readable storage medium, characterized in that, The readable storage medium stores a computer program, and the processor executes the computer program to implement the target detection method for multi-granularity labels as described in any one of claims 1-4.

Images