Weakly supervised object detection method and system based on part mining and overall reconstruction

Abstract

The application relates to a weakly supervised object detection method and system based on component mining and overall reconstruction, which comprises the following steps: mining a plurality of target components of an image-level category label, performing cross-image clustering on detection regions extracted from training images to obtain a plurality of clustering clusters; calculating the distance between the average visual embedding of each clustering cluster and the text embedding of each target component, determining the minimum distance of each average visual embedding, and when the minimum distance is less than a first threshold, determining the target component corresponding to the minimum distance as the component label of each detection region in the corresponding clustering cluster; mapping the component label of the detection region to the training image to obtain a labeled image set; using a component detector to output the component detection frame of each component in the to-be-detected image, performing instance reconstruction based on the component detection frame to obtain the detection frame of the to-be-detected instance in the to-be-detected image, and training the component detector by using a plurality of labeled image sets, wherein the component labels of the labeled image sets belong to different instances. The method can improve the detection accuracy.

Description

Technical Field

[0001] This application relates to the fields of computer vision and pattern recognition technology, and in particular to a weakly supervised target detection method and system based on component mining and overall reconstruction. Background Technology

[0002] With the development of deep learning, fully supervised object detection has made significant progress, but it still relies on a large amount of precise manual annotation, such as bounding boxes and instance masks, resulting in extremely high data acquisition costs. Weakly supervised object detection (WSOD) aims to train an object detector with only image-level class labels, thereby reducing the reliance on tedious and expensive bounding box annotations.

[0003] Currently, existing WSOD (Software-Defined Detection) systems typically employ a multi-instance learning framework, which maps image-level labels to extracted features for multi-instance learning. However, this ignores the fact that image-level labels are composed of semantically meaningful components. Aligning image-level labels forces the detector to rely solely on them for training, making it difficult to learn richer and more discriminative feature representations, thus failing to output accurate detection results. For example, invention application CN114648665A only maps image-level labels to extracted features, making it difficult for the trained weakly supervised detection model to learn richer and more discriminative feature representations, thus failing to output accurate detection boxes. Summary of the Invention

[0004] Therefore, it is necessary to provide a weakly supervised target detection method and system based on component mining and overall reconstruction that can output accurate detection results, addressing the aforementioned technical problems.

[0005] A weakly supervised target detection method based on component mining and overall reconstruction, the method comprising:

[0006] S1. Mine multiple target components of image-level category labels, and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters, wherein the category label of each instance in the multiple training images is consistent with the image-level category label;

[0007] S2. Calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, determine the target component corresponding to the minimum distance as the component label of each detection region in the cluster corresponding to the minimum distance.

[0008] S3. Map the component labels of the detected region to the training image to obtain a set of labeled images with component labels;

[0009] S4. Use a component detector to output the component detection boxes of each component in the image to be tested, and perform instance reconstruction based on each component detection box to obtain the detection boxes of each instance to be tested in the image to be tested. The component detector is trained by multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set.

[0010] In this application, multiple target components are mined from image-level category labels, and cross-image clustering is performed on the detection regions extracted from multiple training images to obtain multiple clusters. The distance between the average visual embedding of each cluster and the text embedding of each target component is calculated, and the minimum distance corresponding to each average visual embedding is determined. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is determined as the component label of each detection region in the cluster corresponding to the minimum distance. The component labels of the detection regions are mapped to the training images to obtain a set of labeled images with component labels. This takes into account the information that the image-level labels themselves are composed of components with semantic meaning, thereby achieving the alignment of component labels and detection regions, rather than the alignment of image-level category labels and detection regions. This allows the component detector to learn richer and more discriminative feature representations when trained based on the labeled image set. As a result, when the component detector outputs the component detection boxes of each component in the test image, it can obtain accurate component detection boxes, thereby improving the accuracy of the detection boxes of each test instance in the test image.

[0011] In one embodiment, the method for excavating the target component in step S1 includes:

[0012] Determine image-level category labels and cue words for components used to mine the image-level category labels, wherein the image-level category labels are text information;

[0013] The image-level category labels and the prompt words are input into the language model, and the candidate components corresponding to the image-level category labels are output.

[0014] Based on the first text embedding of the candidate component and the second text embedding of the parent node of the candidate component, calculate the first cosine distance between the candidate component and the parent node;

[0015] When the first cosine distance is less than the second threshold, the candidate component is eliminated; when the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, the candidate component or its sibling node is eliminated to obtain the target component of each image-level category label.

[0016] The formula for calculating the first cosine distance is: d vis (i,p) represents the first cosine distance. The first text embedding for candidate component i. The second text embedding for the parent node p.

[0017] In this application, a first cosine distance between a candidate component and its parent node is calculated based on the first text embedding of the candidate component and the second text embedding of the parent node of the candidate component. When the first cosine distance is less than a second threshold, the candidate component is eliminated. When the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, either the candidate component or the sibling node is eliminated. This can eliminate visually similar and redundant candidate components, so that the final target components are all visually inconsistent.

[0018] In one embodiment, the process of obtaining the plurality of clusters in step S1 includes:

[0019] Acquire multiple training images and extract multiple detection regions from the multiple training images;

[0020] Each of the detection regions and each of the training images are input into a visual language model, and the visual vectors of each of the detection regions are generated by the image encoder in the visual language model.

[0021] Based on the visual vectors of each of the detection regions, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, each of which includes at least one of the detection regions.

[0022] In this application, by inputting each detection region and each training image into a visual language model, the visual vector of each detection region is generated by the image encoder in the visual language model. Based on the visual vector of each detection region, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, thereby achieving the classification of the detection regions.

[0023] In one embodiment, the process of obtaining the average visual embedding of each cluster in step S2 includes:

[0024] By removing outlier and noise detection regions from each cluster, the core cluster of each cluster is obtained.

[0025] Based on the visual vectors of the detection regions in each core cluster, the average visual embedding of each cluster is calculated.

[0026] The formula for calculating the average visual embedding of each cluster is as follows: ; For cluster c j Average visual embedding, For cluster c j core cluster The visual vector of the detection region r in the image.

[0027] In this application, outlier detection regions and noise detection regions are removed from each cluster to obtain the core cluster of each cluster. The average visual embedding of each cluster is calculated based on the visual vector of each detection region in each core cluster. This avoids the interference of outlier detection regions and noise detection regions on the average visual embedding, thereby obtaining an accurate average visual embedding.

[0028] In one embodiment, the step of using a part detector to output part detection boxes for each part in the image under test includes:

[0029] Based on the detection regions of each type of component label in multiple labeled image sets, binary component detectors corresponding to each type of component label are trained respectively to obtain each trained binary component detector. The component detection boxes of each component in the image to be tested are output using each trained binary component detector.

[0030] Alternatively, based on the detection regions of each type of component label in multiple labeled image sets, a multi-class component detector is trained to obtain the trained multi-class component detector; the trained multi-class component detector is then used to output the component detection boxes of each component in the image to be tested.

[0031] In this application, binary component detectors corresponding to each type of component label are trained based on the detection regions of each type of component label in multiple labeled image sets to obtain trained binary component detectors, or multi-class component detectors are trained based on the detection regions of each type of component label in multiple labeled image sets to obtain trained multi-class component detectors. In this way, the component detection boxes of each component in the image to be tested can be obtained through binary component detectors or multi-class component detectors.

[0032] In one embodiment, the step of reconstructing instances based on the detection bounding boxes of each component to obtain the detection bounding boxes of each instance to be tested in the image to be tested includes:

[0033] Based on the edge weights between the detection boxes of each component, the first detection box belonging to the same test instance is determined, and based on the first detection box corresponding to each test instance, the detection box of each test instance is reconstructed.

[0034] The formula for calculating the edge weight is as follows: ;P ij For component Bi The component detection frame and component B j The edge weights between the component detection boxes For component B i The component detection frame and component B j The crossover ratio between component detection frames, L total It is component B i The center point of the component detection frame and component B j The length of the line connecting the center points of the component detection frame. The connection is located in component B. i The length of the component detection frame, The connection is located in component B. j The length of the component detection frame.

[0035] In this application, by using Calculate the edge weights between the component detection boxes. This allows for a more accurate determination of the edge weights, taking into account the intersection-union ratio.

[0036] In one embodiment, determining the first component detection box belonging to the same test instance, and reconstructing the detection box of each test instance based on the first component detection box corresponding to each test instance, includes:

[0037] When the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection boxes corresponding to the same test instance.

[0038] Determine the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of the detection frame of the first component corresponding to each of the test instances;

[0039] Based on the maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate corresponding to each of the test instances, the detection box of each test instance is reconstructed.

[0040] In this application, when the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection boxes corresponding to the same test instance. This facilitates the combination of the first component detection boxes corresponding to the same test instance to obtain the detection box of the test instance. Furthermore, since this application includes not only the final detection box of the test instance but also the component detection boxes of each component constituting the test instance, the detection process has stronger interpretability and is more robust to occlusion or atypical poses.

[0041] In one embodiment, the method further includes:

[0042] The component detector outputs the component detection boxes of each component in the image under test, and the pre-trained basic weakly supervised model outputs multiple anchor points of the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set.

[0043] Centered on each anchor point, the detection boxes of each component within a preset radius of the anchor point are determined as the second component detection boxes corresponding to the first test instance in the image to be tested, and the detection boxes of each first test instance are reconstructed based on the second component detection boxes corresponding to each first test instance.

[0044] Based on the edge weights between the non-second component detection boxes, the third component detection box corresponding to the second test instance in the test image is determined from the non-second component detection boxes, and the detection box of the second test instance is reconstructed based on the third component detection boxes corresponding to each of the second test instances.

[0045] In this application, a component detector outputs component detection boxes for each component in the image under test, and a pre-trained weakly supervised model outputs multiple anchor points for the image under test. Centered on each anchor point, component detection boxes within a preset radius of the anchor point are identified as second component detection boxes corresponding to the first test instance in the image under test. Based on the second component detection boxes corresponding to each first test instance, the detection boxes for each first test instance are reconstructed. This effectively recalls spatially dispersed component detection boxes that are difficult to reconstruct using edge weights. By determining the third component detection boxes corresponding to the second test instance in the image under test from the non-second component detection boxes according to the edge weights between them, and reconstructing the detection boxes for each second test instance based on their respective third component detection boxes, instance reconstruction operations can be performed on component detection boxes not covered by any anchor points, ensuring that component detection boxes at the edges are not missed.

[0046] A weakly supervised target detection system based on component mining and overall reconstruction, the system comprising:

[0047] The mining module is used to mine multiple target parts with image-level category labels and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters, wherein the category labels of each instance in the multiple training images are consistent with the image-level category labels.

[0048] An alignment module is used to calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and to determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is determined as the component label of each detection region in the cluster corresponding to the minimum distance.

[0049] A mapping module is used to map the component labels of the detected region to the training image to obtain a set of labeled images with component labels;

[0050] The detection module is used to output component detection boxes for each component in the image under test using a component detector, and to perform instance reconstruction based on each component detection box to obtain the detection box for each instance under test in the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set.

[0051] The aforementioned weakly supervised object detection system based on component mining and overall reconstruction mines multiple target components with image-level category labels and performs cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters. It calculates the distance between the average visual embedding of each cluster and the text embedding of each target component, and determines the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is identified as the component label of each detection region in the cluster corresponding to the minimum distance. The component labels of the detection regions are mapped to the training images to obtain a labeled image set with component labels. This takes into account the information that the image-level labels themselves are composed of semantically meaningful components, thus achieving alignment between component labels and detection regions, rather than alignment between image-level category labels and detection regions. This allows the component detector to learn richer and more discriminative feature representations when training on the labeled image set. Consequently, when using the component detector to output the component detection boxes of each component in the test image, accurate component detection boxes can be obtained, improving the accuracy of the detection boxes for each test instance in the test image. Attached Figure Description

[0052] Figure 1 This is an application environment diagram of a weakly supervised target detection method based on component mining and overall reconstruction in one embodiment;

[0053] Figure 2 This is a flowchart illustrating a weakly supervised target detection method based on component mining and overall reconstruction in one embodiment.

[0054] Figure 3 This is a schematic diagram of the component detector output in one embodiment;

[0055] Figure 4This is a schematic diagram of the candidate components output in one embodiment;

[0056] Figure 5 This is a schematic diagram of cross-image clustering analysis in one embodiment;

[0057] Figure 6 This is a schematic diagram of the diagram structure in one embodiment;

[0058] Figure 7 This is a schematic diagram of the detection box of the test instance obtained by edge weight reconstruction in one embodiment;

[0059] Figure 8 This is a schematic diagram of the detection boxes of each instance under test obtained by anchor point reconstruction in one embodiment;

[0060] Figure 9 This is a schematic diagram of the combined output of the detection boxes of the first test instance and the detection boxes of the second test instance in one embodiment;

[0061] Figure 10 This is a schematic diagram of the overall process of a weakly supervised target detection method based on component mining and overall reconstruction in one embodiment;

[0062] Figure 11 This is a block diagram of a weakly supervised target detection system based on component mining and overall reconstruction in one embodiment;

[0063] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] The weakly supervised target detection method based on component mining and overall reconstruction provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 interacts with server 104 via a wired / wireless channel. A data storage system can store the data that server 104 needs to process. Multiple target parts with image-level category labels are mined, and cross-image clustering is performed on the detection regions extracted from multiple training images to obtain multiple clusters. The category labels of each instance in the multiple training images are consistent with the image-level category labels. The distance between the average visual embedding of each cluster and the text embedding of each target part is calculated, and the minimum distance corresponding to each average visual embedding is determined. When the minimum distance is less than a first threshold, the target part corresponding to the minimum distance is determined as the part label of each detection region in the cluster corresponding to the minimum distance. The part labels of the detection regions are mapped to the training images to obtain a set of labeled images with part labels. A part detector is used to output the part detection boxes of each part in the image to be tested, and instance reconstruction is performed based on each part detection box to obtain the detection boxes of each instance to be tested in the image to be tested. The part detector is trained from multiple sets of labeled images, and the instances to which the part labels belong in each set of labeled images are different. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, etc. The server 104 can be a single server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers.

[0066] In one embodiment, such as Figure 2 As shown, a weakly supervised target detection method based on component mining and overall reconstruction is presented, which is then applied to... Figure 1 Taking server 104 as an example, the following steps are included:

[0067] S1. Mine multiple target components of image-level category labels, and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters. The category labels of each instance in multiple training images are consistent with the image-level category labels.

[0068] Image-level category labels are textual information. For example, cat and forklift.

[0069] Target parts refer to the textual information of each part of an instance corresponding to an image-level category label. For example, if the image-level category label is "cat," then the target parts of the corresponding instance "cat" include, but are not limited to, the head, torso, limbs, tail, ears, eyes, nose, and mouth.

[0070] Each training image contains instances with the same category label, and each training image contains instances of exactly one type. For example, every training image may contain only cats, with the image-level category label being "cat," or every training image may contain only forklifts, with the image-level category label being "forklift."

[0071] A detection region is the area containing a part of an instance extracted from a training image using an object detection algorithm. For example, if the instance in the training images is a cat, then the detection regions containing the cat's ears and the detection regions containing the cat's head can be extracted from multiple training images. Object detection algorithms include, but are not limited to, selective search algorithms.

[0072] Cross-image clustering refers to performing cluster analysis on all detection regions extracted from multiple training images together to obtain multiple clusters. Algorithms for cross-image clustering include, but are not limited to, K-Means and DBSCAN (Density-Based Spatial Clustering of Applications with Noise). Each cluster includes at least one detection region.

[0073] S2. Calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than the first threshold, determine the target component corresponding to the minimum distance as the component label of each detection region in the cluster corresponding to the minimum distance.

[0074] The average visual embedding of each cluster can be obtained from the visual vectors of each detection region within the cluster. The visual vectors of each detection region can be obtained from the image encoder of the visual language model.

[0075] The process of obtaining the text embedding of the target component is as follows: The target component is input into the text encoder of the visual language model to obtain its text embedding. The expression for obtaining the text embedding of the target component by inputting it into the text encoder of the visual language model is: e k Text embedding for target component k. A text encoder representing a visual language model. The text information for the target component k.

[0076] The formula for calculating the distance between the average visual embedding of a cluster and the text embedding of the target component is: , The average visual embedding for cluster j and the text embedding e of the target component k k The distance between them.

[0077] The minimum distance corresponding to the average visual embedding refers to the minimum distance between the average visual embedding and the text embeddings of each target component. For example, if the target components include component 1 and component 2, the distance between the average visual embedding and the text embedding of component 1 is A, and the distance between the average visual embedding and the text embedding of component 2 is B. Since A is greater than B, the minimum distance corresponding to the average visual embedding is B.

[0078] The first threshold is a pre-set threshold. Each average visual embedding corresponds to a minimum distance, but not every minimum distance corresponding to an average visual embedding is less than the first threshold.

[0079] The target component corresponding to the minimum distance refers to the target component to which the text embedding at that minimum distance belongs. The cluster corresponding to the minimum distance refers to the cluster to which the average visual embedding at that minimum distance belongs.

[0080] Since the target component is text information, and the component label is also essentially text information, the target component can be directly designated as the component label.

[0081] S3. Map the component labels of the detected regions to the training images to obtain a set of labeled images with component labels;

[0082] In essence, mapping refers to associating the component labels of each detection region with the specific detection regions in the original training image to form a set of labeled images that can be used for training. That is, after the component labels of each detection region are mapped to the training image, each detection region in the training image will have a corresponding component label, instead of just a single detection box.

[0083] For each class of instances, the image-level category label and training images can be used to obtain the labeled image set for that class of instances. By performing S1, S2, and S3 respectively on the image-level category labels and training images for multiple classes of instances, the labeled image set for each class of instances can be obtained, resulting in multiple labeled image sets.

[0084] S4. Use the component detector to output the component detection boxes of each component in the image to be tested, and perform instance reconstruction based on each component detection box to obtain the detection boxes of each instance to be tested in the image to be tested. The component detector is trained from multiple labeled image sets, and the instances to which the component labels belong in each labeled image set are different.

[0085] The part detector is a detector that outputs bounding boxes for each part in the image under test. The part detector can output not only the bounding boxes but also their confidence scores. The part detector can be a multi-class part detector or a binary part detector. For example, Faster R-CNN (Faster Region-based Convolutional Neural Network). A schematic diagram of the part detector output is shown below. Figure 3 As shown.

[0086] Instance reconstruction essentially determines whether each component detection box belongs to the same test instance in the image to be tested. If multiple component detection boxes belong to the same test instance, the component detection boxes belonging to the same test instance are combined to form a detection box that can include all component detection boxes of the test instance.

[0087] The part detector can output the part label of each part detection box. The part label and the target part are equivalent. The target part is the target part with the image-level category label. Therefore, after determining the part label of the part detection box, the image-level category label of the part detection box can be determined directly, and the instance to which the part detection box belongs can be determined.

[0088] In the aforementioned weakly supervised object detection method based on component mining and overall reconstruction, multiple target components with image-level category labels are mined, and the detection regions extracted from multiple training images are clustered across images to obtain multiple clusters. The distance between the average visual embedding of each cluster and the text embedding of each target component is calculated, and the minimum distance corresponding to each average visual embedding is determined. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is determined as the component label of each detection region in the cluster corresponding to the minimum distance. The component labels of the detection regions are mapped to the training images to obtain a set of labeled images with component labels. This takes into account the information that the image-level labels themselves are composed of components with semantic meaning, thereby achieving the alignment of component labels and detection regions, rather than the alignment of image-level category labels and detection regions. This allows the component detector to learn richer and more discriminative feature representations when training based on the labeled image set. As a result, when using the component detector to output the component detection boxes of each component in the test image, accurate component detection boxes can be obtained, improving the accuracy of the detection boxes of each test instance in the test image.

[0089] In one embodiment, the method for excavating the target component in step S1 includes:

[0090] Determine image-level category labels and cue words for components used to mine image-level category labels; image-level category labels are text information.

[0091] The image-level category labels and prompt words are input into the language model, and the candidate parts corresponding to the image-level category labels are output.

[0092] The first cosine distance between the candidate component and its parent node is calculated based on the first text embedding of the candidate component and the second text embedding of the parent node of the candidate component.

[0093] When the first cosine distance is less than the second threshold, candidate components are eliminated; when the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, candidate components or sibling nodes are eliminated to obtain the target components for each image-level category label.

[0094] The formula for calculating the first cosine distance is: d vis (i,p) represents the first cosine distance. The first text embedding for candidate component i. The second text embedding for the parent node p.

[0095] Each candidate component must have at least one component. These candidate components have a hierarchical relationship, forming a tree structure with parent-child nodes. For example, a schematic diagram of the candidate components output for the image-level category label "forklift" is shown below. Figure 4 As shown.

[0096] Cue words can be used to constrain the language model to output names of visible, non-hidden, and well-defined candidate parts. For example, if the cue word is "Please break down the image-level category label 'forklift' into at most two levels of visible external parts based on their appearance structure," then candidate parts such as "wheels" and "cab" will be output, while internal parts such as "engine" will not be output as candidate parts. Language models include, but are not limited to, LLM (Large Language Model).

[0097] The output image-level category labels correspond to candidate parts in the following formats: {image-level category label}-{first-level candidate parts}-{second-level candidate parts}, {image-level category label}-{first-level candidate parts}, and {image-level category label}-atomic.

[0098] When outputting candidate parts corresponding to image-level category labels, the language model follows the following rules:

[0099] 1. Only list external parts that can be directly observed from the image, ignoring the internal or hidden structures of the instance;

[0100] 2. First-level candidate parts must be clearly distinguishable external areas in the instance, such as "body" or "robotic arm";

[0101] 3. Secondary candidate parts are listed only if there are primary candidate parts, such as "wheel-hub";

[0102] 4. Excludes temporary accessories or environmentally related items, such as "forklift-cargo," which are not listed.

[0103] 5. If an instance cannot be divided in appearance, mark it as "atomic".

[0104] Furthermore, the tree structure formed by the output candidate components only includes a two-level structure, namely root node - first-level child node - second-level child node, the first-level child node is the parent node of the second-level child node, the root node is the parent node of the first-level child node, and the root node is the image-level category label.

[0105] A parent node can be an image-level category label or a candidate part. A sibling node is a candidate part that belongs to the same level as the candidate part, and the parent node of the candidate part and its sibling nodes are the same.

[0106] The process for obtaining the first text embedding of a candidate component is as follows: the candidate component is input into the text encoder of the visual language model to obtain the first text embedding of the candidate component. The process for obtaining the second text embedding of a parent node is as follows: the parent node is input into the text encoder of the visual language model to obtain the second text embedding of the parent node.

[0107] Furthermore, in use When calculating the second cosine distance, d vis (i,p) represents the second cosine distance. The first text embedding for candidate component i. The second text embedding of sibling node p is obtained by inputting sibling node p into the text encoder of the visual language model.

[0108] In this embodiment, a first cosine distance between a candidate component and its parent node is calculated based on the first text embedding of the candidate component and the second text embedding of the candidate component's parent node. When the first cosine distance is less than a second threshold, the candidate component is eliminated. When the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, either the candidate component or the sibling node is eliminated. This can eliminate visually similar and redundant candidate components, so that the final target components are all visually inconsistent.

[0109] In one embodiment, the process of obtaining multiple clusters in step S1 includes:

[0110] Acquire multiple training images and extract multiple detection regions from them;

[0111] Each detection region and each training image are input into the visual language model, and the visual vector of each detection region is generated by the image encoder in the visual language model.

[0112] Based on the visual vectors of each detection region, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, each of which includes at least one detection region.

[0113] Among them, visual language models include, but are not limited to, CLIP (Contrastive Language–ImagePretraining) models.

[0114] The visual vector is the feature vector of the detection region. The detection region is a portion of the training image, not just a detection box.

[0115] When generating visual vectors for each detection region through the image encoder in the visual language model, all generated visual vectors are vectors in the same feature space. Therefore, cross-image clustering analysis can be performed directly based on the visual vectors of each detection region using clustering algorithms.

[0116] Cross-image clustering analysis groups visually similar detected regions into the same cluster, with each cluster representing a potential, co-occurring visual pattern across images. A schematic diagram of cross-image clustering analysis is shown below. Figure 5 As shown. Figure 5 In this context, the candidate region is the detection region.

[0117] In this embodiment, by inputting each detection region and each training image into the visual language model, the visual vector of each detection region is generated by the image encoder in the visual language model. Based on the visual vector of each detection region, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, thereby achieving the classification of the detection regions.

[0118] In one embodiment, the process of obtaining the average visual embedding of each cluster in step S2 includes:

[0119] Outlier and noise detection regions are removed from each cluster to obtain the core cluster of each cluster;

[0120] The average visual embedding of each cluster is calculated based on the visual vector of each detection region in each core cluster.

[0121] The formula for calculating the average visual embedding of each cluster is as follows: ; For cluster c jAverage visual embedding, For cluster c j core cluster The visual vector of the detection region r in the image.

[0122] Outlier detection regions are those within a cluster whose distance from other detection regions exceeds a distance threshold. Noise detection regions are those within a cluster whose confidence level is below a threshold. Outlier and noise detection regions within each cluster can be removed using clustering purification algorithms. For example, HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) can be used to remove outlier and noise detection regions from each cluster. HDBSCAN is an automatic clustering algorithm for filtering outlier and noise samples; it only requires setting the minimum cluster size, minimum number of samples, and merging threshold for automatic calculation.

[0123] In this embodiment, by removing outlier detection regions and noise detection regions from each cluster, the core clusters of each cluster are obtained. Based on the visual vectors of each detection region in each core cluster, the average visual embedding of each cluster is calculated. This avoids the interference of outlier detection regions and noise detection regions on the average visual embedding, thereby obtaining an accurate average visual embedding.

[0124] In one embodiment, a part detector is used to output part detection boxes for each part in the image under test, including:

[0125] Based on the detection regions of each type of component label in multiple labeled image sets, binary component detectors corresponding to each type of component label are trained to obtain each trained binary component detector. The component detection boxes of each component in the image to be tested are then output using each trained binary component detector.

[0126] Alternatively, based on the detection regions of each type of component label in multiple labeled image sets, a multi-class component detector is trained to obtain a trained multi-class component detector; the trained multi-class component detector is then used to output the component detection boxes of each component in the image to be tested.

[0127] The binary and multi-class part detectors are trained in a fully supervised mode. The trained binary and multi-class part detectors can output the bounding boxes and confidence scores of each part in the image under test.

[0128] In this embodiment, binary component detectors corresponding to each type of component label are trained based on the detection regions of each type of component label in multiple labeled image sets to obtain trained binary component detectors, or multi-class component detectors are trained based on the detection regions of each type of component label in multiple labeled image sets to obtain trained multi-class component detectors. In this way, the component detection boxes of each component in the image to be tested can be obtained through binary component detectors or multi-class component detectors.

[0129] In one embodiment, instance reconstruction is performed based on the detection bounding boxes of each component to obtain the detection bounding boxes of each instance to be tested in the image to be tested, including:

[0130] Based on the edge weights between the detection boxes of each component, the first component detection box belonging to the same test instance is determined, and the detection box of each test instance is reconstructed based on the first component detection box corresponding to each test instance.

[0131] The formula for calculating the edge weight is as follows: ;P ij For component B i The component detection frame and component B j The edge weights between the component detection boxes For component B i The component detection frame and component B j The crossover ratio between component detection frames, L total It is component B i The center point of the component detection frame and component B j The length of the line connecting the center points of the component detection frame. The connection is located in component B. i The length of the component detection frame, The connection is located in component B. j The length of the component detection frame.

[0132] Edge weights are used to represent the distance, similarity, or other metrics between two component detection boxes. The first component detection box is a component detection box that belongs to the same instance to be tested.

[0133] When the edge weight between two component detection boxes is less than or equal to a preset third threshold, it is determined that the two component detection boxes do not belong to the same test instance.

[0134] Each component detection box can be considered as a node in a graph structure. If multiple component detection boxes belong to the same instance under test, then connect the component detection boxes belonging to the same instance under test. The resulting graph structure diagram is shown below. Figure 6 As shown in the figure. A schematic diagram of the detection box of the test instance obtained by edge weight reconstruction is shown in the figure. Figure 7As shown, Object1 is instance 1 to be tested, and Object2 is instance 2 to be tested.

[0135] In this embodiment, by using Calculate the edge weights between the component detection boxes. This allows for a more accurate determination of the edge weights, taking into account the intersection-union ratio.

[0136] In one embodiment, a first component detection box belonging to the same test instance is determined, and a detection box for each test instance is reconstructed based on the first component detection boxes corresponding to each test instance, including:

[0137] When the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection box corresponding to the same test instance.

[0138] Determine the maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate of the first component detection frame for each instance to be tested;

[0139] Based on the maximum and minimum x-coordinates, maximum and minimum y-coordinates of each test instance, the detection box for each test instance is reconstructed.

[0140] Each test instance corresponds to a maximum x-coordinate, a minimum x-coordinate, a maximum y-coordinate, and a minimum y-coordinate. These coordinates constitute the corner points of the test instance's bounding box. Specifically, the coordinates of the four corner points of the test instance's bounding box are (maximum x-coordinate, maximum y-coordinate), (maximum x-coordinate, minimum y-coordinate), (minimum x-coordinate, maximum y-coordinate), and (minimum x-coordinate, minimum y-coordinate), respectively.

[0141] In this embodiment, when the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection boxes corresponding to the same test instance. This facilitates the combination of the first component detection boxes corresponding to the same test instance to obtain the detection box of the test instance. Furthermore, since this application includes not only the final detection box of the test instance but also the component detection boxes of each component constituting the test instance, the detection process has stronger interpretability and is more robust to occlusion or atypical poses.

[0142] In one embodiment, the method further includes:

[0143] The component detector outputs the component detection boxes of each component in the image under test, and the pre-trained basic weakly supervised model outputs multiple anchor points of the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different in each labeled image set.

[0144] Centered on each anchor point, the component detection boxes within a preset radius of the anchor point are determined as the component detection boxes corresponding to each instance to be tested in the image to be tested. Based on the component detection boxes corresponding to each instance to be tested, the detection boxes of each instance to be tested are reconstructed.

[0145] The schematic diagram of the detection box for each instance to be tested, obtained through anchor point reconstruction, is shown below. Figure 8 As shown.

[0146] This embodiment can effectively recall component detection boxes that are spatially dispersed and difficult to reconstruct using edge weights.

[0147] In one embodiment, the method further includes:

[0148] The component detector outputs the component detection boxes of each component in the image under test, and the pre-trained basic weakly supervised model outputs multiple anchor points of the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different in each labeled image set.

[0149] With each anchor point as the center, the detection boxes of each component within the preset radius of the anchor point are determined as the second component detection boxes corresponding to the first test instance in the image to be tested. Based on the second component detection boxes corresponding to each first test instance, the detection boxes of each first test instance are reconstructed.

[0150] Based on the edge weights between the non-second component detection boxes, the third component detection box corresponding to the second test instance in the test image is determined from the non-second component detection boxes, and the detection box of the second test instance is reconstructed based on the third component detection boxes corresponding to each second test instance.

[0151] Among them, the pre-trained basic weakly supervised models include, but are not limited to, OICR (Online Instance Classifier Refinement) model, WSDDN (Weakly Supervised Deep Detection Network) model, and PCL (Proposal Cluster Learning) model.

[0152] Non-second component detection boxes are component detection boxes other than the second component detection box in each component detection box.

[0153] The calculation method for the edge weights between non-second component detection boxes is the same as the calculation method for the edge weights between component detection boxes.

[0154] The first and second test instances are both test instances in the test image. Further, the output reconstructs the detection boxes of each of the first and second test instances. A schematic diagram of the combined output of the detection boxes of each of the first and second test instances is shown below. Figure 9 As shown. The final detection result is the combination of the detection bounding box of the first test instance (graph-based component aggregation result) and the detection bounding box of the second test instance (anchor-based component aggregation result). The original image is the test image, and the component detection result is the detection result of the component detection bounding box of each component in the test image.

[0155] In this embodiment, a component detector outputs the component detection boxes for each component in the image under test, and a pre-trained weakly supervised model outputs multiple anchor points for the image under test. Using each anchor point as the center, component detection boxes within a preset radius of the anchor point are identified as the second component detection boxes corresponding to the first test instance in the image under test. Based on the second component detection boxes corresponding to each first test instance, the detection boxes for each first test instance are reconstructed. This effectively recalls spatially dispersed component detection boxes that are difficult to reconstruct using edge weights. By determining the third component detection boxes corresponding to the second test instance in the image under test from the non-second component detection boxes according to the edge weights between them, and reconstructing the detection boxes for each second test instance based on their respective third component detection boxes, instance reconstruction operations can be performed on component detection boxes not covered by any anchor point, ensuring that component detection boxes at the edges are not missed.

[0156] In one embodiment, nonmaximum suppression is used to remove duplicate detection boxes, resulting in the coordinates of the final detection boxes and the category of the test instance within each detection box.

[0157] This application also provides an application scenario in which the aforementioned weakly supervised target detection method based on component mining and overall reconstruction is applied. Specifically, the application of the weakly supervised target detection method based on component mining and overall reconstruction in this scenario is as follows:

[0158] like Figure 10As shown, the server determines image-level category labels and prompt words for mining components of the image-level category labels, where the image-level category labels are text information; the image-level category labels and the prompt words are input into a language model, and candidate components corresponding to the image-level category labels are output; based on the first text embedding of the candidate component and the second text embedding of the parent node of the candidate component, a first cosine distance between the candidate component and the parent node is calculated; when the first cosine distance is less than a second threshold, the candidate component is eliminated; when the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, either the candidate component or the sibling node is eliminated, thus obtaining the target components for each image-level category label.

[0159] The server acquires multiple training images and extracts multiple detection regions from them. Each detection region and each training image are input into a visual language model, and the image encoder in the visual language model generates visual vectors for each detection region. Based on the visual vectors of each detection region, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, each cluster including at least one detection region. The server removes outlier and noisy detection regions from each cluster to obtain the core clusters. Based on the visual vectors of each detection region in each core cluster, the average visual embedding of each cluster is calculated. The distance between the average visual embedding of each cluster and the text embedding of each target component is calculated, and a minimum distance corresponding to each average visual embedding is determined. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is identified as the component label of each detection region in the cluster corresponding to the minimum distance. The component labels of the detection regions are mapped to the training images to obtain a labeled image set with component labels.

[0160] The server trains binary component detectors corresponding to each type of component label based on the detection regions of each type of component label in multiple labeled image sets, obtaining trained binary component detectors, and outputs component detection boxes for each component in the image to be tested using the trained binary component detectors; or, based on the detection regions of each type of component label in multiple labeled image sets, it trains multi-class component detectors, obtaining trained multi-class component detectors, and outputs component detection boxes for each component in the image to be tested using the trained multi-class component detectors.

[0161] The server uses a binary or multi-class component detector to output component detection boxes for each component in the image under test. Based on the edge weights between the component detection boxes, when the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection boxes corresponding to the same instance under test. The maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate of the first component detection box corresponding to each instance under test are determined respectively. Based on the maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate of each instance under test, the detection box of each instance under test is reconstructed.

[0162] Alternatively, the server uses a component detector to output component detection boxes for each component in the image under test, and uses a pre-trained weakly supervised model to output multiple anchor points for the image under test. The component detector is trained from multiple labeled image sets, and the instances to which the component labels belong in each labeled image set are different. Centered on each anchor point, each component detection box within a preset radius of the anchor point is determined as a second component detection box corresponding to a first instance under test in the image under test. Based on the second component detection boxes corresponding to each first instance under test, the detection boxes of each first instance under test are reconstructed. According to the edge weights between non-second component detection boxes, a third component detection box corresponding to a second instance under test in the image under test is determined from the non-second component detection boxes. Based on the third component detection boxes corresponding to each second instance under test, the detection boxes of the second instances under test are reconstructed.

[0163] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0164] Based on the same inventive concept, this application also provides a weakly supervised target detection system based on component mining and overall reconstruction for implementing the aforementioned weakly supervised target detection method based on component mining and overall reconstruction. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the weakly supervised target detection system based on component mining and overall reconstruction provided below can be found in the limitations of the weakly supervised target detection method based on component mining and overall reconstruction described above, and will not be repeated here.

[0165] In one embodiment, such as Figure 11 As shown, a weakly supervised target detection system based on component mining and overall reconstruction is provided, including:

[0166] The mining module is used to mine multiple target parts with image-level category labels and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters, wherein the category labels of each instance in the multiple training images are consistent with the image-level category labels.

[0167] An alignment module is used to calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and to determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is determined as the component label of each detection region in the cluster corresponding to the minimum distance.

[0168] A mapping module is used to map the component labels of the detected region to the training image to obtain a set of labeled images with component labels;

[0169] The detection module is used to output component detection boxes for each component in the image under test using a component detector, and to perform instance reconstruction based on each component detection box to obtain the detection box for each instance under test in the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set.

[0170] The modules in the aforementioned weakly supervised target detection system based on component mining and overall reconstruction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0171] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores various types of data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a weakly supervised target detection method based on component mining and overall reconstruction.

[0172] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0174] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0175] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0177] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0178] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A weakly supervised target detection method based on component mining and overall reconstruction, characterized in that, The method includes: S1. Mine multiple target components of image-level category labels, and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters, wherein the category label of each instance in the multiple training images is consistent with the image-level category label; S2. Calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, determine the target component corresponding to the minimum distance as the component label of each detection region in the cluster corresponding to the minimum distance. S3. Map the component labels of the detected region to the training image to obtain a set of labeled images with component labels; S4. Use a component detector to output the component detection boxes of each component in the image to be tested, and perform instance reconstruction based on each component detection box to obtain the detection boxes of each instance to be tested in the image to be tested. The component detector is trained from multiple labeled image sets, and the instances to which the component labels belong in each labeled image set are different. The method for excavating the target component in step S1 includes: Determine image-level category labels and cue words for components used to mine the image-level category labels, wherein the image-level category labels are text information; The image-level category labels and the prompt words are input into the language model, and the candidate components corresponding to the image-level category labels are output. Based on the first text embedding of the candidate component and the second text embedding of the parent node of the candidate component, calculate the first cosine distance between the candidate component and the parent node; When the first cosine distance is less than the second threshold, the candidate component is eliminated; when the first cosine distance is greater than the second threshold and the second cosine distance between the candidate component and its sibling node is less than the second threshold, the candidate component or its sibling node is eliminated to obtain the target component of each image-level category label. The formula for calculating the first cosine distance is: d vis (i,p) represents the first cosine distance. The first text embedding for candidate component i. The second text embedding for the parent node p.

2. The method according to claim 1, characterized in that, The process of obtaining multiple clusters in step S1 includes: Acquire multiple training images and extract multiple detection regions from the multiple training images; Each of the detection regions and each of the training images are input into a visual language model, and the visual vectors of each of the detection regions are generated by the image encoder in the visual language model. Based on the visual vectors of each of the detection regions, a clustering algorithm is used to perform cross-image clustering analysis to obtain multiple clusters, each of which includes at least one of the detection regions.

3. The method according to claim 2, characterized in that, The process of obtaining the average visual embedding of each cluster in step S2 includes: By removing outlier and noise detection regions from each cluster, the core cluster of each cluster is obtained. Based on the visual vectors of the detection regions in each core cluster, the average visual embedding of each cluster is calculated. The formula for calculating the average visual embedding of each cluster is as follows: ; For cluster c j Average visual embedding, For cluster c j core cluster The visual vector of the detection region r in the image.

4. The method according to claim 1, characterized in that, The method of using a component detector to output component detection boxes for each component in the image under test includes: Based on the detection regions of each type of component label in multiple labeled image sets, binary component detectors corresponding to each type of component label are trained respectively to obtain each trained binary component detector. The component detection boxes of each component in the image to be tested are output using each trained binary component detector. Alternatively, based on the detection regions of each type of component label in multiple labeled image sets, a multi-class component detector is trained to obtain the trained multi-class component detector; the trained multi-class component detector is then used to output the component detection boxes of each component in the image to be tested.

5. The method according to claim 1, characterized in that, The step of reconstructing instances based on the detection boxes of each component to obtain the detection boxes of each instance to be tested in the image to be tested includes: Based on the edge weights between the detection boxes of each component, the first detection box belonging to the same test instance is determined, and based on the first detection box corresponding to each test instance, the detection box of each test instance is reconstructed. The formula for calculating the edge weight is as follows: ;P ij For component B i The component detection frame and component B j The edge weights between the component detection boxes For component B i The component detection frame and component B j The crossover ratio between component detection frames, L total It is component B i The center point of the component detection frame and component B j The length of the line connecting the center points of the component detection frame. The connection is located in component B. i The length of the component detection frame, The connection is located in component B. j The length of the component detection frame.

6. The method according to claim 5, characterized in that, The step of determining the first component detection box belonging to the same test instance, and reconstructing the detection box of each test instance based on the first component detection box corresponding to each test instance, includes: When the edge weight between two component detection boxes is greater than a preset third threshold, the two component detection boxes are determined to be the first component detection boxes corresponding to the same test instance. Determine the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of the detection frame of the first component corresponding to each of the test instances; Based on the maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate corresponding to each of the test instances, the detection box of each test instance is reconstructed.

7. The method according to claim 1, characterized in that, The method further includes: The component detector outputs the component detection boxes of each component in the image under test, and the pre-trained basic weakly supervised model outputs multiple anchor points of the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set. Centered on each anchor point, the detection boxes of each component within a preset radius of the anchor point are determined as the second component detection boxes corresponding to the first test instance in the image to be tested, and the detection boxes of each first test instance are reconstructed based on the second component detection boxes corresponding to each first test instance. Based on the edge weights between the non-second component detection boxes, the third component detection box corresponding to the second test instance in the test image is determined from the non-second component detection boxes, and the detection box of the second test instance is reconstructed based on the third component detection boxes corresponding to each of the second test instances.

8. A weakly supervised target detection system based on component mining and overall reconstruction, used to execute the method according to any one of claims 1-7, characterized in that, The system includes: The mining module is used to mine multiple target parts with image-level category labels and perform cross-image clustering on the detection regions extracted from multiple training images to obtain multiple clusters, wherein the category labels of each instance in the multiple training images are consistent with the image-level category labels. An alignment module is used to calculate the distance between the average visual embedding of each cluster and the text embedding of each target component, and to determine the minimum distance corresponding to each average visual embedding. When the minimum distance is less than a first threshold, the target component corresponding to the minimum distance is determined as the component label of each detection region in the cluster corresponding to the minimum distance. A mapping module is used to map the component labels of the detected region to the training image to obtain a set of labeled images with component labels; The detection module is used to output component detection boxes for each component in the image under test using a component detector, and to perform instance reconstruction based on each component detection box to obtain the detection box for each instance under test in the image under test. The component detector is trained from multiple labeled image sets, and the instance to which the component label belongs is different for each labeled image set.

Images