An open world object detection method based on evidence attribute modeling

Abstract

This invention discloses an open-world target detection method based on evidence attribute modeling, comprising: acquiring an image to be detected and dividing it into multiple candidate regions; calculating the matching degree based on a class-independent attribute space to obtain fine-grained attribute responses; converting the fine-grained attribute responses into a Dirichlet evidence distribution, and performing reliability assessment and weighting on the Dirichlet evidence distribution to obtain a disambiguated Dirichlet evidence distribution; based on the disambiguated Dirichlet evidence distribution, jointly calculating the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target in the candidate regions; if the probability of belonging to an unknown target is greater than a preset threshold, the corresponding candidate region is determined to be an unknown target. This invention, through class-independent attribute space matching combined with Dirichlet evidence distribution for uncertainty quantification and disambiguation, effectively overcomes semantic bias and overconfidence problems, significantly improving the accuracy and reliability of unknown target recognition in open-world scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an open-world object detection method based on evidence attribute modeling. Background Technology

[0002] Object detection is a core technology in computer vision, widely used in scenarios such as autonomous driving, intelligent monitoring, and robot navigation. Traditional object detection methods are typically trained and tested based on the closed-world assumption, meaning the model can only recognize predefined known categories in the training set. However, in real-world open-world environments, there are often many unknown categories of objects that were not encountered during the training phase. To enable the detection system to continuously learn and discover new objects, open-world object detection has emerged. This task not only requires the model to accurately detect and classify known objects, but also to effectively identify target regions belonging to unknown categories, so that they can be subsequently incorporated into a knowledge base for incremental learning.

[0003] Existing open-world object detection methods mainly face the following technical challenges: First, the bias problem caused by semantic traction, that is, relying on semantic anchors of known categories makes the model prone to misclassifying unknown targets as similar known categories or ignoring them directly; Second, the confusion between prediction confidence and uncertainty, the normalized probability output by traditional Softmax is prone to overconfident erroneous predictions, and it is difficult to distinguish between low-confidence known targets and high-confidence erroneous predictions; Third, the lack of mapping between fine-grained attributes and category evidence, existing methods lack an effective transformation mechanism from shared attribute responses to interpretable category evidence, and cannot accurately quantify the degree of prediction certainty.

[0004] Therefore, providing a mechanism that can eliminate semantic bias and establish a reasoning mechanism from attributes to categories through evidence-based deep learning theory, thereby enabling accurate discovery of unknown targets in open-world scenarios, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this invention is proposed to provide an open-world target detection method based on evidence attribute modeling that overcomes or at least partially solves the above problems. By constructing a class-independent attribute space and converting fine-grained attribute responses into a disambiguated Dirichlet evidence distribution, the method effectively overcomes semantic bias and overconfidence problems, and achieves accurate identification and high-reliability detection of unknown targets in complex open scenes.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides an open-world object detection method based on evidence attribute modeling, comprising: The image to be detected is acquired and divided into multiple candidate regions. The matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix is ​​calculated based on the class-independent attribute space to obtain the fine-grained attribute response. The fine-grained attribute response is converted into a Dirichlet evidence distribution using a learnable attribute-category evidence mapping matrix; the Dirichlet evidence distribution is then evaluated for reliability and weighted using a Dirichlet evidence disambiguation device to obtain the disambiguated Dirichlet evidence distribution. Based on the disambiguated Dirichlet evidence distribution, the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target are jointly calculated for each candidate region; if the probability of belonging to an unknown target is greater than a preset threshold, the corresponding candidate region is determined to be an unknown target.

[0008] Furthermore, before calculating the matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix based on the class-independent attribute space, the method further includes: constructing the class-independent attribute space, specifically including: Define a set of general visual attribute types, which includes shape, color, size, context, local structure, overall appearance, environment, material, behavior, and texture; Using a large language model, candidate attribute words covering the set of general visual attribute types are generated based on a preset class-independent prompt template; The candidate attribute words are instantiated into natural language descriptions and encoded by a text encoder to obtain class-independent attribute semantic vectors. All class-independent attribute semantic vectors are stacked to form a class-independent attribute semantic matrix.

[0009] Furthermore, the matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix is ​​calculated based on the class-independent attribute space to obtain fine-grained attribute responses; specifically, this includes: Get the Visual embedding of candidate regions Attribute semantic embedding in the class-independent attribute semantic matrix ;in, Indicates the first Semantic embedding corresponding to each attribute text description; calculate and The initial matching score is obtained by calculating the cosine similarity or dot product between the two pairs of samples. The initial matching score is normalized to obtain a fine-grained attribute response. ,in, Indicates the first The candidate region and the first The degree of matching described by each attribute Indicates the candidate region index. Represents a global index for the attribute description.

[0010] Furthermore, the first Visual embedding of candidate regions Feature extraction based on a visual coding network is expressed by the following formula:

[0011] in, Indicates the first Candidate regions, Indicates the first Visual embedding of candidate regions This represents a frozen visual encoding network. Indicates the embedded spatial dimension. This represents the L2 norm normalization operation; The attribute semantic embedding in the class-independent attribute semantic matrix Based on the text encoding network, the corresponding attribute semantics are extracted, which is expressed by the formula:

[0012] in, Indicates the first Each attribute text description, Indicates the first Semantic embedding corresponding to each attribute text description. This indicates a frozen text encoding network.

[0013] Furthermore, the step of converting the fine-grained attribute response into a Dirichlet evidence distribution using a learnable attribute-category evidence mapping matrix specifically includes: Introducing a learnable attribute-category evidence mapping matrix The fine-grained attribute response is input into the attribute-category evidence mapping matrix. Perform a linear transformation and output a non-negative evidence vector using the Softplus activation function; Calculate the parameters of the Dirichlet distribution based on the nonnegative evidence vector. ,in, Indicates the first The candidate region supports the first Non-negative evidence of known categories, denoted as the Dirichlet evidence parameter after incorporating non-information priors, with constant 1 representing non-information priors; Based on parameters Construct the Dirichlet evidence distribution and calculate the total strength of evidence. The probability of being predicted as a known category and uncertainty of evidence The overall strength of evidence It can be expressed by the formula:

[0014] The probability of being predicted as a known category It can be expressed by the formula:

[0015] The uncertainty of the evidence It can be expressed by the formula:

[0016] in, Indicates a known category index. This indicates the number of known categories.

[0017] Furthermore, in the process of transforming the learnable attribute-category evidence mapping matrix into a Dirichlet evidence distribution, a weighted fusion of evidence alignment loss and isotropic evidence regularization term is employed for joint optimization; including: An evidence alignment loss is constructed to align the Dirichlet evidence distribution with supervision information of known categories; the formula for calculating the evidence alignment loss is as follows:

[0018]

[0019] in, Indicates evidence alignment loss, Indicates the first Does the candidate region belong to the first...? One known category; An isotropic evidence regularization term is introduced to suppress the interpretation of background regions, ambiguous candidate regions, and potential unknown targets as known categories; the isotropic evidence regularization term is implemented by calculating the KL divergence, and the calculation formula is as follows:

[0020] in, This indicates the regularity term for isotropic evidence. express KL divergence, Indicates the distribution of Dirichlet evidence. This represents a uniform Dirichlet prior. This represents the Dirichlet parameter vector after being protected by true category evidence.

[0021] Furthermore, the reliability assessment and weighting of the Dirichlet evidence distribution using a Dirichlet evidence disambiguation device are performed to obtain the disambiguated Dirichlet evidence distribution; specifically, this includes: Based on the parameters of the Dirichlet evidence distribution, the expected category entropy is calculated, which is used to characterize the ambiguity of the evidence distribution; A reliability factor is calculated based on the expected category entropy, and the reliability factor is negatively correlated with the expected category entropy. The non-negative evidence vector is weighted and adjusted using the reliability factor to obtain a disambiguated evidence vector, and then converted into a disambiguated evidence distribution based on the disambiguated evidence vector.

[0022] Furthermore, the disambiguated Dirichlet evidence distribution includes: the total evidence strength after disambiguation, the known category prediction probability after disambiguation, and the evidence uncertainty after disambiguation.

[0023] Furthermore, the joint calculation of the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target for each candidate region specifically includes: The target evidence strength is obtained by multiplying the maximum response value in the fine-grained attribute response with the empirical foreground probability; wherein the empirical foreground probability is obtained by mapping the fine-grained attribute response through a multilayer perceptron. The degree of lack of evidence for the known class is obtained by multiplying the maximum predicted probability of the known class after disambiguation with the uncertainty of the evidence after disambiguation. The probability of belonging to an unknown target is obtained by multiplying the strength of the target evidence by the degree of lack of known class evidence. The target evidence strength is used to measure the confidence level of a candidate region as a foreground target, the lack of known class evidence is used to measure the degree to which a candidate region cannot be explained by known categories, and the probability of belonging to an unknown target is combined with a preset threshold to determine that the corresponding candidate region is an unknown target.

[0024] Furthermore, the empirical prospect probability is expressed by the formula:

[0025]

[0026] in, This represents the empirical prospect probability that measures whether the overall attribute response matches the prospect objective. Indicates the total number of attribute text descriptions. A calibration function representing the response to fine-grained properties. , and This represents the learnable calibration parameters. Represents the numerical stability constant; The formula for calculating the probability of belonging to an unknown target is:

[0027] in, Indicates the first The probability that a candidate region belongs to an unknown target. This represents the maximum response value in the fine-grained attribute response. This indicates uncertainty regarding the evidence after disambiguation. This represents the known category prediction after disambiguation.

[0028] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an open-world object detection method based on evidence attribute modeling, which has the following beneficial effects: First, this invention breaks away from the reliance of traditional methods on known category names or strongly related semantic anchors by constructing a class-independent attribute space. It utilizes a large language model to generate a semantic matrix covering common visual attributes such as shape, texture, and material, enabling the model to perform matching based on fine-grained visual features, rather than forcibly classifying unknown targets as known concepts. This "de-categorized" feature extraction method effectively alleviates cognitive bias caused by semantic entanglement, significantly improving the model's feature representation ability and recall rate when facing never-before-seen unknown targets.

[0029] Secondly, this invention introduces the Dirichlet evidence distribution to replace the traditional Softmax probability output, fundamentally solving the problem of confusion between prediction confidence and uncertainty. By mapping attribute responses to non-negative evidence vectors and calculating the total evidence strength and uncertainty, the model can quantify its confidence in the prediction results. Combined with the joint optimization of evidence alignment loss and isotropic regularization terms, it not only enhances the evidence parameters of the target category but also effectively suppresses overconfident predictions in the background and ambiguous regions. It can accurately distinguish between "low-confidence known targets" and "high-confidence erroneous predictions" through a preset threshold, thereby achieving reliable discovery of unknown targets.

[0030] Finally, this invention designs a learnable attribute-category evidence mapping matrix and a Dirichlet evidence disambiguator, establishing an interpretable reasoning mechanism from fuzzy shared attributes to concrete category evidence. This mechanism not only utilizes the cross-category sharing characteristics of attributes but also eliminates unreliable evidence interference through disambiguation processing, ensuring the robustness and accuracy of detection results in complex open scenarios and providing a high-quality data foundation for subsequent incremental learning. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0032] Figure 1 This is a flowchart of the open-world target detection method based on evidence attribute modeling provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the open-world target detection method based on evidence attribute modeling provided in this embodiment of the invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This invention discloses an open-world object detection method based on evidence attribute modeling, referring to... Figure 1 As shown, it includes: The image to be detected is acquired and divided into multiple candidate regions. The matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix is ​​calculated based on the class-independent attribute space to obtain the fine-grained attribute response. By using a learnable attribute-category evidence mapping matrix, fine-grained attribute responses are transformed into Dirichlet evidence distributions; the reliability of the Dirichlet evidence distributions is evaluated and weighted using a Dirichlet evidence disambiguation device to obtain the disambiguated Dirichlet evidence distribution. Based on the disambiguated Dirichlet evidence distribution, the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target are jointly calculated for each candidate region; if the probability of belonging to an unknown target is greater than a preset threshold, the corresponding candidate region is determined to be an unknown target.

[0035] In the actual road driving process of autonomous vehicles, in addition to common "known category" targets such as vehicles, pedestrians, and traffic signs, they often encounter "unknown targets" not included in the training dataset. For example: fallen irregularly shaped cargo, animals on the road, special road obstacles in construction sections, or irregularly shaped engineering vehicles.

[0036] Traditional detectors often suffer from semantic bias, forcibly classifying unknown objects into the most similar known categories (e.g., identifying fallen cargo as a "car" or a "roadblock"), or from overconfidence, assigning a high-probability misclassification. This leads to autonomous driving systems failing to make correct obstacle avoidance decisions, resulting in safety accidents. This embodiment uses evidence attribute modeling for open-world object detection, referring to... Figure 2 As shown, the implementation steps of this embodiment are described in detail below: Step S1: Image acquisition and candidate region segmentation.

[0037] This embodiment uses an onboard camera to acquire real-time road environment images, and a visual Transformer encoder to extract features from the images, obtaining multiple candidate regions and their predicted bounding boxes. For "known categories," this embodiment outputs the category discrimination result corresponding to the candidate regions through a category prediction head. The predicted bounding boxes of each candidate region are represented as follows: ,in, and These represent the x and y coordinates of the top-left corner of the bounding box in the image coordinate system, respectively. and These represent the width and height of the bounding box, respectively. This example assumes that a previously unseen "overturned shopping cart" has been captured on the road surface.

[0038] Step S2: Extraction of class-independent attribute features.

[0039] This embodiment does not directly predict whether the object is a "car" or a "person," but instead enters a "class-independent attribute space." Using predefined general visual attributes, it calculates the matching degree between the visual features of the candidate region and the attribute semantic matrix.

[0040] This embodiment first defines a set of general visual attribute types. Specifically, it is described through visual attributes, including attribute types such as shape, color, size, context, local structure, overall appearance, environment, material, behavior, and texture. For any attribute type... This embodiment uses class-independent prompts to input a large language model to generate candidate attribute words for that attribute type. The prompt in this embodiment can be expressed as: "I am building a visual attribute library for general object detection. Please list several attributes related to the attribute type..." Relevant and typical visual properties that can be used for object detection. This leads to the attribute type. The corresponding set of candidate attribute words:

[0041] in, Indicates the attribute type The generated candidate attribute word set Indicates the first r Each attribute word, This indicates the number of attribute terms generated for this attribute type. Because this attribute generation process depends only on the attribute type and not on any known category names, the resulting attribute terms can better cover the general target semantics in an open environment.

[0042] To convert discrete attribute terms into textual semantic representations that can be encoded by a vision-language model, this embodiment further designs attribute description templates. For each attribute combination... Its natural language description is defined as:

[0043] in, This represents the object-level attribute description instantiated from the attribute term. In this way, the generated attributes are no longer used merely as discrete labels, but are transformed into semantic expressions that can be understood by the vision-language model, providing a foundation for fine-grained matching between subsequent candidate regions and attributes.

[0044] Subsequently, all attribute text descriptions were input into a frozen text encoding network. This yields the corresponding attribute semantic vector. For ease of subsequent representation, this embodiment uniformly denotes all attribute text descriptions as follows: ,in, This represents a global index describing the attribute. Indicates the total number of attribute text descriptions. The semantic embedding representation corresponding to the textual description of each attribute is as follows:

[0045] in, Indicates the first Each attribute text description, Indicates the first Semantic embedding corresponding to each attribute text description. Indicates the embedded spatial dimension. This indicates a frozen text encoding network.

[0046] The final result of this embodiment is Attribute text description, all attribute semantic embedding Stacked as a class-independent attribute semantic matrix:

[0047] in, This represents a class-independent attribute semantic matrix jointly constructed by a large language model and a text encoding network.

[0048] For candidate regions of the input image, this embodiment utilizes a frozen visual coding network. Extract visual features of candidate regions. Let the first candidate region be the first one in the input image. The candidate regions are Then its normalized visual embedding representation is:

[0049] in, Indicates the first Candidate regions, Indicates the first Visual features of each candidate region This represents a frozen visual encoding network. This represents the L2 norm normalization operation.

[0050] The input image in this embodiment contains a total of If there are n candidate regions, then the visual embedding of all candidate regions can be represented as:

[0051] in, Represents the visual feature matrix of the candidate region. This indicates the number of candidate regions.

[0052] Through the above process, the class-independent attribute semantic matrix Visual feature matrix of candidate regions They are mapped into the same visual-linguistic embedding space. This embodiment identifies the candidate region with highly matching attributes, such as fine-grained attribute responses like "metallic texture," "mesh structure," "silver," and "wheels." Because these attributes are generic, this embodiment accurately describes its visual features even if "shopping cart" is not in the "known categories."

[0053] Step S3, evidence attribute modeling.

[0054] Class-independent attribute spaces provide a shared semantic foundation for open-world objectives, but relying solely on attribute activations cannot yield reliable category decisions. Unknown objectives may activate attributes shared by multiple known categories. If such attribute responses are directly compressed into closed-set category probabilities, the model is forced to choose a category from the known categories, leading to the overconfident absorption of unknown objectives into known categories. Therefore, this embodiment treats attribute responses as evidence, rather than ordinary classification scores, and organizes them into a Dirichlet evidence distribution, thereby simultaneously characterizing known category support, total evidence strength, and prediction uncertainty.

[0055] This embodiment utilizes a learnable attribute-category evidence mapping matrix to convert the attribute responses such as "metal" and "grid" into Dirichlet evidence distributions; and calculates the evidence strength of the candidate region belonging to a known category (such as car, truck, pedestrian). Because the attribute combination of "shopping cart" differs significantly from that of "car" or "pedestrian," the evidence strength calculated for the known category in this embodiment is low, while the evidence uncertainty is high.

[0056] For the Visual embedding of candidate regions and the Semantic embedding of attributes The normalized semantic matching function between the two is defined as follows:

[0057] in, Visual embedding of candidate regions With attribute semantic embedding The degree of matching between them For candidate region indexing, A global index for the attribute description.

[0058] No. The candidate region for the first The response described by each attribute is represented as follows:

[0059] in, Indicates the first The candidate region for the first The degree of matching described by each attribute This represents the Sigmoid activation function. This is a temperature coefficient used to adjust the smoothness of the attribute response distribution. A larger value indicates a lower temperature coefficient. This indicates that the candidate region more significantly activates the corresponding attribute semantics.

[0060] To transform attribute responses into category-level evidence, this embodiment introduces a learnable attribute-category evidence mapping matrix. ,in Indicates the total number of attribute text descriptions. This represents the number of known categories at the current stage. This matrix is ​​used to learn the evidence contribution relationships between different attributes and different known categories, where the matrix elements... Indicates the first The attribute description is for the first The evidence contribution weights for each known category. Based on this mapping matrix, the... The candidate region for the first Aggregated evidence for known categories is defined as:

[0061]

[0062] in, Indicates the first The candidate region supports the first Non-negative evidence of known categories; This is used to ensure that the evidence value is non-negative, so that it meets the modeling requirements for evidence accumulation.

[0063] For the For each candidate region, the evidence from all known categories together forms the category evidence vector:

[0064] in, Indicates the first Each candidate region corresponds to a category evidence vector, where each element of the vector represents the evidence strength for a known category. Unlike Softmax probability, the category evidence vector... It does not force a strict competitive relationship between categories, thus preserving the strength of evidence support for each known category for candidate regions. In other words, a candidate region can generate different degrees of evidence response to multiple known categories simultaneously, rather than being immediately compressed into a single closed-set category judgment.

[0065] Subsequently, this embodiment maps categorical evidence to Dirichlet distribution parameters:

[0066] in, This represents the Dirichlet evidence parameter after incorporating non-informational priors, with the constant 1 representing the non-informational prior. When a category lacks additional supporting evidence, its corresponding parameter remains at the basic prior level.

[0067] No. The total evidence strength of each candidate region is defined as:

[0068] in, Indicates the first The total evidence strength of each candidate region across all known categories. When near When this occurs, it indicates that the candidate region is mainly composed of basic priors and lacks effective evidence of known categories; when A larger value indicates that the candidate region has accumulated stronger evidence of known categories.

[0069] Based on the above evidence parameters, we obtain the first... The candidate region belongs to the first Probability of prediction for evidence of known categories:

[0070]

[0071] in, Indicates the first The candidate region was predicted as the [number]th [number]. The probability of a known category. This indicates the number of known categories corresponding to this probability; Indicates the first Uncertainty regarding the evidence for each candidate region This indicates that the uncertainty is derived from the strength of the evidence. A larger [indicator]... This indicates that the overall evidence for the candidate region is weak, making it difficult to form a reliable explanation for the known categories; smaller... This indicates that the candidate region has more sufficient evidence to support it from known categories.

[0072] To align the evidence distribution with known category supervision information, this embodiment constructs an evidence alignment loss:

[0073] in, Indicates the first Does the candidate region belong to the first...? There are several known categories. When a value belongs to a category... ;otherwise, This loss, on the one hand, strengthens the evidentiary parameters corresponding to the target category, and on the other hand, enhances the total evidentiary items. Suppress unconstrained evidence growth to prevent the model from simply increasing evidence across all categories to reduce training loss.

[0074] Open-world object detection includes not only known targets but also background regions, ambiguous candidate regions, and potentially unknown targets. These regions should not be overly confidently interpreted by the model as belonging to a known category. Therefore, this embodiment further introduces an isotropic evidence regularization term:

[0075] in, Denotes KL divergence, Indicates the Dirichlet distribution. This represents a uniform Dirichlet prior. Indicates the first The Dirichlet parameter vector of each candidate region after protection by true class evidence is expressed by the formula:

[0076] This vector consists of protected evidence parameters for each known category. Here, "protection of true category evidence" does not mean deleting a category during the prediction phase, but rather preventing true category evidence from being weakened by the KL regularization term when calculating the regularization term, so that regularization mainly acts on erroneous evidence in non-true categories.

[0077] For a sample with a known category, the target category is masked after the first... Each piece of evidence parameter is defined as:

[0078] in, Indicates the first The candidate region in the first Masked evidence parameters for each category. When At that time, this category is the true target category. Set as a non-information-based prerequisite ;when At that time, the category is not the target category. Reserved as This masking operation sets the parameters corresponding to the true target category to non-informational priors, while retaining the evidence parameters for non-target categories and regularizing them to near a uniform Dirichlet prior. This suppresses excessively strong non-target category evidence, preventing background regions, ambiguous candidate regions, or potential unknown targets from being incorrectly absorbed into known categories.

[0079] Ultimately, the training objective for evidence attribute modeling in this embodiment is:

[0080] in, This represents the weight coefficient of the isotropic evidence regularization term, used to control the regularization strength. Used to align known category supervision Used to suppress the excessive growth of evidence in non-target categories.

[0081] Through the modeling process described above, the attribute response is transformed into a Dirichlet evidence distribution. This distribution can provide predictions for known categories. It can also be assessed through the overall strength of evidence. and uncertainty of evidence We assess whether candidate regions have sufficient and reliable explanations of known categories, thereby providing an evidentiary basis for subsequent reasoning about unknown targets.

[0082] Step S4: Use the Dirichlet evidence disambiguation device to perform evidence disambiguation.

[0083] The Dirichlet evidence distribution obtained through evidence attribute modeling can simultaneously provide known class support and evidence uncertainty. However, the reliability of known class evidence depends not only on the strength of the evidence but also on whether it forms a clear class interpretation. For potential unknown targets, class-independent attribute responses may still activate multiple known class channels simultaneously through the attribute-class evidence mapping matrix, allowing the model to accumulate some known class evidence. However, this evidence is often scattered and difficult to constitute a reliable known class interpretation. Directly using this kind of diffuse evidence may overestimate the known class support of candidate regions, thereby weakening the ability to discover unknown targets.

[0084] To address this, this embodiment proposes a Dirichlet evidence disambiguation device to determine whether the evidence distribution in a candidate region forms a clear, known category interpretation. Unlike methods that rely solely on the maximum class probability or manually set class intervals, the Dirichlet evidence disambiguation device measures the ambiguity of the evidence distribution from the perspective of the class entropy of the Dirichlet evidence distribution. Given the first... The parameter vector of the Dirichlet evidence distribution for each candidate region The corresponding category probability variable is defined as:

[0085] in, This represents the known class probability vector sampled from the Dirichlet evidence distribution. When the evidence set supports a certain known class... It tends to form a sharp distribution; when there is competing evidence among multiple known categories. It then exhibits a more diffuse distribution.

[0086] Based on this, this embodiment will... The ambiguity of evidence for each candidate region is defined as the expected class entropy under the Dirichlet evidence distribution:

[0087] in, Indicates the first The ambiguity of evidence for each candidate region. Smaller. This indicates that the category probability distribution is more concentrated, suggesting that the candidate region has a clearer interpretation of the known category; a larger [value] This indicates that the evidence is scattered across multiple known categories, suggesting that the candidate region lacks reliable support from known classes.

[0088] Subsequently, the ambiguity of the evidence was converted into an evidence reliability factor:

[0089] in, Indicates the first The reliability factor of known class evidence for each candidate region. When the interpretation of known classes is relatively clear. Smaller Maintaining a larger value preserves reliable category evidence; when the interpretation of a known category is ambiguous. Increase This decreases accordingly, thereby suppressing unreliable known class support.

[0090] Based on this reliability factor, the disambiguated category-level evidence is defined as follows:

[0091] in, Indicates the first The candidate region for the first Disambiguated evidence for known categories, This represents the original category-level evidence before disambiguation. This operation adaptively adjusts the known class evidence based on the clarity of the evidence distribution: for candidate regions with clear known class interpretations, their category evidence is preserved; for candidate regions with diffuse evidence distribution, their known class support is weakened overall.

[0092] Disambiguated evidence The known class predictions are remapped back to the Dirichlet evidence space using the same parameterization method as the evidence attribute modeling module, resulting in the disambiguated predictions. and uncertainty of evidence Through the above process, DED can suppress diffuse known class support caused by multiple competing evidence channels, while retaining reliable evidence of real known targets, thus providing a more stable evidentiary basis for subsequent reasoning about unknown targets.

[0093] Step S5: Determine the unknown target.

[0094] Relying solely on category-level evidence distribution is insufficient for reliable unknown target identification. For open-world target detection, an unknown target is neither the region with the lowest known class probability nor should it be simply determined by the highest attribute response. A reliable unknown target should simultaneously satisfy two conditions: first, the candidate region exhibits a clear target attribute pattern; second, the region cannot be fully explained by the currently known categories. Based on this, this embodiment models unknown target reasoning as a joint decision-making process based on the degree of lack of target evidence and known class evidence.

[0095] For the For each candidate region, its maximum response in the class-independent attribute space is used to measure whether the region has activated a valid target attribute:

[0096] in, Indicates the first The candidate region for the first The response is described by each attribute. Indicates the first The candidate region for the first Each attribute describes the maximum response value in the fine-grained attribute response. Indicates the total number of attribute text descriptions. Larger... This indicates that the candidate region has a strong match with at least one basic attribute, and therefore may contain valid target semantics.

[0097] However, the maximum attribute response is easily affected by accidental activations from background texture or local noise. To characterize whether a candidate region has a stable overall foreground attribute pattern, this embodiment further introduces an empirical foreground probability. For any attribute response... Define the shared calibration function as follows:

[0098] in, , and For learnable calibration parameters, is the numerical stability constant. Based on this calibration function, the... The empirical prospect probability of a candidate region is defined as:

[0099] in, An empirical prospect probability that measures whether the overall attribute response matches the prospect objective. This is compared to relying solely on the maximum attribute response. This can reduce spurious predictions caused by isolated background responses.

[0100] Therefore, the first The target evidence for each candidate region is defined as:

[0101] in, This indicates the target evidence for the candidate region. By combining the maximum attribute response and overall foreground consistency, only candidate regions that simultaneously possess significant attribute matching and stable foreground patterns receive higher target evidence.

[0102] On the other hand, the unknown target should also lack a reliable known category explanation. After Dirichlet evidence disambiguation, this embodiment uses the disambiguated known class to predict the probability. and uncertainty of evidence Define the degree of lack of known class evidence:

[0103] in, This indicates the degree of lack of known class evidence for the candidate region. This represents the predicted probability of the known class after disambiguation. This indicates the number of known categories in the current stage. This indicates the uncertainty of evidence after disambiguation. If a candidate region has a high predicted probability of the known class and low uncertainty of evidence, it means that the region can be reliably explained by a certain known class. The smaller the value, the better; conversely, if the candidate region lacks clear known class support or the evidence is highly uncertain, then... Relatively large.

[0104] Finally, the first The probability that a candidate region belongs to an unknown target is defined as:

[0105] in, Indicates the first The probability that a candidate region belongs to an unknown target. Used to suppress accidental property activation in the background region. This is designed to reduce the likelihood of reliable known targets being misclassified as unknown targets. Therefore, known targets are suppressed due to reliable known class support, background regions are suppressed due to a lack of stable target evidence, while true unknown targets receive a higher unknown target probability when there is clear target evidence and a lack of reliable known class explanation.

[0106] This embodiment reconstructs attribute responses in open-world object detection from traditional classification signals into interpretable evidence representations. This embodiment defines unknown targets as a special type of candidate region: a region with clear target attribute evidence but lacking reliable known category interpretations. Based on this definition, this embodiment includes three main stages: attribute generation, evidence attribute modeling, and unknown target reasoning. First, in the attribute generation stage, a class-independent cues guide a large language model to generate general visual attributes, and a class-independent attribute semantic space is constructed using a frozen text encoder. This attribute space does not depend on specific known category names, thus providing a more open foundation for attribute descriptions of potential unknown targets. Second, in the evidence attribute modeling stage, the visual features of candidate regions in the input image are matched with class-independent attribute semantics to obtain fine-grained attribute responses. These responses are further mapped to category-level evidence and a Dirichlet evidence distribution, thereby simultaneously obtaining known category predictions, total evidence strength, and evidence uncertainty. To avoid uncertain evidence being over-interpreted as reliable known category support, this embodiment further proposes a Dirichlet evidence disambiguator to assess the reliability of known category interpretations in the evidence distribution, obtaining disambiguated known category predictions and evidence uncertainty. Finally, in the unknown target reasoning stage, the degree of lack of known class evidence is deduced based on the disambiguated evidence distribution, and this is combined with the target evidence of the candidate region to estimate the probability that the candidate region belongs to an unknown target. Through the above three stages, this embodiment can more reliably discover potential unknown targets with clear target attributes but lacking known category explanations while maintaining the ability to identify known categories.

[0107] Based on the same inventive concept, this invention also provides an open-world target detection system based on evidence attribute modeling. Since the principle of the problem solved by this system is similar to the aforementioned open-world target detection method based on evidence attribute modeling, the implementation of this system can refer to the implementation of the aforementioned method, and the repeated parts will not be described again.

[0108] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An open-world object detection method based on evidence attribute modeling, characterized in that, include: The image to be detected is acquired and divided into multiple candidate regions. The matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix is ​​calculated based on the class-independent attribute space to obtain the fine-grained attribute response. The fine-grained attribute response is converted into a Dirichlet evidence distribution using a learnable attribute-category evidence mapping matrix; the Dirichlet evidence distribution is then evaluated for reliability and weighted using a Dirichlet evidence disambiguation device to obtain the disambiguated Dirichlet evidence distribution. Based on the disambiguated Dirichlet evidence distribution, the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target are jointly calculated for each candidate region; if the probability of belonging to an unknown target is greater than a preset threshold, the corresponding candidate region is determined to be an unknown target.

2. The method as described in claim 1, characterized in that, Before calculating the matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix based on the class-independent attribute space, the method further includes: constructing the class-independent attribute space, specifically including: Define a set of general visual attribute types, which includes shape, color, size, context, local structure, overall appearance, environment, material, behavior, and texture; Using a large language model, candidate attribute words covering the set of general visual attribute types are generated based on a preset class-independent prompt template; The candidate attribute words are instantiated into natural language descriptions and encoded by a text encoder to obtain class-independent attribute semantic vectors. All class-independent attribute semantic vectors are stacked to form a class-independent attribute semantic matrix.

3. The method as described in claim 2, characterized in that, The process involves calculating the matching degree between the visual features of each candidate region and each attribute in the class-independent attribute semantic matrix based on the class-independent attribute space, thereby obtaining a fine-grained attribute response; specifically, this includes: Get the Visual embedding of candidate regions Attribute semantic embedding in the class-independent attribute semantic matrix ;in, Indicates the first Semantic embedding corresponding to each attribute text description; calculate and The initial matching score is obtained by calculating the cosine similarity or dot product between the two pairs of samples. The initial matching score is normalized to obtain a fine-grained attribute response. ,in, Indicates the first The candidate region and the first The degree of matching described by each attribute Indicates the candidate region index. Represents a global index for the attribute description.

4. The method as described in claim 3, characterized in that, The first Visual embedding of candidate regions Feature extraction based on a visual coding network is expressed by the following formula: in, Indicates the first Candidate regions, Indicates the first Visual embedding of candidate regions This represents a frozen visual encoding network. Indicates the embedded spatial dimension. This represents the L2 norm normalization operation; The attribute semantic embedding in the class-independent attribute semantic matrix Based on the text encoding network, the corresponding attribute semantics are extracted, which is expressed by the formula: in, Indicates the first Each attribute text description, Indicates the first Semantic embedding corresponding to each attribute text description. This indicates a frozen text encoding network.

5. The method as described in claim 3, characterized in that, The process of converting the fine-grained attribute response into a Dirichlet evidence distribution using a learnable attribute-category evidence mapping matrix specifically includes: Introducing a learnable attribute-category evidence mapping matrix The fine-grained attribute response is input into the attribute-category evidence mapping matrix. Perform a linear transformation and output a non-negative evidence vector using the Softplus activation function; Calculate the parameters of the Dirichlet distribution based on the nonnegative evidence vector. ,in, Indicates the first The candidate region supports the first Non-negative evidence of known categories, denoted as the Dirichlet evidence parameter after incorporating non-information priors, with constant 1 representing non-information priors; Based on parameters Construct the Dirichlet evidence distribution and calculate the total strength of evidence. The probability of being predicted as a known category and uncertainty of evidence The overall strength of evidence It can be expressed by the formula: The probability of being predicted as a known category It can be expressed by the formula: The uncertainty of the evidence It can be expressed by the formula: in, Indicates a known category index. This indicates the number of known categories.

6. The method as described in claim 5, characterized in that, In the process of transforming a learnable attribute-category evidence mapping matrix into a Dirichlet evidence distribution, a weighted fusion of evidence alignment loss and isotropic evidence regularization term is employed for joint optimization; including: An evidence alignment loss is constructed to align the Dirichlet evidence distribution with supervision information of known categories; the formula for calculating the evidence alignment loss is as follows: in, Indicates evidence alignment loss, Indicates the first Does the candidate region belong to the first...? One known category; An isotropic evidence regularization term is introduced to suppress the interpretation of background regions, ambiguous candidate regions, and potential unknown targets as known categories; the isotropic evidence regularization term is implemented by calculating the KL divergence, and the calculation formula is as follows: in, This indicates the regularity term for isotropic evidence. express KL divergence, Indicates the distribution of Dirichlet evidence. This represents a uniform Dirichlet prior. This represents the Dirichlet parameter vector after being protected by true category evidence.

7. The method as described in claim 5, characterized in that, The process of performing reliability assessment and weighting of the Dirichlet evidence distribution using a Dirichlet evidence disambiguation device to obtain the disambiguated Dirichlet evidence distribution specifically includes: Based on the parameters of the Dirichlet evidence distribution, the expected category entropy is calculated, which is used to characterize the ambiguity of the evidence distribution; A reliability factor is calculated based on the expected category entropy, and the reliability factor is negatively correlated with the expected category entropy. The non-negative evidence vector is weighted and adjusted using the reliability factor to obtain a disambiguated evidence vector, and then converted into a disambiguated evidence distribution based on the disambiguated evidence vector.

8. The method as described in claim 7, characterized in that, The disambiguated Dirichlet evidence distribution includes: the total evidence strength after disambiguation, the known class prediction probability after disambiguation, and the evidence uncertainty after disambiguation.

9. The method as described in claim 8, characterized in that, The joint calculation of the target evidence strength, the degree of lack of known class evidence, and the probability of belonging to an unknown target for each candidate region specifically includes: The target evidence strength is obtained by multiplying the maximum response value in the fine-grained attribute response with the empirical foreground probability; wherein the empirical foreground probability is obtained by mapping the fine-grained attribute response through a multilayer perceptron. The degree of lack of evidence for the known class is obtained by multiplying the maximum predicted probability of the known class after disambiguation with the uncertainty of the evidence after disambiguation. The probability of belonging to an unknown target is obtained by multiplying the strength of the target evidence by the degree of lack of known class evidence. The target evidence strength is used to measure the confidence level of a candidate region as a foreground target, the lack of known class evidence is used to measure the degree to which a candidate region cannot be explained by known categories, and the probability of belonging to an unknown target is combined with a preset threshold to determine that the corresponding candidate region is an unknown target.

10. The method as described in claim 9, characterized in that, The empirical prospect probability is expressed by the formula: in, This represents the empirical prospect probability that measures whether the overall attribute response matches the prospect objective. Indicates the total number of attribute text descriptions. A calibration function representing the response to fine-grained properties. , and This represents the learnable calibration parameters. Represents the numerical stability constant; The formula for calculating the probability of belonging to an unknown target is: in, Indicates the first The probability that a candidate region belongs to an unknown target. This represents the maximum response value in the fine-grained attribute response. This indicates uncertainty regarding the evidence after disambiguation. This represents the known category prediction after disambiguation.

Images