A target detection method, device and equipment

Abstract

The application provides a target detection method, device and equipment, the method comprises the following steps: determining a target entity based on a prompt word text, determining equivalent entities and false positive entities corresponding to the target entity; obtaining a first detection box corresponding to the target entity, a second detection box corresponding to the equivalent entities and a third detection box corresponding to the false positive entities from a to-be-detected image; determining a target detection box based on the overlapping degree between the first detection box, the second detection box and the third detection box; cutting a detection sub-image of the target detection box from the to-be-detected image, determining target attribute information based on the detection sub-image, and outputting the target attribute information and the target detection box. Through the technical scheme of the application, false detection boxes can be effectively eliminated, and the modeling capability of semantic consistency can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a target detection method, apparatus and equipment. Background Technology

[0002] Object detection refers to locating and classifying all entity objects (i.e., targets of interest) in an image and outputting the coordinates of the detection boxes and their class labels. The detection boxes are also called bounding boxes. For example, the output can be the coordinates of the detection boxes, such as (x1, y1) and (x2, y2), where (x1, y1) represents the top-left corner coordinates and (x2, y2) represents the bottom-right corner coordinates, or (x, y, w, h), where (x, y) represents the center point coordinates, w represents the width, and h represents the height. Class labels can also be output, such as "car" or "pedestrian." Furthermore, the confidence score can be output, representing the probability of the entity object's presence.

[0003] To perform object detection, the image to be detected is input into an object detection model, which processes the image and outputs the bounding box coordinates and category labels of the entities. However, object detection models can only detect entities with fixed category labels; they cannot detect entities with arbitrary category labels. For example, an object detection model supports detecting entities labeled "car" or "pedestrian," but not entities labeled "sedan" or "human body." Summary of the Invention

[0004] This application provides a target detection method, the method comprising: determining a target entity based on acquired prompt text, determining an equivalent entity and a false positive entity corresponding to the target entity; wherein the equivalent entity and the target entity correspond to the same entity object, and the false positive entity corresponds to the entity object that falsely reports the target entity; acquiring a first detection box corresponding to the target entity, a second detection box corresponding to the equivalent entity, and a third detection box corresponding to the false positive entity from an acquired image to be detected; determining a target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box; extracting a detection sub-image of the target detection box from the image to be detected, determining target attribute information based on the detection sub-image, and outputting the target attribute information and the target detection box.

[0005] This application provides a target detection device, the device comprising: a determining module, configured to determine a target entity based on acquired prompt text, and determine an equivalent entity and a false alarm entity corresponding to the target entity; wherein the equivalent entity and the target entity correspond to the same entity object, and the false alarm entity corresponds to an entity object that falsely reports the target entity; an acquiring module, configured to acquire a first detection box corresponding to the target entity, a second detection box corresponding to the equivalent entity, and a third detection box corresponding to the false alarm entity from an image to be detected; a determining module, configured to determine a target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box; and a processing module, configured to extract a detection sub-image of the target detection box from the image to be detected, determine target attribute information based on the detection sub-image, and output the target attribute information and the target detection box.

[0006] This application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the target detection method of the example above.

[0007] This application provides a computer program product, which may include a computer program that, when executed by a processor, implements the target detection method of the above example.

[0008] This application provides a machine-readable storage medium storing machine-executable instructions that can be executed by a processor; wherein the processor is configured to execute the machine-executable instructions to implement the target detection method of the example described above when the machine-executable instructions are executed.

[0009] As can be seen from the above technical solutions, in the embodiments of this application, the equivalent entity corresponding to the target entity can be determined. The equivalent entity and the target entity belong to the same entity object. The target detection box is determined based on the degree of overlap between the first detection box of the target entity and the second detection box of the equivalent entity. Thus, target recognition is performed based on the target entity and the equivalent entity. The category of the equivalent entity can be unified with the category of the target entity. Synonymous semantics are integrated to supplement the category, which improves the modeling ability of semantic consistency and solves the problem of the decay of the recognition ability of unknown category attributes. It is especially suitable for entity recognition of unknown category targets in dynamic open environments and joint determination of their multi-dimensional attributes (such as color, action, and spatial position), ensuring cross-modal and synonymous semantic performance. Furthermore, the system identifies the false alarm entities corresponding to the target entity. The false alarm entities are the entity objects that falsely report the target entity. The target detection box is determined based on the degree of overlap between the first detection box of the target entity and the third detection box of the false alarm entity. Target recognition is then performed based on the target entity and the false alarm entities, effectively eliminating false alarm detection boxes, effectively reducing the false detection rate of synonym prompts, effectively reducing the probability of entity recognition errors, significantly reducing the false alarm problem of visually similar objects, and improving detection accuracy and robustness. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a target detection method in one embodiment of this application;

[0011] Figure 2 This is a flowchart illustrating a target detection method in one embodiment of this application;

[0012] Figure 3 This is a schematic diagram of the target detection framework in one embodiment of this application;

[0013] Figure 4 This is a schematic diagram of the target detection process in one embodiment of this application;

[0014] Figure 5 This is a flowchart illustrating automated threshold adjustment in one embodiment of this application;

[0015] Figure 6 This is a schematic diagram of the target detection device in one embodiment of this application;

[0016] Figure 7 This is a hardware structure diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0017] This application proposes a target detection method, which can be applied to electronic devices. See [link to relevant documentation]. Figure 1 The diagram shown is a flowchart of the target detection method, which may include:

[0018] Step 101: Determine the target entity based on the obtained prompt text, and determine the equivalent entity and false alarm entity corresponding to the target entity; the equivalent entity and the target entity correspond to the same entity object (i.e. different expressions for the same entity object), and the false alarm entity corresponds to the entity object that generates a false alarm for the target entity.

[0019] Step 102: Obtain the first detection box corresponding to the target entity, the second detection box corresponding to the equivalent entity, and the third detection box corresponding to the false alarm entity from the acquired image to be detected.

[0020] Step 103: Determine the target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box.

[0021] For example, candidate detection boxes can be determined based on the degree of overlap between a first detection box and a second detection box; wherein, a candidate detection box includes the first detection box, or, a candidate detection box includes the second detection box, or, a candidate detection box includes both the first and second detection boxes. A target detection box is determined based on the degree of overlap between a candidate detection box and a third detection box; wherein, the target detection box includes the remaining candidate detection boxes filtered based on this degree of overlap. That is, for each candidate detection box, it can be used as the target detection box, or the candidate detection box can be filtered and not used as the target detection box.

[0022] Step 104: Extract the detection sub-image of the target detection box from the image to be detected, determine the target attribute information based on the detection sub-image, and output the target attribute information and the target detection box. The target attribute information can represent the attribute of the target detection box, and the target attribute information and the target detection box represent the target recognition result.

[0023] For example, determining a target detection box based on the degree of overlap between a first detection box, a second detection box, and a third detection box may include, but is not limited to: determining a first degree of overlap between the first detection box and the second detection box; if the first degree of overlap is less than a first threshold, then the first detection box and the second detection box can be determined as candidate detection boxes; if the first degree of overlap is not less than the first threshold and the confidence level of the first detection box is not less than the confidence level of the second detection box, then the first detection box can be determined as a candidate detection box; if the first degree of overlap is not less than the first threshold and the confidence level of the first detection box is less than the confidence level of the second detection box, then the second detection box can be determined as a candidate detection box. A second degree of overlap between the candidate detection box and the third detection box is determined, and a false positive score is obtained by weighting the second degree of overlap and the confidence level of the third detection box; if the false positive score is not greater than a second threshold, then the candidate detection box is determined as a target detection box; if the false positive score is greater than the second threshold, then determining the candidate detection box as a target detection box is prohibited.

[0024] For example, before determining the equivalent entity and false positive entity corresponding to the target entity, multiple sample texts can be clustered to obtain multiple clusters. For each cluster, the cluster can include at least one sample text belonging to the same entity object. A dynamic semantic enhancement vocabulary is constructed, which includes multiple first vocabulary entries corresponding to the multiple clusters. For each first vocabulary entry, the first vocabulary entry can include the entity corresponding to each sample text within the cluster. Furthermore, multiple sample images can be identified to obtain the predicted entity of the entity object in each sample image. A false positive suppression vocabulary is constructed, which can include multiple second vocabulary entries corresponding to multiple entity objects. For each second vocabulary entry, the second vocabulary entry can include the correspondence between the real entity of the entity object and the predicted entity of the entity object.

[0025] Based on this, determining the equivalent entity and false positive entity corresponding to the target entity may include, but is not limited to: querying the first term entry corresponding to the target entity from the dynamic semantic enhancement lexicon, and determining the entities within the first term entry as equivalent entities; wherein, at least one entity within the first term entry has a similarity greater than a third threshold with the target entity. Furthermore, querying the second term entry corresponding to the target entity from the false positive suppression lexicon, and determining the predicted entities within the second term entry as false positive entities; wherein, the similarity between the real entities within the second term entry and the target entity may be greater than a fourth threshold.

[0026] For example, determining target attribute information based on a detected sub-image may include, but is not limited to: extracting features from the detected sub-image to obtain first image features; extracting features from the target attribute to obtain first text features, where the target attribute may be determined based on prompt text. A first cross-modal similarity is determined based on the first image features and the first text features; it is determined whether there is a positive sample image corresponding to the target attribute; if so, a first similarity threshold corresponding to the target attribute can be determined based on the positive sample image corresponding to the target attribute; if not, the configured default similarity threshold can be determined as the first similarity threshold corresponding to the target attribute; if the first cross-modal similarity is greater than the first similarity threshold, the target attribute is determined as the target attribute information corresponding to the target detection box.

[0027] For example, determining the first similarity threshold corresponding to the target attribute based on positive sample images may include, but is not limited to: amplifying the positive sample images corresponding to the target attribute to obtain multiple amplified positive sample images; amplifying the negative sample images corresponding to the target attribute to obtain multiple amplified negative sample images; wherein, the entity objects in each positive sample image correspond to the target attribute, and the entity objects in each negative sample image do not correspond to the target attribute. Based on this, the cross-modal similarity of positive samples is determined based on the image features and the first text features of the positive sample images, and the cross-modal similarity of negative samples is determined based on the image features and the first text features of the negative sample images; the first similarity threshold is determined based on the cross-modal similarity of positive samples (multiple positive sample cross-modal similarities) and the cross-modal similarity of negative samples (multiple negative sample cross-modal similarities), and the first similarity threshold is used to distinguish between the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, so that the cross-modal similarity of positive samples corresponding to all positive sample images is greater than the first similarity threshold, and the cross-modal similarity of negative samples corresponding to all negative sample images is less than the first similarity threshold.

[0028] For example, determining the first similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples may include, but is not limited to: determining the current distribution similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples; if there is no historical distribution similarity threshold, the current distribution similarity threshold can be determined as the first similarity threshold, and the first similarity threshold can be used as the historical distribution similarity threshold for the next period; if there is a historical distribution similarity threshold, the divergence value can be determined based on the current distribution similarity threshold and the historical distribution similarity threshold; if the divergence value is greater than the fifth threshold, the current distribution similarity threshold can be determined as the first similarity threshold; if the divergence value is not greater than the fifth threshold, the historical distribution similarity threshold used in the previous period can be determined as the first similarity threshold.

[0029] For example, the divergence value can be determined using the following formula: ;in, This represents the current distribution similarity threshold. Let S represent the similarity threshold of the s-th historical distribution, and let S represent the similarity thresholds of all historical distributions. This represents the divergence value.

[0030] For example, obtaining the first detection box corresponding to the target entity from the acquired image to be detected may include, but is not limited to: for any candidate box in the image to be detected, performing feature extraction on the candidate box to obtain a second image feature, performing feature extraction on the target entity to obtain a second text feature, determining a second cross-modal similarity based on the second image feature and the second text feature; determining whether there is a positive sample image corresponding to the target entity; if so, determining a second similarity threshold corresponding to the target entity based on the positive sample image; if not, determining the configured default similarity threshold as the second similarity threshold corresponding to the target entity; if the second cross-modal similarity is greater than the second similarity threshold, determining the candidate box as the first detection box corresponding to the target entity; if the second cross-modal similarity is not greater than the second similarity threshold, prohibiting the candidate box from being determined as the first detection box corresponding to the target entity.

[0031] For example, determining the second similarity threshold corresponding to the target entity based on positive sample images may include, but is not limited to: amplifying the positive sample images corresponding to the target entity to obtain multiple positive sample images, amplifying the negative sample images corresponding to the target entity to obtain multiple negative sample images; determining the cross-modal similarity of positive samples based on the image features and second text features of the positive sample images, determining the cross-modal similarity of negative samples based on the image features and second text features of the negative sample images; and determining the second similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples.

[0032] As can be seen from the above technical solutions, in the embodiments of this application, the equivalent entity corresponding to the target entity can be determined. The equivalent entity and the target entity belong to the same entity object. The target detection box is determined based on the degree of overlap between the first detection box of the target entity and the second detection box of the equivalent entity. Thus, target recognition is performed based on the target entity and the equivalent entity. The category of the equivalent entity can be unified with the category of the target entity. Synonymous semantics are integrated to supplement the category, which improves the modeling ability of semantic consistency and solves the problem of the decay of the recognition ability of unknown category attributes. It is especially suitable for entity recognition of unknown category targets in dynamic open environments and joint determination of their multi-dimensional attributes (such as color, action, and spatial position), ensuring cross-modal and synonymous semantic performance. Furthermore, the system identifies the false alarm entities corresponding to the target entity. The false alarm entities are the entity objects that falsely report the target entity. The target detection box is determined based on the degree of overlap between the first detection box of the target entity and the third detection box of the false alarm entity. Target recognition is then performed based on the target entity and the false alarm entities, effectively eliminating false alarm detection boxes, effectively reducing the false detection rate of synonym prompts, effectively reducing the probability of entity recognition errors, significantly reducing the false alarm problem of visually similar objects, and improving detection accuracy and robustness.

[0033] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0034] Object detection refers to locating and classifying all entity objects in an image to be detected and outputting the coordinates of the detection bounding boxes of the entity objects and the class labels. To achieve object detection, the image to be detected can be input into an object detection model, which processes the image to be detected and outputs the coordinates of the detection bounding boxes of the entity objects and the class labels. However, there are the following problems in open object detection based on deep learning:

[0035] The semantic sensitivity problem of open object detection. For an open object detection model based on text prompts, it needs to rely on the alignment training of text and images. However, limited by the discrete annotation of training data, it lacks the ability to model semantic consistency for synonymous text prompts. For example, when inputting prompt words such as "human", "people", "person", etc., although these prompt words represent the same semantic concept, the object detection model will regard these prompt words as independent class labels, resulting in deviations in the detection results and reducing robustness.

[0036] False alarm problems caused by visual feature confusion. Elongated cylindrical objects (such as cigarettes, toothpicks, straws, pens, etc.) have high similarity in the visual feature space, and the object detection model lacks an explicit false alarm suppression ability, resulting in the misidentification of non-target entities with similar feature expressions, significantly affecting the detection accuracy.

[0037] Failure of the cross-modal confidence range. The object detection model faces problems such as inaccurate confidence mechanisms and static threshold defects. The object detection model relies on a fixed confidence threshold preset by humans (such as 0.5), resulting in the following problems: sensitive to modal differences: the modal differences between image features and class text semantics cause fluctuations in confidence, and the confidence threshold cannot adapt to new scenarios. Narrow adjustment range: fine-tuning the confidence threshold will cause a sharp increase in the false alarm rate or an increase in the missed detection rate, and it is impossible to balance the trade-off between precision and recall. Lack of dynamic response: lacking a lightweight adaptive mechanism, forcing humans to frequently adjust the confidence threshold, resulting in low operation and maintenance efficiency.

[0038] In view of the above findings, in the embodiments of this application, a scene-adaptive open object detection and attribute recognition method is proposed, which has functions such as "open semantic dynamic enhancement" and "cross-modal threshold adaptive adjustment". While ensuring the accuracy of joint recognition of object attributes in an open environment, it realizes the dynamic stable optimization of the confidence threshold. This embodiment belongs to the cross-technology field of artificial intelligence and computer vision, and is especially suitable for entity recognition of unknown category objects and the joint determination of their multi-dimensional attributes (such as color, action, spatial position) in a dynamically open environment. It can ensure cross-modal and synonymous semantic performance and can be widely applied to scenarios such as autonomous driving, security detection, and industrial quality inspection that need to adapt to changes in open object categories and attributes in real time.

[0039] In this embodiment, a recognition architecture that decouples entities and attributes is designed, and a cascaded topology of requirement parsing function, entity recognition function and attribute recognition function is designed. Through a structured process, high-precision separation and recognition of target entities and attributes in open scenarios is achieved, rather than coupled training of entities and attributes. This solves the problem of attenuation of recognition capability of unknown category attributes, eliminates semantic interference between entities and attributes, and improves the performance of open target detection algorithm.

[0040] In this embodiment, a collaborative mechanism of dynamic semantic enhancement and false alarm suppression vocabulary (DS-FPSV) is designed, integrating a synonym semantic input supplementation mechanism (dynamically expanding entity prompts through a synonym clustering library) and a false alarm entity registration and removal mechanism (constructing an adversarial set of easily confused entities to suppress visual feature confusion), thereby improving the modeling capability of semantic consistency.

[0041] In this embodiment, an adaptive adjustment mechanism for the automated threshold is designed to achieve adaptive optimization of cross-modal confidence based on a very small sample size (e.g., 10 labeled samples). The adaptive optimization of the confidence threshold is achieved through probability-driven amplification preprocessing, cross-modal feature collaborative projection, and closed-loop solution decision boundary calculation, rather than statically configuring the confidence threshold, thereby achieving dynamic and stable optimization of cross-modal confidence.

[0042] In one possible implementation, to achieve the "open semantic dynamic enhancement" function, a DS-FPSV (Dynamic Semantic Enhanced & False-Positive Suppressed Vocabulary) is pre-built. DS-FPSV includes a dynamic semantic enhancement vocabulary and a false-positive suppression vocabulary; the dynamic semantic enhancement vocabulary is DS, and the false-positive suppression vocabulary is FPSV. As a dynamic database, DS-FPSV supports online updates (construction complexity O(N logN)) and seamlessly integrates with the object detection model.

[0043] For the construction process of dynamic semantic enhancement vocabulary, multiple sample texts can be obtained. For each sample text, the sample text can include descriptive information about entity objects. Entity objects can be things that exist in the real world, have independent existence and characteristics, and can be observed, perceived and measured, such as cars, computers, people, etc. The descriptive information of entity objects is the information that describes the entity object.

[0044] Multiple sample texts can be clustered to obtain multiple clusters. For example, for each sample text, its textual features can be extracted. Based on the textual features of different sample texts, all sample texts are clustered to obtain multiple clusters. For each cluster, the cluster includes at least one sample text, and the similarity between the textual features of the sample texts within the cluster is greater than a threshold. For example, the k-means algorithm (i.e., k-means clustering algorithm) or the HDBSCAN algorithm (i.e., density clustering algorithm) can be used to cluster multiple sample texts; this embodiment does not limit the clustering algorithm used.

[0045] When multiple clusters are obtained, each cluster can include at least one sample text, and all sample texts within the cluster belong to the same entity object; that is, these sample texts are descriptive information about the same entity object. For example, sample text 1 is a descriptive information about the entity object "human," sample text 2 is a descriptive information about the entity object "people," and sample text 3 is a descriptive information about the entity object "person." Obviously, these sample texts are all descriptive information about the same entity object "person." When clustering the sample texts, these sample texts can be grouped together to obtain a cluster, meaning that all sample texts within this cluster belong to the same entity object "person."

[0046] A dynamic semantic enhancement vocabulary is constructed, which includes multiple first vocabulary entries corresponding to multiple clusters. For example, the dynamic semantic enhancement vocabulary includes first vocabulary entry 1 corresponding to cluster 1, first vocabulary entry 2 corresponding to cluster 2, and so on. For each first vocabulary entry, the first vocabulary entry can include the entity corresponding to each sample text within the cluster (i.e., the cluster to which the first vocabulary entry corresponds). Here, the entity can be information about the entity object, such as the object name, object number, etc.

[0047] For example, the first term 1 can include entities such as human, person, people, human being, individual person, human being, and person. The entity "human" is obtained by analyzing sample text 1 (the descriptive information for the entity object "human") within cluster 1, the entity "people" is obtained by analyzing sample text 2 within cluster 1, the entity "person" is obtained by analyzing sample text 3 within cluster 1, and so on. Obviously, the first term 1 includes multiple entities for the same entity object "person", and these entities correspond to the same entity object.

[0048] For example, a dynamic semantic augmentation vocabulary can also be called a synonym clustering library (Φ_syn) or a semantic equivalence set. It can be formed by clustering text features (such as using the k-means algorithm or HDBSCAN algorithm). The dynamic semantic augmentation vocabulary can collect and merge semantic expressions with the same meaning across datasets (such as COCO, Visual Genome, or open-image datasets, which are used to provide sample texts). Each cluster can represent a completely unified semantic concept.

[0049] For example, after clustering text features to form a dynamic semantic enhancement vocabulary, the dynamic semantic enhancement vocabulary can be manually verified and reviewed to obtain the final output dynamic semantic enhancement vocabulary. Alternatively, the dynamic semantic enhancement vocabulary can be provided manually without restriction.

[0050] The process of constructing the false positive suppression vocabulary involves acquiring multiple sample images. Each sample image can include image information about the entity object, such as image information about "car". When acquiring sample images, the actual entity within the image can also be obtained. The actual entity can be information about the entity object (e.g., car) within the sample image, such as the object name.

[0051] For each sample image, the predicted entity of the entity object in the sample image can be obtained through recognition. The predicted entity is the predicted identifier of the entity object in the sample image, such as the predicted object name. For example, if the entity object in the sample image is a cigarette, that is, the real entity is "cigarette", then when recognizing the sample image, the predicted entity of the entity object may be "toothpick", "straw", "lollipop", or "pen".

[0052] For each entity object in all sample images, the true entity and the predicted entity of that entity object can be obtained. Based on this, a false positive suppression vocabulary can be constructed. This vocabulary can include multiple second-vocabulary entries corresponding to multiple entity objects, such as second-vocabulary entry 1 corresponding to entity object 1, second-vocabulary entry 2 corresponding to entity object 2, and so on. For each second-vocabulary entry, the second-vocabulary entry can include the correspondence between the true entity of the entity object (i.e., the entity object corresponding to the second-vocabulary entry) and the predicted entity of that entity object.

[0053] For example, the second term 1 could include the correspondence between the real entity "cigarette" and the predicted entities "toothpick, straw, lollipop, pen". Clearly, the predicted entities are those that generate false positives for the real entity "cigarette".

[0054] For example, the false positive suppression vocabulary can also be referred to as the confusion-prone false positive suppression set (Ψ_fp), and can identify groups of confusing objects through visual feature clustering (such as ViT- or CLIP-based visual embeddings). For example, for the entity "cigarette", its confusion set is Ψ_fp = {toothpick, straw, lollipop, pen}, indicating that these objects have high similarity in the visual feature space (such as RGB histogram or Edge feature). The false positive suppression vocabulary can collect and merge semantic expressions that cross datasets (such as datasets like COCO, Visual Genome, or open-image, which are used to provide sample images) but generate false positives.

[0055] For example, after forming the false positive suppression vocabulary through visual feature clustering, the false positive suppression vocabulary can also be manually verified and reviewed to obtain the final output false positive suppression vocabulary. Or, the false positive suppression vocabulary can be directly provided manually, and there is no restriction on the acquisition method of this false positive suppression vocabulary.

[0056] In summary, based on the training data of the open object detection model (such as sample texts and sample images), DS-FPSV can be pre-constructed to perform text and visual feature clustering on the entity corpus, construct synonymous semantic enhancement relationships and confusion-prone false positive suppression relationships, so as to achieve the following based on the open object detection model:

[0057] Synonymous semantic input supplementation mechanism. Establish a synonym clustering library , automatically expand the semantic equivalence set of the input entity prompt word, which can be achieved through a dynamic semantic enhancement vocabulary. For example, when inputting "person", it is dynamically expanded to "human, person, people, human being, individual", and feature alignment is performed by the model in the unified semantic space, and the feature alignment is obtained through similarity retrieval of semantic representations.

[0058] False positive entity registration and elimination mechanism. Construct a confusion-prone entity adversarial set , which can be achieved through the false positive suppression vocabulary. For example, when inputting the target entity "cigarette", the negative sample suppression of its confusion set "toothpick, straw, lollipop, pen" is synchronously activated, and the negative samples are suppressed through feature decoupling and false positive response masks.

[0059] In the above application scenarios, an object detection method is proposed in an embodiment of the present application. This method can be applied to electronic devices, such as personal computers, laptops, Internet of Things devices, cameras, management devices, cloud devices, servers, etc. There is no restriction on the type of these electronic devices, and it can be any device. Refer to Figure 2 As shown, it is a schematic flowchart of the object detection method. This method can include:

[0060] Step 201: Obtain the prompt text and the image to be detected.

[0061] For example, the cue text can also be called the requirement text. For instance, when detecting a person wearing red clothes in an image, the cue text is injected according to the actual requirements, and there are no restrictions on it.

[0062] Step 202: Determine the target entity and target attribute based on the prompt text. The target entity is the information of the entity object in the prompt text, such as "person". The target attribute is the information of the attribute object in the prompt text, such as "wearing red clothes", or "red", or "red clothes", etc.

[0063] For example, the requirement parsing module can implement requirement parsing functionality. This module can be a requirement parsing model, which can be a lightweight language model (such as a distilled version of BERT or a T5-Small model), or other models; there are no restrictions on the type. The input data for the requirement parsing model is the prompt text, and the output data is the target entity and target attributes.

[0064] For example, the requirement parsing model receives input in natural language, i.e., prompt text, such as "detect the black truck in the image." The requirement parsing model decouples entities and attributes in the prompt text through semantic dependency parsing and named entity recognition techniques. For instance, the entity is "truck," and the attribute is "red." The entity is the information of the entity object in the prompt text, hereinafter referred to as the target entity, and the attribute is the attribute object in the prompt text, hereinafter referred to as the target attribute. Through the requirement parsing model, the object ontology and modifying attributes in semantics can be explicitly separated. The object ontology is the target entity, and the modifying attribute is the target attribute, achieving a structured mapping of semantics.

[0065] Step 203: Determine the equivalent entity and false alarm entity corresponding to the target entity. For example, the equivalent entity and the target entity belong to the same entity object, that is, different description information for the same entity object. The false alarm entity corresponds to the entity object that caused the false alarm for the target entity.

[0066] For example, the similarity between the target entity and each entity in the dynamic semantic enhancement vocabulary can be determined. This involves determining the word feature vector of the target entity and the word feature vector of each entity in the dynamic semantic enhancement vocabulary. Then, the similarity (such as cosine similarity) between the word feature vector of the target entity and the word feature vector of each entity in the dynamic semantic enhancement vocabulary can be calculated.

[0067] Then, select the k most similar entities from the dynamic semantic enhancement vocabulary, that is, find the top-k most similar entries through the dynamic semantic enhancement vocabulary. Alternatively, select entities with a similarity greater than a preset threshold (e.g., 0.8) from the dynamic semantic enhancement vocabulary. There are no restrictions on the method of selecting these entities.

[0068] Based on the selected entities, such as k entities, determine the first term to which the k entities belong, and take the first term with the most entities as the first term corresponding to the target entity. For example, if 5 entities are selected from first term 1 and 2 entities are selected from first term 2, then first term 1 is taken as the first term corresponding to the target entity.

[0069] In summary, we can query the first term corresponding to the target entity from the dynamic semantic enhancement lexicon. Based on this, the entities within that first term can be identified as equivalent entities, meaning that the entities within that first term belong to the same entity object as the target entity. Clearly, the entities within that first term have a high similarity to the target entity, such as exceeding the third threshold.

[0070] In this way, a semantic equivalence set can be output. Semantic equivalence set Each entity within is an equivalent entity to the target entity, forming a semantic equivalence set. It is retrieved from a dynamic semantic enhancement vocabulary. For example, the target entity is "person", and the equivalent entities corresponding to the target entity can include, but are not limited to: human, person, people, human being, and individual.

[0071] For example, the similarity between the target entity and each real entity in the false positive suppression vocabulary can be determined. This can be achieved by determining the word feature vector of the target entity and the word feature vector of each real entity in the false positive suppression vocabulary. Then, the similarity (e.g., cosine similarity) between the word feature vector of the target entity and the word feature vector of each real entity in the false positive suppression vocabulary can be calculated.

[0072] Then, the real entity with the highest similarity is selected from the false positive suppression vocabulary. If this maximum similarity is greater than the fourth threshold, it means that the real entity is the same as the target entity, and the second vocabulary item to which the real entity belongs can be used as the second vocabulary item corresponding to the target entity. If this maximum similarity is not greater than the fourth threshold, it means that there is no second vocabulary item corresponding to the target entity in the false positive suppression vocabulary, and there is no false positive entity.

[0073] Taking the case where the maximum similarity is greater than the fourth threshold as an example, we can query the second term corresponding to the target entity from the false positive suppression term list. Based on this, the predicted entity within this second term can be identified as a false positive entity, that is, the entity object that falsely reports the target entity. Obviously, the similarity between the real entity within this second term and the target entity can be greater than the fourth threshold, indicating that the real entity and the target entity are the same.

[0074] This allows the output of a false alarm set. False alarm set Each entity within the set (also known as the easily confused set) is a false alarm entity corresponding to the target entity; the false alarm set... It is obtained from the false positive suppression vocabulary list. For example, the target entity is "person", and the corresponding false positive entities can include, but are not limited to: tree stump, model, robot, doll, sculpture, monkey.

[0075] Step 204: Obtain the first detection box corresponding to the target entity, the second detection box corresponding to the equivalent entity, and the third detection box corresponding to the false alarm entity from the image to be detected.

[0076] For example, the entity recognition module can implement entity recognition functionality. This module can be an object detection model (i.e., an entity detection model). The object detection model can be any model used for object detection, such as the GroundingDINO model or the MQ-Det model; there are no restrictions on the object detection model. The input data of the object detection model are the image to be detected, the target entity, the equivalent entity, and the false positive entity. The output data of the object detection model are the first detection box corresponding to the target entity, the second detection box corresponding to the equivalent entity, and the third detection box corresponding to the false positive entity.

[0077] For example, the image to be detected and the target entity can be input into the target detection model. The target detection model can detect the detection box corresponding to the target entity in the image to be detected (denoted as the first detection box), and output the first detection box corresponding to the target entity (such as the detection box coordinates), the category label of the first detection box, and the confidence score of the first detection box. The confidence score represents the probability that the entity object exists.

[0078] For example, the image to be detected and each equivalent entity (the following explanation will use one equivalent entity as an example) can be input into the object detection model. The object detection model can detect the detection box (denoted as the second detection box) corresponding to the equivalent entity in the image to be detected, and output the second detection box (such as the detection box coordinates), the category label and confidence score of the second detection box corresponding to the equivalent entity.

[0079] For example, the image to be detected and each false alarm entity (the following explanation will use a false alarm entity as an example) can be input into the target detection model. The target detection model can detect the detection box (denoted as the third detection box) corresponding to the false alarm entity in the image to be detected, and output the third detection box (such as the detection box coordinates), the category label of the third detection box, and the confidence score of the false alarm entity.

[0080] In one possible implementation, obtaining the first detection box corresponding to the target entity from the image to be detected (e.g., the target detection model detecting the first detection box in the image to be detected) may include:

[0081] Multiple candidate bounding boxes are divided from the image to be detected, and the method of dividing these candidate bounding boxes is not restricted. For any candidate bounding box in the image to be detected, feature extraction is performed on the candidate bounding box to obtain a second image feature (i.e., the image feature of the candidate bounding box region in the image to be detected). Feature extraction is performed on the target entity to obtain a second text feature (such as the word feature of the target entity). A second cross-modal similarity is determined based on the second image feature and the second text feature (since it is the similarity between image features and text features, it is called cross-modal similarity). In this embodiment, there are no restrictions on how the second cross-modal similarity is determined.

[0082] A second similarity threshold is determined for the target entity. This second similarity threshold can be a pre-configured fixed similarity threshold or an adaptively dynamically adjusted similarity threshold. For details on how to adaptively and dynamically adjust the similarity threshold, please refer to the following embodiments.

[0083] Based on this, if the second cross-modal similarity is greater than the second similarity threshold corresponding to the target entity, then the candidate box is determined as the first detection box corresponding to the target entity; if the second cross-modal similarity is not greater than the second similarity threshold, then determining the candidate box as the first detection box corresponding to the target entity is prohibited. Obviously, after performing the above processing on all candidate boxes in the image to be detected, the first detection box corresponding to the target entity can be obtained, and the first detection box may include at least one candidate box.

[0084] In one possible implementation, for each equivalent entity, obtaining the second detection box corresponding to that equivalent entity from the image to be detected (e.g., the object detection model detects the second detection box) includes:

[0085] Multiple candidate bounding boxes are divided from the image to be detected. For any candidate bounding box within the image, feature extraction is performed to obtain a second image feature (i.e., the image feature of the candidate bounding box region within the image). Feature extraction is performed on the equivalent entity to obtain a second text feature (such as the word feature of the equivalent entity). The second cross-modal similarity is determined based on the second image feature and the second text feature.

[0086] A second similarity threshold is determined for each equivalent entity. This second similarity threshold can be a pre-configured fixed similarity threshold or an adaptively dynamically adjusted similarity threshold. When the similarity threshold is adaptively dynamically adjusted, the second similarity threshold for the equivalent entity and the second similarity threshold for the target entity are determined independently; that is, a second similarity threshold is determined separately for each entity. Therefore, the second similarity thresholds for different entities may be the same or different.

[0087] Based on this, if the second cross-modal similarity is greater than the second similarity threshold corresponding to the equivalent entity, then the candidate box is determined as the second detection box corresponding to the equivalent entity; if the second cross-modal similarity is not greater than the second similarity threshold, then determining the candidate box as the second detection box corresponding to the equivalent entity is prohibited. Obviously, after performing the above processing on all candidate boxes in the image to be detected, the second detection box corresponding to the equivalent entity can be obtained, and the second detection box can include at least one candidate box.

[0088] In one possible implementation, for each falsely reported entity, obtaining the third detection box corresponding to that falsely reported entity from the image to be detected (e.g., the third detection box is detected by an object detection model) includes:

[0089] Multiple candidate bounding boxes are divided from the image to be detected. For any candidate bounding box within the image, feature extraction is performed to obtain a second image feature (i.e., image features of the candidate bounding box region within the image). Feature extraction is then performed on the falsely reported entity to obtain a second text feature (e.g., word features of the falsely reported entity). A second cross-modal similarity is determined based on the second image feature and the second text feature. A second similarity threshold is determined for the falsely reported entity. This second similarity threshold can be a pre-configured fixed similarity threshold or an adaptively dynamically adjusted similarity threshold.

[0090] Based on this, if the second cross-modal similarity is greater than the second similarity threshold corresponding to the falsely reported entity, then the candidate box is determined as the third detection box corresponding to the falsely reported entity; if the second cross-modal similarity is not greater than the second similarity threshold, then determining the candidate box as the third detection box corresponding to the falsely reported entity is prohibited. Obviously, after performing the above processing on all candidate boxes in the image to be detected, the third detection box corresponding to the falsely reported entity can be obtained, and the third detection box may include at least one candidate box.

[0091] Step 205: Determine the target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box.

[0092] For example, candidate detection boxes are determined based on the degree of overlap between a first detection box and a second detection box; wherein, a candidate detection box includes the first detection box, or, a candidate detection box includes the second detection box, or, a candidate detection box includes both the first and second detection boxes; a target detection box is determined based on the degree of overlap between a candidate detection box and a third detection box; wherein, the target detection box includes the remaining candidate detection boxes filtered based on the degree of overlap. For example, the target detection box corresponding to the target entity can be determined using the following steps:

[0093] Step S11: For each second detection box (i.e., multiple equivalent entities will correspond to multiple second detection boxes), determine the first degree of overlap between the first detection box and the second detection box. The first degree of overlap can be the IOU (Intersection over Union) between the two detection boxes.

[0094] Step S12: Select candidate detection boxes from the first detection box and all second detection boxes.

[0095] For example, for each second detection box, if the first overlap degree corresponding to the second detection box is less than the first threshold, that is, the overlap degree between the second detection box and the first detection box is low, then the first detection box and the second detection box are determined as candidate detection boxes, and these two detection boxes are retained.

[0096] If the first overlap degree corresponding to the second detection box is not less than the first threshold, that is, the second detection box has a high degree of overlap with the first detection box, then either the first detection box or the second detection box is determined as a candidate detection box, and only one detection box is retained. For example, if the confidence level of the first detection box is not less than the confidence level of the second detection box, then the first detection box is determined as a candidate detection box. If the confidence level of the first detection box is less than the confidence level of the second detection box, then the second detection box is determined as a candidate detection box.

[0097] In summary, candidate detection boxes can be determined based on the first detection box and all second detection boxes, and there can be at least one candidate detection box. For example, an object detection model can output detection results for multiple synonyms, i.e., a first detection box and multiple second detection boxes. The first detection box and multiple second detection boxes can be merged using Greedy Non-Maximum Suppression (GMS) to output a single candidate detection box, thereby eliminating redundant detections and ensuring that each entity is output only once, or multiple candidate detection boxes can be output.

[0098] Step S13: For each third detection box (i.e., multiple false alarm entities will correspond to multiple third detection boxes), determine the second degree of overlap between the candidate detection box (which can be at least one candidate detection box) and the third detection box. The second degree of overlap can be the IOU between the two detection boxes.

[0099] Step S14: For each third detection box, perform a weighted operation on the second overlap degree corresponding to the third detection box and the confidence of the third detection box to obtain the false positive score of the candidate detection box and the third detection box.

[0100] For example, the following formula can be used to determine the false positive score of the candidate detection box and the third detection box: . represents the false positive score of the candidate detection box and the third detection box, represents the confidence of the third detection box, that is, the confidence of the third detection box output by the target detection model, represents the second overlap degree between the candidate detection box and the third detection box. and are weighting coefficients, and the weighting coefficients can be configured according to actual needs, and the final elimination standard can be obtained through reasonable coefficient ratio.

[0101] Step S15: Select the target detection box corresponding to the target entity from all candidate detection boxes.

[0102] Exemplarily, for each candidate detection box, if the false positive score of the candidate detection box and the third detection box is not greater than the second threshold (i.e., the false positive threshold, such as 0.3, etc.), then the candidate detection box is determined as the target detection box; if the false positive score of the candidate detection box and the third detection box is greater than the second threshold, then it is prohibited to determine the candidate detection box as the target detection box. For example, if the false positive scores of the candidate detection box and all third detection boxes are not greater than the second threshold, then the candidate detection box is determined as the target detection box; if the false positive score of the candidate detection box and any third detection box is greater than the second threshold, it means that the candidate detection box is a false positive detection box, and it can be prohibited to determine the candidate detection box as the target detection box.

[0103] In summary, the target detection box can be selected from all candidate detection boxes, and there is at least one target detection box. For example, use the third detection box corresponding to the false positive entity to filter all candidate detection boxes, achieve the filtering of false positive targets, significantly reduce the false positive rate, and output the refined detection boxes and confidences.

[0104] In summary, it can be seen that the first detection box is optimized by using the second detection box and the third detection box, and finally the target detection box is output, and the ID and class label corresponding to the target detection box are output. The ID indicates which detection box the target detection box is among all detection boxes, and the class label can be the target entity. In the optimization process of the first detection box, through dynamic synonym expansion and unified semantic space alignment (bonded by text feature similarity), consistent detections can be output for different expressions (such as "human" and "人"), reducing detection errors. By suppressing negative samples in the confusion set and decoupling features, detection errors are reduced.

[0105] In one possible implementation, if a first detection box corresponding to the target entity, a second detection box corresponding to an equivalent entity, and a third detection box corresponding to a falsely reported entity can be obtained from the image to be detected, then the target detection box is determined based on the degree of overlap between the first, second, and third detection boxes. If only the first detection box corresponding to the target entity and the second detection box corresponding to an equivalent entity can be obtained from the image to be detected, then the target detection box is determined based on the degree of overlap between the first and second detection boxes, such as determining candidate detection boxes based on the first and second detection boxes and using the candidate detection boxes as the target detection boxes. If only the first detection box corresponding to the target entity and the third detection box corresponding to a falsely reported entity can be obtained from the image to be detected, then the target detection box is determined based on the degree of overlap between the first and third detection boxes, such as using the first detection box as a candidate detection box and using the candidate detection box and the third detection box as the target detection box. If only a second detection box corresponding to an equivalent entity and a third detection box corresponding to a false positive entity can be obtained from the image to be detected, then the target detection box is determined based on the degree of overlap between the second and third detection boxes. For example, the second detection box is used as a candidate detection box, and the candidate detection box and the third detection box are used as the target detection box. If only a first detection box corresponding to a target entity can be obtained from the image to be detected, then the first detection box is used as the target detection box. If only a second detection box corresponding to an equivalent entity can be obtained from the image to be detected, then the second detection box is used as the target detection box. If only a third detection box corresponding to a false positive entity can be obtained from the image to be detected, then no target detection box is output.

[0106] Step 206: Extract the detection sub-image of the target detection box from the image to be detected.

[0107] For example, after obtaining the target detection box, a sub-image of the target detection box can be extracted (cropped) from the image to be detected based on the coordinates of the target detection box. This sub-image is called the detection sub-image.

[0108] Step 207: Determine the target attribute information of the target detection box based on the detected sub-image and target attributes. This target attribute information can represent the attributes of the target detection box. For example, the target attributes can be used as the target attribute information of the target detection box, or the target attributes can be optimized and the optimized target attributes can be used as the target attribute information of the target detection box. There are no restrictions on this.

[0109] For example, the attribute recognition module can implement attribute recognition functionality. This module can be an attribute recognition model, which can be any model used to achieve attribute recognition, such as a cross-modal contrastive learning model (e.g., the CLIP-ViT model or the ALIGN model). There are no restrictions on the attribute recognition model. The input data for the attribute recognition model can be a detection sub-image (i.e., the image region corresponding to the target detection box) and the target attribute (the target attribute can be determined based on the prompt text, such as "wearing red clothes"). The output data of the attribute recognition model can be the target attribute information of the target detection box.

[0110] For example, the detection sub-image and target attributes can be input into an attribute recognition model. The attribute recognition model performs cross-modal feature alignment on the detection sub-image and target attributes, determines the attribute existence confidence, identifies the target attribute information of the target detection box, and outputs the target attribute information. In this way, fine-grained visual and linguistic association capabilities can be achieved, enabling attribute generalization recognition independent of entity categories.

[0111] In one possible implementation, determining the target attribute information of the target detection box based on the detected sub-image and target attributes (such as determining target attribute information using an attribute recognition model) may include:

[0112] Feature extraction is performed on the detected sub-image to obtain first image features (such as the image features corresponding to the detected sub-image). Feature extraction is performed on the target attribute to obtain first text features (such as the word features corresponding to the target attribute). Based on the first image features and the first text features, a first cross-modal similarity is determined (since it is to determine the similarity between image features and text features, it can be called cross-modal similarity). In this embodiment, there are no restrictions on how the first cross-modal similarity is determined.

[0113] A first similarity threshold is determined for the target attribute. This first similarity threshold can be a pre-configured fixed similarity threshold or an adaptively dynamically adjusted similarity threshold. For adaptively dynamically adjusted similarity thresholds, please refer to subsequent embodiments. When adaptively dynamically adjusting the similarity threshold, the first similarity threshold for each attribute is determined independently; that is, a first similarity threshold is determined separately for each attribute. Therefore, the first similarity thresholds for different attributes may be the same or different.

[0114] Based on this, if the first cross-modal similarity is greater than the first similarity threshold, it indicates that the detected sub-image matches the target attribute, and the target attribute is determined as the target attribute information corresponding to the target detection box. If the first cross-modal similarity is not greater than the first similarity threshold, it indicates that the detected sub-image does not match the target attribute, and the target attribute is not determined as the target attribute information corresponding to the target detection box. In this case, the target attribute information corresponding to the target detection box can be generated by the attribute recognition model.

[0115] Step 208: Output target attribute information and target detection box. The target attribute information is the annotation information for the target detection box. At this point, the target recognition process is complete.

[0116] In this embodiment, the semantic sensitivity and visual false alarm issues are addressed through a Dynamic Semantic Enhancement and False Alarm Suppression Word (DS-FPSV). An entity-attribute decoupling architecture is adopted, employing a three-tiered topology of a requirement parsing module, an entity recognition module, and an attribute recognition module. These modules achieve high-precision separation and recognition of target entities and attributes. The end-to-end recognition is performed in the following order: Natural language requirement input → Requirement parsing module (decoupling entity / attribute) → Entity recognition module (detecting entities and outputting target detection boxes) → Attribute recognition module (cropping the image based on the target boxes and verifying attributes) → Outputting target instances with attribute annotations (i.e., target detection boxes).

[0117] In one possible implementation, see Figure 3 The diagram shown is a schematic representation of the target detection framework.

[0118] During object detection, prompt text (i.e., requirement text) can be obtained. This prompt text is then input into a requirement parsing model (such as a language model-based requirement parsing model), which outputs the target entity and target attributes. A DS-FPSV collaborative process is then performed on the target entity to obtain a semantic equivalence set and a set of easily confused false positives. The semantic equivalence set can include equivalent entities corresponding to the target entity, and the easily confused false positive set can include falsely reported entities corresponding to the target entity.

[0119] The system can acquire the image to be detected, input the image to be detected into the entity recognition model (such as developing an object detection model), input the target entity, semantic equivalence set and easily confused false alarm set into the entity recognition model, and output the target instance from the entity recognition model. The target instance includes the target detection box, ID and category label.

[0120] The detection sub-image is obtained by cropping the image to be detected based on the object detection box. The detection sub-image is then input into the attribute recognition model (such as the attribute recognition model based on the cross-modal contrastive learning model). The target attribute is input into the attribute recognition model, and the attribute recognition model outputs the target attribute information of the object detection box. By combining the target attribute information and the target instance, the target instance with the target attribute information can be obtained.

[0121] See also Figure 3As shown, a detection sub-image is obtained by cropping the image to be detected based on the target detection bounding box in the target instance. The detection sub-image with target attribute information can be used as a sample image. Based on the sample image, automatic threshold adaptation adjustment of attributes can be implemented. The first similarity threshold can be updated through the automatic threshold adaptation adjustment process of attributes. The update process of the first similarity threshold is described in subsequent embodiments. Furthermore, automatic threshold adaptation adjustment of entities can be implemented based on the sample image. The second similarity threshold can be updated through the automatic threshold adaptation adjustment process of entities. The update process of the second similarity threshold is described in subsequent embodiments.

[0122] In one possible implementation, see Figure 4 The diagram illustrates the object detection process. It involves obtaining prompt text (i.e., a requirement description). This prompt text is then input into the requirement parsing model, which outputs the target entity and target attributes. A DS-FPSV collaborative process can be performed on the target entity to obtain a semantic equivalence set and a set of easily confused false positives. These are then input into the entity recognition model. Finally, the target attributes are input into the attribute recognition model.

[0123] The system can acquire an image to be detected, input the image to be detected into an entity recognition model, output a target detection box from the entity recognition model, crop the image to be detected based on the target detection box to obtain a detection sub-image, input the detection sub-image into an attribute recognition model, and output the required detection result based on the detection sub-image and the target attribute. The required detection result can be a target instance with target attribute information.

[0124] The demand detection results trigger dynamic detection. During dynamic detection, a small number of scene supervision samples are acquired (e.g., cropping the image to be detected based on the target detection box to obtain a detection sub-image, and the detection sub-image with target attribute information can be used as a sample image). Based on these small number of scene supervision samples, automatic threshold adaptation adjustment for attributes is implemented, automatically updating the first similarity threshold. The first similarity threshold is then input into the attribute recognition model, which performs attribute recognition based on the first similarity threshold. Similarly, based on these small number of scene supervision samples, automatic threshold adaptation adjustment for entities is implemented, automatically updating the second similarity threshold. The second similarity threshold is then input into the entity recognition model, which performs entity recognition based on the second similarity threshold.

[0125] In one possible implementation, see Figure 5 The diagram illustrates the process of automated threshold adjustment. A small number of labeled samples (i.e., sample images) from the application scenario can be obtained. Based on these samples, input amplification preprocessing, cross-modal feature collaborative extraction, and threshold decision boundary optimization are performed to obtain an adaptive threshold. This adaptive threshold is a first similarity threshold and / or a second similarity threshold.

[0126] In one possible implementation, regarding the determination of the first similarity threshold, when determining the first similarity threshold corresponding to the target attribute, it is first determined whether there is a positive sample image corresponding to the target attribute. A positive sample image is a detection sub-image with target attribute information (i.e., obtained by cropping the image to be detected based on the target detection box), and this target attribute information is the target attribute. If so, the first similarity threshold corresponding to the target attribute can be determined based on the positive sample image corresponding to the target attribute; otherwise, the configured default similarity threshold (configured according to actual needs) can be determined as the first similarity threshold corresponding to the target attribute. Furthermore, a minimum number of positive sample images can be constrained, such as 5. If there are 5 positive sample images corresponding to the target attribute, the first similarity threshold corresponding to the target attribute is determined based on these images; if there are fewer than 5 positive sample images corresponding to the target attribute, the configured default similarity threshold is determined as the first similarity threshold.

[0127] For example, the first similarity threshold corresponding to the target attribute can be determined using the following steps:

[0128] Step S21: Obtain m positive sample images corresponding to the target attribute, and obtain n negative sample images corresponding to the target attribute. m and n can be the same or different, such as 5 positive sample images and 5 negative sample images. In each positive sample image, the entity object corresponds to the target attribute (i.e., the target attribute information of the entity object is the target attribute), while in each negative sample image, the entity object does not correspond to the target attribute (i.e., the target attribute information of the entity object is not the target attribute).

[0129] Step S22: Amplify the positive sample images corresponding to the target attribute to obtain multiple amplified positive sample images, such as amplifying 5 positive sample images into 50 positive sample images.

[0130] For example, in the input amplification preprocessing stage, a small number of labeled samples of the target application scenario can be obtained, such as 5 positive sample images. Each positive sample image is a detection sub-image with target attribute information. When a single frame image includes multiple target instances, it corresponds to multiple sample images. For each positive sample image, an amplification strategy can be used to amplify the positive sample image to obtain multiple amplified positive sample images.

[0131] For example, an amplification strategy library can be pre-configured, which may include, but is not limited to, amplification strategies such as illumination perturbation, random occlusion, scale transformation, and adversarial noise injection. Based on this, at least one amplification strategy can be selected from the library and used to amplify the positive sample image. When amplifying the positive sample image using this strategy, multiple positive sample images can be obtained by changing the parameters of the amplification strategy (such as the parameters of illumination perturbation).

[0132] For example, when selecting at least one amplification strategy from the amplification strategy library, the amplification strategy can be selected randomly or by probability, and there is no restriction on the selection method.

[0133] After augmenting the positive sample images, an augmented sample set can be generated. This augmented sample set can include multiple positive sample images, such as 50 positive sample images, and then a scene feature distribution estimation space can be constructed using these positive sample images. For example, the original sample set... Expanding strategy library Such as illumination perturbations, random occlusion, scale transformations, and adversarial noise injection. Amplification probability distribution. Weighted according to operational complexity. Thus, the generation process of multiplex amplification can be represented as: For example, This is an amplification operation using random sampling, thereby ensuring the sample size after amplification. .

[0134] Step S23: Amplify the negative sample images corresponding to the target attribute to obtain multiple amplified negative sample images, such as amplifying 5 negative sample images into 50 negative sample images.

[0135] For example, in the input amplification preprocessing stage, a small number of labeled samples of the target application scenario can be obtained, such as five negative sample images. Each negative sample image is a detection sub-image with target attribute information. For each negative sample image, an amplification strategy can be applied to amplify the negative sample image, resulting in multiple amplified negative sample images. For instance, at least one amplification strategy can be selected from an amplification strategy library, and this amplification strategy can be used to amplify the negative sample images to obtain multiple negative sample images.

[0136] Step S24: Extract features from the target attributes to obtain the first text features (see the above embodiment for this process). Determine the cross-modal similarity of positive samples based on the image features of the positive sample images and the first text features. Determine the cross-modal similarity of negative samples based on the image features of the negative sample images and the first text features.

[0137] For example, in the cross-modal feature collaborative extraction stage, for each positive sample image, feature extraction can be performed on the positive sample image to obtain its image features. Based on the image features of the positive sample image and the first text feature, the cross-modal similarity of the positive sample image is determined, with each positive sample image corresponding to one cross-modal similarity. Similarly, for each negative sample image, feature extraction can be performed on the negative sample image to obtain its image features. Based on the image features of the negative sample image and the first text feature, the cross-modal similarity of the negative sample image is determined, with each negative sample image corresponding to one cross-modal similarity.

[0138] Step S25: Determine the current distribution similarity threshold based on multiple positive sample cross-modal similarities and multiple negative sample cross-modal similarities. The current distribution similarity threshold is used to distinguish between positive sample cross-modal similarities and negative sample cross-modal similarities, so that the positive sample cross-modal similarity of all positive sample images is greater than the current distribution similarity threshold, and the negative sample cross-modal similarity of all negative sample images is less than the current distribution similarity threshold.

[0139] For example, select the minimum positive sample cross-modal similarity from all positive sample images and the maximum negative sample cross-modal similarity from all negative sample images. The current distribution similarity threshold should be less than this minimum positive sample cross-modal similarity and greater than this maximum negative sample cross-modal similarity. For instance, the current distribution similarity threshold could be the average of the minimum positive sample cross-modal similarity and the maximum negative sample cross-modal similarity. Of course, this is just an example; the minimum requirement is that the positive sample cross-modal similarity of all positive sample images is greater than the current distribution similarity threshold, and the minimum negative sample cross-modal similarity of all negative sample images is less than the current distribution similarity threshold.

[0140] For example, in the threshold decision boundary optimization stage, the feature cluster centers and intra-class dispersion of positive sample images in the projection space can be calculated, and the feature cluster centers and intra-class dispersion of negative sample images in the projection space can be calculated. By maximizing the distribution interval between positive and negative sample images through the feature cluster centers and intra-class dispersion, the optimal threshold can be solved. This optimal threshold is the current distribution similarity threshold.

[0141] For example, the feature cluster centers and intra-cluster dispersion of positive sample images can be calculated using the following formula: , . Represents the feature cluster centers, Indicates the within-class dispersion. Indicates the cross-modal similarity of positive samples. This indicates the number of positive sample images.

[0142] For example, the feature cluster centers and intra-cluster dispersion of negative sample images can be calculated using the following formula: , . Represents the feature cluster centers, Indicates the within-class dispersion. Indicates the cross-modal similarity of negative samples. This indicates the number of negative sample images.

[0143] Clearly, the cluster centers and intra-class dispersion of positive sample images can reflect the distribution characteristics of positive sample cross-modal similarity, and the cluster centers and intra-class dispersion of negative sample images can reflect the distribution characteristics of negative sample cross-modal similarity. Based on the distribution characteristics of positive and negative sample cross-modal similarity, an optimal threshold can be found. This optimal threshold can distinguish the distribution characteristics of positive and negative sample cross-modal similarity, and this optimal threshold is the current distribution similarity threshold.

[0144] Step S26: Determine the current distribution similarity threshold as the first similarity threshold corresponding to the target attribute. Alternatively, determine whether there exists a historical distribution similarity threshold corresponding to the target attribute. The historical distribution similarity threshold can be the first similarity threshold of the previous period at the current time. If there is no historical distribution similarity threshold, then determine the current distribution similarity threshold as the first similarity threshold corresponding to the target attribute, and this first similarity threshold is used as the historical distribution similarity threshold for the next period.

[0145] Alternatively, if a historical distribution similarity threshold exists, the divergence value can be determined based on the current distribution similarity threshold and the historical distribution similarity threshold. If the divergence value is greater than the fifth threshold (which can be configured according to actual needs, such as 0.2), then the current distribution similarity threshold is determined as the first similarity threshold corresponding to the target attribute, and this first similarity threshold is used as the historical distribution similarity threshold for the next period. If the divergence value is not greater than the fifth threshold, then the historical distribution similarity threshold used in the previous period can be determined as the first similarity threshold corresponding to the target attribute, i.e., the first similarity threshold remains unchanged.

[0146] For example, the divergence value can be determined using the following formula: ;in, This represents the current distribution similarity threshold. Let S represent the similarity threshold of the s-th historical distribution, and let S represent the similarity thresholds of all historical distributions. This represents the divergence value.

[0147] In summary, when the divergence value is greater than the fifth threshold, it indicates a significant change in the current scene, and the first similarity threshold can be updated, with the current distribution similarity threshold being used as the first similarity threshold. When the divergence value is not greater than the fifth threshold, it indicates that the current scene has not changed significantly, and the first similarity threshold does not need to be updated; the historical distribution similarity threshold used in the previous cycle is used as the first similarity threshold, i.e., the first similarity threshold remains unchanged. In this way, a first similarity threshold optimized for the current scene can be output. During the deployment phase, confidence distribution drift is continuously detected, triggering threshold recalibration. If the divergence value is greater than the fifth threshold, the first similarity threshold is automatically iterated and updated to obtain a threshold adapted to the scene.

[0148] In one possible implementation, regarding the process of determining the second similarity threshold, when determining the second similarity threshold corresponding to the target entity (the determination method for the second similarity threshold corresponding to equivalent entities and false positive entities is the same, and will not be repeated hereafter), it is first determined whether there is a positive sample image corresponding to the target entity. The positive sample image is a detection sub-image of the target entity (i.e., obtained by cropping the image to be detected based on the target detection box), that is, the entity object in the positive sample image corresponds to the target entity. If so, the second similarity threshold corresponding to the target entity can be determined based on the positive sample image corresponding to the target entity; otherwise, the configured default similarity threshold (configured according to actual needs) can be determined as the second similarity threshold corresponding to the target entity. In addition, the minimum number of positive sample images can be constrained, such as 5. If there are 5 positive sample images corresponding to the target entity, the second similarity threshold is determined based on the positive sample images corresponding to the target entity; if there are less than 5 positive sample images corresponding to the target entity, the configured default similarity threshold is determined as the second similarity threshold. For example, the second similarity threshold corresponding to the target entity can be determined using the following steps:

[0149] Step S31: Obtain m positive sample images corresponding to the target entity and n negative sample images corresponding to the target entity. In each positive sample image, the entity object corresponds to the target entity; that is, the entity object in each positive sample image is the target entity. In each negative sample image, the entity object does not correspond to the target entity; that is, the entity object in each negative sample image is not the target entity.

[0150] Step S32: Amplify the positive sample images corresponding to the target entity to obtain multiple amplified positive sample images, such as amplifying 5 positive sample images into 50 positive sample images.

[0151] Step S33: Amplify the negative sample images corresponding to the target entity to obtain multiple amplified negative sample images, such as amplifying 5 negative sample images into 50 negative sample images.

[0152] Step S34: Extract features from the target entity to obtain the second text features (see the above embodiment for this process). Determine the cross-modal similarity of positive samples based on the image features of the positive sample image and the second text features. Determine the cross-modal similarity of negative samples based on the image features of the negative sample image and the second text features.

[0153] Step S35: Determine the current distribution similarity threshold based on the cross-modal similarity of multiple positive samples and the cross-modal similarity of multiple negative samples. The current distribution similarity threshold is used to distinguish between the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, so that the cross-modal similarity of positive samples of all positive sample images is greater than the current distribution similarity threshold, and the cross-modal similarity of negative samples of all negative sample images is less than the current distribution similarity threshold.

[0154] For example, the minimum positive sample cross-modal similarity is selected from the positive sample cross-modal similarities of all positive sample images, and the maximum negative sample cross-modal similarity is selected from the negative sample cross-modal similarities of all negative sample images. The current distribution similarity threshold is less than this minimum positive sample cross-modal similarity and greater than this maximum negative sample cross-modal similarity. For example, the current distribution similarity threshold could be the average of the minimum positive sample cross-modal similarity and the maximum negative sample cross-modal similarity.

[0155] Step S36: Determine the current distribution similarity threshold as the second similarity threshold corresponding to the target entity. Alternatively, determine whether there is a historical distribution similarity threshold corresponding to the target entity. If not, determine the current distribution similarity threshold as the second similarity threshold corresponding to the target entity. Alternatively, if there is a historical distribution similarity threshold, determine the divergence value based on the current distribution similarity threshold and the historical distribution similarity threshold. If the divergence value is greater than the fifth threshold, determine the current distribution similarity threshold as the second similarity threshold corresponding to the target entity. If the divergence value is not greater than the fifth threshold, determine the historical distribution similarity threshold used in the previous period as the second similarity threshold corresponding to the target entity.

[0156] As can be seen from the above technical solutions, in this embodiment, by decoupling the recognition structure of entities and attributes, the fine-grained joint attribute recognition capability in open environments is maintained, improving detection accuracy and robustness, such as an average accuracy improvement of over 20%. The DS-FPSV mechanism effectively reduces the false detection rate of synonym prompts by 40% and the false entity recognition error rate by 35%. Automated threshold adjustment requires only about 10 labeled samples, and near-optimal threshold solution is achieved through small-sample generalization (KL divergence < 0.2), realizing dynamic adaptive efficiency optimization. Ensuring stable multimodal confidence (reduced sensitivity to cross-modal feature drift) can significantly enhance reliability and generalization ability in dynamic open environments. By constructing a scene feature manifold through a compound amplification strategy, and utilizing the linear separability of positive and negative samples in cross-modal space, a closed-form solution is used to approximate the globally optimal threshold in this scene.

[0157] By designing a three-module cascaded topology (requirements analysis module, entity recognition module, and attribute recognition module), a structured process is used to achieve high-precision separation and recognition of target entities and attributes in open scenarios. This overcomes the openness constraint of coupled entity and attribute training and solves the problem of diminishing recognition capability for unknown category attributes. Through a collaborative mechanism of dynamic semantic enhancement and a large vocabulary for false positive suppression (DS-FPSV), integrating a synonymous semantic input supplementation mechanism (dynamically expanding entity prompts through a synonym clustering library) and a false positive entity registration and removal mechanism (constructing adversarial sets of easily confused entities to suppress visual feature confusion), semantic consistency modeling capability is improved, significantly reducing false positives of visually similar objects. Through automated threshold adaptation adjustment, cross-modal confidence is adaptively optimized based on a small number of samples, and online dynamic threshold updates (KL divergence drift triggering threshold updates) are integrated, solving the mismatch problem of static threshold mechanisms in dynamic environments.

[0158] Based on the same concept as the methods described above, this application proposes a target detection device, see [link to relevant documentation]. Figure 6 The diagram shown is a structural schematic of the target detection device, which may include:

[0159] The determination module 61 is used to determine the target entity based on the acquired prompt text, and to determine the equivalent entity and false alarm entity corresponding to the target entity; wherein the equivalent entity and the target entity correspond to the same entity object, and the false alarm entity corresponds to the entity object that falsely reports the target entity; the acquisition module 62 is used to acquire the first detection box corresponding to the target entity, the second detection box corresponding to the equivalent entity, and the third detection box corresponding to the false alarm entity from the image to be detected; the determination module 61 is used to determine the target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box; the processing module 63 is used to extract the detection sub-image of the target detection box from the image to be detected, determine the target attribute information based on the detection sub-image, and output the target attribute information and the target detection box.

[0160] For example, when determining the target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box, the determining module 61 is specifically used to: determine candidate detection boxes based on the degree of overlap between the first detection box and the second detection box; wherein, the candidate detection box includes the first detection box, or, the candidate detection box includes the second detection box, or, the candidate detection box includes both the first detection box and the second detection box; determine the target detection box based on the degree of overlap between the candidate detection box and the third detection box; wherein, the target detection box includes the remaining candidate detection boxes filtered based on the degree of overlap.

[0161] For example, when determining a target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box, the determining module 61 is specifically configured to: determine a first degree of overlap between the first detection box and the second detection box; if the first degree of overlap is less than a first threshold, determine the first detection box and the second detection box as candidate detection boxes; if the first degree of overlap is not less than the first threshold and the confidence level of the first detection box is not less than the confidence level of the second detection box, determine the first detection box as a candidate detection box; if the first degree of overlap is not less than the first threshold and the confidence level of the first detection box is less than the confidence level of the second detection box, determine the second detection box as a candidate detection box; and determine a second degree of overlap between the candidate detection box and the third detection box, and perform a weighted calculation on the second degree of overlap and the confidence level of the third detection box to obtain a false alarm score; if the false alarm score is not greater than the second threshold, determine the candidate detection box as a target detection box; if the false alarm score is greater than the second threshold, prohibit the candidate detection box from being determined as a target detection box.

[0162] For example, the processing module 63 is further configured to cluster the acquired sample texts to obtain multiple clusters, each cluster including at least one sample text belonging to the same entity object; construct a dynamic semantic enhancement vocabulary, the dynamic semantic enhancement vocabulary including multiple first vocabulary entries corresponding to the multiple clusters; for each first vocabulary entry, the first vocabulary entry includes the entity corresponding to each sample text within the cluster; identify the acquired sample images to obtain the predicted entity of the entity object in each sample image; construct a false positive suppression vocabulary, the false positive suppression vocabulary including multiple second vocabulary entries corresponding to multiple entity objects; for each second vocabulary entry, the second vocabulary entry includes the correspondence between the real entity of the entity object and the predicted entity of the entity object.

[0163] For example, when determining the equivalent entity and the false positive entity corresponding to the target entity, the determining module 61 is specifically configured to: query the first term entry corresponding to the target entity from the dynamic semantic enhancement term list, and determine the entity in the first term entry as the equivalent entity; at least one entity in the first term entry has a similarity greater than a third threshold with the target entity; query the second term entry corresponding to the target entity from the false positive suppression term list, and determine the predicted entity in the second term entry as the false positive entity; the real entity in the second term entry has a similarity greater than a fourth threshold with the target entity.

[0164] For example, when the processing module 63 determines the target attribute information based on the detected sub-image, it specifically performs the following steps: extracting features from the detected sub-image to obtain a first image feature; extracting features from the target attribute to obtain a first text feature, wherein the target attribute is determined based on the prompt word text; determining a first cross-modal similarity based on the first image feature and the first text feature; determining whether there is a positive sample image corresponding to the target attribute; if so, determining a first similarity threshold corresponding to the target attribute based on the positive sample image; if not, determining the configured default similarity threshold as the first similarity threshold corresponding to the target attribute; if the first cross-modal similarity is greater than the first similarity threshold, determining the target attribute as the target attribute information.

[0165] For example, when the processing module 63 determines the first similarity threshold corresponding to the target attribute based on the positive sample image, it is specifically used to: amplify the positive sample image corresponding to the target attribute to obtain multiple amplified positive sample images; amplify the negative sample image corresponding to the target attribute to obtain multiple amplified negative sample images; wherein, the entity object in each positive sample image corresponds to the target attribute, and the entity object in each negative sample image does not correspond to the target attribute; determine the cross-modal similarity of positive samples based on the image features of the positive sample image and the first text features, and determine the cross-modal similarity of negative samples based on the image features of the negative sample image and the first text features; determine the first similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, wherein the first similarity threshold is used to distinguish between the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, so that the cross-modal similarity of positive samples corresponding to all positive sample images is greater than the first similarity threshold, and the cross-modal similarity of negative samples corresponding to all negative sample images is less than the first similarity threshold.

[0166] For example, when the processing module 63 determines the first similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, it is specifically used to: determine the current distribution similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples; if there is no historical distribution similarity threshold, determine the current distribution similarity threshold as the first similarity threshold, and use the first similarity threshold as the historical distribution similarity threshold for the next period; if there is a historical distribution similarity threshold, determine the divergence value based on the current distribution similarity threshold and the historical distribution similarity threshold; if the divergence value is greater than the fifth threshold, determine the current distribution similarity threshold as the first similarity threshold; if the divergence value is not greater than the fifth threshold, determine the historical distribution similarity threshold used in the previous period as the first similarity threshold.

[0167] The processing module 63 determines the divergence value using the following formula: ;in, This represents the current distribution similarity threshold. Let S represent the similarity threshold of the s-th historical distribution, and let S represent the similarity thresholds of all historical distributions. This represents the divergence value.

[0168] For example, when the acquisition module 62 acquires the first detection box corresponding to the target entity from the image to be detected, it is specifically used to: extract features from any candidate box in the image to be detected to obtain a second image feature; extract features from the target entity to obtain a second text feature; determine a second cross-modal similarity based on the second image feature and the second text feature; determine whether there is a positive sample image corresponding to the target entity; if so, determine a second similarity threshold corresponding to the target entity based on the positive sample image; if not, determine the configured default similarity threshold as the second similarity threshold corresponding to the target entity; if the second cross-modal similarity is greater than the second similarity threshold, determine the candidate box as the first detection box corresponding to the target entity; if the second cross-modal similarity is not greater than the second similarity threshold, prohibit determining the candidate box as the first detection box corresponding to the target entity.

[0169] For example, when the acquisition module 62 determines the second similarity threshold corresponding to the target entity based on the positive sample image, it is specifically used to: perform amplification processing on the positive sample image corresponding to the target entity to obtain multiple positive sample images, and perform amplification processing on the negative sample image corresponding to the target entity to obtain multiple negative sample images; determine the cross-modal similarity of positive samples based on the image features of the positive sample images and the second text features, determine the cross-modal similarity of negative samples based on the image features of the negative sample images and the second text features; and determine the second similarity threshold corresponding to the target entity based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples.

[0170] Based on the same concept as the above method, this application proposes an electronic device, see [link to previous application]. Figure 7 As shown, the electronic device includes a processor 71 and a machine-readable storage medium 72, the machine-readable storage medium 72 storing machine-executable instructions that can be executed by the processor 71; the processor 71 is used to execute the machine-executable instructions to implement the target detection method disclosed in the above example of this application.

[0171] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the target detection method disclosed in the above examples of this application.

[0172] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0173] Based on the same concept as the method described above, this application also provides a computer program product, which may include a computer program. When executed by a processor, the computer program implements the target detection method disclosed in the examples above.

[0174] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A target detection method, characterized in that, The method includes: Based on the acquired prompt text, the target entity is determined, and the equivalent entity and false alarm entity corresponding to the target entity are determined; wherein, the equivalent entity and the target entity correspond to the same entity object, and the false alarm entity corresponds to the entity object that generated a false alarm for the target entity; Obtain the first detection box corresponding to the target entity, the second detection box corresponding to the equivalent entity, and the third detection box corresponding to the false alarm entity from the acquired image to be detected; A target detection box is determined based on the degree of overlap between a first detection box, a second detection box, and a third detection box; wherein, a candidate detection box is determined based on the degree of overlap between the first detection box and the second detection box; the candidate detection box includes the first detection box, or the candidate detection box includes the second detection box, or the candidate detection box includes both the first detection box and the second detection box; a target detection box is determined based on the degree of overlap between the candidate detection box and the third detection box; wherein, the target detection box includes the remaining candidate detection boxes filtered based on this degree of overlap; Extract a detection sub-image of the target detection box from the image to be detected, determine the target attribute information based on the detection sub-image, and output the target attribute information and the target detection box.

2. The method according to claim 1, characterized in that, The step of determining the target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box includes: Determine the first degree of overlap between the first detection frame and the second detection frame; If the first overlap is less than the first threshold, then the first detection box and the second detection box are determined as candidate detection boxes; if the first overlap is not less than the first threshold and the confidence of the first detection box is not less than the confidence of the second detection box, then the first detection box is determined as a candidate detection box; if the first overlap is not less than the first threshold and the confidence of the first detection box is less than the confidence of the second detection box, then the second detection box is determined as a candidate detection box. The second degree of overlap between the candidate detection box and the third detection box is determined, and the false alarm score is obtained by weighting the second degree of overlap and the confidence of the third detection box. If the false alarm score is not greater than the second threshold, the candidate detection box is determined as the target detection box; if the false alarm score is greater than the second threshold, the candidate detection box is prohibited from being determined as the target detection box.

3. The method according to claim 1, characterized in that, Before determining the equivalent entity and false positive entity corresponding to the target entity, the method further includes: Multiple sample texts that have been acquired are clustered to obtain multiple clusters, and each cluster includes at least one sample text belonging to the same entity object; a dynamic semantic enhancement vocabulary is constructed, which includes multiple first vocabulary entries corresponding to the multiple clusters; for each first vocabulary entry, the first vocabulary entry includes the entity corresponding to each sample text within the cluster; Multiple acquired sample images are identified to obtain the predicted entity of the entity object in each sample image; a false positive suppression vocabulary is constructed, which includes multiple second vocabulary entries corresponding to multiple entity objects; for each second vocabulary entry, the second vocabulary entry includes the correspondence between the real entity of the entity object and the predicted entity of the entity object; Determining the equivalent entity and false positive entity corresponding to the target entity includes: The first term corresponding to the target entity is queried from the dynamic semantic enhancement lexicon, and the entity in the first term is determined as the equivalent entity; wherein, at least one entity in the first term has a similarity to the target entity greater than a third threshold. The second term corresponding to the target entity is queried from the false positive suppression term list, and the predicted entity in the second term list is determined as the false positive entity; wherein, the similarity between the real entity in the second term list and the target entity is greater than a fourth threshold.

4. The method according to claim 1, characterized in that, The step of determining target attribute information based on the detected sub-image includes: Feature extraction is performed on the detected sub-image to obtain first image features; feature extraction is performed on the target attribute to obtain first text features, wherein the target attribute is determined based on the prompt word text; A first cross-modal similarity is determined based on the first image features and the first text features; Determine whether there is a positive sample image corresponding to the target attribute; if so, determine the first similarity threshold corresponding to the target attribute based on the positive sample image; if not, determine the configured default similarity threshold as the first similarity threshold corresponding to the target attribute. If the first cross-modal similarity is greater than the first similarity threshold, then the target attribute is determined as the target attribute information corresponding to the target detection box.

5. The method according to claim 4, characterized in that, Determining the first similarity threshold corresponding to the target attribute based on the positive sample image includes: The positive sample images corresponding to the target attribute are amplified to obtain multiple amplified positive sample images; the negative sample images corresponding to the target attribute are amplified to obtain multiple amplified negative sample images; wherein, the entity objects in each positive sample image correspond to the target attribute, and the entity objects in each negative sample image do not correspond to the target attribute. Positive sample cross-modal similarity is determined based on image features of positive sample images and the first text features, and negative sample cross-modal similarity is determined based on image features of negative sample images and the first text features; The first similarity threshold is determined based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples. The first similarity threshold is used to distinguish between the cross-modal similarity of positive samples and the cross-modal similarity of negative samples, so that the cross-modal similarity of positive samples corresponding to all positive sample images is greater than the first similarity threshold, and the cross-modal similarity of negative samples corresponding to all negative sample images is less than the first similarity threshold.

6. The method according to claim 5, characterized in that, Determining the first similarity threshold based on positive sample cross-modal similarity and negative sample cross-modal similarity includes: The current distribution similarity threshold is determined based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples; If there is no historical distribution similarity threshold, then the current distribution similarity threshold is determined as the first similarity threshold, and the first similarity threshold is used as the historical distribution similarity threshold for the next period. If a historical distribution similarity threshold exists, the divergence value is determined based on the current distribution similarity threshold and the historical distribution similarity threshold. If the divergence value is greater than the fifth threshold, the current distribution similarity threshold is determined as the first similarity threshold. If the divergence value is not greater than the fifth threshold, the historical distribution similarity threshold used in the previous period is determined as the first similarity threshold. The divergence value is determined using the following formula: ; This represents the current distribution similarity threshold. Let S represent the similarity threshold of the s-th historical distribution, and let S represent the similarity thresholds of all historical distributions. This represents the divergence value.

7. The method according to claim 1, characterized in that, The step of obtaining the first detection box corresponding to the target entity from the acquired image to be detected includes: For any candidate box in the image to be detected, feature extraction is performed on the candidate box to obtain a second image feature, feature extraction is performed on the target entity to obtain a second text feature, and a second cross-modal similarity is determined based on the second image feature and the second text feature; Determine whether there is a positive sample image corresponding to the target entity; if so, determine the second similarity threshold corresponding to the target entity based on the positive sample image; if not, determine the configured default similarity threshold as the second similarity threshold corresponding to the target entity. If the second cross-modal similarity is greater than the second similarity threshold, then the candidate box is determined as the first detection box corresponding to the target entity; if the second cross-modal similarity is not greater than the second similarity threshold, then determining the candidate box as the first detection box corresponding to the target entity is prohibited. The step of determining the second similarity threshold corresponding to the target entity based on the positive sample images includes: performing amplification processing on the positive sample images corresponding to the target entity to obtain multiple positive sample images, and performing amplification processing on the negative sample images corresponding to the target entity to obtain multiple negative sample images; determining the cross-modal similarity of positive samples based on the image features of the positive sample images and the second text features, determining the cross-modal similarity of negative samples based on the image features of the negative sample images and the second text features; and determining the second similarity threshold based on the cross-modal similarity of positive samples and the cross-modal similarity of negative samples.

8. A target detection device, characterized in that, The device includes: The determination module is used to determine the target entity based on the acquired prompt text, and to determine the equivalent entity and the false alarm entity corresponding to the target entity; wherein, the equivalent entity and the target entity correspond to the same entity object, and the false alarm entity corresponds to the entity object that generated a false alarm for the target entity; The acquisition module is used to acquire from the image to be detected a first detection box corresponding to the target entity, a second detection box corresponding to the equivalent entity, and a third detection box corresponding to the false alarm entity; A determining module is configured to determine a target detection box based on the degree of overlap between the first detection box, the second detection box, and the third detection box; wherein, candidate detection boxes are determined based on the degree of overlap between the first detection box and the second detection box; the candidate detection box includes the first detection box, or the candidate detection box includes the second detection box, or the candidate detection box includes both the first detection box and the second detection box; a target detection box is determined based on the degree of overlap between the candidate detection box and the third detection box; the target detection box includes the remaining candidate detection boxes filtered based on the degree of overlap. The processing module is used to extract a detection sub-image of the target detection box from the image to be detected, determine the target attribute information based on the detection sub-image, and output the target attribute information and the target detection box.

9. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-7.

Images