A target detection method, medium, device and product

Abstract

Embodiments of the present application disclose a target detection method, a medium, equipment and products. The method comprises: acquiring an image frame; using a first detection model to perform target detection on the image frame to obtain a candidate detection result of a target category; acquiring a key frame in the image frame, sending the key frame to a server, acquiring prior knowledge corresponding to the key frame generated by the server using a second detection model, the prior knowledge comprising a visual prototype feature for the target category, the visual prototype feature being an appearance feature capable of representing the target category; and correcting the candidate detection result based on the visual prototype feature to obtain a target detection result of the target category. The accuracy of target detection can be improved by the present application.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, medium, device and product. Background Technology

[0002] With the development of robotics and artificial intelligence, more and more intelligent robots are being widely used in various scenarios. For example, in embedded vision applications such as indoor companion robots, intelligent robots need to be able to understand the target category described by the user in any natural language in real time, such as recognizing fine-grained objects like dolls with personalized attributes or building blocks of a specific model. Currently, existing technologies typically deploy object detection models based on predefined vocabularies locally, and then use these models to detect objects in image frames. Their detection capabilities are limited by the vocabulary, making it difficult to accurately identify unseen or fine-grained rare objects. Summary of the Invention

[0003] In view of this, this application provides a target detection method, medium, device and product to improve the accuracy of target detection.

[0004] This application provides the following solution: Firstly, a target detection method is provided, applied to an edge device, the edge device being communicatively connected to a server, the method comprising: Acquire image frames; The first detection model is used to perform target detection on the image frame to obtain candidate detection results for the target category; Keyframes in the image frame are obtained and sent to the server. Prior knowledge corresponding to the keyframes generated by the server using the second detection model is obtained. The prior knowledge includes visual prototype features for the target category. The visual prototype features are appearance features that can characterize the target category. The candidate detection results are corrected based on the visual prototype features to obtain the target detection results for the target category.

[0005] Optionally, the prior knowledge also includes a cross-space alignment projection matrix; Before correcting the candidate detection results based on the visual prototype features to obtain the target detection results for the target category, the method further includes: Based on the cross-space alignment projection matrix, the visual prototype features are mapped to the feature space used by the first detection model to obtain the mapped visual prototype features for the execution steps.

[0006] Optionally, the candidate detection results include candidate confidence scores and candidate visual features for at least one object; the step of correcting the candidate detection results based on the visual prototype features to obtain the target detection results for the target category includes: Based on the candidate visual features, an adaptation correction score is determined for the target category; Based on the adaptation correction score and the visual prototype features, the candidate confidence is corrected to obtain the target confidence of the target category, and the target detection result includes the target confidence.

[0007] Optionally, determining the adaptation correction score for the target category based on the candidate visual features includes: Obtain the set of positive example features for the target category and the temporal position of each positive example feature; The temporal weight of each positive example feature in the positive example feature set is determined based on the temporal position of each positive example feature; A first similarity is determined based on the candidate visual features and each positive feature in the set of positive features; The adaptation correction score for the target category is determined based on the temporal weight of each positive feature in the positive feature set and the first similarity.

[0008] Optionally, the step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features to obtain the target confidence score of the target category includes: A second similarity is determined based on the candidate visual features and the visual prototype features; The target confidence level for the target category is determined based on the candidate confidence level, the adaptation correction score, and the second similarity.

[0009] Optionally, determining the target confidence level for the target category based on the candidate confidence level, the adaptation correction score, and the second similarity includes: Based on the number of positive example features, determine the first weight corresponding to the candidate confidence, the second weight corresponding to the adaptation correction score, and the third weight corresponding to the second similarity. The target confidence level for the target category is determined based on the first weight, the second weight, the third weight, the candidate confidence level, the adaptation correction score, and the first similarity.

[0010] Optionally, the step of correcting the candidate confidence based on the adaptation correction score and the visual prototype features includes: Among the objects included in the candidate detection results, those with a corresponding adaptation correction score greater than the first threshold are selected. For the top n objects with the highest corresponding adaptation correction scores among the selected objects, the candidate confidence scores are corrected based on the adaptation correction scores and the visual prototype features, where n is a preset positive integer; for other objects, the adaptation correction scores are used as the target confidence scores.

[0011] Optionally, the step of correcting the candidate confidence based on the adaptation correction score and the visual prototype features includes: If the number of positive features is greater than a preset threshold and the adaptation correction score is greater than a second threshold, the adaptation correction score is used as the target confidence level.

[0012] Optionally, after determining the adaptation correction score for the target category based on the candidate visual features, the method further includes: Obtain visual prototype features for non-target categories; A third similarity is determined based on the candidate visual features and the visual prototype features for non-target categories; If the third similarity is greater than the adaptation correction score, then the adaptation correction score is reduced to obtain the suppressed adaptation correction score. The step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features includes: The candidate confidence score is corrected based on the suppressed adaptation correction score and the visual prototype features.

[0013] Optionally, after determining the target confidence level for the target category, the method further includes: A dynamic detection threshold for the target category is determined based on the number of positive example features. If the target confidence level is greater than or equal to the dynamic detection threshold, the target detection result is determined to be the target category.

[0014] Optionally, the method further includes: The target detection result is obtained from the first image frame number that is a preset non-target category; The number of second image frames whose target detection results were not confirmed as a preset non-target category was obtained; The detection statistics of the preset non-target category are determined based on the first image frame number and the second image frame number; If the detection statistics of the preset non-target category meet the preset old category protection conditions, the dynamic detection threshold of the preset non-target category shall not be less than the preset lower limit threshold.

[0015] Optionally, obtaining the keyframes in the image frame includes: In response to a preset trigger condition, the key frame in the image frame is acquired; The preset triggering conditions include at least one of the following: Received a query for a predefined target category; The confidence level for the room type in the confidence level for the target category is less than the third threshold; The highest confidence level corresponding to a consecutive preset number of image frames is less than the fourth threshold.

[0016] Secondly, a target detection device is provided, applied to an edge device, the edge device being communicatively connected to a server, the device comprising: The image acquisition unit is configured to acquire image frames; An edge target detection unit is configured to perform target detection on the image frame using a first detection model to obtain candidate detection results for the target category; The prior knowledge injection unit is configured to acquire key frames in the image frame, send the key frames to the server, and acquire prior knowledge corresponding to the key frames generated by the server using the second detection model. The prior knowledge includes visual prototype features for the target category, and the visual prototype features are appearance features that can characterize the target category. The detection result correction unit is configured to correct the candidate detection results based on the visual prototype features to obtain the target detection result of the target category.

[0017] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0018] Fourthly, an electronic device is provided, comprising: One or more processors; And a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first aspects above.

[0019] Fifthly, a computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0020] According to the specific embodiments provided in this application, the following technical effects are disclosed: 1) This application introduces keyframe-based visual prototype features from the server to correct the candidate detection results of the edge device, enabling the edge device to maintain real-time detection using the local detection model while acquiring rich visual understanding capabilities from the server to correct the recognition accuracy of the target category, thereby achieving accurate real-time perception of the target category on the edge platform.

[0021] 2) This application performs spatial transformation on the visual prototype features by aligning the projection matrix across spaces, so that the features are in the same manifold as the features of the detection model of the edge device, thereby eliminating the distribution differences of heterogeneous feature spaces and improving the correction accuracy.

[0022] 3) By introducing an adaptation correction score calculated based on local candidate visual features, this application can capture the actual appearance distribution of the target in the current scene and combine it with the visual prototype features provided by the cloud. This integrates two complementary information, visual prototype features and actual appearance distribution, which significantly improves the accuracy and robustness of the target confidence in the face of environmental changes.

[0023] 4) This application adopts a completely non-parametric exponential weighted time-series prototype comparison method, which does not require matrix inversion and backpropagation. The calculation process is numerically stable and has extremely low latency. At the same time, the exponential decay weight makes the latest positive examples contribute more to the score. When the circular queue is full, it automatically covers the old samples, naturally realizing the adaptive update of online distribution, which is suitable for edge devices with limited computing power and memory.

[0024] 5) This application integrates three heterogeneous and complementary confidence sources—candidate confidence, adaptation correction score, and second similarity—to cross-validate candidate objects from three perspectives, significantly improving the reliability of the final detection confidence. At the same time, it can effectively suppress uncertain outputs caused by single sources due to changes in perspective or mismatches.

[0025] 6) This application dynamically adjusts the fusion weight by increasing the number of positive example features, thereby reducing the reliance on candidate confidence in the cold start stage when positive examples are scarce and strengthening visual prototype comparison to ensure caution. In the steady state stage when positive examples are abundant, the weight of the adaptation correction score is increased, giving full play to the local online adaptability and realizing a smooth transition of the detection strategy from caution to confidence, thus optimizing the accuracy of each stage.

[0026] 7) This application excludes low-scoring candidates through coarse screening gating and imposes a quota limit on the number of high-scoring candidates, thereby controlling the number of candidates that require high-cost visual prototype comparison to a very small range, which greatly reduces the overall computational latency of each frame and ensures that the correction process can always meet the requirements of real-time operation on edge devices.

[0027] 8) This application further reduces computational overhead by short-circuiting visual prototype comparison under steady-state high confidence conditions, enabling the system to run with the lowest latency when it has high confidence. The perception latency in the steady-state stage can even be lower than that in the cold start stage, thereby maximizing the real-time frame rate without sacrificing detection accuracy.

[0028] 9) This application uses visual prototype features of other categories to suppress negative examples of the adaptation correction score online. When the candidate feature has a stronger appearance response in the non-target category, the adaptation correction score of the current category is automatically reduced, thereby effectively distinguishing easily confused backgrounds from real targets, significantly reducing the false detection rate caused by similar texture backgrounds, and improving detection accuracy.

[0029] 10) This application associates the detection threshold with the amount of positive examples accumulated, so that the target category adopts a lower threshold in the early stage when positive examples are scarce to ensure detection and recall. As positive examples accumulate, the threshold is gradually increased to suppress false detection, thereby automatically realizing a smooth transition from cautious detection to high-confidence real-time detection without human intervention, balancing recall and precision.

[0030] 11) This application continuously monitors the detection stability of the old category and takes lower limit cutoff protection measures when the reliability of the old category changes, preventing its dynamic threshold from being excessively suppressed. This effectively prevents the catastrophic erosion of the detection capability of the old category by the learning of the new category and ensures the consistency of multi-category detection in long-term operation.

[0031] 12) This application sets up multiple event triggering mechanisms to initiate cloud requests only when the user explicitly queries, the scenario changes significantly, or local detection remains uncertain. This greatly reduces invalid cloud calls and network transmissions, while ensuring timely acquisition of prior knowledge at critical moments when it is truly needed, thus achieving an efficient balance between resource consumption and detection assistance needs.

[0032] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a system architecture diagram applicable to the embodiments of this application.

[0035] Figure 2 This is a flowchart of the target detection method provided in the embodiments of this application.

[0036] Figure 3 This is a schematic block diagram of a target detection device provided in an embodiment of this application.

[0037] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0039] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0041] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0042] Currently, existing technologies typically deploy object detection models based on predefined vocabularies locally, and then use these models to perform object detection. However, their detection capabilities are limited by the vocabulary, making it difficult to accurately identify unseen or finely granular rare objects.

[0043] In view of this, this application provides a new approach. To facilitate understanding of this application, the system architecture on which this application is based will first be described. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture may include: edge devices and servers, with the edge devices communicating with the servers.

[0044] Edge devices can be edge platforms such as NVIDIA Jetson Orin Nano for intelligent robots. Edge devices can perceive target categories described in any natural language in real time, such as detecting user-personalized fine-grained objects like "a doll in animation A" or "building blocks of model B".

[0045] Edge devices can interact with servers via a network. For example, a user can send data such as keyframes collected by a terminal device to a server; or, an edge device can request and obtain prior knowledge based on keyframes from a server.

[0046] The server can be a single server, a server cluster consisting of multiple servers, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.

[0047] It should be understood that Figure 1 The number of edge devices and servers shown is merely illustrative. Depending on implementation needs, there can be any number of edge devices and servers.

[0048] Figure 2 This is a flowchart of a target detection method provided in an embodiment of this application. The method can be... Figure 1 The edge devices in the system shown execute this. For example... Figure 2 As shown, the method may include the following steps: Step 201: Acquire image frames.

[0049] Step 203: Use the first detection model to perform target detection on the image frame and obtain candidate detection results for the target category.

[0050] Step 205: Obtain keyframes from the image frame, send the keyframes to the server, and obtain the prior knowledge corresponding to the keyframes generated by the server using the second detection model. The prior knowledge includes visual prototype features for the target category. The visual prototype features are appearance features that can characterize the target category.

[0051] Step 207: Correct the candidate detection results based on visual prototype features to obtain the target detection results of the target category.

[0052] As can be seen from the above process, this application introduces keyframe-based visual prototype features through the server to correct the candidate detection results of the edge device. This enables the edge device to maintain real-time detection using the local detection model while acquiring rich visual understanding capabilities from the server to correct the recognition accuracy of the target category, thereby achieving accurate real-time perception of the target category on the edge platform.

[0053] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments. It should be noted that the terms "first" and "second" used in this disclosure do not have limitations in terms of size, order, or quantity, but are only used to distinguish them by name. For example, "first detection model" and "second detection model" are used to distinguish the detection models by name.

[0054] First, the above step 201, namely "acquiring image frames", will be described in detail with reference to the embodiments.

[0055] Among them, an image frame can be a continuous sequence of image frames in a video stream.

[0056] For example, the robot can acquire image frames of the indoor environment in real time through an image acquisition unit, which can be a camera or other image sensor that can provide continuous image frames, and then transmit the acquired image frames to the robot's edge device.

[0057] The following describes step 203, namely "using the first detection model to perform target detection on the image frame and obtain candidate detection results of the target category", in detail with reference to the embodiments.

[0058] The first detection model refers to a lightweight model capable of real-time object detection, such as YOLOv11n-seg, YOLO-World, or other models with object detection capabilities. The candidate detection result refers to the initial detection result obtained from object detection on the image frame, which may include, but is not limited to, candidate bounding boxes, candidate visual features, and candidate confidence scores belonging to the target category. The target category refers to any target described by the user in natural language, such as personalized fine-grained items like an animated doll (A) or building blocks (B), or it can include object categories from standard datasets, such as tables and chairs. For example, if a user says, "Please get me the building blocks (B)," the target category could be "building blocks (B)."

[0059] For example, the first detection model configured for the edge device can be the YOLO-World-S model, and the image input resolution of the edge device can be... Edge devices employ TensorRT INT8 quantization and CUDA Graph capture. During initialization, the YOLO-World-S model pre-encodes category names from the offline vocabulary into text embedding matrices using a text encoder. During inference, the text embedding matrices are directly used to calculate visual-text similarity, eliminating the need to repeatedly run the text encoder. During inference, for each input image frame, YOLO-World-S can output candidate detection results at a fixed frequency. ; in, Indicates candidate test results. This represents the candidate box for the i-th object; This represents the candidate visual features of the i-th object. The candidate visual features can be the features output by the YOLO-World backbone layer, which can be directly extracted from the YOLO-World Neck layer without additional inference overhead. The candidate confidence score for the i-th object represents the visual-linguistic similarity between the object within the candidate box and the text description of the target category. This represents the total number of objects. An object is a detected object within an image frame, and each object corresponds to a candidate bounding box. Unlike the target category, which is a category index in the semantic dimension, the object category is a candidate bounding box index in the spatial dimension.

[0060] The following describes in detail step 205, namely, "acquiring key frames in an image frame, sending the key frames to a server, and acquiring prior knowledge corresponding to the key frames generated by the server using the second detection model. The prior knowledge includes visual prototype features for the target category, and the visual prototype features are appearance features that can characterize the target category," with reference to the embodiments.

[0061] Keyframes can be the image frames with the highest clarity in a continuous sequence of image frames. For example, the image frame with the largest Laplacian variance within the current sliding window (such as the most recent 10 frames) can be selected as the keyframe. It is understood that this invention does not upload all image frames to the server, but rather a small number of keyframes, reducing the impact of network latency on target detection.

[0062] Compared to the first detection model, the second detection model has stronger visual semantic understanding capabilities. It can generate prior knowledge based on keyframes to correct candidate detection results for edge devices. For example, the second detection model can be a large visual language model such as GPT-4V or Qwen-VL. Prior knowledge refers to structured information provided by the server in response to edge device requests to enhance edge-side detection capabilities, and may include, but is not limited to, visual prototype features.

[0063] Visual prototype features refer to embedded vectors that can characterize the typical appearance of a target category. For example, by cropping keyframes, an image cropping block containing the target is obtained. The visual prototype embedding, i.e., visual prototype features, can be extracted from the image cropping block by the CLIP visual encoder of the second detection model.

[0064] To avoid frequent and unnecessary network communication and computational resource consumption, this invention can initiate the acquisition of prior knowledge from the server under preset trigger conditions. One possible implementation is to acquire keyframes from image frames in response to preset trigger conditions; wherein the preset trigger conditions include at least one of the following: receiving a statement querying a preset target category; the confidence level of room type for the target category being less than a third threshold; and the highest confidence level corresponding to a consecutive preset number of image frames being less than a fourth threshold.

[0065] The preset target category can be any target described in natural language, such as A anime doll or B type building blocks.

[0066] For example, when a user says "Find my A anime doll" via voice; or when the robot enters a new room and the confidence level of the first detection model for the room type is below 0.3; or when the highest confidence level of the first detection model is below 0.2 within 5 consecutive frames, keyframes in the image frame are acquired, downsampled to 720×480, compressed using JPEG encoding, and then sent to the server via ROS2 DDS bridging or HTTP / 2 protocol. The server can then use a large visual language model to recognize the keyframes and obtain structured prior knowledge. Furthermore, the structured prior knowledge can be: ; in, It represents the target category. A collection of text descriptions, including the main description and Synonymous descriptions include StellaLou plush doll, purple rabbit plush toy, StellaLou plush, etc. , which represents visual prototype features; It represents the target category. Normalized bounding box coordinates in keyframes are used for local scale priors; This indicates that the second detection model is effective against the target. Confidence level of identification; , which represents the cross-space alignment projection matrix used to map visual prototype features to the feature space of the first detection model.

[0067] The text description set can be dynamically injected into the offline vocabulary of the first detection model to improve its ability to detect target categories. Specifically, the edge device can... The text description is input into the CLIP (Contrastive Language-Image Pre-Training) text encoder to obtain encoded text description features. The mean of the encoded text description features is injected into the offline vocabulary of the first detection model to enrich the vocabulary, thereby improving the target detection capability of the first detection model in the device.

[0068] The following describes step 207, namely "correcting the candidate detection results based on visual prototype features to obtain the target detection results of the target category," in detail with reference to the embodiments.

[0069] The target detection result is the corrected detection result, which may include, but is not limited to, the detection box and the target confidence score for the target category.

[0070] Specifically, after acquiring the visual prototype features, the edge device can modify the candidate bounding boxes and / or candidate confidence in the candidate detection results based on the visual prototype features, such as deleting erroneous candidate bounding boxes or increasing candidate confidence, so as to obtain the final target detection result for the target category.

[0071] To further optimize the accuracy of cross-spatial feature matching and solve the problem of mismatch in heterogeneous feature spaces, the present invention provides the following preferred solutions.

[0072] As one possible approach, prior knowledge also includes a cross-spatial alignment projection matrix. Before the step of correcting candidate detection results based on visual prototype features, the method further includes: mapping the visual prototype features to the feature space used by the first detection model based on the cross-spatial alignment projection matrix, obtaining the mapped visual prototype features for execution.

[0073] It is understandable that the candidate visual features in the candidate detection results output by the first detection model have an embedding space biased towards target boundary and scale discrimination, while the visual prototype features provided by the server have an embedding space biased towards semantic and attribute alignment. Directly correcting the candidate detection results based on the visual prototype features results in spatial mismatch and lacks geometric interpretability. Therefore, this invention introduces a cross-spatial alignment projection matrix. The cross-spatial alignment projection matrix can be pre-trained on the server side using contrastive loss, for example, using InfoNCE loss, to align the visual prototype features with the candidate visual features in the candidate detection results in the projected space. After training, the cross-spatial alignment projection matrix is ​​distributed as a constant weight to the edge device and frozen for local use. Specifically, the mapping formula is expressed as: ; in, Represents visual prototype features. Represents the cross-space alignment projection matrix. This represents the mapped visual prototype features, i.e., the aligned visual prototype features. After the visual prototype features are aligned with the candidate visual features, they are in the same feature manifold space, providing a calibration basis for subsequent correction of candidate detection results.

[0074] The InfoNCE loss training for the cross-spatial alignment projection matrix is ​​independent of the second detection model. The specific training process includes: acquiring the COCO / LVIS dataset; extracting visual features v from images in the dataset using the CLIP visual encoder; and extracting detection features f using the YOLO-World backbone. The InfoNCE loss is then trained based on the visual and detection features to obtain the trained cross-spatial alignment projection matrix P. The InfoNCE loss function is as follows: ; in, Indicates matching pairs within a batch. Indicates non-matching pairs within a batch. This represents the temperature coefficient.

[0075] It should be noted that the above design eliminates the need for online training of the projection matrix by edge devices, resulting in zero additional training overhead. Its advantage lies in eliminating the distribution differences in heterogeneous feature spaces, enabling direct and meaningful comparisons between visual prototype features and candidate detection results, thereby significantly improving the reliability and accuracy of the correction process.

[0076] As another possible approach, visual prototype features can be mapped using lightweight neural networks or normalization models to obtain visual prototype features aligned with candidate visual features in the candidate detection results.

[0077] Furthermore, a visual-text bimodal prior cache can be set up on the edge device. This cache can store prior entries for multiple target categories, which can be constructed based on received prior knowledge. It is expressed as follows: ; in, , indicating that The average of all text descriptions encoded by the CLIP text encoder is the unified text prior embedding. This represents the aligned visual prototype features, which are compatible with the YOLO feature space; This represents the positive example feature buffer, which is initially empty. This represents the initial detection threshold, used for target category determination.

[0078] Cache replacement strategy for visual-text bimodal prior cache: Maximum capacity of target category for visual-text bimodal prior cache ,when When this happens, old prior entries can be eliminated based on the least recent query, that is, the least queried prior entry in the cache can be deleted; at the same time, the frequency of occurrence of the target category corresponding to the old prior entry in the current environment can be monitored.

[0079] The steps for constructing the positive example feature buffer include: when a candidate box is first identified as a target... At that time, the candidate visual features corresponding to the candidate box are... Store the positive example feature buffer for this target category: ; in, This represents the number of positive features in the set of positive features, i.e., the cumulative number of positive examples. This represents the maximum number of positive features in the set of positive features. Indicates the first There are 10 positive example features. In this invention, the positive example feature buffer can be a circular queue. When the number of positive example features is greater than the maximum number, the latest positive example feature can overwrite the oldest positive example feature in the positive example feature buffer.

[0080] To improve robustness to online appearance fluctuations caused by changes in perspective, lighting, and motion blur in actual deployments, the present invention provides the following preferred solutions.

[0081] As one possible approach, the candidate detection result includes candidate confidence and candidate visual features of at least one object; the candidate detection result is corrected based on the visual prototype features to obtain the target detection result of the target category, specifically including: determining the adaptation correction score for the target category based on the candidate visual features; and correcting the candidate confidence based on the adaptation correction score and the visual prototype features to obtain the target confidence of the target category, wherein the target detection result includes the target confidence.

[0082] The adaptation correction score refers to a dynamic quantitative indicator used to correct candidate confidence, which can be calculated based on candidate visual features. Specifically, as one possible approach, a set of positive example features for the target category and the temporal position of each positive example feature can be obtained; the temporal weight of each positive example feature in the set of positive example features can be determined based on the temporal position of each positive example feature; a first similarity can be determined based on the candidate visual features and each positive example feature in the set of positive example features; and the adaptation correction score for the target category can be determined based on the temporal weight of each positive example feature in the set of positive example features and the first similarity.

[0083] Here, temporal position refers to the position of a positive example feature within the set of positive example features, such as the j-th positive example feature. Temporal weight refers to the weight of a positive example feature in the temporal dimension. The first similarity is the similarity between the candidate visual feature and the positive example feature, which can be cosine similarity or other similarity metrics.

[0084] For example, the fit correction score can be calculated through a fully nonparametric, exponentially weighted time-series prototype alignment, as shown in the following formula: ; ; in, This represents the temporal weight of the feature of the j-th positive example. This indicates the number of positive features in the set of positive features. Represent candidate visual features, This represents the feature of the j-th positive example. Indicates the adjusted score; .

[0085] It should be noted that the above calculation method supports online incremental updates: whenever a new positive example feature is added, the oldest sample is automatically pushed out of the circular queue. New samples, due to their higher temporal weight, contribute more to the current score, thus achieving adaptive updates of the distribution and automatic forgetting of the old distribution. By employing the above exponentially weighted temporal prototype comparison, this invention enables edge devices to adapt to temporal changes in the target's appearance in real time with extremely low storage and computational overhead, effectively supporting the accuracy requirements of subsequent confidence correction. Furthermore, the above calculation method only involves vector dot products and scalar accumulation, completely avoiding matrix inversion, thus preventing the condition number explosion problem caused by feature correlation in methods such as Ridge Regression.

[0086] Furthermore, by introducing an adaptation correction score calculated based on local candidate visual features, this application can capture the actual appearance distribution of the target in the current scene and combine it with the visual prototype features provided in the cloud. This integrates two complementary pieces of information: visual prototype features and actual appearance distribution, which significantly improves the accuracy and robustness of the target confidence in the face of environmental changes. This better solves the problem that relying solely on fixed visual prototype features is insufficient to cope with online appearance fluctuations.

[0087] Furthermore, in order to more effectively integrate cloud-based visual priors and local adaptation information and overcome the problem of insufficient reliability of a single confidence source, this application adopts a multi-source fusion strategy when correcting candidate confidence based on adaptation correction scores and visual prototype features.

[0088] As another possible approach, the step of correcting the candidate confidence based on the adaptation correction score and visual prototype features to obtain the target confidence may include: determining a second similarity based on the candidate visual features and visual prototype features; and determining the target confidence for the target category based on the candidate confidence, the adaptation correction score, and the second similarity.

[0089] The second similarity can be the similarity between the candidate visual features and the visual prototype features after cross-spatial alignment and projection, and it can be cosine similarity or other similarity. In some embodiments, the second similarity can be the visual prototype ReID (Re-identification) score.

[0090] For example, the formula for calculating the target confidence level can be: ; ; in, Indicates the candidate confidence level. This indicates the adjustment score. Indicates the second similarity. , as well as Indicates weight, Indicates the target confidence level. Represent candidate visual features, This represents the aligned visual prototype features.

[0091] It should be noted that the candidate confidence score, fit correction score, and second similarity score validate the target category from three perspectives: visual-linguistic similarity, online visual distribution, and visual prototype, respectively, thus complementing each other's limitations. The candidate confidence score responds well to common categories but has low confidence for rare, fine-grained categories; the fit correction score is good at capturing online appearance changes but is unusable during cold starts; the second similarity score has strong inter-class discrimination but is insensitive to viewpoint changes. The fusion of these three significantly improves the overall reliability of the target confidence score, effectively suppressing false positives or false negatives caused by a single source.

[0092] To adapt to the changes in the reliability of different confidence sources during the transition of the target category from cold start to steady state, this application proposes the following preferred scheme.

[0093] As one possible approach, the step of determining the target confidence based on candidate confidence, adaptation correction score, and second similarity may further include: determining a first weight corresponding to the candidate confidence, a second weight corresponding to the adaptation correction score, and a third weight corresponding to the second similarity based on the number of positive example features; and determining the target confidence for the target category based on the first weight, second weight, third weight, candidate confidence, adaptation correction score, and second similarity.

[0094] Among them, the number of positive features It can reflect the sufficiency of the target category being confirmed and observed locally on the edge device.

[0095] For example, during the cold start phase ( <8), reduce the impact The dependency can be set. ,set up Improve the understanding Dependencies, settings During the warm start-up phase ( Three-source equalization can be configured. , , , It takes effect immediately. During the steady-state phase ( ), can improve Set the corresponding weights , , Fully trust local adaptation. It has been downgraded to an auxiliary verification.

[0096] It should be noted that this application, by dynamically allocating weights, can achieve a smooth transition from cautiously relying on cloud-based visual prototypes to confidently relying on local adaptations, effectively addressing changes in the reliability of confidence sources at different deployment stages, and further improving the rationality and overall accuracy of the detection strategy.

[0097] To control the computational cost of the correction step and ensure real-time performance when there are multiple candidate boxes, this application proposes the following preferred solution.

[0098] One possible approach is to adjust the candidate confidence based on the adaptation correction score and visual prototype features, including: filtering objects whose corresponding adaptation correction score is greater than a first threshold from the objects included in the candidate detection results; adjusting the candidate confidence based on the adaptation correction score and visual prototype features for the top n objects in the filtered objects, where n is a preset positive integer; and using the adaptation correction score as the target confidence for other objects.

[0099] Here, an object refers to a candidate bounding box in an image frame. Each candidate bounding box may have corresponding adaptation correction scores, visual prototype features, candidate confidence, and other information.

[0100] For example, coarse screening can be performed based on the adaptation correction score: a first threshold is set. When the candidate box corresponds to At that time, the candidate box corresponding to Perform subsequent confidence level adjustments; Below Candidate boxes that pass the initial screening are discarded to avoid unnecessary computation. Furthermore, if more than three candidate boxes pass the initial screening, they are further processed according to... The top 3 candidate boxes are sorted and then subjected to subsequent confidence adjustments, i.e., the second similarity is calculated and subsequent weighted fusion is performed to obtain the target confidence score; the candidate boxes other than the top 3 are then... As the final target confidence level.

[0101] Understandably, an object is considered to have a high probability of belonging to the target category and is worthwhile for more expensive visual prototype comparison and fusion calculations only when its adaptation correction score is higher than the first threshold; candidate boxes below the first threshold are discarded directly. Subsequently, to prevent a surge in computation due to an excessive number of high-scoring candidate boxes, at most the top n candidates for each category in each frame are selected for the complete correction process, i.e., performing the second similarity calculation and dynamic weighted fusion. This hybrid strategy significantly reduces the number of candidates requiring high-cost computation, making the overall computational latency controllable and maintaining real-time performance even in high-density scenes.

[0102] As another feasible approach, a short-circuit mechanism can be further introduced during the steady-state phase. Specifically, the candidate confidence is modified based on the adaptation correction score and visual prototype features, including: when the number of positive features is greater than a preset threshold and the adaptation correction score is greater than a second threshold, the adaptation correction score is used as the target confidence.

[0103] For example, when the number of positive examples in a target category is greater than or equal to 16 and the adaptation correction score is greater than 0.5, the steps of second similarity calculation and dynamic weighted fusion can be skipped, and the adaptation correction score can be directly used as the final target confidence.

[0104] It should be noted that this application further reduces computational overhead by short-circuiting visual prototype comparison under steady-state high confidence conditions, enabling the system to run with the lowest latency when it has high confidence. The perception latency in the steady-state stage can even be lower than that in the cold start stage, thereby maximizing the real-time frame rate without sacrificing detection accuracy.

[0105] To reduce the possibility of misidentification of background regions similar to the target texture, an online negative example suppression mechanism is also provided. This application proposes the following preferred solutions.

[0106] As one possible approach, after determining the adaptation correction score for the target category based on candidate visual features, the method further includes: obtaining visual prototype features for non-target categories; determining a third similarity based on the candidate visual features and the visual prototype features for non-target categories; if the third similarity is greater than the adaptation correction score, reducing the adaptation correction score to obtain a suppressed adaptation correction score; and correcting the candidate confidence based on the adaptation correction score and the visual prototype features, including: correcting the candidate confidence based on the suppressed adaptation correction score and the visual prototype features.

[0107] Here, non-target categories refer to target categories other than the target category in the visual-text bimodal prior cache. For example, the visual-text bimodal prior cache contains target category k and non-target categories. The third similarity refers to the similarity between candidate visual features and visual prototype features of non-target categories, which can be cosine similarity or other similarity metrics.

[0108] For example, if candidate visual features Non-target categories Visual prototype features The cosine similarity is higher than that of the target class k. This indicates that the object's appearance is more similar to that of the non-target category. Then Multiply by the suppression attenuation coefficient Thus obtaining the suppressed And then the suppressed Confidence correction is performed together with visual prototype features, where settings can be configured. .

[0109] It should be noted that this application uses visual prototype features of other categories to suppress negative examples of the adaptation correction score online. When the candidate feature has a stronger appearance response in a non-target category, the adaptation correction score of the current category is automatically reduced, thereby effectively distinguishing easily confused backgrounds from real targets, significantly reducing the false detection rate caused by similar texture backgrounds, and improving detection accuracy.

[0110] After obtaining the target confidence level, the final detection result requires threshold comparison. To balance the high recall requirement in the cold start phase and the high accuracy requirement in the steady-state phase, this invention can employ a dynamic detection threshold. This application proposes the following preferred solution. As one possible implementation, after obtaining the target confidence level for the target category, the method further includes: determining a dynamic detection threshold for the target category based on the number of positive example features; and determining the target detection result as the target category when the target confidence level is greater than or equal to the dynamic detection threshold.

[0111] The dynamic detection threshold can be a detection threshold determined based on the number of positive example features. For example, the formula for determining the dynamic detection threshold can be: ; in, Indicates the dynamic detection threshold; , representing the conservative threshold for cold start; , which represents the normal threshold after full adaptation; This is used to control the transition speed. It can be understood that when a new target category k is first injected into the visual-text bimodal prior cache, there are very few positive features, and the model lacks awareness of its visual distribution. This invention can control the amount of positive examples accumulated. Dynamic adjustment , <8 stages of extreme caution, Build confidence quickly in stages. It then enters a steady state.

[0112] Furthermore, the present invention can also introduce IoU (Intersection over Union) into the dynamic detection threshold to determine the target category.

[0113] For example, candidate boxes Identified as the target category The final output is if and only if: ; in, Indicate target category Normalized bounding box coordinates in the keyframe.

[0114] In some embodiments, if multiple target categories all meet the dynamic detection threshold for the same candidate box, the threshold can be selected from among the multiple target categories. The largest one; if conflicts still exist, the most recently queried target category in the visual-text bimodal prior cache is selected as the final target category of the candidate box.

[0115] It should be noted that this application links the detection threshold with the accumulation of positive examples, so that a lower threshold is used in the early stage when positive examples are scarce in the target category to ensure detection and recall. As positive examples accumulate, the threshold is gradually increased to suppress false detection. Thus, a smooth transition from cautious detection to high-confidence real-time detection is automatically achieved without human intervention, balancing recall and precision.

[0116] To prevent the catastrophic impact of new category learning on the detection capabilities of older categories in a long-term multi-category operating environment, this application also provides protection for the stability of older categories. This application proposes the following preferred solution. As one possible implementation, the method further includes: obtaining a first number of image frames where the target detection result is a preset non-target category; obtaining a second number of image frames where the target detection result is not confirmed as a preset non-target category; determining the detection statistics of the preset non-target category based on the first and second image frame numbers; and ensuring that the dynamic detection threshold of the preset non-target category is not less than a preset lower limit threshold, provided that the detection statistics of the preset non-target category meets the preset older category protection conditions.

[0117] Among them, the preset non-target category is the category that is opposite to the new target category k, that is, the preset non-target category can be the old category.

[0118] For example, determine each old category Detection statistics for the last 10 seconds: ; in, Indicates the first image frame number. Indicates the second image frame number. This represents the pre-defined detection statistics for non-target categories. The object detection results from the most recent 300 frames (10 seconds @ 30 FPS) can be classified as categories. The number of frames; for Furthermore, the target detection results with IoU1>0.3 did not confirm the category. The frame number, IoU1 represents the candidate bounding box relative to the category. The crossover ratio of the Kalman prediction locations, Indicate category The candidate confidence level. If If the condition persists for 5 seconds, the old category protection is triggered: the protection threshold uses a lower limit truncation to avoid threshold collapse caused by superimposed on progressive confidence scheduling. The lower limit truncation protection formula is: ; ; In some embodiments, the category can be upgraded simultaneously. First weight If the confidence level reaches 0.4, the candidate confidence level is re-established and maintained for 3 seconds. If the value increases, the lower limit truncation protection will be lifted.

[0119] To facilitate understanding of the above methods, processes, and technical effects, application examples are provided below: In a room equipped with a service robot, a user says to the robot, "Please find the blue velvet cushion on the TV cabinet." Upon receiving the command, the robot captures image frames of the room environment and sends them to an edge device inside the robot. The edge device performs target detection on each frame using an open-vocabulary detection model. Triggered by the user's voice command, the robot determines that the current local detection results cannot reliably confirm the target and immediately uploads keyframes from the image frames to a cloud server. This allows the cloud server to obtain prior knowledge from a large visual language model. Prior knowledge includes visual prototype features, which encode the cushion's blue hue, velvet texture, and unique shape. The edge device can then use these visual prototype features to correct the detection results of the open-vocabulary detection model, resulting in a reliable detection outcome.

[0120] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0121] According to another embodiment, a target detection device is provided. Figure 3 A schematic block diagram of a target detection apparatus according to one embodiment is shown, the apparatus being disposed in Figure 1 Edge devices in the architecture shown. Figure 3 As shown, the target detection device 300 includes: an image acquisition unit 301, an edge target detection unit 302, a prior knowledge injection unit 303, and a detection result correction unit 304. It may further include a feature alignment unit 305 and an old category protection unit 306. The main functions of each component are as follows: The image acquisition unit 301 is configured to acquire image frames.

[0122] The edge target detection unit 302 is configured to perform target detection on the image frame using a first detection model to obtain candidate detection results for the target category.

[0123] The prior knowledge injection unit 303 is configured to acquire key frames in the image frame, send the key frames to the server, and acquire prior knowledge corresponding to the key frames generated by the server using the second detection model. The prior knowledge includes visual prototype features for the target category, and the visual prototype features are appearance features that can characterize the target category.

[0124] The detection result correction unit 304 is configured to correct the candidate detection results based on the visual prototype features to obtain the target detection result of the target category.

[0125] As one possible approach, the prior knowledge also includes a cross-space alignment projection matrix. The feature alignment unit 305 is configured to map the visual prototype features to the feature space used by the first detection model based on the cross-space alignment projection matrix, and obtain the mapped visual prototype features for the steps to be performed.

[0126] As one possible implementation, the detection result correction unit 304 includes: The adaptation correction score determination subunit is configured to determine an adaptation correction score for the target category based on the candidate visual features.

[0127] The target confidence determination subunit is configured to correct the candidate confidence based on the adaptation correction score and the visual prototype features to obtain the target confidence of the target category, and the target detection result includes the target confidence.

[0128] As one possible implementation, the adaptation correction score determination subunit can be specifically configured to: obtain a set of positive example features for the target category and the temporal position of each positive example feature; determine the temporal weight of each positive example feature in the set of positive example features based on the temporal position of each positive example feature; determine a first similarity based on the candidate visual features and each positive example feature in the set of positive example features; and determine an adaptation correction score for the target category based on the temporal weight of each positive example feature in the set of positive example features and the first similarity.

[0129] As one possible implementation, the target confidence determination subunit can be specifically configured to determine a second similarity based on the candidate visual features and the visual prototype features; and to determine a target confidence for the target category based on the candidate confidence, the adaptation correction score, and the second similarity.

[0130] As one possible implementation, the target confidence determination subunit can be further configured to determine, based on the number of positive example features, a first weight corresponding to the candidate confidence, a second weight corresponding to the adaptation correction score, and a third weight corresponding to the second similarity; and to determine the target confidence for the target category based on the first weight, the second weight, the third weight, the candidate confidence, the adaptation correction score, and the second similarity.

[0131] As one possible approach, the target confidence determination sub-unit can be further configured as follows: Among the objects included in the candidate detection results, those with a corresponding adaptation correction score greater than the first threshold are selected. For the top n objects with the highest corresponding adaptation correction scores among the selected objects, the candidate confidence scores are corrected based on the adaptation correction scores and the visual prototype features, where n is a preset positive integer; for other objects, the adaptation correction scores are used as the target confidence scores.

[0132] As one possible approach, the target confidence determination sub-unit can be further configured as follows: If the number of positive features is greater than a preset threshold and the adaptation correction score is greater than a second threshold, the adaptation correction score is used as the target confidence level.

[0133] As one possible implementation, the detection result correction unit 304 may further include an adaptation correction score suppression subunit, configured to acquire visual prototype features for non-target categories; determine a third similarity based on the candidate visual features and the visual prototype features for non-target categories; and if the third similarity is greater than the adaptation correction score, reduce the adaptation correction score to obtain a suppressed adaptation correction score.

[0134] Correspondingly, the target confidence determination subunit is configured to correct the candidate confidence based on the suppressed adaptation correction score and the visual prototype features.

[0135] As one possible implementation, the detection result correction unit 304 may further include a dynamic detection threshold detection subunit, configured to determine a dynamic detection threshold for the target category based on the number of positive example features; and to determine the target detection result as the target category if the target confidence is greater than or equal to the dynamic detection threshold.

[0136] As one possible implementation, the old category protection unit 306 is configured to: acquire a first number of image frames whose target detection result is a preset non-target category; acquire a second number of image frames whose target detection result is not confirmed as a preset non-target category; determine the detection statistics of the preset non-target category based on the first number of image frames and the second number of image frames; and, if the detection statistics of the preset non-target category meets the preset old category protection conditions, ensure that the dynamic detection threshold of the preset non-target category is not less than a preset lower limit threshold.

[0137] As one possible implementation method, the prior knowledge injection unit 303 can be specifically configured to acquire key frames in the image frame in response to a preset trigger condition; wherein the preset trigger condition includes at least one of the following: receiving a statement querying a preset target category; the confidence of room type in the confidence of the target category is less than a third threshold; the highest confidence corresponding to a preset number of consecutive image frames is less than a fourth threshold.

[0138] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The system and device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0140] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0141] And an electronic device, comprising: One or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0142] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0143] in, Figure 4 An exemplary architecture of an electronic device is shown, which may include a processor 410, a video display adapter 411, a disk drive 412, an input / output interface 413, a network interface 414, and a memory 420. The processor 410, video display adapter 411, disk drive 412, input / output interface 413, network interface 414, and memory 420 can communicate with each other via a communication bus 430.

[0144] The processor 410 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0145] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store the operating system 421 for controlling the operation of the electronic device 400, and the basic input / output system (BIOS) 422 for controlling the low-level operations of the electronic device 400. Additionally, it can store a web browser 423, a data storage management system 424, and a target detection device 300. The target detection device 300 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 420 and executed by the processor 410.

[0146] Input / output interface 413 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0147] Network interface 414 is used to connect a communication module (not shown in the figure) to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0148] Bus 430 includes a pathway for transmitting information between various components of the device, such as processor 410, video display adapter 411, disk drive 412, input / output interface 413, network interface 414, and memory 420.

[0149] It should be noted that although the above-described device only shows the processor 410, video display adapter 411, disk drive 412, input / output interface 413, network interface 414, memory 420, bus 430, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0150] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0151] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A target detection method, characterized in that, Applied to an edge device that communicates with a server, the method includes: Acquire image frames; The first detection model is used to perform target detection on the image frame to obtain candidate detection results of the target category. The candidate detection results include candidate confidence and candidate visual features of at least one object. Keyframes in the image frame are obtained and sent to the server. Prior knowledge corresponding to the keyframes generated by the server using the second detection model is obtained. The prior knowledge includes visual prototype features for the target category. The visual prototype features are appearance features that can characterize the target category. The candidate detection results are corrected based on the visual prototype features to obtain the target detection results for the target category; The step of correcting the candidate detection results based on the visual prototype features to obtain the target detection results for the target category includes: Based on the candidate visual features, an adaptation correction score is determined for the target category; Based on the adaptation correction score and the visual prototype features, the candidate confidence is corrected to obtain the target confidence of the target category, and the target detection result includes the target confidence. The step of determining the adaptation correction score for the target category based on the candidate visual features includes: Obtain the set of positive example features for the target category and the temporal position of each positive example feature; The temporal weight of each positive example feature in the positive example feature set is determined based on the temporal position of each positive example feature; A first similarity is determined based on the candidate visual features and each positive feature in the set of positive features; The adaptation correction score for the target category is determined based on the temporal weight of each positive feature in the positive feature set and the first similarity.

2. The method according to claim 1, characterized in that, The prior knowledge also includes a cross-space alignment projection matrix; Before correcting the candidate detection results based on the visual prototype features to obtain the target detection results for the target category, the method further includes: Based on the cross-space alignment projection matrix, the visual prototype features are mapped to the feature space used by the first detection model to obtain the mapped visual prototype features for the execution steps.

3. The method according to claim 1, characterized in that, The step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features to obtain the target confidence score of the target category includes: A second similarity is determined based on the candidate visual features and the visual prototype features; The target confidence level for the target category is determined based on the candidate confidence level, the adaptation correction score, and the second similarity.

4. The method according to claim 3, characterized in that, The step of determining the target confidence level for the target category based on the candidate confidence level, the adaptation correction score, and the second similarity includes: Based on the number of positive example features, determine the first weight corresponding to the candidate confidence, the second weight corresponding to the adaptation correction score, and the third weight corresponding to the second similarity. The target confidence level for the target category is determined based on the first weight, the second weight, the third weight, the candidate confidence level, the adaptation correction score, and the second similarity.

5. The method according to claim 1, characterized in that, The step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features includes: Among the objects included in the candidate detection results, those with a corresponding adaptation correction score greater than the first threshold are selected. For the top n objects with the highest corresponding adaptation correction scores among the selected objects, the candidate confidence scores are corrected based on the adaptation correction scores and the visual prototype features, where n is a preset positive integer; for other objects, the adaptation correction scores are used as the target confidence scores.

6. The method according to claim 1, characterized in that, The step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features includes: If the number of positive features is greater than a preset threshold and the adaptation correction score is greater than a second threshold, the adaptation correction score is used as the target confidence level.

7. The method according to claim 1, characterized in that, After determining the adaptation correction score for the target category based on the candidate visual features, the method further includes: Obtain visual prototype features for non-target categories; A third similarity is determined based on the candidate visual features and the visual prototype features for non-target categories; If the third similarity is greater than the adaptation correction score, then the adaptation correction score is reduced to obtain the suppressed adaptation correction score. The step of correcting the candidate confidence score based on the adaptation correction score and the visual prototype features includes: The candidate confidence score is corrected based on the suppressed adaptation correction score and the visual prototype features.

8. The method according to claim 1, characterized in that, After determining the target confidence level for the target category, the method further includes: A dynamic detection threshold for the target category is determined based on the number of positive example features. If the target confidence level is greater than or equal to the dynamic detection threshold, the target detection result is determined to be the target category.

9. The method according to claim 1, characterized in that, The method further includes: The target detection result is obtained from the first image frame number that is a preset non-target category; The number of second image frames whose target detection results were not confirmed as a preset non-target category was obtained; The detection statistics of the preset non-target category are determined based on the first image frame number and the second image frame number; When the detection statistics of the preset non-target category meet the preset old category protection conditions, the dynamic detection threshold of the preset non-target category is not less than the preset lower limit threshold. The preset non-target category is an old category, and the preset old category protection conditions are pre-set old category protection trigger conditions.

10. The method according to claim 1, characterized in that, The step of obtaining keyframes from the image frame includes: In response to a preset trigger condition, the key frame in the image frame is acquired; The preset triggering conditions include at least one of the following: Received a query for a predefined target category; For the confidence scores of the target category, the confidence score for room type is less than the third threshold; The highest confidence level corresponding to a consecutive preset number of image frames is less than the fourth threshold.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 10.

12. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 10.

13. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10.

Images