Target detection method and device, storage medium and computer device

Abstract

The application relates to the technical field of computer vision, and particularly provides a target detection method and device, a storage medium and a computer device, wherein the category feature data of basic categories and newly added categories can be recorded through a category feature library. In a detection process, the text vector of the category feature data is taken as a semantic index, the mean of the feature vectors is taken as a feature index, and searching is performed based on an initial detection category and a detection feature vector output by a target detection model, so that the final detection category can be determined from the multiple categories recorded in the category feature library, and target detection is realized. When a new category needs to be added, the category feature library only needs to be updated, the target detection model does not need to be retrained, and the model structure of the target detection model does not need to be modified. In this way, model parameter adjustment can be completely avoided, the detection accuracy of original categories can be ensured, the identification of newly added categories can be supported, and the core demand of dynamic incremental expansion can be adapted.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a target detection method, apparatus, storage medium and computer equipment. Background Technology

[0002] With the continuous development of machine learning technology, object detection has shown broad application prospects in many fields, such as autonomous driving, security monitoring, smart healthcare, and industrial automation. However, the detection categories of object detection models are limited to a fixed set of categories defined during the model training phase. New categories not defined during training cannot be identified at all, failing to meet the essential requirement of ubiquitous object detection. To add new detection categories, sample data for the new categories needs to be collected, and the object detection model needs to be fully retrained to learn the feature information of the new categories. However, full retraining inevitably leads to a catastrophic forgetting problem; after adding new categories, the model's detection accuracy for the original categories significantly decreases, failing to guarantee the cumulative effect of detection capabilities. Summary of the Invention

[0003] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiency that full retraining in the prior art inevitably leads to catastrophic forgetting problems, and that the detection accuracy of the model for the original categories significantly decreases after adding new categories, thus failing to guarantee the cumulative nature of detection capabilities.

[0004] In a first aspect, embodiments of this application provide a target detection method, including:

[0005] Obtain the image to be detected and a category feature library; wherein, the category feature library records category feature data for multiple categories, the category feature data includes text vectors of category texts and the mean of feature vectors determined based on at least two sample images, the multiple categories include base categories and incremental categories;

[0006] The image to be detected is input into the target detection model to obtain an initial detection category and a detection feature vector corresponding to the image to be detected; wherein, the target detection model is obtained by training the model based on the training data of the basic category;

[0007] Calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and calculate the feature similarity between the detected feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of the plurality of categories;

[0008] Based on the semantic similarity and feature similarity, a final detection category is determined among the multiple categories, and the category text of the final detection category is output.

[0009] In some embodiments, the category feature data further includes a covariance matrix; the step of recording the class feature data of the incremental category in the category feature library includes:

[0010] Obtain the category texts of M incremental categories and multiple sample images corresponding to the target incremental category; where M is a positive integer, and the target incremental category is any one of the M incremental categories;

[0011] Based on the feature extraction network of the target detection model, the incremental feature vector of each sample image is determined respectively;

[0012] Under the condition of satisfying the sample quality verification rules, the mean and covariance matrix of the feature vectors of the target incremental category are determined according to each of the incremental feature vectors corresponding to the target incremental category.

[0013] Under the condition that the merging verification rules are met, the distribution consistency score is determined based on the mean of the feature vectors of the M incremental categories, the covariance matrix of the M incremental categories, the mean of the feature vectors of each base category, and the covariance matrix of each base category.

[0014] If the distribution consistency score is lower than a preset score threshold, a first global mean is determined based on the mean of the feature vectors of each of the basic categories, a second global mean is determined based on the mean of the feature vectors of the M incremental categories, and the mean of the feature vectors of the target incremental category is aligned based on the first global mean and the second global mean to obtain an aligned mean.

[0015] Based on the alignment mean of the target incremental category, the covariance matrix of the target incremental category, and the text vector of the category text of the target incremental category, construct the category feature data of the target incremental category;

[0016] The category feature data of the target incremental category is recorded in the category feature library.

[0017] In some embodiments, the verification steps of the sample quality verification rule include:

[0018] Calculate the sample cohesion of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category;

[0019] If the sample cohesion is lower than the preset cohesion threshold, it is determined that the sample quality verification rule is not met, and a sample supplementary check message is pushed.

[0020] If the sample cohesion is not lower than the preset cohesion threshold, then the sample quality verification rule is satisfied.

[0021] In some embodiments, the step of determining the mean of the feature vectors of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category includes:

[0022] The average value of the initial vector is obtained by averaging the incremental feature vectors corresponding to the target incremental category.

[0023] Based on the initial vector mean and the covariance matrix of the target incremental category, determine the confidence coefficient of each incremental feature vector corresponding to the target incremental category;

[0024] Based on each confidence coefficient, the incremental feature vectors corresponding to the target incremental category are weighted and summed to obtain the mean of the feature vectors of the target incremental category.

[0025] In some embodiments, the verification steps of the merged verification rule include:

[0026] The sample consistency of the target incremental category is calculated based on the mean of the feature vectors of the target incremental category and each of the incremental feature vectors corresponding to the target incremental category.

[0027] If the sample consistency is lower than the preset consistency threshold, it is determined that the merging verification rule is not met, and a sample supplementary check message is pushed.

[0028] If the sample consistency is not lower than the preset consistency threshold, then the merge verification rule is satisfied.

[0029] In some embodiments, the step of determining the incremental feature vector for each sample image based on the feature extraction network of the target detection model includes:

[0030] Each of the sample images is input into the feature extraction network to obtain the feature mapping vector corresponding to each sample image;

[0031] The feature mapping corresponding to each sample image is input into the pre-trained global average pooling layer to obtain the incremental feature vector of each sample image.

[0032] In some embodiments, the step of determining the final detection category among the plurality of categories based on the semantic similarity and the feature similarity includes:

[0033] Based on the semantic similarity, the K categories with the highest semantic similarity among the plurality of categories are determined as target categories; where K is a positive integer;

[0034] The semantic similarity and feature similarity of each target category are weighted and summed to obtain the retrieval score for each target category;

[0035] The final detection category is determined based on K retrieval scores.

[0036] Secondly, embodiments of this application provide a target detection device, comprising:

[0037] The acquisition module is used to acquire the image to be detected and the category feature library; wherein, the category feature library records category feature data of multiple categories, the category feature data includes text vectors of category texts and the mean of feature vectors determined based on at least two sample images, and the multiple categories include base categories and incremental categories;

[0038] The model detection module is used to input the image to be detected into the target detection model to obtain an initial detection category and a detection feature vector corresponding to the image to be detected; wherein, the target detection model is obtained by training the model based on the training data of the basic category;

[0039] The similarity calculation module is used to calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and to calculate the feature similarity between the detection feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of the plurality of categories;

[0040] The final category determination module is used to determine the final detection category among the multiple categories based on the semantic similarity and the feature similarity, and output the category text of the final detection category.

[0041] Thirdly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the target detection method described in any of the above embodiments.

[0042] Fourthly, embodiments of this application provide a computer device, which includes: one or more processors, and a memory;

[0043] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the target detection method described in any of the above embodiments.

[0044] In the target detection methods, apparatuses, storage media, and computer devices provided in some embodiments of this application, category feature data of basic categories and newly added categories are recorded through a category feature library. During the detection process, this application uses the text vector of the category feature data as a semantic index and the mean of the feature vectors as a feature index. Based on the initial detection category and detection feature vector output by the target detection model, a retrieval is performed, thereby determining the final detection category from multiple categories recorded in the category feature library, achieving target detection. When a new category needs to be added, this application only needs to update the category feature library; there is no need to retrain the target detection model or modify its model structure. This completely avoids model parameter adjustments, ensuring both the detection accuracy of existing categories and supporting the recognition of new categories. This solves the problem of a significant decrease in the model's detection accuracy for existing categories after adding new categories, and the inability to guarantee the cumulative nature of detection capabilities, thus adapting to the core requirement of dynamic incremental expansion. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the target detection method in some embodiments;

[0047] Figure 2 This is a flowchart illustrating the step of recording incremental category class feature data in a category feature library in some embodiments;

[0048] Figure 3 This is a schematic diagram of the target detection device in some embodiments;

[0049] Figure 4 This is a diagram of the internal structure of a computer device in some embodiments. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this application all comply with the provisions of relevant laws and regulations, and are carried out without violating social morality or hindering public interests.

[0052] In some embodiments, such as Figure 1 As shown, this application provides a target detection method, including:

[0053] S102: Obtain the image to be detected and the category feature library; wherein, the category feature library records category feature data for multiple categories, the category feature data includes the text vector of the category text and the mean of the feature vector determined based on at least two sample images, and the multiple categories include the base category and the incremental category.

[0054] Specifically, the category feature library can record category feature data of (N+M) categories as the dimension. These (N+M) categories may include the N basic categories defined during the training phase of the object detection model, and the M incremental categories added after the object detection model is deployed according to actual application needs, which were not defined during the training phase. It can be understood that N and M are both positive integers, and their specific values ​​can be determined according to the actual situation. This application does not impose specific restrictions on them.

[0055] The category feature library can store category feature data for multiple categories using the same data structure. Each category's feature data can include the text vector of the category's text and the mean of the category's feature vectors. The mean of the feature vectors can be determined from at least two sample images corresponding to that category and is used to characterize the center of the image feature distribution for that category. The text vectors can be converted into fixed-dimensional vector representations of the category text using natural language processing techniques, such as word embeddings or pre-trained language models, to capture the semantic information of the category text.

[0056] For example, when the category feature library stores first category feature data corresponding to the category "cat" and second category feature data corresponding to the category "dog", the first category feature data includes the text vector corresponding to the text "cat" and the average feature vector determined based on multiple cat images, and the second category feature data may include the text vector corresponding to the text "dog" and the average feature vector determined based on multiple dog images.

[0057] It is understood that, in addition to text vectors and the mean of feature vectors, category feature data may include more data, the specific data of which can be determined according to the actual situation, and this application does not impose specific restrictions on this. In some examples, category feature data may also include spatial morphological information corresponding to this category, so as to combine spatial morphological information for detection and classification, thereby further improving detection accuracy.

[0058] Furthermore, spatial morphological information may include a set of segmentation masks. The segmentation mask set, which is derived from the segmentation and masking of sample images corresponding to this category, can be obtained based on the segmented mask images. For example, firstly, geometric features such as the area, perimeter, and aspect ratio of the bounding rectangle of the mask region can be calculated to quantify the overall shape; then, the contour of the mask region is obtained through edge detection algorithms, and Fourier descriptors are used to compress the contour shape and extract contour features to capture local morphology; finally, regional features such as the centroid coordinates and texture features of the mask are calculated to supplement the spatial distribution morphology.

[0059] In this step, at least one of several methods can be used to acquire the image to be detected, and this image can be used as the image to be detected. For example, in the field of industrial quality inspection, image acquisition equipment on the production line can be used to acquire product images at a preset acquisition frequency as the image to be detected; in the field of intelligent transportation, images of vehicles on the road can be acquired through traffic monitoring cameras as the image to be detected.

[0060] S104: Input the image to be detected into the object detection model to obtain the initial detection category and the detection feature vector corresponding to the image to be detected; wherein, the object detection model is obtained by training the model based on the training data of the basic category.

[0061] In this step, the object detection model refers to a model pre-trained using a model training dataset. It can be implemented based on any model structure, and this application does not impose specific restrictions on it; for example, it can be implemented using the YOLo model or the DINO model. The model training dataset includes multiple sets of training data for N basic categories. Each set of training data may include sample images and pre-generated label categories. After model training, the object detection model can detect the input images within the range of N basic categories and output the corresponding category.

[0062] After acquiring the image to be detected, this application can use the image to be detected as the input image of the object detection model. The object detection model performs feature mapping and object detection on the image to be detected, obtaining the initial detection category output by the object detection model and the detection feature vector output by the feature extraction network of the object detection model. It can be understood that the detection feature vector is a feature mapping of the image to be detected, and the image content can be represented using high-dimensional features.

[0063] S106: Calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and calculate the feature similarity between the detection feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of multiple categories.

[0064] In this step, for the (N+M) categories recorded in the category feature library, this application can obtain (N+M) semantic similarities based on the semantic similarity between the initial detection category output by the object detection model and each category in the category feature library in terms of category name. It is understood that this step can employ various methods to calculate semantic similarity, and this application does not impose specific limitations on this. For example, the cosine similarity between the word vector of the initial detection category and the text vector of the category text can be calculated to quantify the semantic association between the category components.

[0065] Furthermore, this application can also calculate the feature similarity between the image to be detected and the sample images corresponding to each category in the category feature library based on the detection feature vector of the image to be detected and the feature mean vector of each category in the category feature library, so as to quantify the degree of similarity between the image to be detected and the sample head image in terms of visual feature distribution. It is understood that this step can use one of a variety of methods to calculate feature similarity, and this application does not impose a specific limitation on it. For example, Mahalanobis distance probability score can be used to quantify the degree of similarity in visual feature distribution.

[0066] S108: Based on semantic similarity and feature similarity, determine the final detection category among multiple categories and output the category text of the final detection category.

[0067] In this step, semantic similarity is used as the semantic index, and the mean of the feature vectors is used as the feature index. The final detection category is determined by combining semantic and feature information. In some examples, this application can also pre-construct a semantic similarity matrix based on the text vectors of (N+M) categories as the semantic index, and construct a tree index structure based on the mean of the feature vectors of (N+M) categories as the feature index. Finally, a semantic-feature hybrid index is constructed by combining a weighted retrieval mechanism of semantic and feature information to ensure feature retrieval efficiency.

[0068] In some embodiments, to improve detection efficiency, S108 may include the following sub-steps:

[0069] Step A1: Based on the semantic similarity, determine the K categories with the highest semantic similarity among multiple categories as the target categories; where K is a positive integer;

[0070] Step A3: Calculate the weighted sum of semantic similarity and feature similarity for each target category to obtain the retrieval score for each target category;

[0071] Step A5: Determine the final detection category based on the K search scores.

[0072] Specifically, firstly, semantic indexing can be used to quickly select K categories with high semantic similarity from (N+M) categories as target categories. Then, the precise feature score (i.e., retrieval score) corresponding to the target category is calculated by combining the feature similarity of the target categories. For example, when the semantic similarity is S1 and the feature similarity is S2, the retrieval score S = a×S1 + (1-a)×S2.

[0073] After determining K retrieval scores, this application can use these scores to determine the final detection category among K target categories. For example, this application can use the target category corresponding to the top P highest retrieval scores as the final detection category, where P is a positive integer, which can be equal to or greater than 1. This mechanism takes into account both semantic relevance and visual consistency, and improves retrieval efficiency by 3-5 times compared to a single index, supporting rapid matching of large-scale feature libraries.

[0074] In this application, category feature data of basic categories and newly added categories are recorded through a category feature library. During the detection process, this application uses the text vector of the category feature data as a semantic index and the mean of the feature vectors as a feature index. Based on the initial detection category and detection feature vector output by the object detection model, a retrieval is performed, thereby determining the final detection category from multiple categories recorded in the category feature library, achieving object detection. When a new category needs to be added, this application only needs to update the category feature library; there is no need to retrain the object detection model or modify its model structure. This completely avoids model parameter adjustments, ensuring both the detection accuracy of existing categories and support for the recognition of new categories. This solves the problem of a significant decrease in the model's detection accuracy for existing categories after adding a new category, and the inability to guarantee the cumulative nature of detection capabilities, thus meeting the core requirement of dynamic incremental expansion.

[0075] In some embodiments, in the initial stage, the category feature library may only record category data for N basic categories, and this category feature library may be denoted as... As practical applications evolve, this application needs to expand the detection categories for target detection. Therefore, it is necessary to record the category feature data of M incremental categories in the category feature library to facilitate the detection of (N+M) categories of targets.

[0076] This forms the initial foundation for achieving object detection, and it can be constructed in any way; this application does not impose any specific restrictions. In some embodiments, in order to achieve... The feature construction lays the foundation for subsequent dynamic incremental category expansion and reduces the difficulty and cost of sample collection. This application can construct the feature through the following steps. :

[0077] Step B1: Obtain multiple original sample images for each base category and the label category for each original sample image.

[0078] This step supports multi-channel image input and can accept images in different formats and label categories in different annotation formats. The image format can be one of JPG, PNG, BMP, etc., and the annotation format can be one of VOC, COCO, YOLO, etc. It is understood that the number of original sample images corresponding to each basic category can be determined according to the actual situation, and this application does not impose specific limitations on this. For example, to reduce the difficulty of sample acquisition, each basic category can correspond to 3 to 5 original sample images.

[0079] Step B3: Perform image preprocessing on each original sample image to obtain a preprocessed image.

[0080] In this step, after receiving the original sample image, it can be preprocessed using preprocessing methods such as anomaly detection, transform enhancement, and normalization.

[0081] The anomaly detection can employ validation rules such as image integrity, annotation validity, and category consistency to filter invalid data. This application uses the general segmentation model SAM2 to automatically convert simple bounding box annotations into high-precision segmentation masks, and performs image transformation enhancement processing on 3-5 original sample images for each category. Processing methods can include geometric transformation enhancement and photometric transformation enhancement. Geometric transformation enhancement can include random rotation (-30°~30°), scaling (0.8~1.2 times), flipping, etc., while photometric transformation enhancement can include brightness (±20%), contrast (±20%), color shift (±10%), etc. After mask processing, this application can eliminate abnormal annotations through spatial consistency verification between the bounding box and the segmentation mask.

[0082] Subsequently, this application can perform data standardization processing. During the standardization process, the original sample image I can first be normalized in size, scaling the image size to a standard dimension. H is the preset image height, and W is the preset image width. Then, this application can process standard-sized images... Pixel normalization is performed using the following formula:

[0083]

[0084] in, and These are the pre-computed global pixel mean and pixel standard deviation for large-scale image datasets (e.g., model training datasets). The sample image after pixel normalization.

[0085] Step B5: Determine the basic feature vector for each preprocessed image based on the feature extraction network of the object detection model.

[0086] In this step, the feature extraction network is pre-trained using a large-scale model training dataset, capable of extracting universal visual features F from images of any category. f represents the feature extraction network, and C represents the number of feature channels.

[0087] Furthermore, this application can also input the general visual feature F into the global average pooling layer and obtain the feature vector output by the global average pooling layer, which is used as the image feature vector (i.e., the basic feature vector) of the preprocessed image. For example, for the first feature vector in the basic library... There are 3 basic categories, and their basic feature vector sets are: ,in , This represents the k-th basic category corresponding to the th basic category. Sample images, For the kth basic category, the corresponding _th The basic feature vector of a sample image This represents the total number of sample images for the k-th basic category. Represents the k-th basic category. The feature vector of a sample image.

[0088] Step B7: Calculate the feature mean vector and covariance matrix of each basic category based on the multiple basic feature vectors corresponding to each basic category.

[0089] For example, for the k-th base category, its feature mean vector can be calculated according to the following expression. Covariance Matrix :

[0090]

[0091]

[0092] Step B9: Calculate the text vector corresponding to the category text of each basic category, and generate the segmentation mask set and its morphological features corresponding to each basic category based on multiple original sample images.

[0093] For a detailed explanation of this step, please refer to the above embodiments; this application will not repeat it here.

[0094] Step B11: Generate category feature data for each base category based on its feature mean vector, covariance matrix, text vector, segmentation mask set, and morphological features, and construct a system based on the N category feature data. .

[0095] In this step, the feature mean vector and covariance matrix can be used as feature distribution parameters, the text vector can be used as semantic description information, and the segmentation mask set and its morphological features can be used as spatial morphological information. The category feature data can simultaneously record the feature distribution parameters, semantic description information, and spatial morphological information of the basic categories, and a basic category feature library is constructed based on N category feature data. .

[0096] Furthermore, in some examples, redundancy can be removed from multiple basic feature vectors of each basic category using algorithms such as spectral clustering. Duplicate or highly similar feature vectors are removed to reduce redundancy based on feature vector similarity within each category, thus simplifying the features. Additionally, the category feature data may also include a set of redundancy-removed basic feature vectors. While reducing the size of the feature library, we ensure the diversity and distribution characteristics of each basic category, and construct a basic feature library containing multi-level information.

[0097] When it is necessary to expand the detection categories of target detection, this application can record the category feature data of M incremental categories into the category feature library through a dynamic incremental category expansion mechanism, so as to realize the target detection of incremental categories through the category feature library.

[0098] In some embodiments, the category feature data for each category may further include a covariance matrix determined based on multiple sample images corresponding to that category. For example... Figure 2 As shown, the steps for recording incremental category class feature data in the category feature library may include:

[0099] S202: Obtain the category text of M incremental categories and multiple sample images corresponding to the target incremental category; where M is a positive integer and the target incremental category is any one of the M incremental categories.

[0100] In this step, the category text of the incremental category to be added and the sample image corresponding to each incremental category can be obtained in any way, and this application does not impose any specific restrictions on this.

[0101] In some examples, this application can use an incremental category data processing channel to process sample images of incremental categories. This incremental category data processing channel is independent of the basic category data processing channel, which can physically isolate incremental data from basic data and avoid interference with historical data. (Sample images for incremental categories) This application can use the same data validation rules as the basic category data for validation, and use the same scale normalization and pixel standardization process as the basic library for image processing to obtain the standardized image. and with As a sample image.

[0102] For detailed instructions on the data verification and standardization process, please refer to the description of step B3 above, which will not be repeated here.

[0103] S204: Based on the feature extraction network of the object detection model, determine the incremental feature vector for each sample image.

[0104] In this step, to ensure the consistency between the incremental category and the base category in the feature space, this application determines the image feature vector of each sample image based on the feature extraction network of the target detection model, thus obtaining the incremental feature vector.

[0105] In some examples, S204 may include the following sub-steps:

[0106] S2041: Input each sample image into the feature extraction network to obtain the feature mapping vector corresponding to each sample image;

[0107] S2043: Input the feature mapping corresponding to each sample image into the pre-trained global average pooling layer to obtain the incremental feature vector of each sample image.

[0108] For example, for sample images This application can extract the general visual features of this sample image through the feature extraction network f(). The general visual features are then input into a global average pooling layer to obtain sample images. Incremental feature vector , i. .

[0109] In this example, a global average pooling layer is used to adapt to scenarios with few samples and dynamic expansion requirements. After processing by the global average pooling layer, this application can reduce dimensionality, retain the core features of each category, eliminate redundant spatial information, and avoid overfitting with few samples. Furthermore, this application can also ensure consistent feature scale through the global average pooling layer, guaranteeing the integrity of the basic library. With incremental library It features compatibility, supports distributed computing and library integration. Furthermore, it enhances robustness, smooths local disturbances, improves cross-scene accuracy, simplifies computation and storage, and adapts to large-scale expansion.

[0110] S206: Under the condition of satisfying the sample quality verification rules, determine the mean and covariance matrix of the feature vectors of the target incremental category based on the incremental feature vectors corresponding to each incremental category.

[0111] The sample quality verification rules are used to assess the quality of sample data in the incremental category, determining whether the number of samples in the incremental category is sufficient and whether the sample quality is adequate for target detection. It is understood that the specific content of the sample quality verification rules can be determined based on the actual situation. In some examples, the verification steps of the sample quality verification rules include:

[0112] S2061: Calculate the sample cohesion of the target incremental category based on the incremental feature vectors corresponding to each target incremental category;

[0113] S2063: If the sample cohesion is lower than the preset cohesion threshold, it is determined that the sample quality verification rule is not met, and a sample supplementary inspection message is pushed.

[0114] S2065: If the sample cohesion is not lower than the preset cohesion threshold, then the sample quality verification rule is satisfied.

[0115] This example uses feature cohesion calculation of similar samples to improve the accuracy of sample quality assessment, thereby enhancing the detection precision of subsequent target detection. For the k-th incremental category, this example calculates the sample cohesion of that incremental category based on the following expression. :

[0116]

[0117] in, Let k be the total number of sample images for the k-th incremental category. Let i be the incremental feature vector corresponding to the i-th sample image of the k-th incremental category. This is the incremental feature vector corresponding to the j-th sample image of the k-th incremental category.

[0118] If the calculated sample cohesion is lower than the preset cohesion threshold, it indicates that the sample quality is low, and therefore it can be determined that the sample quality verification rules are not met, and a sample supplementary check message is pushed. If the sample cohesion is greater than or equal to the preset cohesion threshold, it indicates that the sample quality is sufficient to meet the requirements of target detection, and therefore it can be determined that the sample quality verification rules are met.

[0119] Provided that the sample quality verification rules are met, this application can determine the mean and covariance matrix of the feature vectors of the target incremental category based on the feature vectors corresponding to each incremental category. For example, this application can determine the mean and covariance matrix of the feature vectors of the target incremental category according to the steps shown in step B7. As another example, this application can calculate the mean of the feature vectors of the kth incremental category according to the following expression. Covariance Matrix :

[0120]

[0121]

[0122] In some examples, considering that the quality of multiple sample images corresponding to the same incremental category may vary, with some sample images being more helpful for target detection and others less helpful, this example can introduce a confidence mechanism to determine the mean of the feature vector of the incremental target category, so as to reduce the influence of lower quality samples and thus further improve the detection accuracy of subsequent target detection.

[0123] In this example, the steps for determining the mean of the feature vectors of the target incremental category based on the incremental feature vectors corresponding to each target incremental category include:

[0124] The average value of the initial vector is obtained by averaging the incremental feature vectors corresponding to the target incremental category.

[0125] Based on the initial vector mean and the covariance matrix of the target incremental category, determine the confidence coefficient of each incremental feature vector corresponding to the target incremental category;

[0126] Based on each confidence coefficient, the weighted sum of each incremental feature vector corresponding to the target incremental category is obtained to obtain the mean of the feature vector of the target incremental category.

[0127] For example, for the k-th incremental category, this application can calculate the confidence coefficient of each incremental feature vector in the k-th incremental category according to the following expression:

[0128]

[0129] In the formula, The confidence coefficient is the value of the i-th sample image in the k-th incremental category. Let be the incremental feature vector corresponding to the i-th sample image of k incremental categories. This is the mean of the initial vector corresponding to the kth incremental category.

[0130] After determining each confidence coefficient, this application can calculate the mean of the feature vector of the k-th incremental category according to the following expression. :

[0131]

[0132] S208: Under the condition of satisfying the merging verification rules, determine the distribution consistency score based on the mean eigenvectors of the M incremental categories, the covariance matrix of the M incremental categories, the mean eigenvectors of each base category, and the covariance matrix of each base category.

[0133] Among them, merge validation rules can be used to verify the consistency of sample distributions for similar sample features. It is understood that the specific content of the merge validation rules can be determined based on the actual situation. In some examples, the validation steps of the merge validation rules include:

[0134] S2081: Calculate the sample consistency of the target incremental category based on the mean of the feature vectors of the target incremental category and the incremental feature vectors corresponding to the target incremental category.

[0135] S2083: If the sample consistency is lower than the preset consistency threshold, it is determined that the merge verification rule is not met, and a sample supplementary check message is pushed.

[0136] S2085: If the sample consistency is not lower than the preset consistency threshold, then the merge verification rule is satisfied.

[0137] For example, this application can calculate the sample consistency of the k-th incremental category based on the following expression. :

[0138]

[0139] exist If the sample distribution is below the preset consistency threshold, the k-th incremental category can be considered to be relatively scattered, resulting in poor overall sample quality. Introducing the features of this incremental category into the category feature library may lead to a decrease in detection accuracy. In this case, the merging verification rule is not met, and the feature vector of the k-th incremental category should not be introduced into the category feature library. If the feature distribution of the incremental category is greater than or equal to the preset consistency threshold, it can be considered that the feature distribution is relatively concentrated, and therefore subsequent merging operations can be performed. In this way, local anomalies can be filtered out, ensuring that all newly added categories meet the overall quality standards and ensuring the accuracy of subsequent target detection.

[0140] Under the condition of satisfying the merge verification rules, this application can verify the distributional consistency between the features of the incremental categories and the features of the base categories based on the mean eigenvectors of the M incremental categories, the covariance matrix of the M incremental categories, the mean eigenvectors of each base category, and the covariance matrix of each base category. This distributional consistency can be determined by the distributional consistency score. In some examples, the distributional consistency score comp can be calculated by the following expression:

[0141]

[0142] In the formula, For an incremental library, it can include the initial vector mean and covariance matrix of M incremental categories. In some examples, the incremental library... It can include category feature data for M incremental categories.

[0143] S210: If the distribution consistency score is lower than the preset score threshold, then determine the first global mean based on the mean of the feature vectors of each basic category, determine the second global mean based on the mean of the feature vectors of M incremental categories, and align the mean of the feature vectors of the target incremental category based on the first global mean and the second global mean to obtain the aligned mean.

[0144] In this step, when the distribution consistency score is lower than a preset score threshold, it indicates a distribution offset between the category features of the incremental category and the category features of the base category, such as center shift of similar features or scale inconsistency. Directly merging the category feature data of the incremental category into the category feature library would lead to inconsistent matching standards between the subsequent detection feature vectors and the category features of the two categories, potentially causing detection confusion or a significant drop in accuracy. Therefore, to ensure the effectiveness of feature similarity calculation during detection, this application can achieve feature space alignment by adjusting the distribution of incremental features, ensuring that the features of the incremental category and the features of the base category are in the same metric space.

[0145] During the alignment process, this application can determine a first global mean based on the mean of the feature vectors of N base categories, determine a second global mean based on the mean of the feature vectors of M incremental categories, and perform feature space alignment according to the following expression:

[0146]

[0147] In the formula, The aligned mean of the target incremental categories. The mean of the feature vectors of the target incremental category. The preset alignment coefficient, The first global mean, This is the second global mean.

[0148] Thus, it can be corrected. The distribution offset from the first global mean ensures alignment. It is compatible with the feature distribution of the base library, thereby improving the accuracy and compatibility of incremental category feature distribution and providing a reliable feature center benchmark for subsequent similarity calculation.

[0149] S212: Construct the category feature data of the target incremental category based on the alignment mean of the target incremental category, the covariance matrix of the target incremental category, and the text vector of the category text of the target incremental category.

[0150] After step S210, this application can construct the category feature data of the target incremental category based on the alignment mean of the target incremental category. The category feature data of the target incremental category and the category feature data of the base category can have the same data structure, laying the foundation for subsequent merging.

[0151] S214: Record the category feature data of the target incremental category in the category feature library.

[0152] In this step, the category feature data of the target incremental category can be concatenated into the category feature library to achieve efficient feature library merging. For example, a category dimension concatenation operator ⊕ can be defined to achieve efficient feature library merging: , , Therefore, direct concatenation can be performed according to the category dimension. The operational complexity of this operation is O(N+M), which is significantly lower than traditional retraining methods. D represents the model parameter scale, improving efficiency by more than 100 times and completely avoiding parameter adjustments, thus solving the catastrophic forgetting problem.

[0153] Furthermore, after merging, this application can build an index for the incremental categories, avoiding the overhead of rebuilding the entire index, and coordinate the update of metadata, including automatic recursion of category identifiers and expansion of the semantic association matrix. Finally, this application can also perform the following dual verification mechanism:

[0154] (1) Historical category verification: Use the original test set to verify that the historical category accuracy has not decreased (accuracy loss ≤1%).

[0155] (2) New category validation: Evaluate the detection accuracy of the new category (mAP ≥ 80%);

[0156] Experiments have shown that this application can efficiently expand the detection capability of everything. It only takes 1 hour to add 100 new categories, and each category only requires 3 to 5 sample images. This completely breaks through the expansion bottleneck of traditional technologies and provides a feasible technical path for realizing true detection of everything.

[0157] This application incorporates feature distribution parameters (mean / covariance) into the incremental learning system, enabling structured feature distribution modeling and improving the accuracy of small-sample recognition through statistical characteristic matching. Furthermore, this application utilizes an automatic verification mechanism based on feature distribution similarity to ensure seamless integration of incremental and original data, achieving dynamic compatibility verification.

[0158] The target detection device provided in the embodiments of this application is described below. The target detection device described below and the target detection method described above can be referred to in correspondence.

[0159] In some embodiments, such as Figure 3As shown, this application provides a target detection device 300, comprising:

[0160] The acquisition module 302 is used to acquire the image to be detected and the category feature library; wherein, the category feature library records category feature data of multiple categories, the category feature data includes text vectors of category texts and the mean of feature vectors determined based on at least two sample images, and the multiple categories include basic categories and incremental categories;

[0161] The model detection module 304 is used to input the image to be detected into the target detection model to obtain an initial detection category and a detection feature vector corresponding to the image to be detected; wherein, the target detection model is obtained by training the model based on the training data of the basic category;

[0162] The similarity calculation module 306 is used to calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and to calculate the feature similarity between the detection feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of the plurality of categories;

[0163] The final category determination module 308 is used to determine the final detection category among the plurality of categories based on the semantic similarity and the feature similarity, and output the category text of the final detection category.

[0164] In some embodiments, the category feature data further includes a covariance matrix. The target detection device 300 of this application also includes:

[0165] The incremental category data acquisition module is used to acquire the category text of M incremental categories and multiple sample images corresponding to the target incremental category; wherein, M is a positive integer, and the target incremental category is any one of the M incremental categories;

[0166] The incremental feature extraction module is used to determine the incremental feature vector of each sample image according to the feature extraction network of the target detection model.

[0167] The feature distribution parameter determination module is used to determine the mean and covariance matrix of the feature vectors of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category, provided that the sample quality verification rules are met.

[0168] The distribution consistency score calculation module is used to determine the distribution consistency score based on the mean eigenvectors of the M incremental categories, the covariance matrix of the M incremental categories, the mean eigenvectors of each base category, and the covariance matrix of each base category, provided that the merging verification rules are met.

[0169] The alignment mean determination module is used to determine a first global mean based on the mean of the feature vectors of each of the basic categories if the distribution consistency score is lower than a preset score threshold, determine a second global mean based on the mean of the feature vectors of M incremental categories, and align the mean of the feature vectors of the target incremental category based on the first global mean and the second global mean to obtain the alignment mean.

[0170] The category feature data construction module is used to construct the category feature data of the target incremental category based on the alignment mean of the target incremental category, the covariance matrix of the target incremental category, and the text vector of the category text of the target incremental category.

[0171] The recording module is used to record the category feature data of the target incremental category in the category feature library.

[0172] In some embodiments, the target detection device 300 of this application further includes a sample quality verification module. The sample quality verification module includes:

[0173] The cohesion calculation unit is used to calculate the sample cohesion of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category;

[0174] The first determining unit is used to determine that the sample quality verification rule is not met if the sample cohesion is lower than a preset cohesion threshold, and to push a sample supplementary inspection message.

[0175] The second determining unit is used to determine that the sample quality verification rule is met if the sample cohesion is not lower than a preset cohesion threshold.

[0176] In some embodiments, the feature distribution parameter determination module of this application includes:

[0177] The initial mean calculation unit is used to calculate the average of each incremental feature vector corresponding to the target incremental category to obtain the initial vector mean.

[0178] The confidence calculation tool is used to determine the confidence coefficient of each incremental feature vector corresponding to the target incremental category based on the mean of the initial vector and the covariance matrix of the target incremental category.

[0179] The weighted summation unit is used to perform weighted summation on each incremental feature vector corresponding to the target incremental category based on each confidence coefficient, so as to obtain the mean value of the feature vector of the target incremental category.

[0180] In some embodiments, the target detection device 300 of this application further includes a merging verification module. The merging verification module includes:

[0181] The sample consistency calculation unit is used to calculate the sample consistency of the target incremental category based on the mean of the feature vectors of the target incremental category and each of the incremental feature vectors corresponding to the target incremental category.

[0182] The third determining unit is used to determine that the merging verification rule is not met if the sample consistency is lower than a preset consistency threshold, and to push a sample supplementary check message.

[0183] The fourth determining unit is used to determine that the merging verification rule is satisfied if the consistency of the samples is not lower than a preset consistency threshold.

[0184] In some embodiments, the incremental feature extraction module of this application includes:

[0185] The mapping unit is used to input each of the sample images into the feature extraction network to obtain the feature mapping vector corresponding to each sample image;

[0186] The feature acquisition unit is used to input the feature mapping corresponding to each of the sample images into the pre-trained global average pooling layer to obtain the incremental feature vector of each sample image.

[0187] In some embodiments, the final category determination module 308 of this application includes:

[0188] The target category filtering unit is used to determine the K categories with the highest semantic similarity among the plurality of categories as target categories based on the semantic similarity of each category; where K is a positive integer;

[0189] The retrieval score calculation unit is used to perform weighted summation of semantic similarity and feature similarity for each target category to obtain the retrieval score for each target category.

[0190] The final category determination unit is used to determine the final detection category based on K retrieval scores.

[0191] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the target detection method as described in any embodiment.

[0192] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the target detection method as described in any embodiment.

[0193] Indicatively, Figure 4This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. In one example, the computer device can be a server. (Refer to...) Figure 4 The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions, such as application programs, that can be executed by the processing component 902. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the steps of the target detection method described in any of the above embodiments.

[0194] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0195] Those skilled in the art will understand that the internal structure of the computer device shown in this application is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0196] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0197] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0198] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target detection method, characterized in that, include: Obtain the image to be detected and a category feature library; wherein, the category feature library records category feature data for multiple categories, the category feature data includes text vectors of category texts and the mean of feature vectors determined based on at least two sample images, the multiple categories include base categories and incremental categories; The image to be detected is input into the target detection model to obtain an initial detection category and a detection feature vector corresponding to the image to be detected; wherein, the target detection model is obtained by training the model based on the training data of the basic category; Calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and calculate the feature similarity between the detected feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of the plurality of categories; Based on the semantic similarity and feature similarity, a final detection category is determined among the multiple categories, and the category text of the final detection category is output.

2. The method according to claim 1, characterized in that, The category feature data also includes a covariance matrix; the step of recording the incremental category's class feature data in the category feature library includes: Obtain the category texts of M incremental categories and multiple sample images corresponding to the target incremental category; where M is a positive integer, and the target incremental category is any one of the M incremental categories; Based on the feature extraction network of the target detection model, the incremental feature vector of each sample image is determined respectively; Under the condition of satisfying the sample quality verification rules, the mean and covariance matrix of the feature vectors of the target incremental category are determined according to each of the incremental feature vectors corresponding to the target incremental category. Under the condition that the merging verification rules are met, the distribution consistency score is determined based on the mean of the feature vectors of the M incremental categories, the covariance matrix of the M incremental categories, the mean of the feature vectors of each base category, and the covariance matrix of each base category. If the distribution consistency score is lower than a preset score threshold, a first global mean is determined based on the mean feature vectors of each of the basic categories, a second global mean is determined based on the mean feature vectors of the M incremental categories, and the mean feature vectors of the target incremental category are aligned based on the first global mean and the second global mean to obtain an aligned mean. Based on the alignment mean of the target incremental category, the covariance matrix of the target incremental category, and the text vector of the category text of the target incremental category, construct the category feature data of the target incremental category; The category feature data of the target incremental category is recorded in the category feature library.

3. The method according to claim 2, characterized in that, The verification steps of the sample quality verification rules include: Calculate the sample cohesion of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category; If the sample cohesion is lower than the preset cohesion threshold, it is determined that the sample quality verification rule is not met, and a sample supplementary check message is pushed. If the sample cohesion is not lower than the preset cohesion threshold, then the sample quality verification rule is satisfied.

4. The method according to claim 2, characterized in that, The step of determining the mean of the feature vectors of the target incremental category based on each of the incremental feature vectors corresponding to the target incremental category includes: The average value of the initial vector is obtained by averaging the incremental feature vectors corresponding to the target incremental category. Based on the initial vector mean and the covariance matrix of the target incremental category, determine the confidence coefficient of each incremental feature vector corresponding to the target incremental category; Based on each confidence coefficient, the incremental feature vectors corresponding to the target incremental category are weighted and summed to obtain the mean of the feature vectors of the target incremental category.

5. The method according to claim 4, characterized in that, The verification steps of the merged verification rule include: The sample consistency of the target incremental category is calculated based on the mean of the feature vectors of the target incremental category and each of the incremental feature vectors corresponding to the target incremental category. If the sample consistency is lower than the preset consistency threshold, it is determined that the merging verification rule is not met, and a sample supplementary check message is pushed. If the sample consistency is not lower than the preset consistency threshold, then the merge verification rule is satisfied.

6. The method according to claim 2, characterized in that, The step of determining the incremental feature vector for each sample image based on the feature extraction network of the target detection model includes: Each of the sample images is input into the feature extraction network to obtain the feature mapping vector corresponding to each sample image; The feature mapping corresponding to each sample image is input into the pre-trained global average pooling layer to obtain the incremental feature vector of each sample image.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the final detection category among the plurality of categories based on the semantic similarity and the feature similarity includes: Based on the semantic similarity, the K categories with the highest semantic similarity among the plurality of categories are determined as target categories; where K is a positive integer; The semantic similarity and feature similarity of each target category are weighted and summed to obtain the retrieval score for each target category; The final detection category is determined based on K retrieval scores.

8. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected and the category feature library; wherein, the category feature library records category feature data of multiple categories, the category feature data includes text vectors of category texts and the mean of feature vectors determined based on at least two sample images, and the multiple categories include base categories and incremental categories; The model detection module is used to input the image to be detected into the target detection model to obtain an initial detection category and a detection feature vector corresponding to the image to be detected; wherein, the target detection model is obtained by training the model based on the training data of the basic category; The similarity calculation module is used to calculate the semantic similarity between the text vectors of the initial detection category and the candidate category, and to calculate the feature similarity between the detection feature vector and the mean feature vector of the candidate category; wherein, the candidate category is any one of the plurality of categories; The final category determination module is used to determine the final detection category among the multiple categories based on the semantic similarity and the feature similarity, and output the category text of the final detection category.

9. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the target detection method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the target detection method as described in any one of claims 1 to 7.

Images