Target detection and model training method, device and equipment and storage medium

Abstract

The present disclosure provides a target detection and model training method, device and equipment, and a storage medium, relates to the technical field of artificial intelligence, specifically to the technical fields of computer vision, image processing, deep learning, etc., and can be applied to scenes such as OCR and smart government affairs. The target detection method comprises: performing feature extraction processing on a to-be-processed image to obtain image features of the to-be-processed image; determining position information of a target contained in the to-be-processed image based on the image features; performing embedding processing on the image features to obtain a feature embedding vector of the to-be-processed image; determining a correlation degree between the feature embedding vector and a class center embedding vector of at least one category which is constructed in advance; wherein the class center embedding vector is obtained by processing a classification image of a corresponding category; and determining category information of the target in the at least one category based on the correlation degree. The present disclosure can reduce the labeling amount and improve the classification accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically computer vision, image processing, deep learning and other technical fields, and can be applied to scenarios such as optical character recognition (OCR) and smart government affairs. In particular, it relates to a method, apparatus, device and storage medium for object detection and model training. Background Technology

[0002] Object detection refers to separating the target of interest from the background from a single image or a sequence of images, and determining the target's descriptive information (location information and category information). Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for object detection and model training.

[0004] According to one aspect of this disclosure, a target detection method is provided, comprising: performing feature extraction processing on an image to be processed to obtain image features of the image to be processed; determining the location information of a target contained in the image to be processed based on the image features; performing embedding processing on the image features to obtain a feature embedding vector of the image to be processed; determining the correlation between the feature embedding vector and a pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing a classified image of the corresponding category; and determining the category information of the target in the at least one category based on the correlation.

[0005] According to another aspect of this disclosure, a training method for an object detection model is provided. The model includes a feature extraction network. The method includes: acquiring training data, the training data including: sample images and classification images of at least one category, and true location information and true category information of an object contained in the sample images; using the feature extraction network to perform feature extraction processing on the sample images to obtain image features of the sample images; determining predicted location information of the object based on the image features; embedding the image features to obtain feature embedding vectors of the sample images; processing the classification images of the at least one category to obtain class center embedding vectors of at least one category; determining the correlation between the feature embedding vectors and the class center embedding vectors of the at least one category, and determining predicted category information of the object based on the correlation; constructing a loss function based on the predicted location information, the predicted category information, the true location information, and the true category information, and adjusting the network parameters of the feature extraction network based on the loss function.

[0006] According to another aspect of this disclosure, a target detection apparatus is provided, comprising: an extraction module for performing feature extraction processing on an image to be processed to obtain image features of the image to be processed; a localization module for determining the location information of a target contained in the image to be processed based on the image features; an embedding module for performing embedding processing on the image features to obtain a feature embedding vector of the image to be processed; a relevance module for determining the relevance between the feature embedding vector and a pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing a classification image of the corresponding category; and a classification module for determining the category information of the target in the at least one category based on the relevance.

[0007] According to another aspect of this disclosure, a training apparatus for an object detection model is provided. The model includes a feature extraction network. The apparatus includes: an acquisition module for acquiring training data, the training data including: sample images and at least one category of classification images, and true location information and true category information of an object contained in the sample images; the label data including: true location information and true category information of the object; an extraction module for performing feature extraction processing on the sample images using the feature extraction network to obtain image features of the sample images; a localization module for determining the predicted location information of the object based on the image features; an embedding module for performing embedding processing on the image features to obtain feature embedding vectors of the sample images; a determination module for processing the at least one category of classification images to obtain at least one category of class center embedding vectors; a classification module for determining the correlation between the feature embedding vectors and the class center embedding vectors of the at least one category, and determining the predicted category information of the object based on the correlation; and an adjustment module for constructing a loss function based on the predicted location information, the predicted category information, the true location information, and the true category information, and adjusting the network parameters of the feature extraction network based on the loss function.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the foregoing aspects.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any of the preceding aspects.

[0010] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to any of the preceding aspects.

[0011] According to the technical solution disclosed herein, annotation costs can be reduced and classification accuracy can be improved.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0015] Figure 2 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0016] Figure 3 This is a schematic diagram of the overall architecture of the target detection process provided according to the embodiments of this disclosure;

[0017] Figure 4 This is a schematic diagram according to the second embodiment of the present disclosure;

[0018] Figure 5 This is a schematic diagram according to the third embodiment of the present disclosure;

[0019] Figure 6 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0020] Figure 7 This is a schematic diagram of the overall architecture of the model training process provided according to the embodiments of this disclosure;

[0021] Figure 8 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0022] Figure 9 This is a schematic diagram according to the sixth embodiment of the present disclosure;

[0023] Figure 10 This is a schematic diagram of an electronic device used to implement the target detection method or target detection model training method of the embodiments of this disclosure. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In related technologies, a one-step method is used for target detection. Specifically, a target detection model is used to determine the target's location and category information simultaneously, meaning that localization and classification are performed concurrently.

[0026] The aforementioned method of simultaneous localization and classification requires full annotation, meaning precise annotation of both localized and classified targets in the training set, which incurs high annotation costs. This is especially true when augmenting long-tail data (data with infrequent occurrences), where images contain numerous non-long-tail targets. Performing full annotation for both non-long-tail and long-tail targets would undoubtedly be too costly. Given limited annotation costs, classification accuracy needs improvement.

[0027] To reduce the amount of labeling and improve classification accuracy, this disclosure provides the following embodiments.

[0028] Figure 1 Based on a schematic diagram of the first embodiment of this disclosure, this embodiment provides a target detection method, the method comprising:

[0029] 101. Perform feature extraction processing on the image to be processed to obtain the image features of the image to be processed.

[0030] 102. Based on the image features, determine the location information of the target contained in the image to be processed.

[0031] 103. The image features are embedded to obtain the feature embedding vector of the image to be processed.

[0032] 104. Determine the correlation between the feature embedding vector and the pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing the classification image of the corresponding category.

[0033] 105. Based on the relevance, determine the category information of the target in the at least one category.

[0034] In this embodiment, the target category information is determined based on the correlation between the feature embedding vector and the class center embedding vector. Therefore, if a new category is added, only the class center embedding vector corresponding to the new category needs to be constructed; full annotation of non-new category targets in the training samples is not required, thus reducing the amount of annotation. Furthermore, since the correlation is not limited by annotation costs, classification accuracy can be improved. In addition, the target's location and category information share image features. Compared to using two separate models to extract features corresponding to location and category information, joint optimization can be performed during training to improve the representational power of image features and enhance the accuracy of target detection.

[0035] Image features can be extracted based on preset rules or by using a feature extraction network. The feature extraction network can be, for example, a Convolutional Neural Network (CNN), specifically any of the following networks: Residual Network (ResNet) series, Vision Transformer (ViT) network, or Data-efficient Image Transformers (Deit) series.

[0036] After obtaining image features, these features can be input into the regression branch network. The output of the regression branch network is the location information of the target. The regression branch network can be a regression branch network of a general target detection model, and can specifically include fully connected layers.

[0037] In related technologies, for classification, the classification branch network of a typical object detection model includes a fully connected layer (classification layer). The number of output nodes of the classification layer is a fixed value, which is the same as the total number of categories to be identified. The category information of the target is obtained based on the output value of the output nodes.

[0038] However, adding a new category requires changing the model structure. For example, if the initial number of categories is C1 and the new category is C2, the number of output nodes in the classification layer needs to be changed from C1 to C2. Additionally, the model needs to be retrained, requiring new sample images containing the new category's target. These new sample images need full annotation of the target. Assuming the new category's target is a long-tail target (i.e., a target that appears infrequently), the new sample images typically contain a large number of non-long-tail targets in addition to long-tail targets. However, these non-long-tail targets also need full annotation (location and category), increasing the annotation cost.

[0039] In this embodiment, vector retrieval is used for classification, instead of a classification layer based on fixed output nodes.

[0040] Image features, also known as feature maps, are in matrix form. To perform vector retrieval, these matrix-form image features can first be converted into vector form. Specifically, an embedding layer can be used to embed the input image features. The output of the embedding layer is a vector, called the feature embedding vector. The embedding layer can be a fully connected (FC) layer.

[0041] The size of the vector can be a preset value. For example, if the size (dimension) of the vector is represented by N, and N = 512, then the feature embedding vector can be represented as a column vector of 512*1.

[0042] For vector retrieval, the aforementioned feature embedding vector is used as the query vector, and the search is performed among at least one pre-constructed candidate vector. In this embodiment, the class center embedding vector is used as the candidate vector.

[0043] A class embedding vector corresponds one-to-one with a class and can be obtained by processing the classification images of the corresponding class. Specifically, for each class, the classification images of that class can be obtained; the image features of each classification image of that class can be extracted and converted into a feature embedding vector; the average of the feature embedding vectors of each classification image of that class can be calculated to obtain the class embedding vector of that class.

[0044] A classification image is an image cropped to represent a target of a specific category. For example, if a sample image contains targets of multiple categories, such as milk and cookies, then the image cropped from the milk region can be used as the classification image for the milk category, and the image cropped from the cookie region can be used as the classification image for the cookie category.

[0045] Assuming the number of categories is M (M is a positive integer) and the size of each class core embedding vector is N, then the class core embedding vectors of the M categories can form a matrix of size N*M.

[0046] Vector retrieval can be based on relevance, which can be specifically similarity or distance. Similarity and relevance are positively correlated, meaning the greater the similarity, the greater the relevance; distance and relevance are inversely correlated, meaning the smaller the distance, the greater the relevance.

[0047] Specifically, the feature embedding vector can be normalized to obtain a normalized feature embedding vector; the class center embedding vector of the at least one category can be normalized to obtain a normalized class center embedding vector of the at least one category; the normalized class center embedding vector of the at least one category can be used as a weight vector to weight the feature embedding vector to obtain the relevance.

[0048] Assuming the normalized feature embedding vector is represented by X (e.g., 512*1 dimensional), and the normalized class center embedding vectors of M classes can form a 512*M dimensional matrix W, then the relevance = W T *X, W T It is the transpose of W. The correlation is an M-dimensional vector (M*1 dimension), where each element of the vector represents the correlation between the feature embedding vector and the class center embedding vector of each class.

[0049] After obtaining the relevance, the category information corresponding to the class center embedding vector with the highest relevance can be selected as the category information of the target. For example, the class center embedding vectors include: class center embedding vectors of category A, category B, and category C. Assuming that the relevance between the feature embedding vector of the image to be processed and the class center embedding vector of category A is the highest, then the category of the target in the image to be processed is determined to be category A.

[0050] Specifically, the relevance can be normalized to obtain the probability value of the target belonging to each of the at least one category; based on the probability value, the category information of the target is determined.

[0051] For example, a softmax layer (softmax function) is used to normalize the M-dimensional relevance to obtain M-dimensional probability values. Each probability value corresponds to a category, and the category information corresponding to the highest probability value is selected as the category information of the target.

[0052] The image contains targets of at least one category. The location and category information of the targets of at least one category can be obtained through the above method.

[0053] To better understand this disclosure, the application scenarios of the embodiments of this disclosure will be described.

[0054] Figure 2 This is a schematic diagram illustrating an application scenario corresponding to an embodiment of this disclosure. For example... Figure 2 As shown, users can upload images or videos to electronic device 201. For videos, the electronic device can perform frame extraction to obtain images.

[0055] For images obtained by frame extraction or images uploaded by users, electronic device 201 can perform target detection processing on the images to identify the location and category information of the targets in the images.

[0056] In this context, electronic device 201 can be a user terminal, such as a personal computer, laptop, or mobile device (e.g., a mobile phone). Alternatively, electronic device 201 can also be a server, which can be a cloud server or a local server. Alternatively, the user can send images or videos to the server via the user terminal, where the server processes the images or videos to obtain the target's location and category information, and returns it to the user terminal, which then displays the target's location and category information to the user.

[0057] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0058] When electronic devices perform target detection, they may specifically include: using a target detection model to perform target detection processing on the image to be processed, so as to obtain the location information and category information of the image to be processed.

[0059] like Figure 3 As shown, the target detection model may include: feature extraction network 301, regression branch network 302, and classification branch network 303.

[0060] The feature extraction network 301 is used to perform feature extraction processing on the image to be processed, so as to obtain the image features of the image to be processed. Specifically, it can be a convolutional neural network (CNN).

[0061] The regression branch network 302 is used to determine the location information of the target in the image to be processed based on the image features. Specifically, it may include a linear regression layer (fully connected layer).

[0062] The classification branch network 303 is used to determine the category information of the target based on the image features.

[0063] The classification branch network can specifically include: an embedding layer, a relevance module, and a classification module.

[0064] An embedding layer is used to embed the image features to obtain the feature embedding vector of the image to be processed.

[0065] The correlation module is used to calculate the correlation between the feature embedding vector and the class center embedding vector of the at least one category.

[0066] A classification module is used to determine the category information of the target based on the relevance.

[0067] The embedding layer is used to convert image features into vectors, which can be called feature embedding vectors, such as 512-dimensional vectors. The embedding layer can be a fully connected layer.

[0068] The relevance module is used to calculate the relevance between each class core embedding vector and the feature embedding vector. The relevance can be either similarity or distance; the higher the similarity, the higher the relevance, and vice versa.

[0069] Assuming the feature embedding vector is represented by X (e.g., 512*1 dimensional), and the class center embedding vectors of M classes can form a 512*M dimensional matrix W, then the relevance = W. T *X, W T It is the transpose of W. The correlation is an M-dimensional vector (M*1 dimension), where each element of the vector represents the correlation between the feature embedding vector and the class center embedding vector of each class.

[0070] The classification module is used to determine the category information of the target based on the correlation between the class center embedding vector and the feature embedding vector. For example, the category information corresponding to the class center embedding vector with the highest correlation is used as the category information of the target.

[0071] Specifically, the relevance can be discretized (e.g., using the logit function) to obtain discrete values; the discrete values ​​can be normalized (e.g., using the softmax function) to obtain the probability values ​​of the target belonging to each of the at least one category; based on the probability values, the category information of the target can be determined, such as selecting the category with the highest probability value as the target category.

[0072] In conjunction with the above examples, this disclosure also provides a target detection method.

[0073] Figure 4 Based on the schematic diagram of the second embodiment of this disclosure, this embodiment provides a target detection method, such as... Figure 4 As shown, the method includes:

[0074] 401. The feature extraction network in the pre-trained target detection model is used to perform feature extraction processing on the image to be processed, so as to obtain the image features of the image to be processed.

[0075] The feature extraction network can be a convolutional neural network (CNN), specifically any of the following networks: Residual Network (Resnet) series, Vision Transformer (ViT) network, or Data-efficient Image Transformers (Deit) series.

[0076] In this embodiment, image features are obtained by using a feature extraction network of an object detection model. Compared with rule-based methods, more accurate image features can be obtained, thereby improving the accuracy of object detection.

[0077] 402. Using the regression branch network in the target detection model, the image features are regressed to determine the location information of the target.

[0078] Among them, the regression branch network includes, for example, a linear regression layer, which can be a fully connected layer. Its input is image features and its output is location information (bounding box).

[0079] In this embodiment, the location information of the target is obtained by using the regression branch network of the target detection model. The location information can be determined by using a deep neural network, which can make full use of the advantages of deep neural networks and improve the accuracy of the location information.

[0080] 403. Using the embedding layer in the target detection model, the image features are embedded to obtain the feature embedding vector of the image to be processed.

[0081] The embedding layer can be a fully connected layer. The input of the embedding layer is image features, and the output is a feature embedding vector. The embedding layer can convert the matrix-form image features into a vector-form feature embedding vector.

[0082] In this embodiment, by employing an embedding layer, image features can be converted into feature embedding vectors, which facilitates subsequent vector retrieval processing and improves processing efficiency.

[0083] 404. Determine the correlation between the feature embedding vector and the pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing the classification image of the corresponding category.

[0084] Specifically, this may include: normalizing the feature embedding vector to obtain a normalized feature embedding vector; normalizing the class center embedding vector of the at least one category to obtain a normalized class center embedding vector of the at least one category; and using the normalized class center embedding vector of the at least one category as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

[0085] The normalization process in this step can be performed using the L2 function.

[0086] Assuming the normalized feature embedding vector is represented by X (e.g., 512*1 dimensional), and the normalized class center embedding vectors of M classes can form a 512*M dimensional matrix W, then the relevance = W T *X, W T It is the transpose of W. The correlation is an M-dimensional vector (M*1 dimension), where each element of the vector represents the correlation between the feature embedding vector and the class center embedding vector of each class.

[0087] In this embodiment, the normalized class center embedding vector of at least one category is used as the weight vector to perform weighted processing on the normalized feature embedding vector to obtain the correlation. This allows for a simple calculation of the correlation between the feature embedding vector and each class center embedding vector, thus improving processing efficiency.

[0088] 405. Based on the relevance, determine the category information of the target in the at least one category.

[0089] Specifically, this may include: discretizing the relevance to obtain discrete values; normalizing the discrete values ​​to obtain the probability values ​​of the target belonging to each of the at least one category; and determining the category information of the target based on the probability values.

[0090] Discrete processing can be performed, for example, using the logit function.

[0091] The normalization process in this step can be performed, for example, using the softmax function.

[0092] In this embodiment, the probability value of the target belonging to each category can be obtained based on the relevance, and the category information of the target can be determined based on the probability value, thereby improving the accuracy of the category information.

[0093] The object detection model described above is pre-trained.

[0094] Figure 5 Based on a schematic diagram of the third embodiment of this disclosure, this embodiment provides a method for training an object detection model. The object detection model includes a feature extraction network, and the method includes:

[0095] 501. Obtain training data, the training data including: sample images and classification images of at least one category, as well as the true location information and true category information of the targets contained in the sample images.

[0096] 502. Using the feature extraction network, perform feature extraction processing on the sample image to obtain the image features of the sample image.

[0097] 503. Based on the image features, determine the predicted location information of the target.

[0098] 504. The image features are embedded to obtain the feature embedding vector of the sample image.

[0099] 505. Process the classification image of the at least one category to obtain the class center embedding vector of the at least one category.

[0100] 506. Determine the correlation between the feature embedding vector and the class center embedding vector of the at least one category, and determine the predicted category information of the target based on the correlation.

[0101] 507. Based on the predicted location information, the predicted category information, the actual location information, and the actual category information, construct a loss function, and adjust the network parameters of the feature extraction network based on the loss function until a preset condition is met.

[0102] In this embodiment, the predicted category information of the target is determined based on the correlation between the feature embedding vector and the class center embedding vector. Thus, if a new category is added, only the class center embedding vector corresponding to the new category needs to be constructed; full annotation of targets in non-new categories in the training samples is not required, thereby reducing the amount of annotation. Furthermore, since the correlation is not limited by annotation costs, classification accuracy can be improved. In addition, the target's location and category information share image features. Compared to using two separate models to extract features corresponding to location and category information, joint optimization can be performed during training to improve the representational power of image features and enhance the accuracy of target detection.

[0103] Figure 6 Based on the schematic diagram of the fourth embodiment of this disclosure, this embodiment provides a method for training an object detection model. The object detection model includes: a feature extraction network, an embedding layer, and a regression branch network. The method includes:

[0104] 601. Obtain training data, the training data including: sample images and classification images of at least one category, as well as the true location information and true category information of the target contained in the sample images.

[0105] During the training process, the sample images and classification images can be pre-collected, and the label data can be manually labeled.

[0106] Figure 7 This is a diagram illustrating the overall architecture of the model training process. For example... Figure 7 As shown, the sample image is Figure 7 The image shown above is a classification image. Figure 7 The image shown below.

[0107] like Figure 7 As shown, the classification images are categorized into three types, such as milk, yogurt, and potato chips.

[0108] Furthermore, the targets of at least one category in the sample image can be cropped to obtain a classification image of the at least one category.

[0109] Taking the milk category as an example, the classification image for the milk category is obtained by taking a screenshot of the target milk. That is, the classification image for the milk category only contains images of milk and does not contain images of other targets.

[0110] In this embodiment, by cropping the target in the sample image to obtain the classification image, the sample image can be effectively utilized and the resource utilization rate can be improved. In addition, since the classification image is obtained by cropping the target in the sample image, model training based on the classification image and the sample image can ensure the consistency of the target and improve the accuracy of the model.

[0111] The sample images contain the targets corresponding to the classified images, for example, Figure 7 The sample images contain objects such as milk, yogurt, and potato chips.

[0112] The classification images mentioned above can be... Figure 7 The image is obtained by taking a screenshot of the corresponding target in the sample image shown, or the classification image can be extracted from other sample images.

[0113] 602. Using the feature extraction network, perform feature extraction processing on the sample image to obtain the image features of the sample image.

[0114] The feature extraction network can be a convolutional neural network (CNN), specifically any of the following networks: Residual Network (Resnet) series, Vision Transformer (ViT) network, or Data-efficient Image Transformers (Deit) series.

[0115] See Figure 7 Feature extraction networks in object detection models, such as Figure 7 The top side shows a Convolutional Neural Network (CNN). The input to this feature extraction network is a sample image, and the output is the image features of the sample image.

[0116] For each classified image, a feature extraction network (such as...) can also be used. Figure 7 The CNN shown below extracts image features from the classified images. To distinguish them, the feature extraction network corresponding to the sample image can be represented by a first CNN, and the feature extraction network corresponding to the classified image can be represented by a second CNN. The first CNN and the second CNN are usually different.

[0117] In this embodiment, image features are obtained by using a feature extraction network of an object detection model. Compared with rule-based methods, more accurate image features can be obtained, thereby improving the accuracy of object detection.

[0118] 603. Using the regression branch network, the image features are regressed to obtain the predicted location information.

[0119] like Figure 7 As shown, the object detection model can also include a regression branch network, which can specifically be a fully connected layer. The input to the regression branch network is the image features of the sample image, and the output is the predicted location information of the target contained in the sample image.

[0120] In this embodiment, the location information of the target is obtained by using the regression branch network of the target detection model. The location information can be determined by using a deep neural network, which can make full use of the advantages of deep neural networks and improve the accuracy of the location information.

[0121] 604. Using the embedding layer, the image features are embedded to obtain the feature embedding vector of the sample image.

[0122] like Figure 7As shown, the embedding layer in the object detection model is Figure 7 The FC layer shown above. The input of this FC layer is the image features of the sample image, and the output is the feature embedding vector of the sample image.

[0123] For each classified image, an embedding layer (such as...) can also be used. Figure 7 The FC layer shown below converts the image features of the categorized image into a feature embedding vector for the categorized image. Similar to feature extraction networks, the embedding layer corresponding to the sample image can be represented by the first FC, and the embedding layer corresponding to the categorized image can be represented by the second FC. The first FC and the second FC are usually different.

[0124] In this embodiment, by employing an embedding layer, image features can be converted into feature embedding vectors, which facilitates subsequent vector retrieval processing and improves processing efficiency.

[0125] 605. Process the classification image of the at least one category to obtain the class center embedding vector of the at least one category.

[0126] Specifically, this may include: performing feature extraction processing on the classified image to obtain the image features of the classified image, and converting the image features of the classified image into the feature embedding vector of the classified image; and performing an averaging operation on the feature embedding vectors of the classified images of each category in the at least one category to obtain the class center embedding vector of the at least one category.

[0127] In this embodiment, the feature embedding vectors of the classified images of each category are averaged to obtain the class center embedding vectors of each category. This can improve the expressive power and accuracy of the class center embedding vectors, thereby improving the model performance.

[0128] 606. Determine the correlation between the feature embedding vector and the class center embedding vector of the at least one category, and determine the predicted category information of the target based on the correlation.

[0129] Specifically, this may include: normalizing the feature embedding vector to obtain a normalized feature embedding vector; normalizing the class center embedding vector of the at least one category to obtain a normalized class center embedding vector of the at least one category; and using the normalized class center embedding vector of the at least one category as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

[0130] In this embodiment, the normalized class center embedding vector of at least one category is used as the weight vector to perform weighted processing on the normalized feature embedding vector to obtain the correlation. This allows for a simple calculation of the correlation between the feature embedding vector and each class center embedding vector, thus improving processing efficiency.

[0131] After obtaining the feature embedding vector of the sample image and the class center embedding vectors of M categories, the correlation between the two can be calculated. Assuming the normalized feature embedding vector is represented by X (e.g., 512*1 dimensional), and the normalized class center embedding vectors of the M categories can form a 512*M dimensional matrix W, then the correlation = W. T *X, W T It is the transpose of W. The correlation is an M-dimensional vector (M*1 dimension), where each element of the vector represents the correlation between the feature embedding vector and the class center embedding vector of each class.

[0132] After obtaining the relevance, the relevance can be discretized to obtain discrete values; the discrete values ​​can be transformed to obtain transformed values; and the transformed values ​​can be normalized to obtain the predicted category information.

[0133] like Figure 7 As shown, discretization can be performed, for example, using the logit function.

[0134] This embodiment uses additive angular margin loss (arcmaginloss) for metric learning. The corresponding discrete value is represented by cos(θ), and the transformed value is represented by s*cos(θ+m). Here, θ can be calculated based on the relevance of each dimension, and s and m can be set as needed.

[0135] Normalization can be performed, for example, using the softmax function.

[0136] In this embodiment, the probability values ​​of a target belonging to each category can be obtained based on relevance, and the predicted category information of the target can be determined based on the probability values, thereby improving the accuracy of the category information. Furthermore, by performing size transformation on the discrete values, knowledge can be learned more effectively, improving the model's accuracy.

[0137] 607. Based on the predicted location information, the predicted category information, the actual location information, and the actual category information, construct a loss function, and adjust the network parameters of the feature extraction network based on the loss function until a preset condition is met.

[0138] The loss function L is defined as the first sub-loss function plus the second sub-loss function. The first sub-loss function is constructed based on the predicted location information and the actual location information, such as the L2 loss function.

[0139] The second sub-loss function is constructed based on predicted class information and true class information, such as the CrossEntropy (CE) loss function.

[0140] Once the loss function is obtained, the parameters can be adjusted using that loss function.

[0141] The adjusted parameters can be calculated as: Adjusted parameters = Unadjusted parameters - Learning rate * Gradient value, where the gradient value is obtained by differentiating the loss function with respect to the model parameters, and the learning rate is a preset hyperparameter. The initial values ​​for each parameter can be set; for example, they can be randomized to obtain random initial values.

[0142] Specifically, the feature extraction network can choose a ViT structure and be initialized using the weights of a Contrastive Language-Image Pre-training (CLIP) model that has been pre-trained on massive amounts of data.

[0143] Preset conditions include, for example, reaching a preset number of iterations or meeting convergence conditions (such as the absolute value of the difference between the loss function of the previous round and the loss function of the current round being less than a preset value). A model that meets these preset conditions can be used as the final object detection model, including the final feature extraction network, the final embedding layer, and the final regression branch network. This final object detection model can then be used for object detection during the inference phase.

[0144] The parameters adjusted as described above can specifically include the model parameters of the feature extraction network (first CNN and second CNN), and also the model parameters of the embedding layers (first FC layer and second FC layer). During training, the model parameters of the fully connected layers of other networks (such as the regression branch network) can be fixed, while only the model parameters of the feature extraction network (or the feature extraction network and the embedding layers) are adjusted. This allows the features output by the embedding layers to cluster as close as possible to the class centers, resulting in better representational ability and alignment with the features obtained by the feature extraction model (second CNN).

[0145] The target's category information is determined based on the class embedding. Therefore, when the category changes, there is no need to retrain the model; simply extract the new class embedding and replace it. Since the features output by the embedding layer of the classification branch network can be aligned with the features obtained by the feature extraction model, when encountering targets with untrained categories, the class centers can be extracted by the feature extraction model and added to the class embedding. This allows the model to generalize and recognize new categories by leveraging the correlation between the features output by the embedding layer and the class embedding.

[0146] The training set can be built in the following way: First, the basic training set needs to be precisely labeled with localization and classification; second, the targets in the first step are cut out to obtain the classification dataset; third, for long-tail categories (categories that appear less frequently), targeted collection can be carried out and only long-tail targets are labeled to expand the classification dataset.

[0147] In this embodiment, after adding new long-tail data, only the long-tail data needs to be labeled, without needing to label the non-long-tail data, which can reduce labeling costs. By using metric learning, the model's ability to represent fine-grained categories is enhanced, and categories can be better distinguished. When adding a new category, only the class center embedding vector of that category needs to be added, without adjusting the model structure. This allows for flexible adjustment of the classification layer categories without fine-tuning the model, and can quickly increase the recognition of new categories.

[0148] Figure 8 The schematic diagram is based on the fourth embodiment of the present disclosure. This embodiment provides a target detection device 800, which includes: an extraction module 801, a positioning module 802, an embedding module 803, a correlation module 804, and a classification module 805.

[0149] The extraction module 801 is used to perform feature extraction processing on the image to be processed to obtain image features of the image to be processed; the localization module 802 is used to determine the location information of the target contained in the image to be processed based on the image features; the embedding module 803 is used to perform embedding processing on the image features to obtain the feature embedding vector of the image to be processed; the relevance module 804 is used to determine the relevance between the feature embedding vector and the pre-constructed class center embedding vector of at least one category; wherein, the class center embedding vector is obtained by processing the classification image of the corresponding category; the classification module 805 is used to determine the category information of the target in the at least one category based on the relevance.

[0150] In this embodiment, the target category information is determined based on the correlation between the feature embedding vector and the class center embedding vector. Therefore, if a new category is added, only the class center embedding vector corresponding to the new category needs to be constructed; full annotation of non-new category targets in the training samples is not required, thus reducing the amount of annotation. Furthermore, since the correlation is not limited by annotation costs, classification accuracy can be improved. In addition, the target's location and category information share image features. Compared to using two separate models to extract features corresponding to location and category information, joint optimization can be performed during training to improve the representational power of image features and enhance the accuracy of target detection.

[0151] In some embodiments, the relevance module 804 is further configured to: normalize the feature embedding vector to obtain a normalized feature embedding vector; normalize the class center embedding vector of the at least one category to obtain a normalized class center embedding vector of the at least one category; and use the normalized class center embedding vector of the at least one category as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

[0152] In this embodiment, the normalized class center embedding vector of at least one category is used as the weight vector to perform weighted processing on the normalized feature embedding vector to obtain the correlation. This allows for a simple calculation of the correlation between the feature embedding vector and each class center embedding vector, thus improving processing efficiency.

[0153] In some embodiments, the classification module 805 is further configured to: discretize the relevance to obtain discrete values; normalize the discrete values ​​to obtain probability values ​​of the target belonging to each of the at least one category; and determine the category information of the target based on the probability values.

[0154] In this embodiment, the probability value of the target belonging to each category can be obtained based on the relevance, and the category information of the target can be determined based on the probability value, thereby improving the accuracy of the category information.

[0155] In some embodiments, the extraction module 801 is further configured to: use a feature extraction network in a pre-trained target detection model to perform feature extraction processing on the image to be processed, so as to obtain the image features of the image to be processed.

[0156] In this embodiment, image features are obtained by using a feature extraction network of an object detection model. Compared with rule-based methods, more accurate image features can be obtained, thereby improving the accuracy of object detection.

[0157] In some embodiments, the embedding module 803 is further configured to: use the embedding layer in the target detection model to embed the image features to obtain the feature embedding vector of the image to be processed.

[0158] In this embodiment, by employing an embedding layer, image features can be converted into feature embedding vectors, which facilitates subsequent vector retrieval processing and improves processing efficiency.

[0159] In some embodiments, the positioning module 802 is further configured to: use the regression branch network in the target detection model to perform regression processing on the image features to determine the location information of the target.

[0160] In this embodiment, the location information of the target is obtained by using the regression branch network of the target detection model. The location information can be determined by using a deep neural network, which can make full use of the advantages of deep neural networks and improve the accuracy of the location information.

[0161] Figure 9 This is a schematic diagram according to the fifth embodiment of the present disclosure. This embodiment provides a training device for an object detection model. The model includes a feature extraction network. The device 900 includes an acquisition module 901, an extraction module 902, a localization module 903, an embedding module 904, a determination module 905, a classification module 906, and an adjustment module 907.

[0162] The acquisition module 901 is used to acquire training data, which includes: sample images and classification images of at least one category, as well as the true location information and true category information of the target contained in the sample images; the extraction module 902 is used to perform feature extraction processing on the sample images using the feature extraction network to obtain image features of the sample images; the localization module 903 is used to determine the predicted location information of the target based on the image features; the embedding module 904 is used to perform embedding processing on the image features to obtain feature embedding vectors of the sample images; the determination module 905 is used to process the classification images of the at least one category to obtain class center embedding vectors of at least one category; the classification module 906 is used to determine the correlation between the feature embedding vectors and the class center embedding vectors of the at least one category, and determine the predicted category information of the target based on the correlation; the adjustment module 907 is used to construct a loss function based on the predicted location information, the predicted category information, the true location information, and the true category information, and adjust the network parameters of the feature extraction network based on the loss function.

[0163] In this embodiment, the predicted category information of the target is determined based on the correlation between the feature embedding vector and the class center embedding vector. Thus, if a new category is added, only the class center embedding vector corresponding to the new category needs to be constructed; full annotation of targets in non-new categories in the training samples is not required, thereby reducing the amount of annotation. Furthermore, since the correlation is not limited by annotation costs, classification accuracy can be improved. In addition, the target's location and category information share image features. Compared to using two separate models to extract features corresponding to location and category information, joint optimization can be performed during training to improve the representational power of image features and enhance the accuracy of target detection.

[0164] In some embodiments, the determining module 905 is further configured to: perform feature extraction processing on the classified image to obtain image features of the classified image, and convert the image features of the classified image into feature embedding vectors of the classified image; and perform averaging operation on the feature embedding vectors of the classified images of each category in the at least one category to obtain the class center embedding vector of the at least one category.

[0165] In this embodiment, the feature embedding vectors of the classified images of each category are averaged to obtain the class center embedding vectors of each category. This can improve the expressive power and accuracy of the class center embedding vectors, thereby improving the model performance.

[0166] In some embodiments, the classification module 906 is further configured to: normalize the feature embedding vector to obtain a normalized feature embedding vector; normalize the core embedding vector of the at least one category to obtain a normalized core embedding vector of the at least one category; and use the normalized core embedding vector of the at least one category as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

[0167] In this embodiment, the normalized class center embedding vector of at least one category is used as the weight vector to perform weighted processing on the normalized feature embedding vector to obtain the correlation. This allows for a simple calculation of the correlation between the feature embedding vector and each class center embedding vector, thus improving processing efficiency.

[0168] In some embodiments, the classification module 906 is further configured to: discretize the relevance to obtain discrete values; perform size transformation on the discrete values ​​to obtain transformed values; and normalize the transformed values ​​to obtain the predicted category information.

[0169] In this embodiment, the probability values ​​of a target belonging to each category can be obtained based on relevance, and the predicted category information of the target can be determined based on the probability values, thereby improving the accuracy of the category information. Furthermore, by performing size transformation on the discrete values, knowledge can be learned more effectively, improving the model's accuracy.

[0170] In some embodiments, the target detection model further includes an embedding layer, wherein the embedding module 904 is further configured to: use the embedding layer to perform embedding processing on the image features to obtain a feature embedding vector of the sample image.

[0171] In this embodiment, by employing an embedding layer, image features can be converted into feature embedding vectors, which facilitates subsequent vector retrieval processing and improves processing efficiency.

[0172] In some embodiments, the target detection model further includes a regression branch network, and the localization module 903 is further configured to: use the regression branch network to perform regression processing on the image features to obtain the predicted location information.

[0173] In this embodiment, the location information of the target is obtained by using the regression branch network of the target detection model. The location information can be determined by using a deep neural network, which can make full use of the advantages of deep neural networks and improve the accuracy of the location information.

[0174] In some embodiments, the target is a target of at least one category, and the acquisition module 901 is further configured to: perform cropping processing on the target of the at least one category in the sample image to obtain a classification image of the at least one category.

[0175] In this embodiment, by cropping the target in the sample image to obtain the classification image, the sample image can be effectively utilized and the resource utilization rate can be improved. In addition, since the classification image is obtained by cropping the target in the sample image, model training based on the classification image and the sample image can ensure the consistency of the target and improve the accuracy of the model.

[0176] It is understood that the same or similar content in different embodiments of this disclosure can be referred to each other.

[0177] It is understood that the terms "first" and "second" in the embodiments of this disclosure are only used for distinction and do not indicate the degree of importance or the order of events.

[0178] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0179] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0180] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1000 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0181] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 10010 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0182] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0183] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as object detection methods or object detection model training methods. For example, in some embodiments, the object detection method or object detection model training method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the object detection method or object detection model training method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured by any other suitable means (e.g., by means of firmware) to perform a method for training an object detection method or an object detection model.

[0184] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable load balancing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0188] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0189] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0190] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0191] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A target detection method, comprising: The feature extraction network in the object detection model is used to perform feature extraction processing on the image to be processed, so as to obtain the image features of the image to be processed. Using the regression branch network in the target detection model, the location information of the target contained in the image to be processed is determined based on the image features; The image features are embedded using the embedding layer in the target detection model to obtain the feature embedding vector of the image to be processed. Determine the correlation between the feature embedding vector and the pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing the classification image of the corresponding category; Based on the relevance, the category information of the target is determined in the at least one category; The target detection model is trained based on training data, which includes sample images and classification images of at least one category. The classification images are obtained by cropping targets of at least one category from the sample images. During training, the network parameters of the feature extraction network and the embedding layer are adjusted, while the model parameters of the regression branch network are fixed.

2. The method according to claim 1, wherein, Determining the correlation between the feature embedding vector and the pre-constructed class core embedding vector of at least one category includes: The feature embedding vector is normalized to obtain a normalized feature embedding vector. The class core embedding vectors of the at least one category are normalized to obtain normalized class core embedding vectors of the at least one category. The normalized class center embedding vector of at least one category is used as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

3. The method according to claim 2, wherein, The determination of the target's category information based on the relevance includes: The correlation is discretized to obtain discrete values; The discrete values ​​are normalized to obtain the probability values ​​of the target belonging to each of the at least one category; Based on the probability value, the category information of the target is determined.

4. A method for training an object detection model, the object detection model comprising: Feature extraction The method includes: a network, an embedding layer, and a regression branch network. Acquire training data, which includes: sample images and classification images of at least one category, as well as the true location information and true category information of the targets contained in the sample images; the classification images are obtained by cropping the targets of the at least one category in the sample images; The feature extraction network is used to perform feature extraction processing on the sample image to obtain the image features of the sample image; Using the regression branch network, the predicted location information of the target is determined based on the image features; The image features are embedded using the embedding layer to obtain the feature embedding vector of the sample image; The classification images of the at least one category are processed to obtain the class center embedding vectors of the at least one category; Determine the correlation between the feature embedding vector and the class center embedding vector of the at least one category, and determine the predicted category information of the target based on the correlation. Based on the predicted location information, the predicted category information, the true location information, and the true category information, a loss function is constructed, and the network parameters of the feature extraction network and the embedding layer are adjusted based on the loss function, while the model parameters of the regression branch network are fixed.

5. The method according to claim 4, wherein, The process of processing the classification image of the at least one category to obtain the class center embedding vector of the at least one category includes: The classified image is subjected to feature extraction processing to obtain the image features of the classified image, and the image features of the classified image are converted into the feature embedding vector of the classified image. For each category in the at least one category, the feature embedding vectors of the classification images of each category are averaged to obtain the class center embedding vector of the at least one category.

6. The method according to claim 4, wherein, Determining the correlation between the feature embedding vector and the class center embedding vectors of each category includes: The feature embedding vector is normalized to obtain a normalized feature embedding vector. The class core embedding vectors of the at least one category are normalized to obtain normalized class core embedding vectors of the at least one category. The normalized class center embedding vector of at least one category is used as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

7. The method according to claim 6, wherein, The step of determining the predicted category information of the target based on the relevance includes: The correlation is discretized to obtain discrete values; The discrete values ​​are subjected to size transformation to obtain transformed values; The transformed values ​​are normalized to obtain the predicted category information.

8. A target detection device, comprising: The extraction module is used to perform feature extraction processing on the image to be processed using the feature extraction network in the target detection model, so as to obtain the image features of the image to be processed; The localization module is used to determine the location information of the target contained in the image to be processed based on the image features by employing the regression branch network in the target detection model. An embedding module is used to perform embedding processing on the image features using the embedding layer in the target detection model to obtain the feature embedding vector of the image to be processed. A correlation module is used to determine the correlation between the feature embedding vector and a pre-constructed class center embedding vector of at least one category; wherein the class center embedding vector is obtained by processing the classification image of the corresponding category; A classification module is used to determine the category information of the target in the at least one category based on the relevance. The target detection model is trained based on training data, which includes sample images and classification images of at least one category. The classification images are obtained by cropping targets of at least one category from the sample images. During training, the network parameters of the feature extraction network and the embedding layer are adjusted, while the model parameters of the regression branch network are fixed.

9. The apparatus according to claim 8, wherein, The relevance module is further used for: The feature embedding vector is normalized to obtain a normalized feature embedding vector. The class core embedding vectors of the at least one category are normalized to obtain normalized class core embedding vectors of the at least one category. The normalized class center embedding vector of at least one category is used as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

10. The apparatus according to claim 8, wherein, The classification module is further used for: The correlation is discretized to obtain discrete values; The discrete values ​​are normalized to obtain the probability values ​​of the target belonging to each of the at least one category; Based on the probability value, the category information of the target is determined.

11. A training apparatus for an object detection model, the object detection model comprising: Feature extraction The apparatus includes a network, an embedded layer, and a regression branch network. The acquisition module is used to acquire training data, which includes: sample images and classification images of at least one category, as well as the true location information and true category information of the targets contained in the sample images; the classification images are obtained by cropping the targets of the at least one category in the sample images. The extraction module is used to perform feature extraction processing on the sample image using the feature extraction network to obtain the image features of the sample image; The localization module is used to determine the predicted location information of the target based on the image features using the regression branch network. An embedding module is used to embed the image features using the embedding layer to obtain the feature embedding vector of the sample image; A determination module is used to process the classification image of the at least one category to obtain the class center embedding vector of the at least one category; A classification module is used to determine the correlation between the feature embedding vector and the class center embedding vector of the at least one category, and to determine the predicted category information of the target based on the correlation. The adjustment module is used to construct a loss function based on the predicted location information, the predicted category information, the real location information, and the real category information, and to adjust the network parameters of the feature extraction network and the embedding layer based on the loss function, while fixing the model parameters of the regression branch network.

12. The apparatus according to claim 11, wherein, The determining module is further used for: The classified image is subjected to feature extraction processing to obtain the image features of the classified image, and the image features of the classified image are converted into the feature embedding vector of the classified image. For each category in the at least one category, the feature embedding vectors of the classification images of each category are averaged to obtain the class center embedding vector of the at least one category.

13. The apparatus according to claim 11, wherein, The classification module is further used for: The feature embedding vector is normalized to obtain a normalized feature embedding vector. The class core embedding vectors of the at least one category are normalized to obtain normalized class core embedding vectors of the at least one category. The normalized class center embedding vector of at least one category is used as a weight vector to weight the normalized feature embedding vector to obtain the relevance.

14. The apparatus according to claim 11, wherein, The classification module is further used for: The correlation is discretized to obtain discrete values; The discrete values ​​are subjected to size transformation to obtain transformed values; The transformed values ​​are normalized to obtain the predicted category information.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images