Object detection method and object detection model training method
By learning cross-domain features through a cross-attention network model, the problem of low accuracy in object authenticity detection under different scenarios is solved, and an efficient object anti-counterfeiting effect is achieved in new scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2023-03-16
- Publication Date
- 2026-07-14
AI Technical Summary
Existing object authenticity detection algorithms perform poorly in different scenarios, resulting in low object recognition accuracy and difficulty in adapting to the object anti-counterfeiting requirements of new scenarios.
A cross-attention network model is adopted. By training the model in the original domain scene and using k-means clustering and kNN nearest neighbor methods to assign pseudo-labels to samples in the target domain, the model learns cross-domain features by combining self-attention and cross-attention mechanisms, thereby enhancing the robustness and feature representation ability of the model.
It improves the accuracy of object authenticity detection, can adapt to object detection in different scenarios, and enhances the effect of object recognition.
Smart Images
Figure CN116168274B_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification relate to the field of computer technology, and in particular to an object detection method. Background Technology
[0002] With the development of technology, object recognition technology has been widely used in people's daily lives due to its high reliability and other advantages. However, as the application scope of object recognition technology continues to expand, attacks using non-authentic objects have also emerged. Therefore, object authenticity detection is a crucial issue in the field of object anti-counterfeiting, directly impacting the user experience of object recognition.
[0003] Currently, object authenticity detection algorithms typically extract texture features from the object image first, and then use a classifier to distinguish between real and non-real objects. However, traditional algorithms have weak feature representation capabilities and are easily affected by changes in lighting, which leads to poor performance in object authenticity detection and poor anti-counterfeiting effects in new scenarios, thus affecting the accuracy of object recognition. Summary of the Invention
[0004] In view of the above, embodiments of this specification provide an object detection method. One or more embodiments of this specification also relate to another object detection method, an object detection model training method, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.
[0005] According to a first aspect of the embodiments of this specification, an object detection method is provided, comprising:
[0006] Acquire the image to be detected;
[0007] The image to be detected is input into the object detection model to obtain the object detection result corresponding to the image to be detected;
[0008] The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The sample images in the first sample set and the second sample set are in different scenes, and the sample images include target objects.
[0009] According to a second aspect of the embodiments of this specification, another object detection method is provided, applied to a cloud-side device, comprising:
[0010] The receiving end device sends an object detection request, wherein the object detection request carries an image to be detected;
[0011] The image to be detected is input into the object detection model to obtain the object detection result corresponding to the image to be detected;
[0012] The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The first sample set and the second sample set contain sample images in different scenes. The sample images include target objects.
[0013] The object detection results are sent to the front end for display.
[0014] According to a third aspect of the embodiments of this specification, an object detection model training method is provided, applied to a cloud-based device, comprising:
[0015] Obtain the first sample image in the first sample set and the first object label corresponding to the first sample image;
[0016] Obtain the model parameters from the reference model;
[0017] The first sample image and the model parameters are input into the object detection model to be trained to obtain the first prediction result;
[0018] Based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to obtain the object detection model.
[0019] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising:
[0020] Memory and processor;
[0021] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the above-mentioned object detection method or object detection model training method.
[0022] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the object detection method or object detection model training method described above.
[0023] According to a sixth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the above-described object detection method or object detection method or object detection model training method.
[0024] In one embodiment of this specification, an image to be detected is acquired; the image to be detected is input into an object detection model to obtain an object detection result corresponding to the image to be detected; wherein, the object detection model is trained based on model parameters of a first sample set and a reference model, the reference model is pre-trained based on a second sample set, the first sample set and the second sample set are in different scenes, and the sample images include target objects.
[0025] Specifically, by inputting the acquired image to be detected into the object detection model, the object detection result corresponding to the image is obtained. The object detection model is trained based on the model parameters of a first sample set and a reference model, which in turn is trained based on a second sample set. The sample images in the two sample sets are located in different scenes. This method, where a reference model trained on sample images of one scene helps train the object detection model corresponding to sample images of another scene, allows the object detection model to continuously learn the features of sample images from other scenes. This improves the object detection accuracy in the image scenes of the first sample set, enabling it to adapt to object detection in different scenes, address the issue of object authenticity detection in new scenes, improve the accuracy of object authenticity detection, and enhance the overall object recognition effect. Attached Figure Description
[0026] Figure 1 This is a schematic diagram of a scenario illustrating an object detection method provided in one embodiment of this specification;
[0027] Figure 2 This is a flowchart of an object detection method provided in one embodiment of this specification;
[0028] Figure 3 This is a flowchart of another object detection method provided in one embodiment of this specification;
[0029] Figure 4 This is a flowchart of an object detection model training method provided in one embodiment of this specification;
[0030] Figure 5(a) is a schematic diagram of the processing procedure of an object detection model training method provided in one embodiment of this specification;
[0031] Figure 5(b) is a data flow diagram of an object detection model training method provided in one embodiment of this specification;
[0032] Figure 6 This is a schematic diagram of the structure of an object detection device provided in one embodiment of this specification;
[0033] Figure 7This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation
[0034] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.
[0035] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0036] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."
[0037] First, the terms and concepts used in one or more embodiments of this specification will be explained.
[0038] Object authenticity detection: Determine whether an object in an image is a real object.
[0039] KMean: Given a set of data points and the required number of clusters k, where k is specified by the user, the k-means algorithm repeatedly divides the data into k clusters based on a certain distance function.
[0040] KNN stands for K nearest neighbors, meaning that each sample can be represented by its K nearest neighbors.
[0041] Attention: The attention mechanism assigns different weights to different features to help the model select the features most valuable for accurate classification.
[0042] Object authenticity detection is a crucial issue in object anti-counterfeiting, directly impacting the object recognition experience. Object authenticity detectors are susceptible to scene changes; for example, a model trained on outdoor images may fail to accurately identify genuine objects when directly applied to indoor environments. Object authenticity detection models require continuous adaptation to different scenarios, as poor anti-counterfeiting performance in new scenarios can lead to the inability to identify genuine objects while easily identifying false ones. Therefore, to improve the accuracy of object authenticity detection across various domains, this application proposes a cross-attention-based object authenticity detection method. A cross-attention module is introduced, allowing the model to continuously learn information from new scenes, enhancing feature representation and improving the robustness of the object authenticity detector. Simultaneously, a distillation module learns the cross-attention branch to guide the learning of features in the target domain, achieving high anti-counterfeiting performance in new scenes. To address this, we first train the model in the original domain scene. Then, we use the trained model to extract features from samples in the target domain. Next, we use kMeans clustering to obtain initial class centers. Then, we use kNN nearest neighbors to assign pseudo-labels to the samples in the target domain. Finally, we combine the images from the two domains into a sample pair and feed them into a cross-attention network structure. In this way, the network can learn the features of the cross domain, continuously enhancing the network's capabilities. The cross-attention module can effectively align the features of the source and target domains, thereby improving the robustness of the model.
[0043] This specification provides an object detection method, and also relates to another object detection method, an object detection model training method, an object detection device, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.
[0044] See Figure 1 , Figure 1 A schematic diagram of a scenario for an object detection method provided according to an embodiment of this specification is shown.
[0045] Figure 1 The left side of the image shows object images for various scene types, including object image 1 for an indoor scene, object image 2 for an outdoor scene without sunlight, and object image 3 for an outdoor scene with sunlight. By inputting these three images into the object detection model, the detection results can be obtained, which of the object images 1-3 are images with real objects, such as images of living animals or plants, and which are images without real objects, such as planar images of animals or plants (non-living images of animals or plants).
[0046] It should be noted that, in order to adapt to object image realism detection in different scenarios, this embodiment proposes a cross-attention network, which can continuously learn information from new scenarios, enhance the expressive power of features, and improve the robustness of object realism detection. This addresses the problem of low object realism detection capability in different domains. In different environments, object realism detection is easily affected by different scenarios, such as indoor and outdoor scenes, day and night scenes, and different lighting conditions. The object realism detection model can achieve high accuracy in one scenario, but will show low pass rate and low realism detection capability in another scenario.
[0047] See Figure 2 , Figure 2 A flowchart of an object detection method according to an embodiment of this specification is shown, which specifically includes the following steps.
[0048] It should be noted that the object detection method provided in this embodiment can be applied to object authenticity detection scenarios to solve the problem of low object authenticity detection capability in different fields. In different environments, object authenticity detection is easily affected by different scenarios, such as indoor and outdoor scenarios, daytime and nighttime scenarios, and different lighting conditions.
[0049] Step 202: Obtain the image to be detected.
[0050] The image to be detected can be understood as any image containing a target object, including but not limited to faces, animals, plants, and objects. The image to be detected can also be any image containing a scene, such as an indoor scene object image or an outdoor scene object image.
[0051] In practical applications, the executing entity can acquire the image to be detected in order to accurately identify the target object in the image; for example, acquiring an image of an object in a sunlight scene and detecting the object in that image.
[0052] Step 204: Input the image to be detected into the object detection model to obtain the object detection result corresponding to the image to be detected.
[0053] The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The sample images in the first sample set and the second sample set are in different scenes, and the sample images include target objects.
[0054] In practical applications, when an image to be detected is obtained, it can be used as the input of an object detection model, which can output the object detection result corresponding to the image to be detected. It should be noted that the object detection model is pre-trained. Specifically, it can be trained using the model parameters in the first sample set and the reference model, while the reference model is trained using the second sample set. The training process can be referred to the description of the following embodiments.
[0055] Furthermore, the pre-training process of the object detection model can be referred to as follows; specifically, before inputting the image to be detected into the object detection model, the process further includes:
[0056] Obtain the first sample image in the first sample set and the first object label corresponding to the first sample image;
[0057] Obtain the model parameters from the reference model;
[0058] The first sample image and the model parameters are input into the object detection model to be trained to obtain the first prediction result;
[0059] Based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to obtain the object detection model.
[0060] The first sample set can be understood as consisting of a large number of first sample images and their corresponding first object labels. A first sample image can be understood as a sample image containing a target object within the target domain scene to be trained, such as an object image in an outdoor scene. A first object label can be understood as the detection result label for the first sample image, such as whether the detected object in an outdoor scene is a real object or a non-real object. It should be noted that the target domain can be understood as the scene domain in which the object detection model is primarily trained. For example, there is already an accurate model for detecting indoor object images, but this model cannot accurately detect object images in outdoor scenes. Therefore, the outdoor scene can be understood as the target domain. The purpose of this object detection model is to pre-train a model that can accurately detect object images in outdoor scenes.
[0061] The reference model can be understood as a reference model for accurate detection of object images in the source domain scene. This reference model is obtained in advance by training based on object images in the source domain scene. The model parameters of the reference model are also parameters adjusted through iterative training. It should be noted that the source domain can be understood as any scene domain. Using the previous example, the source domain can be understood as an indoor scene.
[0062] In practical applications, the first sample image and the corresponding first object label in the first sample set can be obtained first. In order for the object detection model to learn the image features in the source domain reference model, the model parameters in the reference model can be obtained, and the first sample image and the model parameters can be input into the object detection model to be trained to obtain the first prediction result. The first prediction result can be understood as the initial detection result predicted by the object detection model to be trained for the first sample image. Further, based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to complete the training process of the object detection model to be trained and obtain the object detection model.
[0063] Furthermore, the first object label corresponding to the first sample image can be obtained through a model or by manual labeling. However, in order to efficiently obtain the labels corresponding to sample images in various scenarios, this embodiment provides a method of constructing a preset feature extraction model. Specifically, obtaining the first object label corresponding to the first sample image includes:
[0064] Based on a preset feature extraction model, features are extracted from the first sample image to obtain a first feature vector;
[0065] Obtain the central feature vector corresponding to the second sample set, and calculate the vector distance between the first feature vector and the central feature vector;
[0066] Based on the vector distance, adjust the parameters in the preset feature extraction model to obtain the target feature extraction model;
[0067] Based on the target feature extraction model, the first object label corresponding to the first sample image is obtained.
[0068] In specific implementation, a second sample set, which is in a different scenario than the first sample set, can be used to participate in the process of determining the first object label. First, features can be extracted from the first sample image using a preset feature extraction model to obtain the first feature vector. Then, the central feature vector corresponding to the second sample set can be obtained, and the vector distance between the first feature vector and the central feature vector can be calculated. It should be noted that the central feature vector is the cluster center vector of all the second sample images in the second sample set after clustering. Furthermore, after obtaining the vector distance, the parameters in the preset feature extraction model can be adjusted according to the vector distance, and training can be carried out iteratively to obtain the target feature extraction model. Then, based on the trained target feature extraction model, the first object label corresponding to the first sample image can be accurately obtained.
[0069] In practical applications, this method of determining the labels of sample images in the target domain is to narrow the distance between the target domain corresponding to the first sample set and the source domain corresponding to the second sample set. Features of the sample images in the target domain can be extracted through a preset feature extraction model. Then, the class center can be calculated based on multiple sample images in the source domain, thereby determining the feature distance between the sample images in the target domain and the class center. Pseudo-labels are then assigned to the sample images in the target domain. The preset feature extraction model is then iteratively trained to make the labels corresponding to the sample images in the target domain increasingly accurate, thereby improving the training accuracy of the object detection model.
[0070] It should be noted that the K-means algorithm can be used to calculate the class centers in this embodiment, but it is not a limitation. Please refer to the following formula 1:
[0071]
[0072] Where ck is the class center; ft represents the features of image t; Let represent the probability that image t belongs to class k; we use the KNN strategy to label the target domain image by taking the nearest class center. Additionally, the new class centers can be obtained using the newly generated labels, as shown in Formulas 2 and 3 below:
[0073]
[0074]
[0075] Based on this, if the labels generated by the trained model are consistent with the labels corresponding to the source domain images, it can be considered that the training of the model is complete, and the final first object label is relatively accurate.
[0076] Specifically, the object detection model to be trained includes a cross-attention layer and a self-attention layer. The cross-attention layer is mainly used to learn model parameters in other domains and apply the learned model information to its own training. These two attention mechanisms can learn each other's feature information very well. It should be noted that the cross-attention layer and the self-attention layer are two parallel model processing layers. The sample images output by the two processing layers are the same, and both can be understood as the first sample image in the input target domain scene.
[0077] Accordingly, the step of inputting the first sample image and the model parameters into the object detection model to be trained to obtain the first prediction result includes:
[0078] The first sample image is input into the self-attention layer to obtain the first image vector;
[0079] Obtain the intra-domain parameters in the self-attention layer, and input the first sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the second image vector;
[0080] Based on the first image vector and the second image vector, a first target loss value is calculated, and a first prediction result is determined according to the first target loss value.
[0081] In practical applications, the first sample image can be input into the self-attention layer, and after processing by the self-attention layer, the first image vector can be obtained. Then, the intra-domain parameters in the self-attention layer, such as Qt, are obtained, representing the Q vector in the object detection model to be trained in the target domain scene. Simultaneously, the first sample image, the intra-domain parameters, and the model parameters in the reference model can be input into the cross-attention layer to obtain the second image vector. It should be noted that the model parameters in the reference model can be understood as the model parameters of a model in a source domain different from the target domain, i.e., out-of-domain parameters, such as Ks and Vs, representing the K and V vectors in the reference model in the source domain scene. Finally, based on the first and second image vectors, the first target loss value is calculated, and the first prediction result is determined based on the first target loss value.
[0082] This training method employs a self-attention mechanism to focus on information within the domain, where Q, K, and V information all come from the same domain, making the module more focused on information within the domain; it also employs a cross-attention mechanism to focus on information between domains, thus enabling the cross-attention module to have a strong feature representation ability.
[0083] Furthermore, considering that the first object label corresponding to the first sample image is obtained using a model, this embodiment also employs a distillation loss function, using a cross-attention branch for distillation, to impose certain constraints and supervision on the target domain from the perspective of category distribution; specifically, the calculation of the first target loss value based on the first image vector and the second image vector includes:
[0084] Based on the first image vector and the second image vector, a first distillation loss value is calculated, and based on the first image vector and the first object label, a first reference loss value is calculated.
[0085] A first target loss value is determined based on the first distillation loss value and the first reference loss value.
[0086] In practical applications, a first distillation loss value is calculated based on a first image vector and a second image vector, and the first image vector is supervised by the second image vector. Further, a first reference loss value is calculated based on the first image vector and the first object label, and then the first target loss value corresponding to the object detection model to be trained is determined based on the two loss values.
[0087] The distillation loss value can be achieved using the loss function, as shown in Formula 4 below:
[0088]
[0089] The overall loss function is as follows: Loss = α * L s +β*L t +γ*L s→t +δ*L dtl Where α, β, γ, δ, the sum of which equals 1; L s This represents the loss of the source domain; the others are similar.
[0090] Based on this, cross-attention branches are used for distillation to impose certain constraints and supervision on the target domain in terms of category distribution, and further allow the branches of the target domain to incorporate some information from the source domain, thereby enhancing the feature capabilities of the branches of the target domain.
[0091] The following embodiments provide a detailed description of the pre-training of the reference model. It should be noted that in this embodiment, the object detection model and the reference model learn information within the same domain from each other, thereby narrowing the distance between the target domain features and the source domain features, allowing for better feature alignment and enhancing the network's expressive power. Specifically, the reference model is pre-trained based on a second sample set and includes:
[0092] Obtain the second sample image and the second object label corresponding to the second sample image from the second sample set;
[0093] Obtain the model parameters from the object detection model;
[0094] The second sample image and the model parameters are input into the reference model to be trained to obtain the second prediction result;
[0095] Based on the second prediction result and the second object label, the model parameters in the reference model to be trained are adjusted to obtain the reference model.
[0096] The second sample set can be understood as consisting of a large number of second sample images and the second object labels corresponding to the second sample images. The second sample images can be understood as sample images with target objects in the source domain scene, such as object images in an indoor scene. The second object labels can be understood as the detection result labels of the second sample images, such as whether the object image in the indoor scene is a real object or a non-real object.
[0097] In practical applications, to train the reference model, we first obtain the second sample image and the second object label corresponding to the second sample image in the second sample set. At the same time, in order to improve the training accuracy of the reference model, we can also obtain the model parameters shared by the object detection model. We input the second sample image and the model parameters in the object detection model into the reference model to be trained to obtain the second prediction result. Based on the second prediction result and the second object label, we adjust the model parameters in the reference model to be trained. After iterative training, we obtain the reference model.
[0098] It should be noted that the training method of the reference model can also refer to the parameters trained in the object detection model, so that the two models can learn effective information from each other, thereby improving the processing accuracy of the two models.
[0099] Furthermore, the reference model to be trained has the same model structure as the object detection model mentioned above, and both can include a cross-attention layer and a self-attention layer. The cross-attention mechanism can continuously and effectively extract domain information of the target domain, which improves the shortcomings of the attention mechanism that only learns source domain information. Specifically, the reference model to be trained includes a cross-attention layer and a self-attention layer.
[0100] Accordingly, the step of inputting the second sample image and the model parameters into the reference model to be trained to obtain the second prediction result includes:
[0101] The second sample image is input into the self-attention layer to obtain the third image vector;
[0102] Obtain the intra-domain parameters in the self-attention layer, and input the second sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the fourth image vector;
[0103] Based on the third image vector and the fourth image vector, a second target loss value is calculated, and a second prediction result is determined according to the second target loss value.
[0104] In practical applications, the second sample image can be input into the self-attention layer, and after processing by the self-attention layer, a third image vector can be obtained. Then, the intra-domain parameters in the self-attention layer, such as Qs, are obtained, representing the Q vector in the reference model to be trained in the source domain scene. Simultaneously, the second sample image, intra-domain parameters, and model parameters from the object detection model can be input into the cross-attention layer to obtain a fourth image vector. It should be noted that the model parameters in the object detection model can be understood as the model parameters of the model in the target domain, which is different from the source domain, i.e., out-of-domain parameters, such as Kt and Vt, representing the K and V vectors in the reference model in the target domain scene. Finally, based on the third and fourth image vectors, the second target loss value is calculated, and the second prediction result is determined based on the second target loss value.
[0105] Furthermore, the step of calculating the second target loss value based on the third image vector and the fourth image vector includes:
[0106] Based on the third image vector and the fourth image vector, a second distillation loss value is calculated, and based on the third image vector and the second object label, a second reference loss value is calculated.
[0107] The second target loss value is determined based on the second distillation loss value and the second reference loss value.
[0108] In practical applications, the training process of the reference model can also refer to the above-mentioned training object detection model, using distillation loss value to calculate the final target loss value, thereby imposing certain constraints and supervision on the source domain from the perspective of category distribution, and further allowing the branches of the source domain to incorporate some information from the target domain, thereby enhancing the feature capabilities of the source domain branches.
[0109] Furthermore, this embodiment also provides a method for fine-tuning model parameters, which can further improve the accuracy of model processing; specifically, after obtaining the object detection result corresponding to the image to be detected, it also includes:
[0110] Receive feedback information from the front end corresponding to the detection results of the object;
[0111] Based on the feedback information, the model parameters in the object detection model are adjusted.
[0112] In practical applications, feedback information regarding object detection results sent from the front end can also be received. For example, users can score the object detection results or mark the results as correct or incorrect on the front end. This embodiment does not limit this. Furthermore, based on the feedback information, the model parameters in the object detection model can be fine-tuned to obtain a more accurate object detection model.
[0113] In summary, the object detection method provided in this application proposes a cross-attention model to improve the accuracy of object authenticity detection in different domains. This model can continuously learn information in new scenarios, enhance the expressive power of features, and improve the robustness of object authenticity detection. At the same time, it learns the cross-attention branches through distillation loss values to guide the features of the target domain, thereby achieving a high anti-counterfeiting effect in new scenarios.
[0114] See Figure 3 , Figure 3 A flowchart of another object detection method according to an embodiment of this specification is shown, which specifically includes the following steps.
[0115] It should be noted that the object detection method provided in this embodiment is applied to cloud-side devices; for ease of understanding, the object detection method is explained using the application scenario of object authenticity detection as an example.
[0116] Step 302: Receive an object detection request sent by the receiving end device, wherein the object detection request carries an image to be detected.
[0117] The image to be detected can be understood as any image containing a target object, including but not limited to faces, animals, plants, and objects. The image to be detected can also be any image containing a scene, such as an indoor scene object image or an outdoor scene object image.
[0118] In practical applications, after receiving an object detection request from the end device, the cloud-side device can perform subsequent object detection processing on the image to be detected carried in the object detection request, such as detecting whether the target object in the image to be detected is a real object.
[0119] Step 304: Input the image to be detected into the object detection model to obtain the object detection result corresponding to the image to be detected; wherein, the object detection model is trained based on the model parameters of the first sample set and the reference model, the reference model is pre-trained based on the second sample set, the first sample set and the second sample set are in different scenes, and the sample images include target objects.
[0120] The first sample set can be understood as consisting of a large number of first sample images and the first object labels corresponding to the first sample images; the first sample images can be understood as sample images with target objects corresponding to the target domain scene to be trained, such as object images in outdoor scenes; the target objects can be understood as the content in the sample images, including but not limited to faces, animals, plants, objects, etc.; the second sample set can be understood as consisting of a large number of second sample images and the second object labels corresponding to the second sample images; the second sample images can be understood as sample images with target objects in the source domain scene, such as object images in indoor scenes.
[0121] In practical applications, when an image to be detected is obtained, it can be used as the input of an object detection model, which can output the object detection result corresponding to the image to be detected. It should be noted that the object detection model is pre-trained. Specifically, it can be trained using the model parameters in the first sample set and the reference model, while the reference model is trained using the second sample set. The training process can be referred to the description in the above embodiments, and will not be elaborated further here.
[0122] Step 306: Send the object detection results to the front end and display them.
[0123] In practical applications, after obtaining the object detection results, the results can be fed back to the front end for display.
[0124] In summary, the object detection method provided in this application uses a reference model trained on sample images of one scene to help train an object detection model corresponding to sample images of another scene. This allows the object detection model to continuously learn the features of sample images in other scenes, thereby improving the object detection accuracy in the image scene of the first sample set. It can adapt to object detection in different scenes, solve the problem of object authenticity detection in new scenes, improve the accuracy of object authenticity detection, and enhance the object recognition effect.
[0125] See Figure 4 , Figure 4 A flowchart of an object detection model training method according to an embodiment of this specification is shown, which specifically includes the following steps.
[0126] It should be noted that the object detection model training method provided in this embodiment is applied to cloud-side devices; after the object detection model is trained, it can be stored in the cloud-side device for subsequent use.
[0127] Step 402: Obtain the first sample image in the first sample set and the first object label corresponding to the first sample image.
[0128] The first sample set can be understood as consisting of a large number of first sample images and the first object labels corresponding to the first sample images; the first sample image can be understood as a sample image with a target object corresponding to the target domain scene to be trained, such as an object image in an outdoor scene; the first object label can be understood as the detection result label of the first sample image, such as whether the detection result of the object image in an outdoor scene is a real object or a non-real object.
[0129] Step 404: Obtain the model parameters from the reference model.
[0130] The reference model can be understood as a reference model for accurately detecting object images in the source domain scene. This reference model is obtained by training in advance based on object images in the source domain scene. The model parameters of the reference model are also adjusted after iterative training. It should be noted that the source domain can be understood as any scene domain. Using the previous example, the source domain can be understood as an indoor scene.
[0131] Step 406: Input the first sample image and the model parameters into the object detection model to be trained to obtain the first prediction result.
[0132] The object detection model to be trained may include a cross-attention layer and a self-attention layer.
[0133] Step 408: Based on the first prediction result and the first object label, adjust the model parameters in the object detection model to be trained to obtain the object detection model.
[0134] In practical applications, the first sample image and the corresponding first object label in the first sample set can be obtained first. In order for the object detection model to be trained to learn the image features in the source domain reference model, the model parameters in the reference model can be obtained, and the first sample image and the model parameters can be input into the object detection model to be trained to obtain the first prediction result. The first prediction result can be understood as the initial detection result predicted by the object detection model to be trained on the first sample image. Further, based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted, thereby completing the training process of the object detection model to be trained and obtaining the object detection model.
[0135] In summary, the object detection model training method provided in this application uses a reference model trained on sample images of one scene to help train an object detection model corresponding to sample images of another scene. This allows the object detection model to continuously learn the features of sample images in other scenes, thereby improving the object detection accuracy in the image scene of the first sample set. It can adapt to object detection in different scenes, solve the object anti-counterfeiting effect in new scenes, improve the accuracy of object detection, and enhance the object recognition effect.
[0136] The following is in conjunction with the appendix Figure 5(a) and 5(b) Figure 5(a) shows a schematic diagram of the processing procedure of an object detection model training method provided in an embodiment of this specification, and Figure 5(b) shows a schematic diagram of the data flow of an object detection model training method provided in an embodiment of this specification.
[0137] Figure 5(a) includes two images, namely Image 1 and Image 2, which are object images in different scene domains. In this embodiment, a cross-attention network is trained to enable the network to accurately identify images in different domains.
[0138] Specifically, image 1 is input into the corresponding cross-attention layer and self-attention layer, respectively. After feature processing by these two layers, the fs vector and fs-t vector are obtained. Similarly, image 2 is input into the corresponding cross-attention layer and self-attention layer, respectively. After feature processing by these two layers, the ft-s vector and ft vector are obtained. During the training process, the two domains exchange model parameters based on a weight-sharing mechanism, allowing them to learn feature information from different domains. As shown in Figure 5(b), the self-attention layer uses its own domain-specific information to process the input, while the cross-attention layer uses external domain information, which is not information from the cross-attention layer itself.
[0139] In practical applications, a transformer structure is adopted, and patch blocks are fed into the cross-attention module. The cross-attention network includes two self-attention mechanisms and two cross-attention mechanisms. The two self-attention mechanisms can learn information from their respective domains well, while the two cross-attention mechanisms can learn information from the cross-domain. For example, in the s->t branch, the Q component of the source domain comes from the source domain, and the K and V components come from the target domain. The other cross-attention mechanism is symmetrical to this, so the cross-attention learns information from both domains. The cross-attention network adopts a weight-sharing mechanism, which reduces the computational cost of parameters. It can learn intermediate states of different modalities and has strong feature alignment capabilities. The self-attention mechanism focuses on intra-domain information, as Q, K, and V information all come from the same domain, making the module more focused on intra-domain information. The cross-attention mechanism focuses on inter-domain information, thus giving the cross-attention module strong feature representation capabilities.
[0140] It should be noted that the images, models, sample sets, and other information and data involved in the above method embodiments are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0141] Corresponding to the above method embodiments, this specification also provides embodiments of object detection devices. Figure 6 A schematic diagram of an object detection device according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:
[0142] Image acquisition module 602 is configured to acquire an image to be detected;
[0143] The object detection module 604 is configured to input the image to be detected into the object detection model to obtain the object detection result corresponding to the image to be detected.
[0144] The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The sample images in the first sample set and the second sample set are in different scenes, and the sample images include target objects.
[0145] Optionally, the device further includes:
[0146] The first model training module is configured to obtain a first sample image in the first sample set and a first object label corresponding to the first sample image;
[0147] Obtain the model parameters from the reference model;
[0148] The first sample image and the model parameters are input into the object detection model to be trained to obtain the first prediction result;
[0149] Based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to obtain the object detection model.
[0150] Optionally, the object detection model to be trained includes a cross-attention layer and a self-attention layer;
[0151] Optionally, the first model training module is further configured as follows:
[0152] The first sample image is input into the self-attention layer to obtain the first image vector;
[0153] Obtain the intra-domain parameters in the self-attention layer, and input the first sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the second image vector;
[0154] Based on the first image vector and the second image vector, a first target loss value is calculated, and a first prediction result is determined according to the first target loss value.
[0155] Optionally, the first model training module is further configured as follows:
[0156] Based on the first image vector and the second image vector, a first distillation loss value is calculated, and based on the first image vector and the first object label, a first reference loss value is calculated.
[0157] A first target loss value is determined based on the first distillation loss value and the first reference loss value.
[0158] Optionally, the device further includes:
[0159] The second model training module is configured as follows:
[0160] Obtain the second sample image and the second object label corresponding to the second sample image from the second sample set;
[0161] Obtain the model parameters from the object detection model;
[0162] The second sample image and the model parameters are input into the reference model to be trained to obtain the second prediction result;
[0163] Based on the second prediction result and the second object label, the model parameters in the reference model to be trained are adjusted to obtain the reference model.
[0164] Optionally, the reference model to be trained includes a cross-attention layer and a self-attention layer;
[0165] Optionally, the second model training module is configured as follows:
[0166] The second sample image is input into the self-attention layer to obtain the third image vector;
[0167] Obtain the intra-domain parameters in the self-attention layer, and input the second sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the fourth image vector;
[0168] Based on the third image vector and the fourth image vector, a second target loss value is calculated, and a second prediction result is determined according to the second target loss value.
[0169] Optionally, the second model training module is configured as follows:
[0170] Based on the third image vector and the fourth image vector, a second distillation loss value is calculated, and based on the third image vector and the second object label, a second reference loss value is calculated.
[0171] The second target loss value is determined based on the second distillation loss value and the second reference loss value.
[0172] Optionally, the first model training module is further configured as follows:
[0173] Based on a preset feature extraction model, features are extracted from the first sample image to obtain a first feature vector;
[0174] Obtain the central feature vector corresponding to the second sample set, and calculate the vector distance between the first feature vector and the central feature vector;
[0175] Based on the vector distance, adjust the parameters in the preset feature extraction model to obtain the target feature extraction model;
[0176] Based on the target feature extraction model, the first object label corresponding to the first sample image is obtained.
[0177] Optionally, the device further includes:
[0178] The parameter adjustment module is configured to receive feedback information sent by the front end corresponding to the detection result of the object;
[0179] Based on the feedback information, the model parameters in the object detection model are adjusted.
[0180] The object detection device provided in this embodiment obtains the object detection result corresponding to the image to be detected by inputting the acquired image to be detected into the object detection model. The object detection model is trained based on the model parameters of a first sample set and a reference model, while the reference model is trained based on a second sample set. The sample images in the two sample sets are located in different scenes. This reference model, trained with sample images from one scene, helps train the object detection model corresponding to sample images from another scene. This allows the object detection model to continuously learn the features of sample images from other scenes, thereby improving the object detection accuracy in the image scenes of the first sample set. It can adapt to object detection in different scenes, solve the problem of object authenticity detection in new scenes, improve the accuracy of object authenticity detection, and enhance the object recognition effect.
[0181] The above is a schematic scheme of an object detection device according to this embodiment. It should be noted that the technical solution of this object detection device and the technical solution of the object detection method described above belong to the same concept. For details not described in detail in the technical solution of the object detection device, please refer to the description of the technical solution of the object detection method described above.
[0182] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.
[0183] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.
[0184] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.
[0185] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.
[0186] The processor 720 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the object detection method described above.
[0187] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the object detection method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the object detection method described above.
[0188] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the object detection method described above.
[0189] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the object detection method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the object detection method described above.
[0190] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the object detection method described above.
[0191] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the object detection method described above belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the object detection method's technical solution.
[0192] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0193] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0194] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.
[0195] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0196] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.
Claims
1. An object detection method, comprising: Acquire the image to be detected; The image to be detected is input into the object detection model to obtain the object detection result corresponding to the image to be detected; The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The sample images in the first and second sample sets are in different scenes. The sample images include target objects. The object detection model includes a cross-attention layer and a self-attention layer. The cross-attention layer is used to learn the model parameters in the reference model. The self-attention layer is used to learn the information of the first sample image in the first sample set. The cross-attention layer and the self-attention layer learn each other's feature information.
2. The method according to claim 1, further comprising, before inputting the image to be detected into the object detection model: Obtain the first sample image in the first sample set and the first object label corresponding to the first sample image; Obtain the model parameters from the reference model; The first sample image and the model parameters are input into the object detection model to be trained to obtain the first prediction result; Based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to obtain the object detection model.
3. The method according to claim 2, wherein the object detection model to be trained comprises a cross-attention layer and a self-attention layer; Accordingly, the step of inputting the first sample image and the model parameters into the object detection model to be trained to obtain the first prediction result includes: The first sample image is input into the self-attention layer to obtain the first image vector; Obtain the intra-domain parameters in the self-attention layer, and input the first sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the second image vector; Based on the first image vector and the second image vector, a first target loss value is calculated, and a first prediction result is determined according to the first target loss value.
4. The method according to claim 3, wherein calculating the first target loss value based on the first image vector and the second image vector includes: Based on the first image vector and the second image vector, a first distillation loss value is calculated, and based on the first image vector and the first object label, a first reference loss value is calculated. A first target loss value is determined based on the first distillation loss value and the first reference loss value.
5. The method according to any one of claims 1-4, wherein the reference model is pre-trained based on a second sample set, comprising: Obtain the second sample image and the second object label corresponding to the second sample image from the second sample set; Obtain the model parameters from the object detection model; The second sample image and the model parameters are input into the reference model to be trained to obtain the second prediction result; Based on the second prediction result and the second object label, the model parameters in the reference model to be trained are adjusted to obtain the reference model.
6. The method according to claim 5, wherein the reference model to be trained comprises a cross-attention layer and a self-attention layer; Accordingly, the step of inputting the second sample image and the model parameters into the reference model to be trained to obtain the second prediction result includes: The second sample image is input into the self-attention layer to obtain the third image vector; Obtain the intra-domain parameters in the self-attention layer, and input the second sample image, the intra-domain parameters, and the model parameters into the cross-attention layer to obtain the fourth image vector; Based on the third image vector and the fourth image vector, a second target loss value is calculated, and a second prediction result is determined according to the second target loss value.
7. The method according to claim 6, wherein calculating the second target loss value based on the third image vector and the fourth image vector comprises: Based on the third image vector and the fourth image vector, a second distillation loss value is calculated, and based on the third image vector and the second object label, a second reference loss value is calculated. The second target loss value is determined based on the second distillation loss value and the second reference loss value.
8. The method according to claim 2, wherein obtaining the first object label corresponding to the first sample image includes: Based on a preset feature extraction model, features are extracted from the first sample image to obtain a first feature vector; Obtain the central feature vector corresponding to the second sample set, and calculate the vector distance between the first feature vector and the central feature vector; Based on the vector distance, adjust the parameters in the preset feature extraction model to obtain the target feature extraction model; Based on the target feature extraction model, the first object label corresponding to the first sample image is obtained.
9. The method according to claim 1, further comprising, after obtaining the object detection result corresponding to the image to be detected: Receive feedback information from the front end corresponding to the detection results of the object; Based on the feedback information, the model parameters in the object detection model are adjusted.
10. An object detection method, applied to cloud-side devices, comprising: The receiving end device sends an object detection request, wherein the object detection request carries an image to be detected; The image to be detected is input into the object detection model to obtain the object detection result corresponding to the image to be detected; The object detection model is trained based on the model parameters of a first sample set and a reference model. The reference model is pre-trained based on a second sample set. The first sample set and the second sample set contain sample images in different scenes. The sample images include target objects. The object detection model includes a cross-attention layer and a self-attention layer. The cross-attention layer is used to learn the model parameters in the reference model. The self-attention layer is used to learn the information of the first sample image in the first sample set. The cross-attention layer and the self-attention layer learn each other's feature information. The object detection results are sent to the front end for display.
11. A method for training an object detection model, applied to cloud-based devices, comprising: Obtain the first sample image in the first sample set and the first object label corresponding to the first sample image; Obtain the model parameters from the reference model; The first sample image and the model parameters are input into the object detection model to be trained to obtain a first prediction result. The object detection model includes a cross attention layer and a self attention layer. The cross attention layer is used to learn the model parameters in the reference model, and the self attention layer is used to learn the information of the first sample image. The cross attention layer and the self attention layer learn each other's feature information. Based on the first prediction result and the first object label, the model parameters in the object detection model to be trained are adjusted to obtain the object detection model.
12. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the method according to any one of claims 1 to 11.
13. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method of any one of claims 1 to 11.
14. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the method described in any one of claims 1 to 11.