Anchors-based dense embedding generation method, device and storage medium

Abstract

The application discloses an anchor point-based dense embedding generation method and device and a storage medium, which can be widely applied to embedding generation technology. The method comprises the following steps: obtaining a target image to be searched; inputting the target image into a target model; mapping the target image into a feature space by using the target model, comparing according to the distance in the feature space, and obtaining a search result; wherein the target model is obtained by training a deep neural network model by using dense embedding generated by an anchor point. The application uses the feature space near the anchor point to densely generate embedding features without corresponding data points for training, so that more accurate image search is realized, and the problem that existing sampling cannot sample effective samples due to the limitation of computing resources in deep metric learning is solved. In addition, the application can be directly integrated into an existing deep metric learning framework without any additional modification, and has significant technical effects.

Description

TECHNICAL FIELD

[0001] The present application relates to embedding generation technology, and in particular, to an anchor-based dense embedding generation method, device and storage medium. BACKGROUND

[0002] Deep metric learning is the basis of various applications, including face recognition, verification, image retrieval, image clustering, image classification, few-shot learning, video representation learning, and sound generation. Since its inception, it has attracted considerable interest in the research community, with scholars providing various methods and making substantial progress. The goal of deep metric learning is to learn a deep model that can map semantically similar data points to similar embeddings in the embedding space. To achieve this, most existing methods train the deep model using a loss function that brings embeddings from semantically similar data points closer to each other. However, some embeddings can contribute little or no improvement to training the deep model, or even cause undesirable local minima (such as model collapse) early in training. Therefore, sampling informative and stable embeddings is very important to facilitate the training of deep models. Therefore, improving sample quality is of great significance to achieve effective deep metric learning. There are two common measures to achieve this goal: designing more effective sampling methods or providing more embeddings.

[0003] Existing work has made great progress in designing effective sampling methods on embedding pairs or complete batch embeddings. These methods usually perform sampling on a batch of embeddings, but often result in inaccurate sampling results due to the following reasons. First, the batch size is usually limited by the memory of a single GPU, as the sampling process usually cannot be performed across different GPU devices; second, even if a GPU has enough memory to support a larger batch size, there can still be "barren" regions in the feature space embedded by deep models due to the lack of data points, resulting in the "missing embedding" problem as shown in Figure 1 Therefore, a limited number of embeddings can compromise sample quality and deep metric learning performance.

[0004] Recently, some attempts have been made to overcome the problem of inaccurate sampling caused by the lack of data points through pseudo-embedding generation. Difficult embedding generation methods use additional generative adversarial networks or autoencoders to generate difficult embeddings from simple embeddings. Embedding extension generation methods interpolate between embeddings to achieve feature space enhancement. Cross-batch memory preservation methods preserve the embeddings from the previous iteration and consider them still meaningful for sampling in the current batch. However, these methods either utilize additional subnetworks, introducing additional training costs, or require further modifications to the standard deep metric learning sampling and loss calculation process, which can limit their applicability in other tasks. SUMMARY

[0005] To at least partially solve one of the technical problems existing in the prior art, the present application aims to provide an anchor point-based dense embedding generation method, device and storage medium.

[0006] The technical solution adopted by the present application is:

[0007] An anchor point-based dense embedding generation method, comprising the following steps:

[0008] Obtaining a target image to be searched;

[0009] Inputting the target image into a target model;

[0010] Mapping the target image into a feature space by using the target model, and obtaining an image corresponding to a vector closest to each other according to a Euclidean distance between two vectors in the feature space as a search result;

[0011] The target model is obtained by training a deep neural network model using dense embeddings generated by anchor point embedding.

[0012] Further, the target model is obtained by training in the following way:

[0013] Inserting a pseudo-embedding generation layer before a sampling layer of the deep neural network model;

[0014] During the training process, the images obtained by sampling are first used as anchor points after being input into the deep neural network model, and the feature space near the anchor point embedding is used to generate pseudo-embeddings without data points;

[0015] By combining the embeddings with data points and the pseudo-embeddings without data points, more embeddings are provided to facilitate the sampling process, so that the sampling process can sample more effective sample pairs to train the deep neural network model, and obtain the target model.

[0016] Further, the pseudo-embeddings are generated in the following way, comprising:

[0017] Using scaling and shifting of features to generate pseudo-embeddings in the feature space around the anchor point embedding.

[0018] Further, the scaling and shifting of features to generate pseudo-embeddings in the feature space around the anchor point embedding, comprises:

[0019] Using a most discriminative feature scaling module to generate pseudo-embeddings in the feature space around a single anchor point embedding;

[0020] Using a past feature transformation shifting module to generate pseudo-embeddings in the feature space around multiple anchor point embeddings.

[0021] Further, the using most discriminative feature scaling module generates pseudo embeddings around a single anchor embedding using a feature space, including:

[0022] Utilizing an effective semantic information of an embedding is often highly activated, and the channels with the top K high activation values are identified as the effective semantic features of the anchor embedding;

[0023] The effective semantic features are randomly scaled to generate new pseudo embeddings.

[0024] Further, the using past feature transformation shift module generates pseudo embeddings around multiple anchor embeddings using a feature space, including:

[0025] Utilizing the semantic difference between similar embeddings that can be added to other embeddings to generate effective embeddings, for anchor embeddings of the same category, the feature displacement between two anchor embeddings is calculated as a semantic feature transformation;

[0026] The semantic feature transformation is stored in a memory bank;

[0027] The semantic feature transformation is obtained from the memory bank and added to the anchor embedding to generate a new pseudo embedding.

[0028] Further, the training process further includes the following steps:

[0029] Randomly select a training sample, and after generating a pseudo embedding, use a stochastic gradient descent algorithm to optimize to make a preset target loss function L converge. Wherein, the loss function L can be any existing deep metric learning loss function.

[0030] Another technical solution adopted by the present application is:

[0031] An anchor-based dense embedding generation device, comprising:

[0032] A target image acquisition module for acquiring a target image to be searched;

[0033] A target image input module for inputting the target image into a target model;

[0034] A search module for mapping the target image into a feature space using the target model, and obtaining an image corresponding to the closest vector as a search result according to the Euclidean distance between two vectors in the feature space;

[0035] Wherein, the target model is obtained by training a deep neural network model using dense embeddings generated by anchor embeddings.

[0036] Another technical solution adopted by the present application is:

[0037] An anchor point based dense embedding generation device, comprising:

[0038] at least one processor;

[0039] at least one memory for storing at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0041] Another technical solution adopted by the present application is:

[0042] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the method of claim.

[0043] The beneficial effects of the present application are: the present application uses the feature space near the anchor point to densely generate the embedding features without corresponding data points for training, so as to realize more accurate image retrieval, solve the problem that due to the limitation of computing resources in deep metric learning, the existing sampling cannot sample effective samples. In addition, the present application can be directly integrated into the existing deep metric learning framework without any additional modification, and has significant technical effects. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments of the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0045] Figure 1 is a schematic diagram of the missing embedding problem;

[0046] Figure 2 is a schematic diagram of the anchor point based dense embedding generation algorithm for deep metric learning in the embodiments of the present application;

[0047] Figure 3 is a schematic diagram of the most discriminative feature scaling module in the embodiments of the present application;

[0048] Figure 4 is a schematic diagram of the past feature transformation shift module in the embodiments of the present application. DETAILED DESCRIPTION

[0049] Embodiments of the present application are described below in detail with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only for the purpose of explaining the present application, and should not be understood as limiting the present application. For the step numbers in the following embodiments, they are only set for the convenience of illustrating the description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0050] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by the upper, lower, front, rear, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0051] In the description of the present application, several meanings are one or more, and the meaning of multiple is two or more, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If it is described as first, second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of indicated technical features.

[0052] In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more. The association relationship of the associated objects is described as "and / or", which means that there can be three relationships, for example, A and / or B can represent three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the front and rear associated objects.

[0053] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0054] In order to solve the existing technical problems, the present application proposes an anchor point based dense embedding generation algorithm to solve the problem that effective samples cannot be sampled due to the limitation of computing resources in deep metric learning. In order to realize more accurate image retrieval, the algorithm regards the features with corresponding data points as "anchor points" embedding, and utilizes the feature space near the anchor points to densely generate embedding features without corresponding data points for training.

[0055] The embodiment provides an anchor point based dense embedding generation method, comprising the following steps:

[0056] S1, obtaining a target image to be retrieved;

[0057] S2, inputting the target image into a target model;

[0058] S3, mapping the target image into a feature space by using the target model, and obtaining an image corresponding to a vector closest to the target image as a retrieval result according to a comparison of Euclidean distances between two vectors in the feature space.

[0059] The target model is obtained by training a deep neural network model using dense embeddings generated by anchor point embedding.

[0060] As an optional implementation, the target model is obtained by training in the following manner:

[0061] A1, initializing a frequency record matrix P and a memory base B for each class of training, and the two structures are used to track the feature distribution and historical transformation information of each class.

[0062] A2, obtaining a current batch of images and inputting them into a deep neural network to obtain current batch embeddings v, which represent the positions of each image in the feature space.

[0063] A3, updating the frequency record matrix P according to the current batch embeddings v, which enables the algorithm to track the semantic distribution of each class in the feature space.

[0064] A4, calculating a binary channel mask M according to the frequency record matrix P, which is used to identify the most discriminative features in the embeddings v.

[0065] A5, updating the memory base B according to the transformation between embeddings of the same class within the current batch, which enables the algorithm to obtain cross-batch and different-class feature transformations without being limited by GPU memory.

[0066] A6, scaling the most discriminative features of the embeddings v according to the binary channel mask M, and shifting the embeddings v according to the memory base B, to finally generate pseudo embeddings v ′ , which provide a dense and effective representation for training a deep metric learning model.

[0067] A7, concatenating the pseudo embeddings v ′ with the original embeddings v, and training them using both the real anchor point embeddings and the pseudo embeddings generated by the application, and jointly sampling and calculating a loss function.

[0068] A8, optimizing the model parameters by a stochastic gradient descent (SGD) algorithm, which is a commonly used optimization method suitable for training deep metric learning models.

[0069] A9、training is completed, the target model is obtained. The model can be used to realize accurate image retrieval or face recognition tasks.

[0070] At the same time, the pre-trained model described in steps A1-A9 refers to any deep metric learning neural network model, and the loss function refers to any deep metric learning loss function.

[0071] Wherein, the pseudo embedding is generated by the following way:

[0072] Using the most discriminative feature scaling module, the feature space around a single anchor embedding is used to generate pseudo embedding;

[0073] Using the past feature transformation shift module, the feature space around multiple anchor embeddings is used to generate pseudo embedding.

[0074] As an optional implementation, see Figure 3 , using an embedded effective semantic information is often highly activated, the feature space around a single anchor embedding is used to generate pseudo embedding, including:

[0075] Identify the channel with the top K high activation value as the effective semantic feature of the anchor embedding.

[0076] Randomly scale the above effective semantic feature to generate a new pseudo embedding.

[0077] As an optional implementation, see Figure 4 , using the semantic difference between similar embeddings can be added to other embeddings to generate effective embeddings, the feature space around multiple anchor embeddings is used to generate pseudo embedding, including:

[0078] For anchor embeddings of the same category within the batch, the feature displacement between each other is calculated as the semantic feature transformation.

[0079] Store the semantic feature transformation in the memory bank.

[0080] Add the semantic transformation from the memory bank to the anchor embedding to generate a new pseudo embedding.

[0081] The above method is described in detail below in combination with the drawings and specific embodiments.

[0082] As shown in Figure 2 , the embodiment provides an anchor-based dense embedding generation method for deep metric learning, including the following steps:

[0083] (a) Symbolic explanation

[0084] Representing a deep neural network (such as a convolutional deep neural network) as follows, and letting the input image be I, the embedding generated after passing through the deep neural network is represented as: Where d is the feature dimension of the embedding. In this embodiment, both v and v′ are normalized, i.e., ||v||², ||v′||² = 1. C represents the number of classes in the training set.

[0085] (b) Initialize the frequency record matrix and memory for each class in the training set.

[0086] Specifically, for the Frequency Recording Matrix (FRM), we first... Initialize to a zero matrix. For each class c, a set of embeddings is represented as {v|y v =c}.

[0087] For memory libraries, due to significant inter-class differences, most inter-class transformations are typically not transferable. Therefore, we only consider intra-class embeddings to obtain transformations. In deep metric learning, sampling only two images per class in a batch consistently yields good performance, resulting in a very limited number of transformations obtainable in a single batch (i.e., two transformations). To address this issue, we construct a memory library B to store transformations from previous iterations, ensuring transformation diversity. Specifically, we construct a transformation library B∈R C×Z×d , where Z is the memory capacity of each class.

[0088] It is important to note that each memory bank B is a first-in-first-out (FIFO) queue. When the queue is full, the head of the queue (the earliest transformation to enter the queue) will be dequeued.

[0089] (c) Update the frequency record matrix for the current batch embedding and calculate the binary channel mask.

[0090] Given a set of embeddings of class c, denoted as {v|y v =c}, the present invention updates P in the following manner:

[0091]

[0092] Here, Top(v,K) is an operator used to select the top K elements from vector v. During training, we continuously update the frequency record matrix by recording the positions of highly activated neurons from the embeddings of the same class. In this way, the updated frequency record matrix can serve as a stable, accurate, and efficient semantic identifier for the training class. Given the frequency record matrix, we compute the binary channel mask M∈R for each class. C×d :

[0093]

[0094] (d) Update the memory bank with the current batch of embeddings

[0095] During training, once we have a set of embeddings V c = {v | y v＝c}, we compute the transformations between them by the following equation:

[0096] t z = v i - v j , v i , v j ∈ V c , i≠j.

[0097] Then, we queue the semantic transformations into B according to the first-in-first-out principle to ensure that the transformations in the bank are in a relatively new state:

[0098] B[y v , z] = t z , z ∈ {1, 2, …, Z}.

[0099] Note that z is reset to 1 when the queue is full.

[0100] (e) For the current batch of embeddings, compute the semantic scaling and shifting factors

[0101] Given an embedding v, we compute the semantic scaling

[0102] s = γ ⊙ M[y v ] + 1 d ⊙ (1 - M[y v ])

[0103] where γ ∈ R d , and γ ~ Uniform[1 - r s , 1 + r s ] d , where r s ∈ (0, 1) is a hyperparameter to be set. Note that we only randomly scale the distinctive features, while preserving the indistinctive ones. To produce multiple scaling factors, we repeatedly sample γ from the uniform distribution.

[0104] Finally, with the help of the memory bank B, given an embedding v, we retrieve the semantic shifting factor as follows:

[0105] b = r b t, t ~ {B[y v , z] | v = 1, 2, …, Z}.

[0106] w b is a hyperparameter. Multiple shifting factors are formed by repeated sampling.

[0107] (f) For the current batch of embeddings, generate pseudo-embeddings using semantic scaling and shifting factors

[0108] Specifically, we first suggest exploring the embedding space around a single embedding by enhancing or weakening the semantics of the single embedding. We call this process semantic scaling.

[0109] Second, we suggest adding (intra-class differences (semantic transformations) in the embeddings to exploit the embedding space between multiple embeddings. We call this process semantic shifting. Based on the above analysis, the pseudo-embedding r' generation formula is:

[0110]

[0111] Subsequently, the pseudo-embedding v' and the embedding v are input to the subsequent sampling step together.

[0112] (g) Training the model

[0113] The overall algorithm of the present application is detailed in (a)-(f) above. Given the anchor point label pair {(v, y v )}, we generate the pseudo-embedding label pair {(v', y' v )} without corresponding data points by the scheme proposed in the present application, where y' v = y v , because the class semantics are preserved. Then, the embedding label pairs with or without data points are fed into the sampling module to obtain the positive embedding set and the negative embedding set (e.g. pairs, triplets, etc. specified by the sampling and loss function): {(P, N)} = Sample({(v, y v )}∪{(v', y' v )}). Finally, given a deep learning loss function L DML , it is noted that the present application is compatible with existing deep metric learning sampling methods and loss functions. The deep metric learning objective function based on the present application is formulated as:

[0114] The above technical solution only embodies the preferred technical solution of the technical solution of the present application, and some variations of certain parts made by the person skilled in the art of the present application all embody the principle of the present application and belong to the protection scope of the invention.

[0115] We apply the above anchor-based dense embedding generation algorithm to the classic three image retrieval datasets: CUB-200 (CUB), which is a fine-grained bird dataset, the first 100 classes are used for training, and the other 100 classes are used for testing. CARS-196 (CARS), a fine-grained vehicle dataset, the first 98 classes are used for training, and the other 98 classes are used for testing. SOP, a large online product dataset, the training and test partitions are 11318 and 11316 classes, respectively. The anchor-based dense embedding generation algorithm proposed in this embodiment is used to compare with existing methods.

[0116] The recognition results are shown in Tables 1 and 2 as follows:

[0117] Table 1: Comparison results with the latest method

[0118]

[0119] Table 2: Comparison results with pair-based methods

[0120]

[0121] In combination with Tables 1 and 2 above, it can be seen that the accuracy of the results obtained by calculation confirms that the technical solution provided by the embodiment of the present application has practicality and feasibility.

[0122] In summary, the method of the present embodiment has at least the following advantages and beneficial effects compared with the prior art:

[0123] (1) The present application introduces a most discriminative feature scaling module and a past feature transformation shift module, which can generate more effective pseudo-embeddings for sampling.

[0124] (2) The present application does not need to further modify the sampling and loss calculation process of the standard deep metric learning, and can be directly integrated into the existing deep metric learning framework.

[0125] (3) The present application uses pseudo-embeddings v' and original embeddings v together for sampling, which can fully utilize the feature space without corresponding data points, without relying on a large amount of data, thereby improving the generalization ability of the model on real data.

[0126] (4) Since the feature space near the anchor embedding is fully utilized, and the diversity of pseudo-embedding generation can be controlled by controlling the hyperparameters, the present application can better capture the subtle differences between different categories, thereby improving the discrimination ability of the model.

[0127] (5) The application is applicable to any existing deep metric learning framework and can improve performance. This means that the algorithm has high universality and can be applied to different types of deep metric learning tasks, thereby helping to achieve more accurate image retrieval or face recognition and the like.

[0128] The embodiment also provides an anchor point-based dense embedding generation device, comprising:

[0129] at least one processor;

[0130] at least one memory for storing at least one program;

[0131] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 2 the method.

[0132] The anchor point-based dense embedding generation device of the embodiment can execute the anchor point-based dense embedding generation method provided by the method embodiment of the application, can execute the implementation steps of any combination of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0133] The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 2 the method.

[0134] The embodiment also provides a storage medium storing instructions or programs that can execute the anchor point-based dense embedding generation method provided by the method embodiment of the application. When the instructions or programs are executed, the implementation steps of any combination of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method are possessed.

[0135] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0136] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary to an understanding of the application. Rather, the actual implementation of the modules, in combination with their attributes, functions, and internal relationships, are to be understood within the context of the devices disclosed herein. Thus, those skilled in the art with access to patents, scientific journals, and other public sources known by those skilled in the art will be able, using ordinary skill, to practice the application as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and that the scope of the present application is to be determined by the entire scope of the claims, along with all equivalents of the claims and their equivalents.

[0137] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products, which can be incorporated into a computer- readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0138] The logic and / or steps represented in the flowcharts and / or otherwise described herein, for example, can be embodied in non-transitory computer-readable media, which can be executed by an instruction execution system, apparatus, or device such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0139] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0140] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above described embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or combinations thereof, can be used with the necessary modifications: a discrete logic circuit(s) having logic gates for implementing logic functions upon an application of data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), and / or the like.

[0141] In the above description of the present specification, reference to the description of the terms "one embodiment", "another embodiment", or "certain embodiments" or the like means that a particular feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above described terms in the present specification are not necessarily referred to the same embodiment or example. Also, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0142] Although embodiments of the present application have been shown and described, it would be recognized by those of ordinary skill in the art that various changes, modifications, alternatives, and variations can be made thereto without departing from the spirit and scope of the application, which would be limited only by the scope of the claims and the equivalents thereof.

[0143] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above-described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope of the claims of the present application.

Claims

1. An anchor-based dense embedding generation method, characterized in that, The method comprises the following steps: obtaining a target image to be searched; inputting the target image into a target model; mapping the target image into a feature space by using the target model, and obtaining an image corresponding to a vector closest to another vector in the feature space as a search result according to comparison of Euclidean distances between the two vectors; wherein the target model is obtained by training a deep neural network model using dense embeddings generated by anchor point embeddings; the target model is obtained by training in the following way: inserting a pseudo-embedding generation layer before a sampling layer of the deep neural network model; in the training process, after an image obtained by sampling is input into the deep neural network model, first, an embedding with data points is used as an anchor point, and a feature space near the anchor point embedding is used to generate a pseudo-embedding without data points; by combining the embedding with data points and the pseudo-embedding without data points, more embeddings are provided to promote the sampling process, so that the sampling process can sample more effective samples to train the deep neural network model, and obtain the target model; the pseudo-embedding is generated in the following way, comprising: generating pseudo-embeddings in the feature space around the anchor point embedding using scaling and shifting of features; the generating of the pseudo-embeddings in the feature space around the anchor point embedding using scaling and shifting of features comprises: generating pseudo-embeddings around a single anchor point embedding using the feature space by using a most discriminative feature scaling module; generating pseudo-embeddings around multiple anchor point embeddings using the feature space by using a past feature transformation shifting module.

2. The anchor-based dense embedding generation method of claim 1, wherein, the generating of the pseudo-embeddings around a single anchor point embedding using the feature space by using the most discriminative feature scaling module comprises: identifying channels with top K high activation values as effective semantic features of the anchor point embedding; randomly scaling the effective semantic features to generate new pseudo-embeddings.

3. The anchor-based dense embedding generation method of claim 1, wherein, the generating of the pseudo-embeddings around multiple anchor point embeddings using the feature space by using the past feature transformation shifting module comprises: for anchor point embeddings of the same category, respectively calculating feature displacements between two anchor point embeddings as semantic feature transformations; storing the semantic feature transformations in a memory bank; adding the semantic feature transformations to the anchor point embeddings to generate new pseudo-embeddings.

4. The anchor-based dense embedding generation method of claim 1, wherein, the training process further comprises the following steps: After generating pseudo-embedding, the random gradient descent algorithm is used for optimization to make the preset target loss function converge.

5. An anchor-based dense embedding generation apparatus for implementing the method of any one of claims 1 to 4, characterized in that, comprising: a target image obtaining module for obtaining a target image to be searched; a target image inputting module for inputting the target image into a target model; a searching module for mapping the target image into a feature space by using the target model, and obtaining an image corresponding to a vector closest to another vector in the feature space as a search result according to comparison of Euclidean distances between the two vectors; wherein the target model is obtained by training a deep neural network model using dense embeddings generated by anchor point embeddings.

6. An anchor-based dense embedding generation apparatus, comprising: comprising: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the method of any one of claims 1 to 4.

Images