A trademark similarity retrieval system and method based on multi-dimensional feature fusion

Abstract

The present invention provides a trademark similarity retrieval method with multi-dimensional feature fusion, which fuses the features of three dimensions: convolutional neural network, visual word bag, and graphic elements, to achieve a trademark similarity retrieval method with more semantic information and recognition. First, based on the convolutional neural network, multi-scale convolutional neural network features are extracted, and the multi-scale convolutional neural network features are optimized and trained based on the triple loss function. At the same time, the visual word bag features of the trademark image are extracted, and combined with the trademark graphic elements, the complementary information between the features of multiple dimensions is considered, and based on the features of three dimensions: multi-scale convolutional neural network features, visual word bag features, and graphic element features, the correlation and complementarity between features of different dimensions are fully utilized and mined, thereby improving the comprehensive trademark retrieval effect.

Description

Technical Field

[0001] The present invention relates to the field of image search technology, and in particular to a trademark similarity retrieval system and method based on multi-dimensional feature fusion. Background Art

[0002] A trademark is a distinctive sign that identifies a product, service, or a specific individual or company related to it. A registered trademark is a legal property that needs to be protected to prevent trademark infringement. The Trademark Office is responsible for trademark review and registration, administrative adjudication, etc. With the rapid development of the economy and the increasing number of companies, the number of trademark registrations has continued to grow, making it more difficult for the Trademark Office to review and manage trademarks.

[0003] For new trademark applications, the Trademark Office will review the trademark to ensure that the new trademark does not imitate the registered trademark and has sufficient differences from the registered trademark. At present, the Trademark Office's method of trademark review is mainly to search for trademarks through manually annotated text information and graphic codes. The search accuracy and efficiency of this method are relatively limited, and the workload of manual annotation and review is very large, which makes the processing of the growing number of trademark registration applications face important challenges.

[0004] With the development of computer image processing technology, retrieval methods based on trademark image content have emerged. This method does not rely on manually annotated information, but extracts corresponding image features from trademark images, performs similarity matching through image features, and then retrieves similar trademarks. Most existing methods use traditional image retrieval methods based on simple single image features. However, trademark images are usually composed of abstract graphics and symbols, which are highly abstract and complex. The semantic gap between computer representation of trademark images and human cognition makes it difficult for existing methods to understand trademark images, which in turn affects the accuracy and efficiency of trademark retrieval.

[0005] Therefore, in response to the above-mentioned problems existing in trademark retrieval, the present invention proposes a trademark similarity retrieval method and system with multi-dimensional feature fusion, which utilizes the three-dimensional features of convolutional neural network, visual word bag, and graphic elements to provide a trademark similarity retrieval with more semantic information and recognition, and achieves better trademark retrieval effect. Summary of the invention

[0006] In order to solve the problem that when a single image feature is used in the prior art, since trademark images are usually composed of abstract graphics and symbols and have strong abstractness and complexity, there is a semantic gap between the computer's representation of trademark images and human cognition, which makes it difficult for existing methods to understand trademark images, thereby affecting the accuracy and efficiency of trademark retrieval, the present invention provides a trademark similarity retrieval system and method based on multi-dimensional feature fusion, which utilizes the features of three dimensions of convolutional neural network, visual word bag and graphic elements to provide trademark similarity retrieval with more semantic information and recognition, achieves better trademark retrieval effect, and solves the above-mentioned problem.

[0007] The present invention provides a trademark similarity search system based on multi-dimensional feature fusion, including a trademark database, a feature search module, and a feature extraction module. The trademark database is connected to the feature extraction module, and the feature extraction module is connected to the feature search module.

[0008] The feature extraction module includes a neural network feature module, a visual word bag feature module and a graphic element module. The neural network feature module is connected to the trademark database and the feature retrieval module. The visual word bag feature module is connected to the trademark database and the feature retrieval module. The graphic element module is connected to the trademark database. The neural network feature module is used to extract image multi-scale convolutional neural network features through the trademark database and optimize the neural network based on the triple metric loss function and output the multi-scale convolutional neural network features to the feature retrieval module. The visual word bag feature module is used to construct a visual dictionary based on the extracted image key point features through the trademark database and output the image visual word bag features extracted based on the visual dictionary to the feature retrieval module. The graphic element module is used to establish an index library of registered trademark graphic elements and manually input the graphic element features to the feature retrieval module through the query personnel.

[0009] Neural network feature module: This module is responsible for extracting multi-scale convolutional neural network features of images and optimizing the neural network training based on the triple metric loss function;

[0010] Visual word bag feature module: This module is responsible for extracting image key point features and building a visual dictionary, and finally outputs the image visual word bag features extracted based on the visual dictionary;

[0011] Graphic element feature module: This module is responsible for establishing an index library of registered trademark graphic elements through a full-text search engine, and the graphic element features of registered trademarks are manually input by search personnel;

[0012] Feature retrieval module: This module is responsible for calling the feature module to extract the multi-scale convolutional neural network features, visual word bag features and graphic element features of the trademark image to be registered during the retrieval stage, and calculates the similarity between the trademark to be registered and the trademarks in the registered trademark library, and returns the retrieval results by sorting the similarities.

[0013] The present invention provides a trademark similarity retrieval system based on multi-dimensional feature fusion. As a preferred embodiment, the neural network feature module includes a residual network, a first fully connected layer, a second fully connected layer, a first convolutional layer, a second convolutional layer, a first pooling layer and a second pooling layer. The residual network, the first convolutional layer and the second convolutional layer are all connected to the trademark database, the residual network data is connected to the first fully connected layer, the first convolutional layer data is connected to the first pooling layer, the second convolutional layer data is connected to the second pooling layer, the first fully connected layer, the first pooling layer and the second pooling layer are all connected to the second fully connected layer, and the second fully connected layer transmits the multi-scale convolutional neural network features to the feature retrieval module.

[0014] The present invention provides a trademark similarity search method based on multi-dimensional feature fusion, comprising the following steps:

[0015] S1. Establish a trademark database;

[0016] S2. Input the trademark to be registered through the feature search module;

[0017] S3. According to the trademark to be registered, the neural network feature module extracts the multi-scale convolutional neural network features of the trademark image through the trademark database and transmits them to the feature retrieval module, the visual word bag feature module extracts the visual word bag features of the trademark image through the trademark database and transmits them to the feature retrieval module, and the graphic element module extracts the graphic element features of the trademark image through the trademark database and transmits them to the feature retrieval module;

[0018] S4. Based on the multi-scale convolutional neural network features, visual word bag features and graphic element features, the trademark to be registered is matched with the trademarks in the registered trademark database to obtain the fusion retrieval results of the three dimensional features.

[0019] The trademark similarity search method based on multi-dimensional feature fusion of the present invention is a preferred method. In step S4, the trademark to be registered is matched with the trademarks in the registered trademark database for similarity, and the specific calculation formula for obtaining the fusion search result of the three-dimensional features is:

[0020] Score(a,d i )

[0021] =α*Score CNN (a,d i )+β*Score BoVW (a,d i)+Score element (a,d i )

[0022] Among them, Score (a, d i ) is the trademark a to be registered and the i-th trademark d in the registered trademark database i The total similarity, Score CNN is the similarity of multi-scale convolutional neural network features, Score BoVW is the similarity of the visual word bag feature, Score element is the similarity of graphic element features, and α and β are weight parameters.

[0023] The trademark similarity search method based on multi-dimensional feature fusion of the present invention is preferably used in step S3 where the neural network feature module extracts the multi-scale convolutional neural network features of the trademark image and transmits them to the feature search module in the following specific manner:

[0024] S311, the trademark image is input into the residual network, the first convolutional layer and the second convolutional layer respectively, the residual network is initialized with ImageNet pre-training parameters, and the Average Pooling layer in the residual network is extracted to obtain the first feature through the first fully connected layer;

[0025] S312: the first convolution layer and the second convolution layer use different Padding values ​​and Stride values, and obtain the second feature and the third feature through the first pooling layer and the second pooling layer respectively;

[0026] S313, the first feature, the second feature and the third feature are all L2 regularized;

[0027] S314, the regularized first feature, the second feature, and the third feature are concatenated and input into the second fully connected layer;

[0028] S315, the second fully connected layer obtains multi-scale convolutional neural network features through linear mapping.

[0029] The trademark similarity search method based on multi-dimensional feature fusion of the present invention, as a preferred embodiment, the neural network feature module in step S3 extracts the multi-scale convolutional neural network features of the trademark image and transmits them to the feature search module, further comprising:

[0030] S316. Optimizing the parameters of the multi-scale convolutional neural network through a triplet metric loss function to mine and learn the semantic information of the registered trademark image;

[0031] The loss function is expressed as:

[0032] Loss(p,q,r)

[0033] =max(0,margin+D(f(q),f(p))-D(f(q),D(f(n))))

[0034] Among them, Loss(p, q, r) is the metric loss function of the triple (p, q, r), p, q, r are the current trademark, the trademark similar to the current trademark and the non-similar trademark to the current trademark respectively, f(·) is the output feature of the multi-scale convolutional neural network, D(·) is the cosine distance between the two input features, and margin is the margin parameter.

[0035] In the trademark similarity search method based on multi-dimensional feature fusion of the present invention, as a preferred method, the specific method in which the visual word bag feature module in step S3 extracts the visual word bag features of the trademark image and transmits them to the feature search module is:

[0036] S321, use Harris, Hessian, and Kaze local detectors to extract key points of the image;

[0037] S322, extracting key point features from image key points through Sift descriptors;

[0038] S323, key point features are clustered through Kmeans to build a visual dictionary;

[0039] S324, extracting visual bag-of-words features from the image through a visual dictionary.

[0040] On the one hand, this method extracts multi-scale convolutional neural network features based on convolutional neural networks, and further models the semantic information of trademarks by optimizing and training the multi-scale convolutional neural network features based on a triplet loss function. At the same time, the present invention takes into account that a large amount of complementary information is contained between multiple dimensional features, and based on the features of the three dimensions of multi-scale convolutional neural network features, visual word bag features, and graphic element features, it fully utilizes and mines the correlation and complementarity between features of different dimensions, thereby achieving better retrieval accuracy.

[0041] The beneficial effects of the present invention are as follows:

[0042] (1) The residual network in the multi-scale convolutional neural network focuses on learning high-level semantic features, while the other two shallow networks focus on learning the content features of the image itself. At the same time, based on the optimization training of the triplet metric loss function, the multi-scale convolutional neural network features can better model the relative similarity of trademarks, thereby achieving better retrieval results;

[0043] (2) The bag of visual words focuses on the local features of the image content, and the graphic encoding reflects people's high-level semantic understanding of the trademark. The fusion of three dimensional features, multi-scale convolutional neural features, bag of visual words features, and graphic encoding features, enables the advantages of different dimensional features to complement each other, thereby improving the comprehensive trademark retrieval effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of a trademark similarity search system based on multi-dimensional feature fusion;

[0045] Figure 2 It is a schematic diagram of a feature extraction module of a trademark similarity retrieval system based on multi-dimensional feature fusion;

[0046] Figure 3 It is a schematic diagram of the structure of a neural network feature module of a trademark similarity retrieval system based on multi-dimensional feature fusion;

[0047] Figure 4 The figure is a flow chart of a trademark similarity retrieval method based on multi-dimensional feature fusion.

[0048] Reference numerals:

[0049] 1. Trademark database; 2. Feature retrieval module; 3. Feature extraction module; 31. Neural network feature module; 311. Residual network; 312. First fully connected layer; 313. Second fully connected layer; 314. First convolutional layer; 315. Second convolutional layer; 316. First pooling layer; 317. Second pooling layer; 32. Visual word bag feature module; 33. Graphic element module. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0051] Example 1

[0052] like Figure 1 As shown, it includes a trademark database 1, a feature retrieval module 2, and a feature extraction module 3. The trademark database 1 is connected to the feature extraction module 3, and the feature extraction module 3 is connected to the feature retrieval module 2.

[0053] like Figure 2As shown, the feature extraction module 3 includes a neural network feature module 31, a visual word bag feature module 32 and a graphic element module 33. The neural network feature module 31 is connected to the trademark database 1 and the feature retrieval module 2. The visual word bag feature module 32 is connected to the trademark database 1 and the feature retrieval module 2. The graphic element module 33 is connected to the trademark database 1. The neural network feature module 31 is used to extract the image multi-scale convolutional neural network features through the trademark database 1 and optimize the neural network based on the triple metric loss function and output the multi-scale convolutional neural network features to the feature retrieval module 2. The visual word bag feature module 32 is used to construct a visual dictionary based on the extracted image key point features through the trademark database 1 and output the image visual word bag features extracted based on the visual dictionary to the feature retrieval module 2. The graphic element module 33 is used to establish an index library of registered trademark graphic elements and output graphic element features to the feature retrieval module 2.

[0054] like Figure 3 As shown, the neural network feature module 31 includes a residual network 311, a first fully connected layer 312, a second fully connected layer 313, a first convolutional layer 314, a second convolutional layer 315, a first pooling layer 316 and a second pooling layer 317. The residual network 311, the first convolutional layer 314 and the second convolutional layer 315 are all connected to the trademark database 1. The residual network 311 data is connected to the first fully connected layer 312, the first convolutional layer 314 data is connected to the first pooling layer 316, the second convolutional layer 315 data is connected to the second pooling layer 317, the first fully connected layer 312, the first pooling layer 316 and the second pooling layer 317 are all connected to the second fully connected layer 313, and the second fully connected layer 313 transfers the multi-scale convolutional neural network features to the feature retrieval module 2.

[0055] like Figure 4 As shown, a trademark similarity search method based on multi-dimensional feature fusion includes the following steps:

[0056] S1. Establishing a trademark database 1

[0057] Convert all images of registered trademarks and trademarks to be registered into grayscale images to eliminate the interference of color on search results, and store all images in JPG format;

[0058] S2. Input the trademark to be registered through the feature search module 2;

[0059] S3. According to the trademark to be registered, the neural network feature module 31 extracts the multi-scale convolutional neural network features of the trademark image through the trademark database 1 and transmits them to the feature retrieval module 2. The visual word bag feature module 32 extracts the visual word bag features of the trademark image through the trademark database 1 and transmits them to the feature retrieval module 2. The graphic element module 33 extracts the graphic element features of the trademark image through the trademark database 1 and transmits them to the feature retrieval module 2.

[0060] S4. Based on the multi-scale convolutional neural network features, visual word bag features and graphic element features, the trademark to be registered is matched with the trademarks in the registered trademark database to obtain the fusion retrieval results of the three dimensional features.

[0061] The specific calculation formula for similarity matching between the trademark to be registered and the trademarks in the registered trademark database to obtain the fusion search results of the three-dimensional features is:

[0062] Score(a,d i )

[0063] =α*Score CNN (a,d i )+β*Score BoVW (a,d i )+Score element (a,d i )

[0064] Among them, Score (a, d i ) is the trademark a to be registered and the i-th trademark d in the registered trademark database i The total similarity, Score CNN is the similarity of multi-scale convolutional neural network features, Score BoVW is the similarity of the visual word bag feature, Score element is the similarity of the graphic element features, α and β are weight parameters, in the implementation, α is taken as 0.6, and β is taken as 0.4. i ) Sort the trademarks in the registered trademark database from large to small, and return the sorted result as the search result of the trademark to be registered.

[0065] In the specific implementation, the MAP indicator is used to evaluate the accuracy of trademark search. MAP is a common statistical indicator of search results, and its definition is as follows:

[0066]

[0067] Where R is the total number of positive samples in the trademark database, R k Indicates the number of positive samples in the first k returned results, rel k Indicates whether the kth returned result is a positive sample, if it is a positive sample, it is 1, otherwise it is 0. The above formula defines the AP value of a single query, and the MAP indicator is the average AP value of all queries.

[0068] In this embodiment, the image is first unified into a size of 224*224 pixels. The first path uses the structure of ResNet connecting the first fully connected layer 313. In this implementation, the ResNet101 network initialized with pre-trained parameters of the ImageNet data set (www.image-net.org) is used, and the 2048-dimensional features of the Average Pooling layer in the network are extracted and input into the first fully connected layer 313. The first fully connected layer 313 uses a linear mapping of the number of hidden neurons of 4096. The other two paths both use the structure of a convolutional layer plus a pooling layer, but the Padding values ​​of the convolutional layer are 1 and 4 respectively, and the Stride values ​​are 16 and 32 respectively, and both obtain features with a dimension of 1536 through the pooling layer. The features obtained by the three-way network are all L2 regularized, and then spliced ​​and input into the second fully connected layer 313. The second fully connected layer 313 also uses a linear mapping of the number of hidden neurons of 4096, and finally obtains a 4096-dimensional multi-scale convolutional neural network feature.

[0069] In step S3, the neural network feature module 31 extracts the multi-scale convolutional neural network features of the trademark image and transmits them to the feature retrieval module 2 in the following specific manner:

[0070] S311, the trademark image is respectively input into the residual network 311, the first convolutional layer 314 and the second convolutional layer 315, the residual network 311 initialized with the ImageNet pre-training parameters, and the first feature of the Average Pooling layer in the residual network 311 is extracted into the first fully connected layer 312;

[0071] S312, the first convolution layer 314 and the second convolution layer 315 use different Padding values ​​and Stride values, and obtain the second feature and the third feature through the first pooling layer 316 and the second pooling layer 317 respectively;

[0072] S313, the first feature, the second feature and the third feature are all L2 regularized;

[0073] S314, the regularized first feature, the second feature and the third feature are concatenated and input into the second fully connected layer 313;

[0074] S315, the second fully connected layer 313 obtains multi-scale convolutional neural network features through linear mapping.

[0075] S316. Optimizing the parameters of the multi-scale convolutional neural network through a triplet metric loss function to mine and learn the semantic information of the registered trademark image;

[0076] The loss function is expressed as:

[0077] Loss(p,q,r)

[0078] =max(0,margin+D(f(q),f(p))-D(f(q),D(f(n))))

[0079] Among them, Loss(p, q, r) is the metric loss function of the triple (p, q, r), p, q, r are the current trademark, the trademark similar to the current trademark and the non-similar trademark to the current trademark respectively, f(·) is the output feature of the multi-scale convolutional neural network, D(·) is the cosine distance between the two input features, margin is the boundary parameter, and the value of margin is 0.1 during the implementation process.

[0080] In the implementation process, the triples are constructed according to the existing annotation information, and the data samples are randomly selected for training. The training is optimized by the stochastic gradient descent algorithm, with an initial learning rate of 0.001, a momentum parameter of 0.9, and a single training data size of 64.

[0081] In step S3, the visual bag of words feature module 32 extracts the visual bag of words features of the trademark image and transmits them to the feature retrieval module 2 in the following specific manner:

[0082] S321, use Harris, Hessian, and Kaze local detectors to extract key points of the image;

[0083] S322, extracting key point features from image key points through Sift descriptors;

[0084] S323, key point features are clustered through Kmeans to build a visual dictionary;

[0085] S324, extracting visual bag-of-words features from the image through a visual dictionary.

[0086] In this implementation, three local detectors, Harris, Hessian, and Kaze, are first used to extract the key points of the image.

[0087] Based on the extracted image key points, the key point features are extracted respectively through Sift descriptor, and the feature dimension of each key point is 128.

[0088] A visual dictionary is established based on Sift key point features using the Kmeans clustering algorithm, with a dictionary size of 2000 words.

[0089] Finally, for each image, the word frequency and inverse document frequency are calculated based on the visual dictionary, and a 2000-dimensional visual bag-of-words feature is output.

[0090] In this embodiment, an index library of graphic elements of registered trademarks is established, and the graphic element features of the trademark to be registered are manually input by the search personnel.

[0091] This step uses full-text search engine libraries such as Lucene and Whoosh to create an index library for the graphic elements of the registered trademark to support rapid retrieval of the graphic elements in the next step. The index fields of the index library include the registration number of the trademark and the set of graphic codes contained in the trademark image.

[0092] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A trademark similarity search system based on multi-dimensional feature fusion, characterized by: It comprises a trademark database (1), a feature retrieval module (2), and a feature extraction module (3), wherein the trademark database (1) is connected to the feature extraction module (3), and the feature extraction module (3) is connected to the feature retrieval module (2); The feature extraction module (3) comprises a neural network feature module (31), a visual word bag feature module (32) and a graphic element module (33), wherein the neural network feature module (31) is connected to the trademark database (1) and the feature retrieval module (2), the visual word bag feature module (32) is connected to the trademark database (1) and the feature retrieval module (2), and the graphic element module (33) is connected to the trademark database (1). The neural network feature module (31) is used to extract image multi-scale convolutional neural network features through the trademark database (1), optimize the neural network based on the triple metric loss function, and output the multi-scale convolutional neural network features to the feature retrieval module (2), the visual word bag feature module (32) is used to construct a visual dictionary based on the extracted image key point features through the trademark database (1), and output the image visual word bag features extracted based on the visual dictionary to the feature retrieval module (2), and the graphic element module (33) is used to establish an index library of registered trademark graphic elements and allow the query personnel to manually input the graphic element features to the feature retrieval module (2); The trademark similarity search method of the trademark similarity search system based on multi-dimensional feature fusion includes the following steps: S1. Establishing the trademark database (1); S2. Input the trademark to be registered through the feature search module (2); S3, according to the trademark to be registered, the neural network feature module (31) extracts the multi-scale convolutional neural network features of the trademark image through the trademark database (1) and transmits them to the feature retrieval module (2), the visual word bag feature module (32) extracts the visual word bag features of the trademark image through the trademark database (1) and transmits them to the feature retrieval module (2), and the graphic element module (33) extracts the graphic element features of the trademark image through the trademark database (1) and transmits them to the feature retrieval module (2); S4. Based on the multi-scale convolutional neural network features, the visual word bag features and the graphic element features, the trademark to be registered is matched with the trademarks in the registered trademark database to obtain a fusion retrieval result of the three-dimensional features; The specific calculation formula for fusion search results in step S4 is: Score(a,d i ) =α*Score CNN (a,d i )+β*Score BoVW (a,d i )+Score element (a,d i ) Among them, Score (a, d i ) is the trademark a to be registered and the i-th trademark d in the registered trademark database i The total similarity, Score CNN is the similarity of multi-scale convolutional neural network features, Score BoVW is the similarity of the visual word bag feature, Score element is the similarity of graphic element features, and α and β are weight parameters.

2. The trademark similarity search system based on multi-dimensional feature fusion according to claim 1 is characterized by: The neural network feature module (31) comprises a residual network (311), a first fully connected layer (312), a second fully connected layer (313), a first convolutional layer (314), a second convolutional layer (315), a first pooling layer (316) and a second pooling layer (317); the residual network (311), the first convolutional layer (314) and the second convolutional layer (315) are all connected to the trademark database (1); the residual network (311) data is connected to the first fully connected layer (312); the first convolutional layer (314) data is connected to the first pooling layer (316); the second convolutional layer (315) data is connected to the second pooling layer (317); the first fully connected layer (312), the first pooling layer (316) and the second pooling layer (317) are all connected to the second fully connected layer (313); the second fully connected layer (313) transmits the multi-scale convolutional neural network features to the feature retrieval module (2).

3. The trademark similarity search system based on multi-dimensional feature fusion according to claim 1 is characterized by: In step S3, the neural network feature module (31) extracts the multi-scale convolutional neural network features of the trademark image and transmits them to the feature retrieval module (2) in the following specific manner: S311, the trademark image is respectively input into the residual network (311), the first convolutional layer (314) and the second convolutional layer (315), and the average pooling layer in the residual network (311) is extracted through the first fully connected layer (312) to obtain the first feature; S312: the first convolution layer (314) and the second convolution layer (315) use different convolution operation parameters, and obtain the second feature and the third feature through the first pooling layer (316) and the second pooling layer (317) respectively; S313: the first feature, the second feature and the third feature are all subjected to L2 regularization; S314, the regularized first feature, the second feature and the third feature are concatenated and input into a second fully connected layer (313); S315. The second fully connected layer (313) obtains the multi-scale convolutional neural network features through linear mapping.

4. The trademark similarity search system based on multi-dimensional feature fusion according to claim 3 is characterized by: The neural network feature module (31) in step S3 extracts the neural network features of the trademark image and transmits them to the feature retrieval module (2), further comprising: S316. Optimizing the parameters of the multi-scale convolutional neural network through a triplet metric loss function to mine and learn the semantic information of the registered trademark image; The loss function is expressed as: Loss(p,q,r) =max(0,margin+D(f(q),f(p))-D(f(q),D(f(n)))) Among them, Loss(p, q, r) is the metric loss function of the triple (p, q, r), p, q, r are the current trademark, the trademark similar to the current trademark and the non-similar trademark to the current trademark respectively, f(·) is the output feature of the multi-scale convolutional neural network, D(·) is the cosine distance between the two input features, and margin is the margin parameter.

5. The trademark similarity search system based on multi-dimensional feature fusion according to claim 1 is characterized by: The specific method in which the bag-of-visual-words feature module (32) in step S3 extracts the bag-of-visual-words features of the trademark image and transmits them to the feature retrieval module (2) is: S321, use Harris, Hessian, and Kaze local detectors to extract key points of the image; S322, extracting key point features from image key points through Sift descriptors; S323, key point features are clustered through Kmeans to build a visual dictionary; S324, extracting visual bag-of-words features from the image through a visual dictionary.

Images