Logo retrieval method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence, and relates to a l ogo retrieval method, which comprises the following steps of: pre-processing all l ogo images in a l ogo material library in advance to obtain a material image; rotating the material image at different angles, and performing feature extraction on the material image at different rotation angles through a preset multi-modal model to obtain feature vectors of the material image at different rotation angles; combining all the feature vectors as index vectors of the corresponding material images, and constructing a vector database based on the index vectors; obtaining a query request, and extracting text features of the query request to obtain a query vector; and calculating the cosine similarity between the query vector and each index vector in the vector database, and determining a query result according to the cosine similarity. The invention further provides a logo retrieval device, computer equipment and a storage medium. According to the method, the index vector of the material image in the logo material library is determined in advance, and only the text vector of the query request needs to be calculated during each retrieval, so that the retrieval time is greatly shortened.

Description

Technical Field

[0001] The present application relates to the field of digital finance and image processing technology, and in particular to a logo retrieval method, device, computer equipment and storage medium. Background Art

[0002] The supply chain system is a major system of the bank that serves small and micro business owners. It enables merchants to design logos, generate copy and other related scenarios in AI. In recent years, banks have continued to provide convenient services to merchants and accelerate the digital development of banks. Companies such as China Merchants Bank, CITIC Bank, and Bank of Beijing have successively launched one-stop LOGO query and proofreading solutions to empower a large number of merchant users. Among them, in order to protect the interests of merchants, it is necessary to prevent and control LOGO-related infringement and counterfeiting. At present, the process of discovering and confirming infringement and counterfeiting generally adopts the method of manual retrieval and monitoring, which takes a long time, especially when the business scale is large, it takes longer time, and it is impossible to quickly discover and confirm infringement, so as to quickly enter the disposal stage and take corresponding measures to control risks, and thus fail to protect the interests of enterprises well. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a logo retrieval method, apparatus, computer device and storage medium to solve the technical problem of slow retrieval speed of logo query systems in the prior art.

[0004] In order to solve the above technical problems, the present application embodiment provides a logo retrieval method, which adopts the following technical solution:

[0005] Pre-process all logo images in the logo material library to obtain material images;

[0006] Rotating the material image at different angles, extracting features of the material image at different rotation angles using a preset multimodal model, and obtaining feature vectors of the material image at different rotation angles;

[0007] Combining all feature vectors as index vectors corresponding to the material image, and constructing a vector database based on the index vectors;

[0008] Obtaining a query request and extracting text features of the query request to obtain a query vector;

[0009] The cosine similarity between the query vector and each index vector in the vector database is calculated, and a query result is determined according to the cosine similarity.

[0010] Furthermore, the pre-processing of all logo images in the logo material library includes:

[0011] Normalizing all images in the logo material library to obtain normalized images;

[0012] Perform image enhancement processing on the normalized image to obtain the material image.

[0013] Furthermore, the feature extraction of the material image at different rotation angles by using a preset multimodal model to obtain feature vectors of the material image at different rotation angles includes:

[0014] Extracting features of the output image at the initial angle using the multimodal model to obtain a first feature vector;

[0015] Rotating the material image to a first angle, and performing feature extraction on an output image at a second angle using the multimodal model to obtain a second feature vector;

[0016] Rotating the material image to a second angle, and performing feature extraction on the output image at a third angle using the multimodal model to obtain a third feature vector;

[0017] The material image is rotated to a third angle, and features of the output image at a fourth angle are extracted using the multimodal model to obtain a fourth feature vector.

[0018] Furthermore, the calculating the cosine similarity between the query vector and each index vector in the vector database, and determining the query result according to the cosine similarity, includes:

[0019] Calculating cosine similarities between the query vector and the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector respectively to obtain a first similarity, a second similarity, a third similarity, and a fourth similarity;

[0020] Calculating an average of the first similarity, the second similarity, the third similarity and the fourth similarity to obtain an average similarity between the query vector and each of the index vectors;

[0021] Compare all average similarities and take the material image corresponding to the highest average similarity as the query result.

[0022] Furthermore, constructing a vector database based on the index vector includes:

[0023] Performing low-dimensional division on the high-dimensional index vector to obtain multiple low-dimensional subspaces;

[0024] Using a K-means clustering algorithm to cluster the low-dimensional index vectors in the low-dimensional subspace to obtain cluster centers;

[0025] The high-dimensional index vector is mapped to the cluster center according to product quantization to construct the vector database.

[0026] Furthermore, after mapping the high-dimensional index vector to the cluster center according to product quantization, the method further includes:

[0027] Constructing a vector space based on the index vector, and dividing the vector space into a dense area and a sparse area according to the distribution of the index vector;

[0028] According to a fine-grained indexing strategy, the dense area is divided into a plurality of sub-areas, and indexes are constructed for the sub-areas;

[0029] According to the coarse-grained indexing strategy, a plurality of the sparse regions are merged into one region, and an index is constructed for the merged region.

[0030] Furthermore, after calculating the cosine similarity between the query vector and each index vector in the vector database and determining the query result according to the cosine similarity, the method further comprises:

[0031] Recording the query result corresponding to the query vector and generating a query log;

[0032] A clustering algorithm is used to mine and analyze the query log, and the index strategy of the vector database is optimized according to the mining analysis.

[0033] In order to solve the above technical problems, the embodiment of the present application further provides a logo search device, which adopts the following technical solution:

[0034] A preprocessing module is used to preprocess all images in the logo material library to obtain material images;

[0035] An image feature extraction module is used to rotate the material image at different angles, extract features of the material image at different rotation angles through a preset multimodal model, and obtain feature vectors of the material image at different rotation angles;

[0036] A construction module, used to combine all feature vectors as index vectors corresponding to the material image, and construct a vector database based on the index vectors;

[0037] A text feature extraction module, used to obtain a query request and extract text features of the query request to obtain a query vector;

[0038] The query module is used to calculate the cosine similarity between the query vector and each index vector in the vector database, and determine the query result according to the cosine similarity.

[0039] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0040] The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the logo retrieval method described in any one of the above items are implemented.

[0041] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0042] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the logo retrieval method described in any one of the above items are implemented.

[0043] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by pre-rotating the logo image in the logo material library at multiple angles and then extracting features, the feature vectors of multiple angles extracted are combined to obtain the index vector of the material image, and a vector database is constructed based on the index vector. When searching, only the query vector of the query request needs to be obtained, and the cosine similarity between the query vector and each index vector is calculated, and the query result is determined according to the pre-similarity. By pre-calculating the index vector of the material image in the logo material library, only the text vector of the query request needs to be calculated for each search, which greatly shortens the search time, and the index vector is a combination of the feature vectors of the material image at different angles, which can greatly improve the accuracy during the search. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0046] Figure 2 A flowchart of an embodiment of a logo retrieval method according to the present application;

[0047] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S202;

[0048] Figure 4 yes Figure 2A flowchart of a specific implementation of step S205;

[0049] Figure 5 yes Figure 2 A flowchart of a specific implementation of step S203;

[0050] Figure 6 yes Figure 5 A flowchart of a specific implementation of step S503;

[0051] Figure 7 yes Figure 2 A flowchart of a specific implementation of step S205;

[0052] Figure 8 is a structural schematic diagram of an embodiment of a logo retrieval device according to the present application;

[0053] Fig. 9 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0057] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0058] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0059] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0060] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0061] It should be noted that the logo retrieval method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the logo retrieval device is generally set in the server / terminal device.

[0062] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0063] Continue to refer Figure 2 , shows a flow chart of an embodiment of a logo search method according to the present application. The logo search method comprises the following steps:

[0064] Step S201, pre-processing all logo images in the logo material library to obtain material images;

[0065] In this embodiment, the electronic device on which the logo retrieval method is executed (eg Figure 1 The server / terminal device shown in the figure) can receive a request for obtaining a paired sample image and a description text through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future. Among them, the logo material library stores the logo image of the merchant, and the logo image can be called or downloaded from the bank's local merchant logo database or the Internet.

[0066] Step S202, rotating the material image at different angles, extracting features of the material image at different rotation angles using a preset multimodal model, and obtaining feature vectors of the material image at different rotation angles;

[0067] In this embodiment, the logo image can show different features at different angles. Therefore, in order to capture more image features of the logo image, the preprocessed material image is rotated at different angles, and the feature vectors of the material image at the initial position and at different rotation angles are recorded.

[0068] Step S203, combining all feature vectors as index vectors corresponding to the material image, and constructing a vector database based on the index vectors;

[0069] In this embodiment, feature vectors of the material image at different angles are extracted and recorded as vector, then vector1, vector2, vector3, vector4 are obtained, and all feature vectors are combined as index vectors corresponding to the material image, that is, the index vector is {vector1, vector2, vector3, vector3}.

[0070] Step S204, obtaining a query request, and extracting text features of the query request to obtain a query vector;

[0071] Step S205 , calculating the cosine similarity between the query vector and each index vector in the vector database, and determining the query result according to the cosine similarity.

[0072] In this embodiment, a vector database is constructed based on all index vectors, and a received query request is obtained during retrieval. The text features of the query request are extracted through a transformer model to obtain a text vector of the query request, i.e., the query vector. The cosine similarity between the query vector and each index vector in the vector database is calculated, and the query result is determined based on the cosine similarity.

[0073] The present application extracts features by rotating the logo image in the logo material library at multiple angles in advance, combines the extracted feature vectors at multiple angles, obtains the index vector of the material image, builds a vector database based on the index vector, and only needs to obtain the query vector of the query request during retrieval, calculates the cosine similarity between the query vector and each index vector, and determines the query result based on the pre-similarity. Since the index vector of the material image in the logo material library is determined in advance, each retrieval only needs to calculate the text vector of the query request, and then perform retrieval based on the similarity between the text vector and the index vector, which greatly shortens the retrieval time, and the index vector is a combination of the feature vectors of the material image at different angles, which can greatly improve the accuracy during retrieval.

[0074] In some optional implementations of this embodiment, step 201 includes the following steps:

[0075] Normalizing all images in the logo material library to obtain normalized images;

[0076] Perform image enhancement processing on the normalized image to obtain the material image.

[0077] In this embodiment, all logo images in the logo material library are preprocessed, and the preprocessing includes normalizing the logo images and enhancing the logo images. Among them, all logo images are normalized to ensure that all logo images have the same size, for example, all logo images are processed to a uniform size of 224*224; the logo images are enhanced, such as improving the contrast and brightness of the logo images, improving the quality of the logo images, and facilitating the extraction of more logo image features when extracting features from the logo images later.

[0078] like Figure 3 As shown, in some optional implementations of this embodiment, step 202 includes the following steps:

[0079] Step S301, extracting features of the output image at the initial angle using the multimodal model to obtain a first feature vector;

[0080] Step S302, rotating the material image to a first angle, and performing feature extraction on an output image at a second angle using the multimodal model to obtain a second feature vector;

[0081] Step S303, rotating the material image to a second angle, and performing feature extraction on the output image at a third angle using the multimodal model to obtain a third feature vector;

[0082] Step S304: Rotate the material image to a third angle, and perform feature extraction on the output image at a fourth angle using the multimodal model to obtain a fourth feature vector.

[0083] In this embodiment, in order to extract as many features of the material image as possible, a multimodal model (slip model) is pre-trained, and the image features of the material image at the initial angle, the first rotation angle, the second rotation angle, and the third rotation angle are respectively extracted through the multimodal model (slip model), and the feature vectors of the material image after the initial angle, the first rotation angle, the second rotation angle, and the third rotation angle are respectively obtained, that is, the first feature vector, the second vector, and the fourth vector. For example, rotate 90°, 180°, and 270°, and then extract the image features of the image at the original position, rotate 90°, rotate 180°, and rotate 270°. Among them, in actual applications, the feature extraction of the material image at different rotation angles can be appropriately increased or decreased according to the situation. In order to ensure the accuracy as much as possible, the image features of the material image at three different rotation angles are extracted in this embodiment. For tasks with low accuracy requirements, the image feature extraction of the material image at different angles can be appropriately reduced. For tasks with higher accuracy requirements, the image feature extraction of the material image at different angles can be appropriately increased.

[0084] This embodiment extracts the feature vectors of the rotated material image at different angles, fully exploits the features of the logo image at different angles, and combines the feature vectors of the material image at different angles as index vectors, which is beneficial to improve the accuracy of retrieval.

[0085] like Figure 4 As shown, in some optional implementations of this embodiment, step 205 includes the following steps:

[0086] Step S401, respectively calculating cosine similarities between the query vector and the first feature vector, the second feature vector, the third feature vector and the fourth feature vector to obtain a first similarity, a second similarity, a third similarity and a fourth similarity;

[0087] Step S402, calculating an average of the first similarity, the second similarity, the third similarity and the fourth similarity to obtain an average similarity between the query vector and each of the index vectors;

[0088] Step S403: compare all average similarities, and take the material image corresponding to the highest average similarity as the query result.

[0089] In this embodiment, in order to improve the accuracy of retrieval, the cosine similarity between the query vector and each feature vector in each index vector is calculated, and then based on the average value of all cosine similarities, that is, the average similarity, the average similarities are arranged from large to small, and the material image corresponding to the index vector with the highest average similarity is determined as the query result. In this embodiment, the index vector includes the first feature vector, the second feature vector, the third feature vector and the fourth feature vector, so the cosine similarity between the query vector and the first feature vector, the second feature vector, the third feature vector and the fourth feature vector is calculated respectively.

[0090] This embodiment calculates the cosine similarity between the query vector and each feature vector of the index vector to compare the similarity between the query vector and the feature vectors of each angle of the material image, and determines the similarity between the index vector and the query vector by the average value of all cosine similarities, thereby effectively improving the accuracy of retrieval.

[0091] like Figure 5 As shown, in some optional implementations of this embodiment, step 203 includes the following steps:

[0092] Step S501, performing low-dimensional division on the high-dimensional index vector to obtain multiple low-dimensional subspaces;

[0093] Step S502, clustering the low-dimensional index vectors in the low-dimensional subspace using a K-means clustering algorithm to obtain cluster centers;

[0094] Step S503: Map the high-dimensional index vector to the cluster center according to product quantization to construct the vector database.

[0095] In this embodiment, in order to reduce the storage overhead of index vectors, an index compression method based on product quantization is adopted to divide the high-dimensional index vector into multiple low-dimensional subspaces, and the K-means clustering algorithm is used to perform clustering in each subspace to establish compact clustering centers. The high-dimensional index vectors are mapped to the clustering centers according to product quantization, which greatly reduces the index storage overhead. At the same time, the original features are approached, the quantization error is controlled within an acceptable range, and the index quality is guaranteed.

[0096] For example, a data set containing 10 million logo images can be used, and a 512-dimensional index vector is extracted for each logo image. According to the dimension 512 of the index vector, it can be divided into 8 64-dimensional low-dimensional subspaces. For each 64-dimensional low-dimensional subspace, the K-means clustering algorithm is used to cluster the index vectors in the subspace, and the number of cluster centers is set to 256. After 20 iterations, a compact cluster center index is obtained. Through the product quantization method, the original 512-dimensional high-dimensional feature vector is mapped to the cluster centers of 8 64-dimensional low-dimensional subspaces to approximate the original features. In the quantization process, the average value of the quantization error is controlled within 1 by minimizing the objective function of the reconstruction error.

[0097] like Figure 6 As shown, in some optional implementations of this embodiment, step 503 includes the following steps:

[0098] Step S601, constructing a vector space based on the index vector, and dividing the vector space into a dense area and a sparse area according to the distribution of the index vector;

[0099] Step S602: dividing the dense area into multiple sub-areas according to a fine-grained indexing strategy, and indexing the sub-areas;

[0100] Step S603: according to the coarse-grained indexing strategy, a plurality of the sparse regions are merged into one region, and an index is constructed for the merged region.

[0101] In this embodiment, in view of the problem of uneven distribution of logo feature space, an adaptive multi-index fusion mechanism is designed, different index granularities and algorithms are used for logos in different areas, fine-grained indexing is used to improve precision in dense areas, and coarse-grained indexing is used to improve recall in sparse areas to balance query performance. Among them, the fine-grained indexing strategy refers to decomposing a complex function or business logic into multiple smaller and more specific index units, each of which is responsible for handling specific situations or conditions, thereby achieving high flexibility and scalability. The coarse-grained indexing strategy is a simplified indexing strategy suitable for filtering scenarios with large amounts of data. It simplifies the complexity of the index by dividing the data into multiple partitions, each of which is managed independently. The difference between the two is that the coarse-grained indexing strategy is a lightweight index, which is characterized by the fact that the establishment and maintenance of the index has almost zero impact on system resource occupation and performance. This index is automatically established and automatically updated as the data changes, without manual intervention. The coarse-grained index greatly reduces the scope of query and data materialization, and can quickly locate the target data set records. For example, when storing data, every 65536 rows of data are packaged into a data packet. Each data packet automatically establishes an intelligent index when the data is entered into the database. When querying data, statistical values ​​and data filtering can be obtained without unpacking, thereby reducing input / output and significantly optimizing complex queries. The fine-grained index strategy is more flexible and can dynamically allocate resources as needed. Unlike the coarse-grained mode, the fine-grained mode allocates resources on demand, and resources can be released immediately after each task is completed. This mode is similar to the current cloud computing, and the idea is to allocate on demand. For example, in open source projects, the fine-grained mode allows each task to dynamically allocate resources, and resources can be released immediately after each task is completed. This mode facilitates resource control and isolation, but the short job operation delay is large. Based on the index vector, the vector space is constructed, the distribution of the vector space is analyzed, and the vector space is divided into dense areas and sparse areas according to the degree of aggregation of the index vectors in the vector space. For the dense area, a fine-grained index strategy is adopted to further divide the area into multiple sub-areas, and an index is constructed in each sub-area to improve the precision of the logo in the area. For sparse areas, a coarse-grained indexing strategy is adopted to merge multiple sparse small areas into a large area, and an index is built on the merged large area to improve the recall rate of logos in the area. When querying, the spatial area to which the query belongs is determined based on the text features of the query request, and the corresponding index granularity is selected for search and matching. The search results of multiple indexes are merged and comprehensively sorted. If there is overlap in the candidate sets obtained by different indexes, deduplication is performed to obtain the final query result.

[0102] like Figure 7 As shown, in some optional implementations of this embodiment, after step 205, the following steps are included:

[0103] Step S701, recording the query result corresponding to the query vector and generating a query log;

[0104] Step S702: using a clustering algorithm to perform mining analysis on the query log, and optimizing the indexing strategy of the vector database according to the mining analysis.

[0105] In this embodiment, in order to continuously optimize the index and query strategy and improve the performance and user experience of logo retrieval, the query results corresponding to the query vector are recorded to generate a query log, and a clustering algorithm is used to mine and analyze the query log. The user's retrieval pattern and preference are discovered based on the mining and analysis, and the index strategy of the vector database is optimized according to the user's retrieval pattern.

[0106] For example, frequently searched logo images are given a higher weight in the index. Based on user preferences, the similarity calculation method between different logo images is dynamically adjusted to make the recommendation results more in line with the user's potential needs. User feedback on search results, including clicks, dwell time, etc., is collected, and the relationship between user feedback and relevance is learned through machine learning models, which are used to guide query optimization and algorithm improvement, continuously improving the performance and user experience of logo image retrieval.

[0107] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0108] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0110] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0111] Further references Figure 8 , as a response to the above Figure 2 The present application provides an embodiment of a logo retrieval device, and the embodiment of the device is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0112] like Figure 8 As shown, the logo retrieval device 800 described in this embodiment includes: a pre-processing module 801, an image feature extraction module 802, a construction module 803, a text feature extraction module 804 and a query module 805. Among them:

[0113] The preprocessing module 801 is used to preprocess all logo images in the logo material library to obtain material images;

[0114] An image feature extraction module 802 is used to rotate the material image at different angles, extract features of the material image at different rotation angles through a preset multimodal model, and obtain feature vectors of the material image at different rotation angles;

[0115] A construction module 803 is used to combine all feature vectors as index vectors corresponding to the material image, and construct a vector database based on the index vectors;

[0116] A text feature extraction module 804 is used to obtain a query request and extract text features of the query request to obtain a query vector;

[0117] The query module 805 is used to calculate the cosine similarity between the query vector and each index vector in the vector database, and determine the query result according to the cosine similarity.

[0118] In some optional implementations of this embodiment, the preprocessing module 801 includes:

[0119] A normalization unit, used for normalizing all logo images in the logo material library to obtain normalized images;

[0120] The enhancement unit is used to perform image enhancement processing on the normalized image to obtain the material image.

[0121] In some optional implementations of this embodiment, the image feature extraction module 802 includes:

[0122] A first feature extraction unit, configured to extract features from the output image at the initial angle using the multimodal model to obtain a first feature vector;

[0123] a second feature extraction unit, configured to rotate the material image to the first angle, and perform feature extraction on the output image at the second angle through the multimodal model to obtain a second feature vector;

[0124] a third feature extraction unit, configured to rotate the material image to a second angle, and perform feature extraction on the output image at the third angle through the multimodal model to obtain a third feature vector;

[0125] a fourth feature extraction unit, configured to rotate the material image to a third angle, and perform feature extraction on the output image at the fourth angle using the multimodal model to obtain a fourth feature vector;

[0126] A vector combining unit is used to combine the first eigenvector, the second eigenvector, the third eigenvector and the fourth eigenvector to obtain the index vector.

[0127] In some optional implementations of this embodiment, the query module 805 includes:

[0128] a similarity calculation unit, configured to respectively calculate cosine similarities between the query vector and the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector to obtain a first similarity, a second similarity, a third similarity, and a fourth similarity;

[0129] an average value calculating unit, configured to calculate an average value of the first similarity, the second similarity, the third similarity and the fourth similarity to obtain an average similarity between the query vector and each of the index vectors;

[0130] The comparison unit is used to compare all average similarities and take the material image corresponding to the highest average similarity as the query result.

[0131] In some optional implementations of this embodiment, the construction module 803 includes:

[0132] A space division unit, used for performing low-dimensional division on the high-dimensional index vector to obtain a plurality of low-dimensional subspaces;

[0133] A clustering unit, used for clustering the low-dimensional index vectors in the low-dimensional subspace using a K-means clustering algorithm to obtain cluster centers;

[0134] A mapping unit is used to map the high-dimensional index vector to the cluster center according to product quantization to construct the vector database.

[0135] In some optional implementations of this embodiment, the construction module 803 further includes:

[0136] A region division unit, used for constructing a vector space based on the index vector, and dividing the vector space into a dense region and a sparse region according to the distribution of the index vector;

[0137] A first index building unit, configured to divide the dense area into a plurality of sub-areas according to a fine-grained indexing strategy, and to build indexes for the sub-areas;

[0138] The second index building unit is used to merge the multiple sparse regions into one region according to the coarse-grained indexing strategy, and to build an index for the merged region.

[0139] In some optional implementations of this embodiment, the logo retrieval device includes:

[0140] A recording module, used to record the query result corresponding to the query vector and generate a query log;

[0141] The optimization module is used to perform mining analysis on the query log using a clustering algorithm, and optimize the index strategy of the vector database according to the mining analysis.

[0142] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0143] The computer device 9 includes a memory 91, a processor 92, and a network interface 93 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 9 with a memory 91, a processor 92, and a network interface 93, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0144] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0145] The memory 91 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 91 can be an internal storage unit of the computer device 9, such as a hard disk or memory of the computer device 9. In other embodiments, the memory 91 can also be an external storage device of the computer device 9, such as a plug-in hard disk equipped on the computer device 9, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 91 can also include both the internal storage unit of the computer device 9 and its external storage device. In this embodiment, the memory 91 is generally used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions of the logo retrieval method, etc. In addition, the memory 91 can also be used to temporarily store various types of data that have been output or are to be output.

[0146] The processor 92 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 92 is generally used to control the overall operation of the computer device 9. In this embodiment, the processor 92 is used to run computer-readable instructions stored in the memory 91 or process data, such as computer-readable instructions for running the logo retrieval method.

[0147] The network interface 93 may include a wireless network interface or a wired network interface. The network interface 93 is generally used to establish a communication connection between the computer device 9 and other electronic devices.

[0148] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the logo retrieval method as described above.

[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0150] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

[0151] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

Claims

1. A logo search method, characterized in that: The steps include: Pre-process all logo images in the logo material library to obtain material images; Rotating the material image at different angles, extracting features of the material image at different rotation angles using a preset multimodal model, and obtaining feature vectors of the material image at different rotation angles; Combining all feature vectors as index vectors corresponding to the material image, and constructing a vector database based on the index vectors; Obtaining a query request and extracting text features of the query request to obtain a query vector; The cosine similarity between the query vector and each index vector in the vector database is calculated, and a query result is determined according to the cosine similarity.

2. The logo retrieval method according to claim 1, characterized in that: The pre-processing of all images in the logo material library includes: Normalizing all logo images in the logo material library to obtain normalized images; Perform image enhancement processing on the normalized image to obtain the material image.

3. The logo retrieval method according to claim 1, characterized in that: The method of extracting features of the material image at different rotation angles by using a preset multimodal model to obtain feature vectors of the material image at different rotation angles includes: Extracting features of the output image at the initial angle using the multimodal model to obtain a first feature vector; Rotating the material image to a first angle, and performing feature extraction on an output image at a second angle using the multimodal model to obtain a second feature vector; Rotating the material image to a second angle, and performing feature extraction on the output image at a third angle using the multimodal model to obtain a third feature vector; The material image is rotated to a third angle, and features of the output image at a fourth angle are extracted using the multimodal model to obtain a fourth feature vector.

4. The logo retrieval method according to claim 3, characterized in that: The calculating the cosine similarity between the query vector and each index vector in the vector database, and determining the query result according to the cosine similarity, includes: Calculating cosine similarities between the query vector and the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector respectively to obtain a first similarity, a second similarity, a third similarity, and a fourth similarity; Calculating an average of the first similarity, the second similarity, the third similarity and the fourth similarity to obtain an average similarity between the query vector and each of the index vectors; Compare all average similarities and take the material image corresponding to the highest average similarity as the query result.

5. The logo retrieval method according to claim 1, characterized in that: The constructing a vector database based on the index vector includes: Performing low-dimensional division on the high-dimensional index vector to obtain multiple low-dimensional subspaces; Using a K-means clustering algorithm to cluster the low-dimensional index vectors in the low-dimensional subspace to obtain cluster centers; The high-dimensional index vector is mapped to the cluster center according to product quantization to construct the vector database.

6. The logo retrieval method according to claim 5, characterized in that: After mapping the high-dimensional index vector to the cluster center according to product quantization, the method further comprises: Constructing a vector space based on the index vector, and dividing the vector space into a dense area and a sparse area according to the distribution of the index vector; According to a fine-grained indexing strategy, the dense area is divided into a plurality of sub-areas, and indexes are constructed for the sub-areas; According to the coarse-grained indexing strategy, a plurality of the sparse regions are merged into one region, and an index is constructed for the merged region.

7. The logo retrieval method according to claim 1, characterized in that: After calculating the cosine similarity between the query vector and each index vector in the vector database and determining the query result according to the cosine similarity, the method further comprises: Recording the query result corresponding to the query vector and generating a query log; A clustering algorithm is used to mine and analyze the query log, and the index strategy of the vector database is optimized according to the mining analysis.

8. A logo search device, characterized in that: include: A preprocessing module is used to preprocess all logo images in the logo material library to obtain material images; An image feature extraction module is used to rotate the material image at different angles, extract features of the material image at different rotation angles through a preset multimodal model, and obtain feature vectors of the material image at different rotation angles; A construction module, used to combine all feature vectors as index vectors corresponding to the material image, and construct a vector database based on the index vectors; A text feature extraction module, used to obtain a query request and extract text features of the query request to obtain a query vector; The query module is used to calculate the cosine similarity between the query vector and each index vector in the vector database, and determine the query result according to the cosine similarity.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the logo retrieval method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the logo retrieval method according to any one of claims 1 to 7 are implemented.

Images