Seal library construction and seal retrieval

Abstract

Disclosed in embodiments of the present disclosure are a seal library construction method, a seal retrieval method, a storage medium, and a terminal. The seal library construction method comprises: acquiring a seal image of a seal to be stored into a library, and extracting text information of said seal on the basis of the seal image; inputting the seal image into a preset feature extractor, and determining an image morphological feature of said seal outputted by the preset feature extractor on the basis of the seal image; and correspondingly storing unique identification information, the text information, and the image morphological feature of said seal into the seal library. As text information is one of the most important pieces of information of a seal, and an image morphological feature can explain detail information of the seal, when a seal to be stored into a library is stored into the library, the text information and the image morphological feature of said seal are extracted, and the two types of information of said seal are stored into the seal library, thereby improving the accuracy of retrieval results in application scenarios.

Description

Heraldry Database Construction and Heraldry Retrieval Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the construction of heraldry libraries and heraldry retrieval. Background Technology

[0002] In some online service scenarios, users are typically required to provide corresponding documents, such as certificates and seals. Users upload these documents as images to the platform, and after the platform verifies their authenticity, the relevant transactions can be processed. However, some users upload altered or falsified documents to mislead the platform's review process and gain undue advantage, with forged seals being a particularly frequent method. Therefore, to avoid the negative impact of false materials on the service platform and other users, it is necessary to verify the authenticity of the materials. Summary of the Invention

[0003] This disclosure provides a method for constructing a heraldry library, a heraldry retrieval method, a storage medium, and a terminal, which can solve the technical problem in related technologies of being unable to accurately identify the authenticity of seals.

[0004] In a first aspect, embodiments of this disclosure provide a method for constructing a heraldry library. The method includes: acquiring a heraldry image of a heraldry to be added to the library; extracting text information of the heraldry to be added to the library based on the heraldry image; inputting the heraldry image into a preset feature extractor; determining the image morphological features of the heraldry to be added to the library output by the preset feature extractor based on the heraldry image; and saving the unique identifier information, text information, and image morphological features of the heraldry to be added to the heraldry library accordingly.

[0005] Secondly, embodiments of this disclosure provide a heraldic retrieval method, which includes: determining the heraldic image corresponding to a target heraldic image; extracting target text information from the heraldic image; comparing the target text information with the text information of each heraldic image in the heraldic database to obtain the text similarity between the target heraldic image and each heraldic image in the database; determining whether the text similarity meets preset filtering conditions; if it does, extracting the target image morphological features from the heraldic image; comparing the target image morphological features with the image morphological features of each heraldic image in the heraldic database to obtain the morphological similarity between the target heraldic image and each heraldic image in the database; calculating the overall similarity between the target heraldic image and each heraldic image in the database based on the text similarity and morphological similarity between the target heraldic image and each heraldic image in the database; and determining the retrieval result of the target heraldic image in the heraldic database based on the overall similarity; wherein the heraldic database is the heraldic database provided in the heraldic database construction method.

[0006] Thirdly, embodiments of this disclosure provide an apparatus for constructing a heraldry library. The apparatus includes: a text information extraction module, used to acquire an image of a heraldry to be added to the library, and extract text information of the heraldry to be added to the library based on the heraldry image; a morphological feature extraction module, used to input the heraldry image into a preset feature extractor, and determine the image morphological features of the heraldry to be added to the library output by the preset feature extractor based on the heraldry image; and a library storage module, used to save the unique identifier information, text information, and image morphological features of the heraldry to be added to the library in a corresponding manner to the heraldry library.

[0007] Fourthly, this disclosure provides a heraldic retrieval device, comprising: a text feature comparison module, used to determine the heraldic image corresponding to a target heraldic image, extract target text information from the heraldic image, compare the target text information with the text information of each heraldic image in the heraldic library, and obtain the text similarity between the target heraldic image and each heraldic image in the library; a morphological feature comparison module, used to determine whether the text similarity meets preset filtering conditions, and if so, extract the target image morphological features from the heraldic image, compare the target image morphological features with the image morphological features of each heraldic image in the heraldic library, and obtain the morphological similarity between the target heraldic image and each heraldic image in the library; a similarity calculation module, used to calculate the overall similarity between the target heraldic image and each heraldic image in the library based on the text similarity and morphological similarity between the target heraldic image and each heraldic image in the library; and a retrieval result output module, used to determine the retrieval result of the target heraldic image in the heraldic library based on the overall similarity; wherein the heraldic library is the heraldic library provided in the heraldic library construction method.

[0008] Fifthly, embodiments of this disclosure provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method described above.

[0009] Sixthly, embodiments of this disclosure provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0010] In a seventh aspect, embodiments of this disclosure provide a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being adapted to be loaded by the processor and to execute the steps of the method described above.

[0011] The beneficial effects of the technical solutions provided by some embodiments of this disclosure include at least the following: This disclosure provides a method for constructing a heraldry library, which involves obtaining an image of a heraldry to be added to the library, extracting text information of the heraldry based on the heraldry image, inputting the heraldry image into a preset feature extractor, determining the image morphological features of the heraldry to be added to the library output by the preset feature extractor based on the heraldry image, and saving the unique identifier, text information, and image morphological features of the heraldry to the heraldry library accordingly. Since text information is one of the most important pieces of information about a heraldry, and image morphological features can describe the detailed information of the heraldry, when adding a heraldry to the library, extracting both text information and image morphological features, and saving both types of information to the heraldry library, allows for comparison and retrieval based on the multi-dimensional information of the heraldry during subsequent searches using the heraldry library, thus improving the accuracy of search results in application scenarios. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 is an exemplary system architecture diagram of a heraldry library construction method or heraldry retrieval method provided in an embodiment of this disclosure;

[0014] Figure 2 is a flowchart illustrating a method for constructing an emblem library according to an embodiment of this disclosure;

[0015] Figure 3 is a flowchart illustrating a method for constructing an emblem library according to another embodiment of this disclosure;

[0016] Figure 4 is a flowchart illustrating a heraldry retrieval method provided in an embodiment of this disclosure;

[0017] Figure 5 is a flowchart illustrating a heraldry retrieval method according to another embodiment of this disclosure;

[0018] Figure 6 is a structural block diagram of an emblem library construction device provided in an embodiment of this disclosure;

[0019] Figure 7 is a structural block diagram of a heraldry retrieval device provided in an embodiment of this disclosure;

[0020] Figure 8 is a schematic diagram of the structure of a terminal provided in an embodiment of this disclosure;

[0021] Figure 9 is a schematic diagram of the structure of a terminal provided in another embodiment of this disclosure. Detailed Implementation

[0022] To make the features and advantages of the embodiments of this disclosure more apparent and understandable, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.

[0023] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this disclosure as detailed in the appended claims.

[0024] With the rapid development of computers and the internet, the storage and transmission of digital images have become incredibly simple. Digital images are ubiquitous on the internet and in the real world, and multimedia has become a crucial means of information delivery and sharing. Compared to text, transmitting information through images is more intuitive and credible. Initially, it was widely believed that images were captured and generated instantly, eliminating the possibility of fabrication. Many online services requiring user documentation relied on this trust in image information, allowing users to upload images as proof of documents, replacing paper documents or written descriptions.

[0025] However, with the continuous development of digital media technology, this credibility is constantly being undermined. In the era of multimedia information explosion, almost everyone has the ability to create and transmit large amounts of digital images. At the same time, the ease of use of many image editing software programs has made image modification increasingly easy. Ordinary people can easily process and modify images, and with the development of technology, forged images have become increasingly difficult to detect, even to the point of being indistinguishable from the real thing. In daily life, people often modify images for aesthetic or entertainment purposes, which does not have a negative impact. However, in some cases, maliciously tampered images, once disseminated, can affect people's judgment of objective things, and in severe cases, can even have a negative impact on society. For online service scenarios that require providing corresponding services based on real information, it is even more important to pay attention to the authenticity of images uploaded by users. Therefore, image tamper detection is conducted to identify whether an image is genuine, avoiding the identification of fake images as real images, which could lead to negative impacts on the service platform and other users.

[0026] Leveraging the internet, various platforms can conveniently and efficiently meet user needs by providing online services. Some of these services require verification of user documents, allowing users to upload materials in image format to prove their eligibility. For example, in financial service scenarios, users sometimes need to submit documents in image format, such as proof of poverty or medical certificates required in loan repayment negotiations. However, in some cases, users may use common image editing techniques to tamper with real images to construct fake documents for the purpose of unjust enrichment.

[0027] Especially in various consumer finance scenarios, users often need to upload various types of materials for review. Verifying the authenticity of these materials is a crucial step in the review process. Among current methods of document forgery, the most frequent is the forgery of undisclosed seals, primarily involving electronic seals and counterfeit seals. Electronic seals refer to fake seals generated by programmers and then composited into real photos. Counterfeit seals involve forging a physical seal by hand and stamping it onto the material. For composite images, we can use deep neural networks based on spatial and frequency domains to detect and identify partial tampering. However, for counterfeit seals, since there is no forgery at the image level, image recognition techniques are ineffective against them.

[0028] Therefore, this disclosure provides a method for constructing a heraldry library and a heraldry retrieval method to solve the aforementioned technical problem of being unable to accurately identify the authenticity of seals.

[0029] Please refer to Figure 1, which is an exemplary system architecture diagram of a heraldry library construction method or heraldry retrieval method provided in an embodiment of this disclosure.

[0030] As shown in Figure 1, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth, Wireless-Fidelity (Wi-Fi), or microwave communication links.

[0031] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0032] Server 103 can be a business server providing various services. It should be noted that server 103 can be either hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0033] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this disclosure. That is, the method provided in the embodiments of this disclosure can be applied to a system structure that only includes terminal 101. The embodiments of this disclosure do not limit this.

[0034] In this embodiment of the disclosure, based on an exemplary system architecture diagram of the heraldry library construction method, terminal 101 acquires the heraldry image of the heraldry to be added to the library, and extracts the text information of the heraldry to be added to the library based on the heraldry image; further, terminal 101 inputs the heraldry image into a preset feature extractor, and determines the image morphological features of the heraldry to be added to the library output by the preset feature extractor based on the heraldry image; at this time, terminal 101 can save the unique identification information, text information and image morphological features of the heraldry to be added to the heraldry library accordingly.

[0035] In this embodiment of the disclosure, based on an exemplary system architecture diagram of the heraldry retrieval method, terminal 101 determines the heraldry image corresponding to the target heraldry, extracts the target text information of the heraldry image, compares the target text information with the text information of each heraldry in the heraldry library, and obtains the text similarity between the target heraldry and each heraldry in the library; further, terminal 101 determines whether the text similarity meets the preset filtering conditions. If it does, it extracts the target image morphological features of the heraldry image, compares the target image morphological features with the image morphological features of each heraldry in the heraldry library, and obtains the morphological similarity between the target heraldry and each heraldry in the library; at this time, terminal 101 can calculate the overall similarity between the target heraldry and each heraldry in the library based on the text similarity and morphological similarity between the target heraldry and each heraldry in the library; finally, terminal 101 determines the retrieval result of the target heraldry in the heraldry library based on the overall similarity, wherein the heraldry library is the heraldry library provided in the above-mentioned heraldry library construction method.

[0036] It should be understood that the number of terminals, networks, and servers in Figure 1 is only illustrative, and can be any number of terminals, networks, and servers depending on the implementation needs.

[0037] Please refer to Figure 2, which is a flowchart illustrating a method for constructing an emblem library according to an embodiment of this disclosure. The execution entity in this embodiment can be a terminal executing the emblem library construction, a processor within the terminal executing the emblem library construction method, or an emblem library construction service within the terminal executing the emblem library construction method. For ease of description, the following uses the processor within the terminal as an example to illustrate the specific execution process of the emblem library construction method.

[0038] As shown in Figure 2, the method for constructing an emblem library may include at least the following steps.

[0039] S202. Obtain the heraldic image of the heraldic emblem to be added to the database, and extract the text information of the heraldic emblem to be added to the database based on the heraldic image.

[0040] Optionally, when verifying the authenticity of a coat of arms, it's important to consider that coats of arms are typically forged using two methods: image forgery and physical seal forgery. For image forgery, image tampering detection algorithms are generally employed, but tampering destroys the image's inherent features. Physical seal forgery, on the other hand, often involves fabricating textual information or altering details of the original seal, making it difficult to detect using image tampering detection methods. However, counterfeit seals are often used multiple times, appearing repeatedly in various scenarios and documents. Therefore, for identified counterfeit seals or seals that have appeared in other users' uploaded materials, a coat of arms database can be established. When new documents requiring verification appear, the coat of arms image is extracted and compared with the seals in the database to see if it matches a counterfeit or previously used seal, thus mitigating the inherent limitations of purely image-based authentication methods to some extent.

[0041] Optionally, the text information in a coat of arms constitutes the majority of the information conveyed by the coat of arms. When two coats of arms have completely identical text, even if their image textures and shades differ, they will still be considered the same coat of arms. Therefore, when building a coat of arms library, the coat of arms images to be added to the library can be obtained, and then the text information of the coat of arms to be added to the library can be extracted based on the coat of arms images and saved to the coat of arms library. In this way, when searching for coats of arms uploaded by users later, the text information of the uploaded coat of arms can be compared with that of the coats of arms already in the library, thereby determining whether the uploaded coat of arms matches a coat of arms already in the library.

[0042] S204. Input the heraldry image into the preset feature extractor and determine the image morphological features of the heraldry to be added to the database output by the preset feature extractor based on the heraldry image.

[0043] Optionally, in addition to the text information, the heraldic information is also reflected in the morphological information of the heraldry. Since the characters in the heraldry are curved and have different directions, and are often damaged, it may not be possible to guarantee that the characters in the heraldry can be extracted completely and accurately when extracting the characters. In this case, morphological information is needed to assist in the retrieval.

[0044] Optionally, when extracting the image morphological features of the herald to be included in the database, a preset feature extractor can be pre-trained. The preset feature extractor is obtained by training a neural network based on a deep learning algorithm. After deep learning, the preset feature extractor converges and can output the image morphological features of the herald image in the form of embedded vectors. That is, after the herald image is input into the preset feature extractor, the image morphological features of the herald to be included in the database output by the preset feature extractor based on the herald image are determined.

[0045] S206. Save the unique identifier, text information, and image morphological features of the emblem to be added to the emblem library.

[0046] Optionally, after extracting the text information and image morphological features of the herald to be added to the database, the unique identifier, text information, and image morphological features of the herald are saved to the herald database accordingly, completing the herald addition. Thus, when constructing the herald database for retrieval, the text and form of the herald are combined. Subsequent retrieval using the herald database allows for comparison and retrieval based on the multi-dimensional information of the herald, improving the accuracy of retrieval results in application scenarios.

[0047] This disclosure provides a method for constructing a heraldry library. The method involves acquiring an image of a heraldry to be added to the library, extracting textual information from the heraldry image, inputting the heraldry image into a preset feature extractor, determining the image morphological features of the heraldry to be added based on the heraldry image, and saving the unique identifier, textual information, and image morphological features of the heraldry to the heraldry library. Since textual information is one of the most important pieces of information about a heraldry, and image morphological features can describe the detailed information of the heraldry, when adding a heraldry to the library, both textual information and image morphological features are extracted and saved to the heraldry library. Subsequent retrieval using the heraldry library allows for comparison and retrieval based on the multi-dimensional information of the heraldry, improving the accuracy of retrieval results in application scenarios.

[0048] Please refer to Figure 3, which is a flowchart illustrating a method for constructing an emblem library according to another embodiment of this disclosure.

[0049] As shown in Figure 3, the method for constructing an emblem library may include at least the following steps.

[0050] S302. Obtain the heraldry image of the heraldry to be added to the database, and divide the heraldry image into an upper circle image and a middle and lower circle image.

[0051] Optionally, the text within a heraldry typically includes an upper, middle, and lower circle of text. The upper circle text is severely curved, with curvature reaching up to 270 degrees. The middle and lower circle texts are relatively straight. Based on this, the upper and lower circle texts are processed separately. That is, the heraldic image is divided into an upper circle image and a middle and lower circle image.

[0052] S304. Perform polar coordinate expansion on the upper circle image to obtain a straightened upper circle image. Extract the upper circle text of the emblem to be added to the database from the straightened upper circle image based on the image text recognition algorithm.

[0053] Optionally, if the text in the upper circle image is highly curved, to improve the accuracy of text recognition, the upper circle image can be expanded using polar coordinates to obtain a straightened upper circle image. After straightening the upper circle image, the upper circle text of the emblem to be added to the database can be extracted from the straightened upper circle image based on an image text recognition algorithm. The image text recognition algorithm can employ OCR (Optical Character Recognition) technology. OCR technology processes and analyzes the image file, ultimately recognizing the text in the image and converting it into text format.

[0054] S306. Extract the middle and lower circle text of the emblem to be added to the database from the middle and lower circle images based on image text recognition algorithms.

[0055] Optionally, for relatively straight image text, the OCR algorithm can accurately extract the text information. Therefore, in this embodiment of the disclosure, for relatively straight middle and lower circle text, the image text recognition algorithm can be directly used to extract the middle and lower circle text of the emblem to be added to the database in the middle and lower circle image.

[0056] S308. Combine the upper circle text, middle circle text, and lower circle text to obtain the text information of the emblem to be added to the database.

[0057] Optionally, after obtaining the upper, middle, and lower circle text of the coat of arms, the three segments of text are concatenated using semicolons in the order of upper, middle, and lower, and stored in the database as the text information of the coat of arms to be entered into the database for later retrieval.

[0058] S310. Input the heraldry image into the preset feature extractor and determine the image morphological features of the heraldry to be added to the database output by the preset feature extractor based on the heraldry image.

[0059] Optionally, it is also necessary to extract the image morphological features of the heraldry to be included in the database based on the heraldry image. In order to accurately extract the image morphological features of the heraldry to be included in the database, it is necessary to select appropriate training samples and training methods when training the preset feature extractor so that the preset feature extractor can achieve the desired performance.

[0060] Furthermore, when training the preset feature extractor, an initial feature extractor can be prepared first. Multiple sample forged emblems are generated, and the initial feature extractor is trained based on the sample emblem images of each sample forged emblem until the initial feature extractor converges to the desired performance, thus obtaining the preset feature extractor. In this way, through a large number of sample forged emblems, the initial feature extractor can learn how to obtain accurate image morphological features.

[0061] Specifically, in real-world scenarios, the images of emblems may exhibit rotation, unclear texture, or blurriness. Therefore, when training the initial feature extractor, diverse sample data should be prepared to facilitate the initial feature extractor's learning of diverse sample features. In practice, each sample forged emblem can undergo a preset number of detail variation processing steps to obtain multiple sample randomly varied emblems. These detail variations include changes in rotation angle, texture, and clarity / blurring. The initial feature extractor is then trained based on the sample emblem images of each randomly varied emblem until the initial feature extractor converges to the desired performance.

[0062] In one feasible implementation, the number of sample forged emblems can be 500. Each sample forged emblem undergoes the following detailed modification: random rotation by 10 angles, random texture generation, and random erosion blurring, resulting in 5000 randomly varied sample emblems. A large amount of diverse sample data is beneficial for the model's performance convergence.

[0063] Alternatively, when selecting a training method, the initial feature extractor can undergo contrastive learning. Contrastive learning enables the feature extractor to learn the subtle differences between emblems, and the results of matching experiments are superior to feature extraction methods that use general direct classification tasks as feature learning methods.

[0064] Specifically, contrastive learning requires grouping individual sample images into multiple sets of triples. That is, the sample herald image with randomly varying heraldry for each sample is used as the first sample herald image. The sample triples for each first sample herald image are determined. Each sample triple includes the first sample herald image, the second sample herald image which comes from the same sample forged herald as the first sample herald image, and the third sample herald image which comes from a different sample forged herald image than the first sample herald image. Then, based on each sample triple, the initial feature extractor is subjected to contrastive learning. The loss value during model training is calculated using various loss algorithms such as arcface loss and cross entropy loss. The network parameters of the initial feature extractor are continuously adjusted according to the loss value until the initial feature extractor converges to the desired performance.

[0065] Optionally, considering that the heraldic image may be rotated, in order to ensure the accuracy of the image morphology features, the rotation angle of the heraldic image can be identified and corrected before extracting the image morphology features, and then the feature extraction can be performed. That is, it is also necessary to pre-train a rotation angle predictor based on sample heraldic images of randomly varied heraldic images for each sample to determine the rotation angle of the heraldic image and correct it.

[0066] S312. If the emblem to be added to the database is a historical emblem or a counterfeit emblem, the unique identification information, text information and image morphological features corresponding to each historical emblem shall be saved to the first emblem database, and the unique identification information, text information and image morphological features corresponding to each counterfeit emblem shall be saved to the second emblem database.

[0067] Optionally, the types of emblems to be added to the database can be various. One type is historical emblems uploaded by other users whose authenticity is not entirely certain; the other type is counterfeit seals that have been investigated and confirmed to be fake. Both types of emblems can be used as emblems to be added to the database. When saving emblems to be added to the database, different emblem databases need to be divided according to the different types of emblems. That is, the unique identification information, text information, and image morphological features corresponding to each historical emblem are saved to the first emblem database, and the unique identification information, text information, and image morphological features corresponding to each counterfeit emblem are saved to the second emblem database. If a subsequent emblem to be verified is the same as an emblem in the first emblem database, it means that the emblem to be verified has also been used by other users; if a emblem to be verified is the same as an emblem in the second emblem database, it means that the emblem to be verified is also a counterfeit emblem.

[0068] This disclosure provides a method for constructing a heraldic library. When recognizing the text information of a heraldic emblem, the heraldic image is divided into an upper circle image and middle and lower circle images. The upper circle image is expanded using polar coordinates to obtain a straightened upper circle image. Based on an image text recognition algorithm, the upper circle text of the heraldic emblem to be included in the library is extracted from the straightened upper circle image, thereby accurately identifying text with a high degree of curvature. Before extracting image morphological features, a feature extractor is pre-trained using contrastive learning. By generating diverse samples to forge heraldic emblems, the feature extractor learns the extraction knowledge of diverse image morphological features. For various types of heraldic emblems to be included in the library, they are stored in different heraldic libraries according to their type for later retrieval.

[0069] Please refer to Figure 4, which is a flowchart illustrating a heraldry retrieval method provided in an embodiment of this disclosure.

[0070] As shown in Figure 4, the emblem retrieval method may include at least the following steps.

[0071] S402. Determine the heraldry image corresponding to the target heraldry, extract the target text information from the heraldry image, compare the target text information with the text information of each heraldry in the heraldry library, and obtain the text similarity between the target heraldry and each heraldry in the library.

[0072] Optionally, in real-world scenarios, when user-submitted materials contain heraldry, the authenticity of the target heraldry needs to be verified. This can be achieved by searching for the target heraldry to determine whether it is a previously known heraldry or a counterfeit. It should be noted that the search database used during the search is the heraldry database obtained through the heraldry database construction method described in the above embodiments. This database includes the textual information and image morphological features of several heraldry in the database.

[0073] In this embodiment of the disclosure, the heraldry image corresponding to the target heraldry is first determined, the target text information is extracted based on the heraldry image, and the target text information is compared with the text information of each heraldry in the heraldry library to obtain the text similarity between the target heraldry and each heraldry in the library.

[0074] S404. Determine whether the text similarity meets the preset filtering conditions. If it does, extract the target image morphological features of the herald image, compare the target image morphological features with the image morphological features of each herald in the herald library, and obtain the morphological similarity between the target herald and each herald in the library.

[0075] Optionally, if the text similarity between the target emblem and each emblem in the database is very low, it indicates that the target emblem is unlikely to be a duplicate or forged emblem. In this case, it is unnecessary to compare image morphological features, saving unnecessary computational resources. Therefore, preset filtering conditions need to be set for text similarity to filter target emblems that require subsequent image morphological feature comparison and to directly end the search for target emblems that do not require further comparison. The preset filtering conditions can be set according to the needs of the actual scenario and / or empirical values; this embodiment does not limit this.

[0076] Optionally, if the text similarity meets the preset screening conditions, it is necessary to further extract the target image morphological features of the herald image, and compare the target image morphological features with the image morphological features of each herald in the herald library to obtain the morphological similarity between the target herald and each herald in the library.

[0077] S406. Based on the textual and morphological similarity between the target coat of arms and each coat of arms in the database, calculate the overall similarity between the target coat of arms and each coat of arms in the database.

[0078] Optionally, after obtaining the textual and morphological similarity between the target coat of arms and each coat of arms in the database, the overall similarity between the target coat of arms and each coat of arms in the database can be calculated by combining the textual and morphological similarity between the target coat of arms and the same coat of arms in the database.

[0079] S408. Determine the retrieval results of the target emblem in the emblem database based on the overall similarity.

[0080] Optionally, the final retrieval results for the target coat of arms are obtained based on the overall similarity between the target coat of arms and all existing coats of arms in the database. These results include information on which existing coats of arms the target coat of arms is similar to, and the degree of similarity. Based on these results, the system can make a corresponding judgment on the authenticity of the target coat of arms. By comparing the textual and image morphological information of the target coat of arms within the database, and through multi-dimensional feature comparison, the authenticity of the current coat of arms can be determined more accurately.

[0081] This disclosure provides a heraldic retrieval method. The method involves: determining the heraldic image corresponding to a target heraldic emblem; extracting the target text information from the heraldic image; comparing the target text information with the text information of each heraldic emblem in the heraldic database to obtain the text similarity between the target heraldic emblem and each heraldic emblem in the database; determining whether the text similarity meets preset filtering conditions; if so, extracting the target image morphological features from the heraldic image; comparing the target image morphological features with the image morphological features of each heraldic emblem in the heraldic database to obtain the morphological similarity between the target heraldic emblem and each heraldic emblem in the database; calculating the overall similarity between the target heraldic emblem and each heraldic emblem in the database based on the text similarity and morphological similarity; and determining the retrieval result of the target heraldic emblem in the heraldic database based on the overall similarity. Since the database contains textual information and image morphological features of the coat of arms, when using the coat of arms database for retrieval, both textual and morphological factors of the coat of arms are taken into account, resulting in better retrieval results. The database is directly searched and compared based on the textual and image morphological information of the target coat of arms, and the authenticity of the current coat of arms is more accurately determined through multi-dimensional feature comparison.

[0082] Please refer to Figure 5, which is a flowchart illustrating a heraldry retrieval method according to another embodiment of this disclosure.

[0083] As shown in Figure 5, the emblem retrieval method may include at least the following steps.

[0084] S502. Determine the heraldry image corresponding to the target heraldry and extract the target text information from the heraldry image.

[0085] In this embodiment of the disclosure, the heraldry image corresponding to the target heraldry is first determined, and the target text information is extracted based on the heraldry image so that the target text information can be compared with the text information of each heraldry in the heraldry library to obtain the text similarity between the target heraldry and each heraldry in the library.

[0086] S504. Based on a preset hash algorithm, convert the target text information and the text information of each herald in the herald library into hash values, and calculate the text similarity between the target herald and each herald in the library based on the hash values ​​of the target text information and the hash values ​​of the text information of each herald in the library.

[0087] Alternatively, when calculating text similarity, a hash algorithm (simhash algorithm) can be used to calculate the similarity between text information. Simhash is a locality-sensitive hash; it assumes that two strings have a certain degree of similarity, and this similarity is maintained after hashing. Therefore, it is very suitable for calculating the similarity between two pieces of text. The simhash algorithm converts the target text information and the text information of each herald in the heraldry database into hash values. Based on the distance between the hash values ​​of the target text information and the hash values ​​of the text information of each herald in the database, the text similarity between the target herald and each herald in the database is obtained.

[0088] S506. Determine whether the text similarity meets the preset filtering conditions. The preset filtering conditions are that there is a text in the database that is the same as the target emblem or the text similarity between the target emblem and at least M text in the database reaches a preset ratio, where M is a positive integer.

[0089] Optionally, as described in the above embodiments, preset filtering conditions need to be set for text similarity to filter target emblems that require subsequent comparison of image morphological features, and to directly end the search for target emblems that do not require subsequent comparison of image morphological features. That is, after calculating the text similarity between the target emblem and each emblem in the database, it is further determined whether the text similarity meets the preset filtering conditions.

[0090] In this embodiment, the preset filtering condition is the existence of an in-stock emblem identical to the target emblem, or the target emblem having a text similarity of at least M in-stock emblems reaching a preset ratio, where M is a positive integer. That is, if an in-stock emblem is exactly the same as the target emblem, it indicates a high risk of forgery for the target emblem; and if a large number of in-stock emblems have a high similarity to the target emblem, it also indicates a high risk of forgery for the target emblem. When setting the preset filtering condition, M can be 5, and the preset ratio is 90%. In other words, when at least 5 in-stock emblems have a similarity of 90% to the target emblem, it proves that the target emblem has a very high risk of forgery.

[0091] Optionally, if the target emblem does not meet the preset filtering conditions, the search result for the target emblem is determined to be a no-match, that is, the target emblem can be considered to be a real emblem at this time.

[0092] S508. If satisfied, extract the target image morphological features of the heraldry image, compare the target image morphological features with the image morphological features of each heraldry in the heraldry library, and obtain the morphological similarity between the target heraldry and each heraldry in the library.

[0093] Optionally, if the target heraldry meets the preset filtering conditions, it indicates that there are many heraldry in the database with high similarity to the text information of the target heraldry, and further comparison of image morphological features is required. That is, extract the target image morphological features of the heraldry image, compare the target image morphological features with the image morphological features of each heraldry in the database, and obtain the morphological similarity between the target heraldry and each heraldry in the database.

[0094] S510. Determine the first weight corresponding to text similarity and the second weight corresponding to morphological similarity.

[0095] Optionally, the overall similarity between the target emblem and each emblem in the database can be calculated by combining text similarity and morphological similarity. Specifically, text similarity and morphological similarity can be weighted, firstly determining the first weight corresponding to text similarity and the second weight corresponding to morphological similarity. It should be noted that the first weight and the second weight can both be 0.5, or they can be unequal. The specific weight allocation can be adjusted according to the needs of the application scenario, and this embodiment does not limit it.

[0096] S512. Multiply the text similarity between the target emblem and each emblem in the database by the first weight to obtain the first product between the target emblem and each emblem in the database. Multiply the morphological similarity between the target emblem and each emblem in the database by the second weight to obtain the second product between the target emblem and each emblem in the database. Add the first product and the second product between the target emblem and each emblem in the database to obtain the overall similarity between the target emblem and each emblem in the database.

[0097] Optionally, the textual similarity between the target emblem and each of the existing emblems is multiplied by a first weight to obtain a first product between the target emblem and each of the existing emblems. The morphological similarity between the target emblem and each of the existing emblems is multiplied by a second weight to obtain a second product between the target emblem and each of the existing emblems. The first and second products are then added together to obtain the overall similarity between the target emblem and each of the existing emblems. By using a weighted approach, combining textual and morphological similarity, the overall similarity between the target emblem and each of the existing emblems is calculated.

[0098] S514. Sort each heraldry in the database from high to low similarity based on the overall similarity, and determine the top K target heraldry in the database as the retrieval results of the target heraldry in the heraldry database; K is a positive integer.

[0099] Optionally, after obtaining the overall similarity, the in-database heralds can be sorted from high to low based on their overall similarity, and the top K target in-database heralds can be determined as the retrieval results of the target herald in the herald database; K is a positive integer, thus obtaining the in-database heralds with high similarity to the target herald. In one feasible implementation, the top 3 in-database heralds with the highest similarity can be selected as the target in-database heralds, thus obtaining the retrieval results of the target herald.

[0100] Optionally, when constructing the heraldry library, the types of heraldry included in the library may be historical heraldry uploaded by users or forged heraldry identified as fake. In this case, the heraldry library includes a first heraldry library and a second heraldry library. The first heraldry library is used to store the unique identification information, text information and image morphological features corresponding to historical heraldry, while the second heraldry library is used to store the unique identification information, text information and image morphological features corresponding to forged heraldry.

[0101] Optionally, since the emblems in the first emblem library are historical emblems uploaded by users, these emblems may be shared by multiple people. Therefore, if the target emblem matches a large number of emblems whose authenticity cannot be denied, it indicates that the authenticity of the current target emblem needs to be verified. On the other hand, the second emblem library contains counterfeit emblems that are confirmed to be fake. Therefore, if the target emblem matches a large number of emblems in the second emblem library, it indicates that the target emblem must be a counterfeit emblem.

[0102] Specifically, when determining whether a target emblem is a counterfeit emblem or an emblem to be verified, the number of emblems belonging to the second emblem library among the top K target emblems in the database can be used as a basis. That is, if the number of emblems belonging to the second emblem library among the top K target emblems in the database is greater than N, then the search result for the target emblem is determined to be a counterfeit emblem. N is a positive integer and less than or equal to K. For example, N can be equal to K. When K is 3, N is also 3. Then, when the top 3 emblems with the highest similarity to the target emblem all belong to the second emblem library, it means that the hit rate and similarity to counterfeit emblems are very high. At this time, the target emblem can be considered a counterfeit emblem. If the number is less than or equal to N, it means that among the most similar emblems in the database, some come from the first emblem database and some come from the second emblem database. In this case, it is not possible to directly determine that the target emblem is a counterfeit emblem. It is necessary to mark the target emblem according to the number of hits in the first and second emblem databases so that specialized technical personnel can conduct manual or other methods to verify the authenticity of the target emblem based on the markings.

[0103] This disclosure provides a heraldic retrieval method. Based on a preset hash algorithm, target text information and the text information of each herald in the heraldic database are converted into hash values. The text similarity between the target herald and each herald in the database is calculated based on the hash values ​​of the target text information and the text information of each herald in the database. It is then determined whether the text similarity meets preset filtering conditions. These preset filtering conditions are either the existence of a herald in the database identical to the target herald, or the target herald having a text similarity with at least M heralds in the database reaching a preset ratio, where M is a positive integer. If satisfied, the target image morphological features of the herald image are extracted, and these features are compared with the image morphological features of each herald in the database to obtain the morphological similarity between the target herald and each herald in the database. Based on the overall similarity, each herald in the database is sorted from high to low similarity, and the top K target heralds in the database are determined as the retrieval results of the target herald in the heraldic database. When using the heraldry database for retrieval, both the textual and morphological factors of the heraldry are taken into account, resulting in better retrieval performance. The database is directly searched and compared based on the textual and image morphological information of the target heraldry, and the authenticity of the current heraldry is more accurately determined through multi-dimensional feature comparison.

[0104] Please refer to Figure 6, which is a structural block diagram of an heraldic library construction device provided in an embodiment of this disclosure. As shown in Figure 6, the heraldic library construction device 600 includes: a text information extraction module 610, used to acquire the heraldic image of the heraldic emblem to be added to the library, and extract the text information of the heraldic emblem to be added to the library based on the heraldic image; a morphological feature extraction module 620, used to input the heraldic image into a preset feature extractor, and determine the image morphological features of the heraldic emblem to be added to the library output by the preset feature extractor based on the heraldic image; and a library storage module 630, used to save the unique identification information, text information, and image morphological features of the heraldic emblem to be added to the heraldic library.

[0105] Optionally, the text information extraction module 610 is further configured to divide the heraldic image into an upper circle image and a middle and lower circle image; perform polar coordinate expansion processing on the upper circle image to obtain a straightened upper circle image; extract the upper circle text of the heraldic image to be included in the database based on an image text recognition algorithm; extract the middle circle text and lower circle text of the heraldic image to be included in the database based on an image text recognition algorithm; and concatenate the upper circle text, middle circle text, and lower circle text to obtain the text information of the heraldic image to be included in the database.

[0106] Optionally, the emblem library construction device 600 further includes: a feature extractor training module, used to generate multiple sample forged emblems, and train an initial feature extractor based on the sample emblem images of each sample forged emblem until the initial feature extractor converges to the desired performance, thereby obtaining a preset feature extractor.

[0107] Optionally, the emblem library construction device 600 further includes: a sample variation processing module, used to perform a preset number of detail variation processing on each sample forged emblem to obtain multiple sample randomly varied emblems, the detail variations including rotation angle variation, texture variation, and sharpness blur variation; and a feature extractor training module, used to train an initial feature extractor based on the sample emblem images of each sample randomly varied emblem until the initial feature extractor converges to the desired performance.

[0108] Optionally, the feature extractor training module is further configured to use the sample herald image of each randomly varied herald as the first sample herald image, determine the sample triplet of each first sample herald image, the sample triplet including the first sample herald image, the second sample herald image from the same sample forged herald as the first sample herald image, and the third sample herald image from a different sample forged herald as the first sample herald image; and perform comparative learning on the initial feature extractor based on each sample triplet until the initial feature extractor converges to the desired performance.

[0109] Optionally, the heraldry library construction device 600 further includes: a rotation predictor training module, used to train a rotation angle predictor based on sample heraldry images of randomly varying heraldry images, the rotation angle predictor being used to determine the rotation angle of the heraldry and turn the heraldry upright.

[0110] Optionally, the emblems to be added to the database are historical emblems or counterfeit emblems. The database storage module 630 is also used to save the unique identification information, text information and image morphological features corresponding to each historical emblem to the first emblem database, and to save the unique identification information, text information and image morphological features corresponding to each counterfeit emblem to the second emblem database.

[0111] In this embodiment, a heraldic library construction apparatus is provided, comprising: a text information extraction module for acquiring heraldic images of heraldic images to be added to the library, and extracting text information of the heraldic images; a morphological feature extraction module for inputting the heraldic images into a preset feature extractor, and determining the image morphological features of the heraldic images output by the preset feature extractor based on the heraldic images; and a library storage module for storing the unique identifier information, text information, and image morphological features of the heraldic images to be added to the library. Since text information is one of the most important pieces of information about a heraldic image, and image morphological features can describe the detailed information of the heraldic image, when adding a heraldic image to the library, both text information and image morphological features of the heraldic image are extracted, and both types of information are stored in the heraldic library. Subsequent retrieval using the heraldic library allows for comparison and retrieval based on the multi-dimensional information of the heraldic images, improving the accuracy of retrieval results in application scenarios.

[0112] Please refer to Figure 7, which is a structural block diagram of a heraldic retrieval device provided in an embodiment of this disclosure. As shown in Figure 7, the heraldic retrieval device 700 includes: a text feature comparison module 710, used to determine the heraldic image corresponding to the target heraldic image, extract the target text information of the heraldic image, compare the target text information with the text information of each heraldic image in the heraldic database, and obtain the text similarity between the target heraldic image and each heraldic image in the database; a morphological feature comparison module 720, used to determine whether the text similarity meets the preset screening conditions, and if so, extract the target image morphological features of the heraldic image, compare the target image morphological features with the image morphological features of each heraldic image in the heraldic database, and obtain the morphological similarity between the target heraldic image and each heraldic image in the database; a similarity calculation module 730, used to calculate the overall similarity between the target heraldic image and each heraldic image in the database based on the text similarity and morphological similarity between the target heraldic image and each heraldic image in the database; and a retrieval result output module 740, used to determine the retrieval result of the target heraldic image in the heraldic database based on the overall similarity.

[0113] Optionally, the heraldry library includes a first heraldry library and a second heraldry library. The first heraldry library is used to store the unique identification information, text information, and image morphological features corresponding to historical heraldry, and the second heraldry library is used to store the unique identification information, text information, and image morphological features corresponding to counterfeit heraldry. The preset filtering condition is that there is a heraldry in the library that is the same as the target heraldry, or the text similarity between the target heraldry and at least M heraldry in the library reaches a preset ratio, where M is a positive integer.

[0114] Optionally, the retrieval result output module 740 is also used to sort each in-database heraldry from high to low similarity based on the overall similarity, and determine the top K target in-database heraldry as the retrieval results of the target heraldry in the heraldry database; K is a positive integer.

[0115] Optionally, the retrieval result output module 740 is further configured to determine the number of in-stock emblems belonging to the second emblem library among the top K target in-stock emblems. If the number is greater than N, the retrieval result of the target emblem is determined to be a hit forged emblem; N is a positive integer and less than or equal to K; if the number is less than or equal to N, the target emblem is marked according to the hit count of the target emblem in the first emblem library and the second emblem library, so that the authenticity of the target emblem can be verified according to the marking of the target emblem.

[0116] Optionally, the text feature comparison module 710 is also used to convert the target text information and the text information of each herald in the herald library into hash values ​​based on a preset hash algorithm, and to calculate the text similarity between the target herald and each herald in the library based on the hash value of the target text information and the hash value of the text information of each herald in the library.

[0117] Optionally, the similarity calculation module 730 is further used to determine the first weight corresponding to the text similarity and the second weight corresponding to the morphological similarity; multiply the text similarity between the target emblem and each emblem in the database by the first weight to obtain the first product between the target emblem and each emblem in the database; multiply the morphological similarity between the target emblem and each emblem in the database by the second weight to obtain the second product between the target emblem and each emblem in the database; and add the first product and the second product between the target emblem and each emblem in the database to obtain the overall similarity between the target emblem and each emblem in the database.

[0118] Optionally, the heraldry retrieval device 700 further includes a text retrieval result output module, used to determine that the retrieval result for the target heraldry is not found if the conditions are not met.

[0119] In this embodiment of the disclosure, a heraldic retrieval device is provided, wherein: a text feature comparison module is used to determine the heraldic image corresponding to the target heraldic image, extract the target text information of the heraldic image, and compare the target text information with the text information of each heraldic image in the heraldic database to obtain the text similarity between the target heraldic image and each heraldic image in the database; a morphological feature comparison module is used to determine whether the text similarity meets the preset screening conditions, and if it does, extract the target image morphological features of the heraldic image, and compare the target image morphological features with the image morphological features of each heraldic image in the heraldic database to obtain the morphological similarity between the target heraldic image and each heraldic image in the database; a similarity calculation module is used to calculate the overall similarity between the target heraldic image and each heraldic image in the database based on the text similarity and morphological similarity between the target heraldic image and each heraldic image in the database; and a retrieval result output module is used to determine the retrieval result of the target heraldic image in the heraldic database based on the overall similarity. Since the database contains textual information and image morphological features of the coat of arms, when using the coat of arms database for retrieval, both textual and morphological factors of the coat of arms are taken into account, resulting in better retrieval results. The database is directly searched and compared based on the textual and image morphological information of the target coat of arms, and the authenticity of the current coat of arms is more accurately determined through multi-dimensional feature comparison.

[0120] This disclosure provides a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of any of the methods described above.

[0121] This disclosure also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the steps of any of the methods described in the above embodiments.

[0122] Please refer to Figure 8, which is a schematic diagram of the structure of a terminal provided in an embodiment of this disclosure. As shown in Figure 8, the terminal 800 may include: at least one terminal processor 801, at least one network interface 804, a user interface 803, a memory 805, and at least one communication bus 802.

[0123] The communication bus 802 is used to enable communication between these components.

[0124] The user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.

[0125] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0126] The terminal processor 801 may include one or more processing cores. The terminal processor 801 connects to various parts within the terminal 800 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling data stored in the memory 805. Optionally, the terminal processor 801 may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The terminal processor 801 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the terminal processor 801.

[0127] The memory 805 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 805 may include a non-transitory computer-readable storage medium. The memory 805 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned terminal processor 801. As shown in FIG8, the memory 805, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an emblem library construction program.

[0128] In the terminal 800 shown in Figure 8, the user interface 803 is mainly used to provide an input interface for the user and obtain the user input data; while the terminal processor 801 can be used to call the heraldry library construction program stored in the memory 805 and specifically perform the following operations: obtain the heraldry image of the heraldry to be added to the library, extract the text information of the heraldry to be added to the library based on the heraldry image; input the heraldry image into the preset feature extractor, determine the image morphological features of the heraldry to be added to the library output by the preset feature extractor based on the heraldry image; and save the unique identification information, text information and image morphological features of the heraldry to be added to the heraldry library accordingly.

[0129] In some embodiments, when the terminal processor 801 extracts the text information of the herald to be stored based on the herald image, it specifically performs the following steps: dividing the herald image into an upper circle image and a middle and lower circle image; performing polar coordinate expansion processing on the upper circle image to obtain a straightened upper circle image; extracting the upper circle text of the herald to be stored from the straightened upper circle image based on an image text recognition algorithm; extracting the middle circle text and the lower circle text of the herald to be stored from the middle and lower circle images based on an image text recognition algorithm; and concatenating the upper circle text, the middle circle text, and the lower circle text to obtain the text information of the herald to be stored.

[0130] In some embodiments, the terminal processor 801 further performs the following steps: generating multiple sample forged emblems, training an initial feature extractor based on the sample emblem images of each sample forged emblem until the initial feature extractor converges to the desired performance, and obtaining a preset feature extractor.

[0131] In some embodiments, after generating multiple sample forged emblems, the terminal processor 801 further performs the following steps: performing detail change processing on each sample forged emblem a preset number of times to obtain multiple sample randomly changed emblems, the detail changes including rotation angle changes, texture changes, and sharpness blur changes; when the terminal processor 801 trains an initial feature extractor based on the sample emblem images of each sample forged emblem until the initial feature extractor converges to the desired performance, the terminal processor 801 further performs the following steps: training an initial feature extractor based on the sample emblem images of each sample randomly changed emblem until the initial feature extractor converges to the desired performance.

[0132] In some embodiments, when the terminal processor 801 trains an initial feature extractor based on sample emblem images of each sample forged emblem until the initial feature extractor converges to the desired performance, it specifically performs the following steps: taking the sample emblem image of each randomly varied emblem as the first sample emblem image, determining sample triples for each first sample emblem image, wherein the sample triples include the first sample emblem image, a second sample emblem image originating from the same sample forged emblem as the first sample emblem image, and a third sample emblem image originating from a different sample forged emblem than the first sample emblem image; performing comparative learning on the initial feature extractor based on each sample triple until the initial feature extractor converges to the desired performance.

[0133] In some embodiments, the terminal processor 801 further performs the following steps: training a rotation angle predictor based on the sample herald images of randomly changing heralds for each sample, wherein the rotation angle predictor is used to determine the rotation angle of the herald and turn the herald upright.

[0134] In some embodiments, the emblem to be added to the database is a historical emblem or a counterfeit emblem. When the terminal processor 801 saves the unique identification information, text information and image morphological features of the emblem to be added to the emblem database, it specifically performs the following steps: saving the unique identification information, text information and image morphological features corresponding to each historical emblem to the first emblem database, and saving the unique identification information, text information and image morphological features corresponding to each counterfeit emblem to the second emblem database.

[0135] Please refer to Figure 9, which is a schematic diagram of the structure of a terminal provided in another embodiment of this disclosure. As shown in Figure 9, the terminal 900 may include: at least one terminal processor 901, at least one network interface 904, a user interface 903, a memory 905, and at least one communication bus 902.

[0136] The communication bus 902 is used to enable communication between these components.

[0137] The user interface 903 may include a display screen and a camera. Optionally, the user interface 903 may also include a standard wired interface and a wireless interface.

[0138] The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0139] The terminal processor 901 may include one or more processing cores. The terminal processor 901 connects to various parts within the terminal 900 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 905, and by calling data stored in the memory 905. Optionally, the terminal processor 901 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The terminal processor 901 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the terminal processor 901.

[0140] The memory 905 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 905 may include a non-transitory computer-readable storage medium. The memory 905 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 905 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 905 may also be at least one storage device located remotely from the aforementioned terminal processor 901. As shown in FIG9, the memory 905, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an emblem retrieval program.

[0141] In the terminal 900 shown in Figure 9, the user interface 903 is mainly used to provide an input interface for the user and obtain the user input data; while the terminal processor 901 can be used to call the heraldry retrieval program stored in the memory 905 and specifically perform the following operations: determine the heraldry image corresponding to the target heraldry, extract the target text information of the heraldry image, compare the target text information with the text information of each heraldry in the heraldry library to obtain the text similarity between the target heraldry and each heraldry in the library; determine whether the text similarity meets the preset filtering conditions, if it does, extract the target image morphological features of the heraldry image, compare the target image morphological features with the image morphological features of each heraldry in the heraldry library to obtain the morphological similarity between the target heraldry and each heraldry in the library; calculate the overall similarity between the target heraldry and each heraldry in the library based on the text similarity and morphological similarity between the target heraldry and each heraldry in the library; determine the retrieval result of the target heraldry in the heraldry library based on the overall similarity; wherein, the heraldry library is the heraldry library provided in the above heraldry library construction method.

[0142] In some embodiments, the heraldry library includes a first heraldry library and a second heraldry library. The first heraldry library is used to store the unique identification information, text information and image morphological features corresponding to historical heraldry, and the second heraldry library is used to store the unique identification information, text information and image morphological features corresponding to counterfeit heraldry. The preset filtering condition is that there is a heraldry in the library that is the same as the target heraldry or the text similarity between the target heraldry and at least M heraldry in the library reaches a preset ratio, where M is a positive integer.

[0143] In some embodiments, when the terminal processor 901 performs the following steps to determine the retrieval results of the target emblem in the emblem library based on the overall similarity: sorting the emblems in the library from high to low similarity based on the overall similarity, and determining the top K target emblems in the library as the retrieval results of the target emblem in the emblem library; K is a positive integer.

[0144] In some embodiments, when the terminal processor 901 executes the retrieval result of determining the top K target emblems in the emblem library as the target emblem in the emblem library, it specifically performs the following steps: determining the number of emblems in the second emblem library among the top K target emblems; if the number is greater than N, then determining that the retrieval result of the target emblem is a hit forged emblem; N is a positive integer and less than or equal to K; if the number is less than or equal to N, then marking the target emblem according to the hit count of the target emblem in the first emblem library and the second emblem library, so that the authenticity of the target emblem can be verified according to the marking of the target emblem.

[0145] In some embodiments, when the terminal processor 901 compares the target text information with the text information of each herald in the heraldry library to obtain the text similarity between the target heraldry and each heraldry in the library, it specifically performs the following steps: converting the target text information and the text information of each heraldry in the library into hash values ​​based on a preset hash algorithm, and calculating the text similarity between the target heraldry and each heraldry in the library based on the hash value of the target text information and the hash value of the text information of each heraldry in the library.

[0146] In some embodiments, when the terminal processor 901 calculates the overall similarity between the target emblem and each emblem in the database based on the text similarity and morphological similarity between the target emblem and each emblem in the database, it specifically performs the following steps: determining the first weight corresponding to the text similarity and the second weight corresponding to the morphological similarity; multiplying the text similarity between the target emblem and each emblem in the database by the first weight to obtain the first product of the target emblem and each emblem in the database; multiplying the morphological similarity between the target emblem and each emblem in the database by the second weight to obtain the second product of the target emblem and each emblem in the database; and adding the first product and the second product of the target emblem and each emblem in the database to obtain the overall similarity between the target emblem and each emblem in the database.

[0147] In some embodiments, after determining whether the text similarity meets the preset filtering conditions, the terminal processor 901 further performs the following steps: if it does not meet the conditions, the search result for the target emblem is determined to be a miss.

[0148] In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0149] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0150] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0151] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0152] Furthermore, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, historical emblems and counterfeit emblems involved in this disclosure were obtained under full authorization.

[0153] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0154] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0155] The above is a description of a method for constructing an emblem library, an emblem retrieval method, an apparatus, a storage medium, and a terminal provided by the embodiments of this disclosure. For those skilled in the art, based on the ideas of the embodiments of this disclosure, there will be changes in the specific implementation methods and application scope. Therefore, the content of this disclosure should not be construed as a limitation on the embodiments of this disclosure.

Claims

1. A coat of arms library construction method, the method comprising: acquiring a coat of arms image of a to-be-stored coat of arms, and extracting text information of the to-be-stored coat of arms based on the coat of arms image; inputting the coat of arms image into a preset feature extractor, and determining image form feature of the to-be-stored coat of arms output by the preset feature extractor according to the coat of arms image; correspondingly storing unique identification information, text information and image form feature of the to-be-stored coat of arms into a coat of arms library.

2. The method of claim 1, wherein the text information of the to-be-stored coat of arms is extracted based on the coat of arms image, comprising: dividing the coat of arms image into an upper circle image and a middle-lower circle image; performing polar coordinate unfolding processing on the upper circle image to obtain a straightened upper circle image, and extracting upper circle text of the to-be-stored coat of arms in the straightened upper circle image based on an image text recognition algorithm; extracting middle circle text and lower circle text of the to-be-stored coat of arms in the middle-lower circle image based on the image text recognition algorithm; splicing the upper circle text, the middle circle text and the lower circle text to obtain the text information of the to-be-stored coat of arms.

3. The method of claim 1, further comprising: generating a plurality of sample fake coats of arms, and training an initial feature extractor based on sample coat of arms images of the sample fake coats of arms until the initial feature extractor converges to a desired performance to obtain a preset feature extractor.

4. The method of claim 3, after the plurality of sample fake coats of arms are generated, further comprising: performing preset number of detail change processing on each sample fake coat of arms to obtain a plurality of sample random change coats of arms, wherein the detail change includes rotation angle change, texture change and clarity blurring change; the initial feature extractor is trained based on sample coat of arms images of the sample fake coats of arms until the initial feature extractor converges to a desired performance, comprising: the initial feature extractor is trained based on sample coat of arms images of the sample random change coats of arms until the initial feature extractor converges to a desired performance.

5. The method of claim 4, wherein the initial feature extractor is trained based on sample coat of arms images of the sample fake coats of arms until the initial feature extractor converges to a desired performance, comprising: taking the sample coat of arms image of each sample random change coat of arms as a first sample coat of arms image, determining a sample triple group of each first sample coat of arms image, wherein the sample triple group includes the first sample coat of arms image, a second sample coat of arms image derived from the same sample fake coat of arms as the first sample coat of arms image, and a third sample coat of arms image derived from a different sample fake coat of arms as the first sample coat of arms image; performing contrast learning on the initial feature extractor based on each sample triple group until the initial feature extractor converges to a desired performance.

6. The method of claim 4, further comprising: training a rotation angle predictor based on sample coat of arms images of the sample random change coats of arms, wherein the rotation angle predictor is used to judge a coat of arms rotation angle and correct the coat of arms.

7. The method of claim 1, wherein the to-be-archived coat of arms is a historical coat of arms or a counterfeit coat of arms, and the corresponding saving of the unique identification information, the text information and the image form features of the to-be-archived coat of arms into the coat of arms library comprises: corresponding saving of the unique identification information, the text information and the image form features of each historical coat of arms into a first coat of arms library, and corresponding saving of the unique identification information, the text information and the image form features of each counterfeit coat of arms into a second coat of arms library.

8. A coat of arms retrieval method, comprising: determining a coat of arms image corresponding to a target coat of arms, extracting target text information of the coat of arms image, comparing the target text information with text information of each in-library coat of arms in a coat of arms library, to obtain a text similarity of the target coat of arms and each in-library coat of arms; judging whether the text similarity meets a preset screening condition, and if so, extracting a target image form feature of the coat of arms image, comparing the target image form feature with image form features of each in-library coat of arms in the coat of arms library, to obtain a form similarity of the target coat of arms and each in-library coat of arms; based on the text similarity and the form similarity of the target coat of arms and each in-library coat of arms, calculating an overall similarity of the target coat of arms and each in-library coat of arms; determining a retrieval result of the target coat of arms in the coat of arms library according to each overall similarity; wherein the coat of arms library is the coat of arms library of any one of claims 1-7.

9. The method of claim 8, wherein the coat of arms library comprises a first coat of arms library and a second coat of arms library, the first coat of arms library is used to store the unique identification information, the text information and the image form features of the historical coat of arms, and the second coat of arms library is used to store the unique identification information, the text information and the image form features of the counterfeit coat of arms; the preset screening condition is that there is an in-library coat of arms identical to the target coat of arms or the text similarity of the target coat of arms and at least M in-library coats of arms reaches a preset proportion, M being a positive integer.

10. The method of claim 9, wherein the determining of the retrieval result of the target coat of arms in the coat of arms library according to each overall similarity comprises: ranking each in-library coat of arms in a descending order of similarity according to each overall similarity, and determining the first K target in-library coats of arms as the retrieval result of the target coat of arms in the coat of arms library; K being a positive integer.

11. The method of claim 10, wherein the determining of the first K target in-library coats of arms as the retrieval result of the target coat of arms in the coat of arms library comprises: determining a number of in-library coats of arms belonging to the second coat of arms library among the first K target in-library coats of arms, and if the number is greater than N, determining that the retrieval result of the target coat of arms is a hit counterfeit coat of arms; N being a positive integer and less than or equal to K; and if the number is less than or equal to N, marking the target coat of arms according to a hit number of the target coat of arms in the first coat of arms library and the second coat of arms library, so as to enable authenticity verification of the target coat of arms according to the marking of the target coat of arms. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 12. The method of claim 8, wherein the comparing the target text information with the text information of each in-library emblem in the emblem library to obtain the text similarity between the target emblem and each in-library emblem comprises: converting the target text information and the text information of each in-library emblem in the emblem library into hash values based on a preset hash algorithm, and calculating the text similarity between the target emblem and each in-library emblem according to the hash value of the target text information and the hash value of the text information of each in-library emblem.

13. The method of claim 8, wherein the calculating the overall similarity between the target emblem and each in-library emblem based on the text similarity and the shape similarity between the target emblem and each in-library emblem comprises: determining a first weight corresponding to the text similarity and a second weight corresponding to the shape similarity; multiplying the text similarity between the target emblem and each in-library emblem by the first weight to obtain a first product of the target emblem and each in-library emblem, multiplying the shape similarity between the target emblem and each in-library emblem by the second weight to obtain a second product of the target emblem and each in-library emblem, and adding the first product and the second product of the target emblem and each in-library emblem to obtain the overall similarity between the target emblem and each in-library emblem.

14. The method of claim 8, wherein after the judging whether the text similarity satisfies the preset screening condition, the method further comprises: determining the search result of the target emblem as a miss if the text similarity does not satisfy the preset screening condition.

15. An emblem library construction device, comprising: a text information extraction module configured to acquire an emblem image of a to-be-stored emblem, and extract text information of the to-be-stored emblem based on the emblem image; a shape feature extraction module configured to input the emblem image into a preset feature extractor, and determine image shape features of the to-be-stored emblem output by the preset feature extractor according to the emblem image; and a storage module configured to store unique identification information, text information and image shape features of the to-be-stored emblem into an emblem library.

16. An emblem search device, comprising: a text feature comparison module configured to acquire an emblem image of a target emblem, extract target text information of the emblem image, and compare the target text information with text information of each in-library emblem in an emblem library to obtain text similarity between the target emblem and each in-library emblem; a shape feature comparison module configured to judge whether the text similarity satisfies a preset screening condition, and if the text similarity satisfies the preset screening condition, extract target image shape features of the emblem image, and compare the target image shape features with image shape features of each in-library emblem in the emblem library to obtain shape similarity between the target emblem and each in-library emblem; a similarity calculation module configured to calculate overall similarity between the target emblem and each in-library emblem based on the text similarity and the shape similarity between the target emblem and each in-library emblem; and a search result output module configured to determine a search result of the target emblem in the emblem library according to each overall similarity; wherein the emblem library is any one of the emblem libraries in claims 1-7. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 17. A computer program product comprising instructions which, when the computer program product is executed on a computer or a processor, cause the computer or the processor to carry out the steps of the method according to any one of claims 1-7 or 8-14.

18. A computer storage medium having stored thereon a plurality of instructions adapted to be loaded and executed by a processor to perform the steps of the method according to any one of claims 1-7 or 8-14.

19. A terminal comprising a memory, a processor, and a computer program stored on the memory and adapted to be executed on the processor, the processor implementing the steps of the method according to any one of claims 1-7 or 8-14 when executing the computer program.

Images