Identification method, device, electronic device and computer-readable storage medium

Abstract

The present disclosure relates to a method, device, electronic device and computer-readable storage medium for identifying a logo, the method comprising: performing area detection on an image containing a logo to be identified, obtaining a target area containing the logo to be identified; performing feature extraction on the target area, obtaining a feature vector to be processed of the target area; matching the feature vector to be processed with a preset feature vector library; if a match is determined, using a preset logo name corresponding to a matched preset feature vector subset as the logo name of the logo to be identified. Since the logos in the image are different, or the forms of the same logo are different, the extracted feature vectors are also different. Therefore, different feature vectors can be extracted for different logos and different forms of the same logo. The present disclosure not only improves the recognition rate of known logos, but also improves the recognition rate of new logos.

Description

Technical Field

[0001] The present disclosure relates to image recognition, and more particularly to a method, device, electronic device, and computer-readable storage medium for identifying an image. Background Art

[0002] Logo recognition involves identifying brand logos from product images / videos / live broadcasts. Compared with general object detection tasks, logo recognition has the following difficulties: different brand logos are highly similar; and the same brand logos have different shapes.

[0003] In the related art, the recognition method is similar to target detection, that is, different brand logos are identified as categories. However, due to the above difficulties, the recognition rate of this method is still low, especially for brand-new brand logos. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an identification recognition method, device, electronic device and storage medium. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a method for identifying an identifier, including:

[0006] Performing region detection on an image containing a mark to be identified, to obtain a target region containing the mark to be identified;

[0007] Extracting features of the target area to obtain a feature vector to be processed of the target area;

[0008] Matching the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identification name one by one;

[0009] If it is determined that there is a match, the preset identification name corresponding to the matched preset feature vector subset is used as the identification name of the identification to be identified.

[0010] Optionally, performing region detection on the image containing the mark to be identified to obtain a target region containing the mark to be identified includes:

[0011] Using a trained identification region detection model to detect the image, and determining target position information of the identification to be identified in the image;

[0012] The region of the to-be-recognized mark is segmented based on the target position information to obtain a target region including the to-be-recognized mark.

[0013] Optionally, the trained identification area detection model is trained in the following manner:

[0014] Inputting the first sample image containing the region annotation into a preset identification region detection model, so that the preset first identification region detection model detects the first sample image, determines the first position information of the target identification in the first sample image, and segments the region of the target identification based on the first position information to obtain a predicted region containing the target identification;

[0015] Calculating a first loss based on the region annotation and the predicted region;

[0016] The preset identification area detection model is trained based on the first loss to obtain a trained identification area detection model.

[0017] Optionally, extracting features from the target area to obtain a feature vector to be processed of the target area includes:

[0018] The trained marker recognition model is used to extract features of the target area to obtain a feature vector to be processed of the target area.

[0019] Optionally, the trained identification model is trained in the following manner:

[0020] Inputting the second sample image into a preset identification recognition model so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification;

[0021] Detecting the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier;

[0022] When the prediction result does not match the label, obtaining a second feature vector belonging to the same identifier as the label from the feature vector library, and obtaining a third feature vector belonging to a different identifier from the label;

[0023] Grouping the first eigenvector, the second eigenvector and the third eigenvector into a triple;

[0024] Calculate a second loss based on the triple;

[0025] The preset identification recognition model is trained based on the second loss to obtain a trained identification recognition model.

[0026] Optionally, the calculating a second loss based on the triplet includes:

[0027] Substituting the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substituting the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance;

[0028] Calculating a distance difference between the first distance and the second distance;

[0029] A second loss is calculated based on the distance difference and a preset distance difference threshold.

[0030] Optional,

[0031] The matching of the feature vector to be processed with a preset feature vector library includes:

[0032] Performing a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset;

[0033] If there are search results, it is determined that there is a match;

[0034] If there are no search results, it is determined that there is no match.

[0035] The identification method according to claim 1, characterized in that the method further comprises:

[0036] If it is determined that there is no match, the to-be-identified identifier is determined as the to-be-identified identifier;

[0037] Generate prompt information that the identifier to be identified is an identifier to be identified;

[0038] The prompt information is displayed.

[0039] Optionally, the method further includes:

[0040] Obtain the target identifier name of the identifier to be identified;

[0041] Determining a target feature vector corresponding to the target identifier;

[0042] Establishing an association relationship between the target identification name and the target feature vector;

[0043] Based on the association relationship and the target feature vector, the preset feature vector library is updated.

[0044] Optionally, each of the preset feature vector subsets has corresponding object category information; matching the feature vector to be processed with a preset feature vector library includes:

[0045] Determining target object category information of the image based on content information in the image;

[0046] Determining a subset of candidate feature vectors having the target object category information in the feature vector library;

[0047] The feature vector to be processed is matched with the subset of candidate feature vectors.

[0048] According to a second aspect of an embodiment of the present disclosure, there is provided an identification recognition device, including:

[0049] A region detection unit is configured to perform region detection on an image containing a mark to be identified, and obtain a target region containing the mark to be identified;

[0050] A feature extraction unit is configured to extract features from the target area to obtain a feature vector to be processed for the target area;

[0051] A matching unit is configured to match the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identification name one by one;

[0052] The first determination unit is configured to use the preset identification name corresponding to the matching preset feature vector subset as the identification name of the identification to be identified if it is determined that there is a match.

[0053] Optionally, the area detection unit includes:

[0054] A first calling subunit is configured to call a trained identification area detection model to detect the image and determine target position information of the identification to be identified in the image;

[0055] The marking subunit is configured to segment the area of ​​the to-be-recognized mark based on the target position information to obtain a target area containing the to-be-recognized mark.

[0056] Optionally, the device further comprises:

[0057] A first input unit is configured to input a first sample image containing region annotations into a preset identification region detection model, so that the preset identification region detection model detects the first sample image, determines first position information of a target identification in the first sample image, and segments a region of the target identification based on the first position information to obtain a predicted region containing the target identification;

[0058] A first calculation unit is configured to calculate a first loss based on the region annotation and the predicted region;

[0059] The first training unit is configured to train the preset identification area detection model based on the first loss to obtain a trained identification area detection model.

[0060] Optionally, the feature extraction unit is specifically configured as follows:

[0061] The trained identification model is called to extract features of the target area to obtain a feature vector to be processed of the target area.

[0062] Optionally, the device further comprises:

[0063] a second input unit configured to input a second sample image into a preset identification recognition model, so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification;

[0064] A prediction unit is configured to detect the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier;

[0065] an acquisition unit, configured to acquire, when the prediction result does not match the label, a second feature vector belonging to the same identifier as the label from the feature vector library, and acquire a third feature vector belonging to a different identifier from the acquired feature vector;

[0066] a combining unit, configured to combine the first feature vector, the second feature vector and the third feature vector into a triple;

[0067] A second calculation unit is configured to calculate a second loss based on the triple;

[0068] The second training unit is configured to train the preset identification model based on the second loss to obtain a trained identification model.

[0069] Optionally, the second computing unit is specifically configured to:

[0070] Substituting the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substituting the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance;

[0071] Calculating a distance difference between the first distance and the second distance;

[0072] A second loss is calculated based on the distance difference and a preset distance difference threshold.

[0073] Optional,

[0074] The matching unit comprises:

[0075] A search subunit, configured to perform a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset;

[0076] The processing subunit is configured to determine that a match exists if a search result exists; and to determine that a match does not exist if no search result exists.

[0077] Optionally, the device further comprises:

[0078] A second determining unit is configured to determine the to-be-identified identifier as the to-be-identified identifier if it is determined that there is no match;

[0079] a prompt information generating unit, configured to generate prompt information indicating that the identifier to be identified is an identifier to be identified;

[0080] The display unit is configured to display the prompt information.

[0081] Optionally, the device further comprises:

[0082] An identification name acquisition unit, configured to acquire a target identification name of the identification to be identified;

[0083] A feature vector determination unit, configured to determine a target feature vector corresponding to the target identifier;

[0084] An establishing unit, configured to establish an association relationship between the target identification name and the target feature vector;

[0085] An updating unit is configured to update the preset feature vector library based on the association relationship and the target feature vector.

[0086] Optionally, each of the preset feature vector subsets has corresponding object category information;

[0087] The matching unit comprises:

[0088] an object category information determination subunit, configured to determine target object category information of the image based on content information in the image;

[0089] A feature vector determination subunit is configured to determine a subset of candidate feature vectors having the target object category information in the feature vector library;

[0090] The matching subunit is configured to match the feature vector to be processed with the subset of candidate feature vectors.

[0091] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0092] processor;

[0093] a memory for storing processor-executable instructions;

[0094] The processor is configured to execute the instructions to implement the identification method of the first aspect.

[0095] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a mobile terminal, the electronic device can perform the identification method of the first aspect.

[0096] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, which includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the identification method of the first aspect is implemented.

[0097] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: performing area detection on an image containing a mark to be identified, obtaining a target area containing the mark to be identified, then extracting features from the target area, obtaining a feature vector to be processed for the target area, and then matching the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identifier name one by one; if it is determined that there is a match, the preset identifier name corresponding to the matched preset feature vector subset is used as the identifier name of the mark to be identified. In this way, since the marks in the image are different, or the forms of the same mark are different, the extracted feature vectors are also different, so different feature vectors can be extracted regardless of different marks or different forms of the same mark, and the higher the accuracy of the extracted feature vectors, the higher the recognition rate based on the feature vectors. Based on this feature, this embodiment not only improves the recognition rate of known marks, but also improves the recognition rate of new marks.

[0098] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0100] Figure 1 The figure is a flow chart of a method for identifying a logo according to an exemplary embodiment.

[0101] Figure 2 is a schematic diagram showing the detection effect of a target area according to an exemplary embodiment.

[0102] Figure 3 The figure is a logical diagram of a training identification area detection model according to an exemplary embodiment.

[0103] Figure 4 It is a logical schematic diagram of a training mark recognition model according to an exemplary embodiment.

[0104] Figure 5 The figure is a block diagram of a device for identifying an identity according to an exemplary embodiment.

[0105] Figure 6 It is a block diagram of a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0106] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0107] Figure 1 is a flow chart of a method for identifying a logo according to an exemplary embodiment. Figure 1 As shown, the identification recognition method is used in a terminal, and the terminal may have the following characteristics:

[0108] (1) In terms of hardware system, the device has a central processing unit, memory, input components and output components, that is, the device is often a microcomputer device with communication function. In addition, it can also have multiple input methods, such as keyboard, mouse, touch screen, microphone and camera, and the input can be adjusted as needed. At the same time, the device often has multiple output methods, such as receiver, display screen, etc., which can also be adjusted as needed;

[0109] (2) In terms of software system, the device must have an operating system, such as Windows Mobile, Symbian, Palm, Android, iOS, etc. At the same time, these operating systems are becoming more and more open, and personalized applications developed based on these open operating system platforms are emerging in an endless stream, such as address books, calendars, notepads, calculators, and various games, which greatly meet the needs of personalized users;

[0110] (3) In terms of communication capabilities, the device has flexible access methods and high-bandwidth communication performance, and can automatically adjust the selected communication method according to the selected service and the environment, so as to facilitate user use. The device can support 3GPP (3rd Generation Partnership Project), 4GPP (4th Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access) mobile communications, computer network communications based on TCP / IP (Transmission Control Protocol / Internet Protocol) and UDP (User Datagram Protocol) protocols, and short-range wireless transmission methods based on Bluetooth and infrared transmission standards. It not only supports voice services, but also supports a variety of wireless data services;

[0111] (4) In terms of functional use, the equipment pays more attention to humanization, personalization and multi-functionality. With the development of computer technology, the equipment has moved from a "device-centric" model to a "people-centric" model, integrating embedded computing, control technology, artificial intelligence technology and biometric authentication technology, fully reflecting the people-oriented purpose. Due to the development of software technology, the equipment can adjust the settings according to personal needs and become more personalized. At the same time, the equipment itself integrates a lot of software and hardware, and its functions are becoming more and more powerful.

[0112] The method comprises the following steps.

[0113] In step S11, region detection is performed on the image containing the mark to be recognized to obtain a target region containing the mark to be recognized.

[0114] The logo may be a visual graphic with certain meaning and understandable to people, such as a LOGO, a trademark, etc. In this embodiment, the logo area may be detected for the image containing the logo to be identified to obtain the target area to be identified.

[0115] In this embodiment, performing region detection on an image containing a mark to be identified to obtain a target region containing the mark to be identified includes:

[0116] Using a trained identification region detection model to detect the image, and determining target position information of the identification to be identified in the image;

[0117] The region of the to-be-recognized mark is segmented based on the target position information to obtain a target region including the to-be-recognized mark.

[0118] Specifically, a trained identification area detection model can be deployed in the terminal. When the terminal obtains an image to be detected, the trained identification area detection model can be called to detect the image to detect whether the image contains a logo. If it does, the position information of the logo (i.e., the logo to be identified) in the image is determined and recorded as "target position information", where the position information includes but is not limited to coordinates, length, width, etc.

[0119] After the position information is determined, the region of the mark can be segmented from the image to be detected based on the position information to obtain a region containing the mark, that is, a target region containing the mark to be identified.

[0120] If the image does not contain the logo, then the process can be terminated.

[0121] It should be noted that in step S11, when the logo region detection model is used to detect the logo in the image, it is sufficient to detect whether the logo exists in the image, and it is not necessary to identify the logo. In other words, it is only necessary to detect whether the image contains the logo, and it is not necessary to detect what the logo is specifically. In this way, by detecting the image through the trained logo region detection model, it is possible to accurately determine that the image contains the target region to be identified, thereby ensuring the readiness of subsequent feature extraction of the target region.

[0122] For example, Figure 2 As shown, for the image to be identified shown in the left figure, it is determined through detection that the image contains the mark "A", then segmentation is performed based on the position information of the mark in the image to obtain the target area shown in the right figure.

[0123] The trained identification area detection model is trained in the following manner:

[0124] Inputting the first sample image containing the region annotation into a preset identification region detection model, so that the preset first identification region detection model detects the first sample image, determines the first position information of the target identification in the first sample image, and segments the region of the target identification based on the first position information to obtain a predicted region containing the target identification;

[0125] Calculating a first loss based on the region annotation and the predicted region;

[0126] The preset identification area detection model is trained based on the first loss to obtain a trained identification area detection model.

[0127] Specifically, before training, a network with target detection function such as Faster-RCNN, YOLO, RetinaNet, etc. can be selected as the initial landmark area detection model, recorded as the preset landmark area detection model. At the same time, a sample image containing region annotations is prepared. For example, the target area containing the logo has been marked in the sample image, which is recorded as the first sample image.

[0128] Further, refer to Figure 3 , showing a logical schematic diagram of the training mark area detection model. Specifically, the first sample image can be input into the preset mark area detection model, and the preset mark area detection model will detect the first sample image and determine the position information of the target mark in the first sample image, which is recorded as the first position information, wherein the first position information includes but is not limited to coordinates, length, width, etc. After determining the first position information, the area of ​​the target mark can be segmented based on the first position information to obtain the predicted area containing the target mark.

[0129] Then, a first loss is obtained by using the region annotation and prediction region calculation of the first sample image, and a preset identification region detection model is trained based on the first loss to obtain a trained identification region detection model.

[0130] In the above training process, since there is no need to identify the logo, a logo area detection model with good effect can be trained through a small number of sample images, which ensures the accuracy of the logo area detection model in identifying the target area in the image.

[0131] In step S12, feature extraction is performed on the target area to obtain a feature vector to be processed of the target area.

[0132] In this embodiment, extracting features from the target area to obtain a feature vector to be processed of the target area includes:

[0133] The trained marker recognition model is used to extract features of the target area to obtain a feature vector to be processed of the target area.

[0134] Specifically, after the target area is detected, the trained identification model can be used to extract features of the target area to obtain a feature vector of the target area, which is recorded as a feature vector to be processed. In this way, by using the trained identification model to extract features of the target area, the feature vector of the target area can be accurately extracted, thereby ensuring the accuracy of the subsequent feature vector matching.

[0135] The trained identification model is trained in the following way:

[0136] Inputting the second sample image into a preset identification recognition model so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification;

[0137] Detecting the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier;

[0138] When the prediction result does not match the label, obtaining a second feature vector belonging to the same identifier as the label from the feature vector library, and obtaining a third feature vector belonging to a different identifier from the label;

[0139] Grouping the first eigenvector, the second eigenvector and the third eigenvector into a triple;

[0140] Calculate a second loss based on the triple;

[0141] The preset identification recognition model is trained based on the second loss to obtain a trained identification recognition model.

[0142] Specifically, before training, the spatial transformer layer and SE-Resnet50 can be selected as the initial identification model, which is recorded as the preset identification model. At the same time, a sample image containing a sample identification and a label of the sample identification is prepared, which is recorded as the second sample image. The sample identification can be the identification contained in the sample image, and the label of the sample identification can be the name of the identification. For example, assuming Figure 2 The left image is the second sample image, so the "A" in the image is the sample logo, and the label of the sample logo is "Brand A" (not shown in the figure).

[0143] Further, refer to Figure 4 , showing a logical schematic diagram of the training identification model. Specifically, the second sample image can be input into the preset identification model, and the preset identification model extracts features from the second sample image to obtain a feature vector of the second sample image, which is recorded as a first feature vector. Then, the first feature vector is detected based on a preset feature vector library to obtain a prediction result of the sample identification, and the prediction result is used to indicate the identification name corresponding to the sample identification; wherein, the specific detection method is detailed in step S13.

[0144] When the prediction result does not match the label of the second sample image, a second feature vector with the same identifier as the label is obtained from the feature vector library, and a third feature vector with a different identifier from the label is obtained, and the first feature vector, the second feature vector and the third feature vector are constructed to obtain a triple (x, x + ,x - ), and then the second loss is calculated by triplet calculation. In this way, by constructing triplet to train the identification model, it is possible to obtain a highly accurate trained identification model even when there are few sample images, thereby ensuring the accuracy of the feature vector extracted from the target area.

[0145] In this embodiment, the calculating the second loss based on the triplet includes:

[0146] Substituting the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substituting the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance;

[0147] Calculating a distance difference between the first distance and the second distance;

[0148] A second loss is calculated based on the distance difference and a preset distance difference threshold.

[0149] Specifically, the second loss can be expressed as formula (1):

[0150] L = max{d(x,x + )-D(x,x - )+α,0} Formula (1)

[0151] Among them, x is the first eigenvector, x + is the second eigenvector, x - is the third eigenvector; α is the distance difference threshold, and the distance difference is x and x + The distance between x and x + The distance between A) and x and x - The distance between x and x - The distance between A and B is the distance between A and B); 0 is used to keep L ≥ 0; d(x, y) is the distance metric network, that is, the parameters that need to be learned, where y uses x - 、x + Substitution.

[0152] Then, the preset identification recognition model is trained based on the second loss to obtain a trained identification recognition model.

[0153] Since the triplet contains feature vectors belonging to the same identifier as the label and feature vectors belonging to different identifiers, the use of triples for model training can not only improve the recognition rate of the same identifier in different forms, but also improve the recognition rate between different identifiers, ensuring that the overall recognition rate of the identifier is improved when the number of sample images remains unchanged. Specifically, in practical applications, researchers learned through testing that the recognition rate of the relevant detection scheme is about 82%, while the recognition rate of the scheme of this embodiment is about 89%. Especially when the number of sample images is small, the recognition rate of the scheme of this embodiment is more improved than the recognition rate of the relevant detection scheme. In this way, since the triplet includes not only feature vectors of the same identifier but also feature vectors of different identifiers, the identification recognition model can be trained by triples. Not only can the recognition rate of the same identifier in different forms be improved, but also the recognition rate between different identifiers can be improved, ensuring that the overall recognition rate of the identifier is improved when the number of sample images remains unchanged.

[0154] It should be noted that, in addition to Spatial Transformer Layer and SE-Resnet50, other networks with specific recognition functions are also applicable to this embodiment. In practical applications, they can be adjusted according to actual needs, and this embodiment does not limit this.

[0155] In step S13, the feature vector to be processed is matched with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, and each preset feature vector subset corresponds to a preset identification name one by one.

[0156] After extracting the feature vector to be processed, a preset feature vector library may be used to detect the feature vector to be processed, wherein the feature vector library includes at least two preset feature vector subsets, and each preset feature vector set corresponds to a preset identification name one by one.

[0157] In this embodiment, matching the feature vector to be processed with a preset feature vector library includes:

[0158] Performing a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset;

[0159] If there are search results, it is determined that there is a match;

[0160] If there are no search results, it is determined that there is no match.

[0161] Specifically, a plurality of preset feature vector subsets are stored in the feature vector library, each preset feature vector subset includes a plurality of preset feature vectors, and each preset feature vector subset has a one-to-one corresponding preset identification name. Therefore, the feature vector to be processed can be searched for neighbors with each preset feature vector in each preset feature vector subset. For example, it is detected whether the distance between the feature vector to be processed and each preset feature vector in each preset feature vector subset exceeds a distance threshold. If not, it indicates that there is a search item, thereby determining that there is a match; if the distance between the feature vector to be processed and each preset feature vector in each preset feature vector subset exceeds the distance threshold, it indicates that there is no search item, thereby determining that there is no match. In this way, by searching the feature vector in the target area with each feature vector in the feature vector library, the feature vector in the target area can be quickly searched, thereby improving the search efficiency.

[0162] In this embodiment, each of the preset feature vector subsets has corresponding object category information;

[0163] The matching of the feature vector to be processed with a preset feature vector library includes:

[0164] Determining target object category information of the image based on content information in the image;

[0165] Determining a candidate feature vector having the target object category information in the feature vector library;

[0166] The feature vector to be processed is matched with the candidate feature vector.

[0167] Specifically, each preset feature vector subset in the preset feature vector library has one-to-one corresponding object category information. For example, the object category information corresponding to the preset feature vector subset A is "clothes", and the object category information corresponding to the preset feature vector subset B is "quilt".

[0168] Based on this, when matching the feature vector to be processed with a preset feature vector library, the target object category information of the image can be determined based on the content information in the image, and then the candidate feature vector with the target object category information in the feature vector library is determined, and the feature vector to be processed is matched with each feature vector in the candidate feature vector subset.

[0169] When determining the target object category information of the image based on the content information in the image, the text in the image can be extracted, and then the text is subjected to natural language processing to determine the corresponding target object category information. Of course, in addition to this, determining the target object category information of the image by other methods is also applicable to this embodiment. In actual applications, it can be set according to actual needs, and this embodiment does not limit this.

[0170] In this way, preliminary recognition can be performed by determining the object category information of the image, and then precise recognition can be performed through the feature vector, which not only improves the accuracy of recognition, but also improves the speed of recognition.

[0171] In step S14, if it is determined that there is a match, the preset identification name corresponding to the matched preset feature vector subset is used as the identification name of the identification to be identified.

[0172] Specifically, if it is determined that there is a match, the preset identification name corresponding to the preset feature vector subset containing the search result can be used as the identification name of the identification to be identified.

[0173] In this embodiment, after step S14, the following steps may also be included:

[0174] If it is determined that there is no match, the identifier to be identified is determined as a non-matching identifier;

[0175] Generate prompt information that the identifier to be identified is a non-matching identifier;

[0176] The prompt information is displayed.

[0177] Specifically, if it is determined that there is no match, it means that the identification to be identified cannot be identified, so a prompt message that the identification to be identified is a non-matching identification is generated, and then the prompt message can be displayed. In this way, when the identification fails, the user can be prompted that the identification in the image is a new identification, thereby realizing the identification of the new identification.

[0178] In this embodiment, after displaying the prompt information, the following may also be included:

[0179] Get the target identifier name without matching identifier;

[0180] Determining a target feature vector corresponding to the unmatched identifier;

[0181] Establishing an association relationship between the target identification name and the target feature vector;

[0182] Based on the association relationship and the target feature vector, the preset feature vector library is updated.

[0183] Specifically, after seeing the prompt message, the administrator can enter the identification name of the unmatched identification in the device, recorded as the target identification name, and then the device performs feature extraction on the unmatched identification to obtain the corresponding feature vector, recorded as the target feature vector, and then establishes an association between the target identification name and the target feature vector, and updates the preset feature vector library based on the association and the target feature vector, thereby realizing dynamic update of the preset feature vector library, so that the next time the identification is identified, the corresponding identification result is obtained.

[0184] In this embodiment, an area detection is performed on an image containing a mark to be identified to obtain a target area containing the mark to be identified, and then a feature extraction is performed on the target area to obtain a feature vector to be processed for the target area, and then the feature vector to be processed is matched with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, and each preset feature vector subset corresponds to a preset identifier name one by one; if it is determined that there is a match, the preset identifier name corresponding to the matched preset feature vector subset is used as the identifier name of the mark to be identified. In this way, since the marks in the image are different, or the forms of the same mark are different, the extracted feature vectors are also different. Therefore, different feature vectors can be extracted regardless of different marks or different forms of the same mark. Moreover, the higher the accuracy of the extracted feature vector, the higher the recognition rate based on the feature vector. Based on this feature, this embodiment not only improves the recognition rate of known marks, but also improves the recognition rate of new marks.

[0185] Figure 5 is a block diagram of a device for identifying an identifier according to an exemplary embodiment. Figure 5 The device includes a region detection unit 511, a feature extraction unit 512, a matching unit 513 and a first determination unit 514.

[0186] The region detection unit 511 is configured to perform region detection on the image containing the mark to be recognized, and obtain a target region containing the mark to be recognized.

[0187] The feature extraction unit 512 is configured to extract features from the target area to obtain a feature vector to be processed of the target area.

[0188] The matching unit 513 is configured to match the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identification name one by one;

[0189] The first determining unit 514 is configured to use the preset identification name corresponding to the matching preset feature vector subset as the identification name of the identification to be identified if it is determined that there is a match.

[0190] Optionally, the area detection unit includes:

[0191] A first calling subunit is configured to call a trained identification area detection model to detect the image and determine target position information of the identification to be identified in the image;

[0192] The marking subunit is configured to segment the area of ​​the to-be-recognized mark based on the target position information to obtain a target area containing the to-be-recognized mark.

[0193] Optionally, the device further comprises:

[0194] A first input unit is configured to input a first sample image containing region annotations into a preset identification region detection model, so that the preset identification region detection model detects the first sample image, determines first position information of a target identification in the first sample image, and segments a region of the target identification based on the first position information to obtain a predicted region containing the target identification;

[0195] A first calculation unit is configured to calculate a first loss based on the region annotation and the predicted region;

[0196] The first training unit is configured to train the preset identification area detection model based on the first loss to obtain a trained identification area detection model.

[0197] Optionally, the feature extraction unit is specifically configured as follows:

[0198] The trained identification model is called to extract features of the target area to obtain a feature vector to be processed of the target area.

[0199] Optionally, the device further comprises:

[0200] a second input unit configured to input a second sample image into a preset identification recognition model, so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification;

[0201] A prediction unit is configured to detect the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier;

[0202] an acquisition unit, configured to acquire, when the prediction result does not match the label, a second feature vector belonging to the same identifier as the label from the feature vector library, and acquire a third feature vector belonging to a different identifier from the acquired feature vector;

[0203] a combining unit, configured to combine the first feature vector, the second feature vector and the third feature vector into a triple;

[0204] A second calculation unit is configured to calculate a second loss based on the triple;

[0205] The second training unit is configured to train the preset identification model based on the second loss to obtain a trained identification model.

[0206] Optionally, the second computing unit is specifically configured to:

[0207] Substituting the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substituting the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance;

[0208] Calculating a distance difference between the first distance and the second distance;

[0209] A second loss is calculated based on the distance difference and a preset distance difference threshold.

[0210] Optional,

[0211] The matching unit comprises:

[0212] A search subunit, configured to perform a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset;

[0213] The processing subunit is configured to determine that a match exists if a search result exists; and to determine that a match does not exist if no search result exists.

[0214] Optionally, the device further comprises:

[0215] A second determining unit is configured to determine the to-be-identified identifier as the to-be-identified identifier if it is determined that there is no match;

[0216] a prompt information generating unit, configured to generate prompt information indicating that the identifier to be identified is an identifier to be identified;

[0217] The display unit is configured to display the prompt information.

[0218] Optionally, the device further comprises:

[0219] An identification name acquisition unit, configured to acquire a target identification name of the identification to be identified;

[0220] A feature vector determination unit, configured to determine a target feature vector corresponding to the target identifier;

[0221] An establishing unit, configured to establish an association relationship between the target identification name and the target feature vector;

[0222] An updating unit is configured to update the preset feature vector library based on the association relationship and the target feature vector.

[0223] Optionally, each of the preset feature vector subsets has corresponding object category information;

[0224] The matching unit comprises:

[0225] an object category information determination subunit, configured to determine target object category information of the image based on content information in the image;

[0226] A feature vector determination subunit is configured to determine a subset of candidate feature vectors having the target object category information in the feature vector library;

[0227] The matching subunit is configured to match the feature vector to be processed with the subset of candidate feature vectors.

[0228] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0229] Figure 6 6 is a block diagram of an apparatus 600 for identification according to an exemplary embodiment. For example, the apparatus 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0230] Reference Figure 6 , the device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0231] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0232] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0233] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0234] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0235] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0236] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0237] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0238] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0239] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0240] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0241] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0242] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for identifying a logo, characterized in that: include: Performing region detection on an image containing a mark to be identified, to obtain a target region containing the mark to be identified; Extracting features of the target area to obtain a feature vector to be processed of the target area; Matching the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identification name one by one; If it is determined that there is a match, the preset identification name corresponding to the matched preset feature vector subset is used as the identification name of the identification to be identified; The step of extracting features from the target area to obtain a feature vector to be processed of the target area includes: Using the trained marker recognition model to extract features from the target area to obtain a feature vector to be processed of the target area; The trained identification model is trained in the following way: Inputting the second sample image into a preset identification recognition model so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification; Detecting the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier; When the prediction result does not match the label, obtaining a second feature vector belonging to the same identifier as the label from the feature vector library, and obtaining a third feature vector belonging to a different identifier from the label; Grouping the first eigenvector, the second eigenvector and the third eigenvector into a triple; Substituting the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substituting the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance; Calculating a distance difference between the first distance and the second distance; Calculating a second loss based on the distance difference and a preset distance difference threshold; The preset identification recognition model is trained based on the second loss to obtain a trained identification recognition model.

2. The identification method according to claim 1, characterized in that: The performing of region detection on the image containing the mark to be identified to obtain the target region containing the mark to be identified includes: Using a trained identification region detection model to detect the image, and determining target position information of the identification to be identified in the image; The region of the to-be-recognized mark is segmented based on the target position information to obtain a target region including the to-be-recognized mark.

3. The identification method according to claim 2, characterized in that: The trained landmark region detection model is trained in the following manner: Inputting a first sample image containing region annotations into a preset identification region detection model, so that the preset identification region detection model detects the first sample image, determines first position information of a target identification in the first sample image, and segments a region of the target identification based on the first position information to obtain a predicted region containing the target identification; Calculating a first loss based on the region annotation and the predicted region; The preset identification area detection model is trained based on the first loss to obtain a trained identification area detection model.

4. The identification method according to claim 1, characterized in that: The matching of the feature vector to be processed with a preset feature vector library includes: Performing a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset; If there are search results, it is determined that there is a match; If there are no search results, it is determined that there is no match.

5. The identification method according to claim 1, characterized in that: The method further comprises: If it is determined that there is no match, the identifier to be identified is determined as a non-matching identifier; Generate prompt information that the identifier to be identified is a non-matching identifier; The prompt information is displayed.

6. The identification method according to claim 5, characterized in that: The method further comprises: Get the target identifier name without matching identifier; Determining a target feature vector corresponding to the unmatched identifier; Establishing an association relationship between the target identification name and the target feature vector; Based on the association relationship and the target feature vector, the preset feature vector library is updated.

7. The identification method according to any one of claims 1 to 6, characterized in that: Each of the preset feature vector subsets has corresponding object category information; matching the feature vector to be processed with a preset feature vector library includes: Determining target object category information of the image based on content information in the image; Determining a subset of candidate feature vectors having the target object category information in the feature vector library; The feature vector to be processed is matched with the subset of candidate feature vectors.

8. A marking recognition device, characterized in that: include: A region detection unit is configured to perform region detection on an image containing a mark to be identified, and obtain a target region containing the mark to be identified; A feature extraction unit is configured to extract features from the target area to obtain a feature vector to be processed for the target area; A matching unit is configured to match the feature vector to be processed with a preset feature vector library; the feature vector library includes at least two preset feature vector subsets, each preset feature vector subset corresponds to a preset identification name one by one; A first determining unit is configured to use a preset identification name corresponding to the matched preset feature vector subset as the identification name of the identification to be identified if it is determined that there is a match; Wherein, the feature extraction unit is specifically configured as follows: Calling the trained identification model to extract features of the target area to obtain a feature vector to be processed of the target area; The device also includes: a second input unit configured to input a second sample image into a preset identification recognition model, so that the preset identification recognition model performs feature extraction on the second sample image to obtain a first feature vector of the second sample image; the second sample image includes a sample identification and a label of the sample identification; A prediction unit is configured to detect the first feature vector based on a preset feature vector library to obtain a prediction result of the sample identifier; the prediction result is used to indicate an identifier name corresponding to the sample identifier; an acquisition unit, configured to acquire, when the prediction result does not match the label, a second feature vector belonging to the same identifier as the label from the feature vector library, and acquire a third feature vector belonging to a different identifier from the acquired feature vector; a combining unit, configured to combine the first feature vector, the second feature vector and the third feature vector into a triple; The second calculation unit is configured to substitute the first feature vector and the second feature vector into a preset distance measurement network to calculate a first distance, and substitute the first feature vector and the third feature vector into the preset distance measurement network to calculate a second distance; calculate a distance difference between the first distance and the second distance; and calculate a second loss based on the distance difference and a preset distance difference threshold; The second training unit is configured to train the preset identification model based on the second loss to obtain a trained identification model.

9. The identification device according to claim 8, characterized in that: The area detection unit comprises: A first calling subunit is configured to call a trained identification area detection model to detect the image and determine target position information of the identification to be identified in the image; The marking subunit is configured to segment the area of ​​the to-be-recognized mark based on the target position information to obtain a target area containing the to-be-recognized mark.

10. The identification device according to claim 9, characterized in that: The device also includes: A first input unit is configured to input a first sample image containing region annotations into a preset identification region detection model, so that the preset identification region detection model detects the first sample image, determines first position information of a target identification in the first sample image, and segments a region of the target identification based on the first position information to obtain a predicted region containing the target identification; A first calculation unit is configured to calculate a first loss based on the region annotation and the predicted region; The first training unit is configured to train the preset identification area detection model based on the first loss to obtain a trained identification area detection model.

11. The identification device according to claim 8, characterized in that: The matching unit comprises: A search subunit, configured to perform a nearest neighbor search on the feature vector to be processed and each preset feature vector in each preset feature vector subset; The processing subunit is configured to determine that a match exists if a search result exists; and to determine that a match does not exist if no search result exists.

12. The identification device according to claim 8, characterized in that: The device also includes: A second determining unit is configured to determine the to-be-identified identifier as the to-be-identified identifier if it is determined that there is no match; a prompt information generating unit, configured to generate prompt information indicating that the identifier to be identified is an identifier to be identified; The display unit is configured to display the prompt information.

13. The identification device according to claim 12, characterized in that: The device also includes: An identification name acquisition unit, configured to acquire a target identification name of the identification to be identified; A feature vector determination unit, configured to determine a target feature vector corresponding to the target identifier; An establishing unit, configured to establish an association relationship between the target identification name and the target feature vector; An updating unit is configured to update the preset feature vector library based on the association relationship and the target feature vector.

14. The identification recognition device according to any one of claims 8 to 13, characterized in that: Each of the preset feature vector subsets has corresponding object category information; The matching unit comprises: an object category information determination subunit, configured to determine target object category information of the image based on content information in the image; A feature vector determination subunit is configured to determine a subset of candidate feature vectors having the target object category information in the feature vector library; The matching subunit is configured to match the feature vector to be processed with the subset of candidate feature vectors.

15. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the identification method according to any one of claims 1 to 7. 16 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the identification recognition method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program or computer instructions, characterized in that When the computer program or computer instruction is executed by a processor, the identification method according to any one of claims 1 to 7 is implemented.

Images