Model training method, usage method, related device and storage medium
By obtaining different small sample data sets and the combination of loss functions, the mark feature extraction model is trained, which solves the problem of low accuracy in recognition of uncommon mark type, and realizes high-precision and fast response image recognition effect, while taking into account the model scalability.
Patent Information
- Application Number
- CN202010221116.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-03-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-03-26
AI Technical Summary
In the image recognition of uncommon mark types, the classification recognition accuracy is low and the cost is high, making it difficult to build large-scale data sets, especially the mark types themselves are not common, resulting in a small sample data volume and the inability to effectively train deep learning models.
By obtaining two different small sample data sets, model training is performed on image sample data of uncommon sign types, and a combination of cross entropy loss function and triple loss function is used to generate a mark feature extraction model to improve classification recognition accuracy , and improve the scalability of the model through augmentation processing and detector replacement.
Without increasing the cost of data set, the image recognition accuracy and response speed of unusual logo types are improved, the scalability of the model is enhanced, and the logo type recognition of small sample data sets is suitable for logo type recognition.
Smart Images

Figure CN113449750B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a model training method, a use method, a related device and a storage medium. Background Art
[0002] As a mainstream method in image recognition algorithms, deep learning has excellent classification effects in the case of large-scale data sets. However, in reality, projects that need to use image recognition algorithms generally find it difficult to pay for the construction costs of large-scale data sets, and sometimes the total amount of data itself is very small. For example, some types of signs are not common, and the sample data collected is very small, and it is impossible to collect the amount of data required for training deep learning models, resulting in low classification and recognition accuracy of the sign type and poor results. Summary of the invention
[0003] The embodiments of the present invention provide a model training method, a usage method, a related device and a storage medium, which fully utilize a small sample training set for training without increasing the cost of the data set, thereby achieving higher classification and recognition accuracy and rapid response, while taking into account the scalability of the model.
[0004] On the one hand, the present application provides a landmark feature extraction model training method, the landmark feature extraction model training method comprising:
[0005] Acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type;
[0006] Inputting the first sample data set into a preset network model for training to generate a first landmark feature extraction model;
[0007] Acquire a second sample data set of the mark, the second sample data set and the first sample data set are sample data sets of the same type, all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different;
[0008] A small sample training is performed based on the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0009] In some embodiments of the present application, the first sample data set includes image sample data of multiple sign types, and the multiple sign types include target sign types; the loss function in the first sign feature extraction model is a cross entropy loss function;
[0010] The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises:
[0011] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model;
[0012] The second sample data set is input into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
[0013] In some embodiments of the present application, the acquiring of the second sample data set of the mark includes:
[0014] The sample data set is obtained by filtering the sample data of the target marker type from the first sample data set.
[0015] In some embodiments of the present application, the acquiring of the second sample data set of the mark includes:
[0016] Acquire small sample data of the target marker type;
[0017] The small sample data is augmented to obtain a second sample data set, wherein the second sample data set contains at least 300 target mark type images.
[0018] In some embodiments of the present application, the first sample data set includes image sample data that only includes the target mark type; the loss function in the first mark feature extraction model is a cross entropy loss function;
[0019] The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises:
[0020] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a fifth landmark feature extraction model;
[0021] The second sample data set is input into the fifth landmark feature extraction model for training to generate a second landmark feature extraction model.
[0022] In some embodiments of the present application, the preset network model includes the first detector for detecting the mark area; the first mark feature extraction model includes the second detector for detecting the mark area;
[0023] The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises:
[0024] Replacing the first detector in the preset network model with the second detector in the first marker feature extraction model to obtain a fourth marker feature extraction model;
[0025] The second sample data set is input into the fourth marker feature extraction model for training to generate a second marker feature extraction model.
[0026] On the other hand, the present application also provides a method for using the model, the method further comprising:
[0027] Obtaining a logo image to be recognized;
[0028] The logo image to be recognized is input into a second logo feature extraction model to obtain a first classification feature of the logo image to be recognized.
[0029] In some embodiments of the present application, the method further includes:
[0030] Establishing a plurality of feature libraries, wherein the plurality of feature libraries correspond to a plurality of marker types;
[0031] After acquiring the first classification feature of the logo image to be identified, determining the logo type of the logo image to be identified;
[0032] If the logo type of the logo image to be identified is the first logo type, the first classification feature is stored in a first feature library corresponding to the first logo type.
[0033] In some embodiments of the present application, the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0034] The determining the logo type of the logo image to be identified includes:
[0035] respectively calculating similarity values between the first classification feature and a second classification feature stored in a feature library of each mark type;
[0036] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0037] In another aspect, the present application provides a method for identifying a sign, the method comprising:
[0038] Obtaining a logo image to be recognized;
[0039] Inputting the logo image to be identified into a second logo feature extraction model to obtain classification features of the logo image to be identified;
[0040] After obtaining the classification features of the logo image to be recognized, the logo type of the logo image to be recognized is determined.
[0041] In some embodiments of the present application, multiple sign types correspond to multiple feature libraries, and the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0042] The determining the logo type of the logo image to be identified includes:
[0043] respectively calculating the similarity value between the classification feature of the logo image to be identified and a second classification feature stored in the feature library of each logo type;
[0044] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0045] On the other hand, the present application provides a landmark feature extraction model training device, the landmark feature extraction model training device comprising:
[0046] A first acquisition unit, configured to acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type;
[0047] A generating unit, configured to input the first sample data set into a preset network model for training to generate a first landmark feature extraction model;
[0048] A second acquisition unit is used to acquire a second sample data set of the mark, the second sample data set is a sample data set of the same type as the first sample data set, all data in the second sample data set are image sample data of the target mark type, and the first sample data set is different from the second sample data set;
[0049] A training unit is used to perform small sample training according to the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0050] In some embodiments of the present application, the first sample data set includes image sample data of multiple sign types, and the multiple sign types include target sign types; the loss function in the first sign feature extraction model is a cross entropy loss function;
[0051] The training unit is specifically used for:
[0052] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model;
[0053] The second sample data set is input into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
[0054] In some embodiments of the present application, the second acquisition unit is specifically used to:
[0055] The sample data set is obtained by filtering the sample data of the target marker type from the first sample data set.
[0056] In some embodiments of the present application, the second acquisition unit is specifically used to:
[0057] Acquire small sample data of the target marker type;
[0058] The small sample data is augmented to obtain a second sample data set, wherein the second sample data set contains at least 300 target mark type images.
[0059] In some embodiments of the present application, the first sample data set includes image sample data that only includes the target mark type; the loss function in the first mark feature extraction model is a cross entropy loss function;
[0060] The training unit is specifically used for:
[0061] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a fifth landmark feature extraction model;
[0062] The second sample data set is input into the fifth landmark feature extraction model for training to generate a second landmark feature extraction model.
[0063] In some embodiments of the present application, the preset network model includes the first detector for detecting the mark area; the first mark feature extraction model includes the second detector for detecting the mark area;
[0064] The training unit is specifically used for:
[0065] Replacing the first detector in the preset network model with the second detector in the first marker feature extraction model to obtain a fourth marker feature extraction model;
[0066] The second sample data set is input into the fourth marker feature extraction model for training to generate a second marker feature extraction model.
[0067] On the other hand, the present application also provides a model using device, the device also includes:
[0068] An acquisition unit, used for acquiring a logo image to be recognized, wherein the logo image to be recognized is an image of a target logo type;
[0069] The classification unit is used to input the logo image to be identified into the second logo feature extraction model to obtain the first classification feature of the logo image to be identified.
[0070] In some embodiments of the present application, the device further includes:
[0071] An establishing unit, used for establishing a plurality of feature libraries, wherein the plurality of feature libraries correspond to a plurality of marker types;
[0072] a determination unit, configured to determine the logo type of the logo image to be identified after acquiring the first classification feature of the logo image to be identified;
[0073] A storage unit is configured to store the first classification feature in a first feature library corresponding to the first mark type if the mark type of the mark image to be identified is a first mark type.
[0074] In some embodiments of the present application, the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0075] The determining unit is specifically used for:
[0076] respectively calculating similarity values between the first classification feature and a second classification feature stored in a feature library of each mark type;
[0077] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0078] On the other hand, the present application provides a sign recognition device, the sign recognition device comprising:
[0079] An acquisition unit, used for acquiring a logo image to be recognized;
[0080] a recognition unit, configured to input the logo image to be recognized into a second logo feature extraction model to obtain classification features of the logo image to be recognized;
[0081] The determination unit is used to determine the logo type of the logo image to be identified after obtaining the classification feature of the logo image to be identified.
[0082] In some embodiments of the present application, multiple sign types correspond to multiple feature libraries, and the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0083] The determining unit is specifically used for:
[0084] respectively calculating the similarity value between the classification feature of the logo image to be identified and a second classification feature stored in the feature library of each logo type;
[0085] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0086] On the other hand, the present application also provides a computer device, the computer device comprising:
[0087] one or more processors;
[0088] Memory; and
[0089] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the landmark feature extraction model training method, or to execute the landmark recognition method, or to execute the steps in the model usage method.
[0090] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in the landmark feature extraction model training method described in any one of the first aspects, or to execute the steps in the landmark recognition method, or to execute the steps in the model usage method.
[0091] The embodiment of the present application obtains a first sample data set of the mark, wherein the first sample data set includes image sample data of an uncommon target mark type; inputs the first sample data set into a preset network model for training to generate a first mark feature extraction model; obtains a second sample data set of the mark, wherein the second sample data set is a sample data set of the same type as the first sample data set, and the second sample data set only includes image sample data of the target mark type; performs small sample training based on the second sample data set and the first mark feature extraction model to obtain a second mark feature extraction model. Based on the low classification and recognition accuracy and poor effect of uncommon marks in the prior art, the present application sequentially trains models using two different sample data sets of image sample data of the target mark type of the small sample mark type to generate a mark feature extraction model, and fully utilizes the small sample training set for training without increasing the data set cost, thereby achieving the purpose of higher classification and recognition accuracy and rapid response, while taking into account the scalability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0092] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0093] Figure 1 It is a schematic diagram of a scenario of a landmark feature extraction model training system provided by an embodiment of the present invention;
[0094] Figure 2 It is a schematic diagram of an embodiment of the flow chart of the landmark feature extraction model training method provided in an embodiment of the present invention;
[0095] Figure 3 is a schematic flow chart of an embodiment of step 204 in an embodiment of the present invention;
[0096] Figure 4 is another embodiment flow chart of step 204 in the embodiment of the present invention;
[0097] Figure 5 In the embodiment of the present invention, Figure 1 A flowchart of another embodiment of a method for training a landmark feature extraction model based on the embodiment shown;
[0098] Figure 6 is a schematic diagram of a flow chart of an embodiment of saving classification features provided in an embodiment of the present invention;
[0099] Figure 7is a schematic flow chart of an embodiment of a method for identifying a sign provided in an embodiment of the present invention;
[0100] Figure 8 It is a schematic diagram of the structure of an embodiment of a landmark feature extraction model training device provided in an embodiment of the present invention;
[0101] Fig. 9 is a schematic structural diagram of an embodiment of a sign recognition device provided in an embodiment of the present invention;
[0102] Fig.10 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0103] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0104] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0105] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.
[0106] The embodiments of the present invention provide a model training method, a use method, a related device and a storage medium, which are described in detail below.
[0107] See also Figure 1 , Figure 1 Schematic diagram of a scene of a landmark feature extraction model training system provided by an embodiment of the present invention. The landmark feature extraction model training system may include a computer device 100, in which a landmark feature extraction model training device is integrated, such as Figure 1 Computer equipment in.
[0108] In the embodiment of the present invention, the computer device 100 is mainly used to obtain a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type; the first sample data set is input into a preset network model for training to generate a first mark feature extraction model; a second sample data set of the mark is obtained, wherein the second sample data set and the first sample data set are sample data sets of the same type, and all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different; small sample training is performed according to the second sample data set and the first mark feature extraction model to obtain a second mark feature extraction model.
[0109] Furthermore, the computer device 100 can also be used to obtain a logo image to be identified; input the logo image to be identified into a second logo feature extraction model to obtain classification features of the logo image to be identified; of course, the computer device 100 can also be used to determine the logo type of the logo image to be identified after obtaining the classification features of the logo image to be identified.
[0110] In the embodiment of the present invention, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present invention includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.
[0111] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the logo feature extraction model training system can also include one or more other computer devices, for example, another computer device, which can be used to obtain the logo image to be identified; input the logo image to be identified into the second logo feature extraction model to obtain the classification features of the logo image to be identified; after obtaining the classification features of the logo image to be identified, determine the logo type of the logo image to be identified. Therefore, the specific number of computing devices can be set according to actual conditions and is not limited here.
[0112] In addition, if Figure 1 As shown, the logo feature extraction model training system may also include a memory 200 for storing logo data, such as a sample data set of logos, or logo images to be identified.
[0113] It should be noted that Figure 1 The scenario diagram of the landmark feature extraction model training system shown is only an example. The landmark feature extraction model training system and scenario described in the embodiment of the present invention are intended to more clearly illustrate the technical solution of the embodiment of the present invention, and do not constitute a limitation on the technical solution provided by the embodiment of the present invention. Ordinary technicians in this field can know that with the evolution of the landmark feature extraction model training system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present invention is also applicable to similar technical problems.
[0114] First, an embodiment of the present invention provides a method for training a landmark feature extraction model, wherein the execution subject of the method is a landmark feature extraction model training device, and the landmark feature extraction model training device is applied to a computer device, and the method comprises: obtaining a first sample data set of the landmark, wherein the first sample data set includes image sample data of a target landmark type, and the target landmark type is a small sample landmark type; inputting the first sample data set into a preset network model for training to generate a first landmark feature extraction model; obtaining a second sample data set of the landmark, wherein the second sample data set is a sample data set of the same type as the first sample data set, and all the data in the second sample data set is image sample data of the target landmark type, and the first sample data set is different from the second sample data set; performing small sample training according to the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0115] like Figure 2 FIG. 1 is a flow chart of an embodiment of a method for training a landmark feature extraction model according to an embodiment of the present invention. The method for training a landmark feature extraction model comprises:
[0116] 201. Obtain a first sample data set of a mark.
[0117] A sign is a symbol used by people in daily life to indicate the characteristics of something. It uses simple, obvious, and easily recognizable images, graphics, or text symbols as an intuitive language. In addition to indicating something or replacing something, it also has the functions of expressing meaning, emotion, and instructing actions.
[0118] Among them, the sample data set is a collection of several sample data, and the first sample data set includes image sample data of the target sign type. The image sample data in the first sample data set can be collected by an image acquisition terminal (such as a vehicle-mounted camera device), or directly searched and obtained from an open sign image database, which is not limited here.
[0119] The target flag type described in the present application is a small sample flag type. It should be noted that in the embodiment of the present application, the small sample data refers to a data sample with a small number of samples, generally referring to the original collected data (without augmentation processing or other data processing methods) with less than 50 data samples. Since the amount of sample data of some types is huge, it is not applicable to simply calculate by number. Therefore, in the embodiment of the present application, the small sample data can also be a data sample with a value lower than the preset ratio of the same type of data (the value of the preset ratio is less than or equal to 1 / 10), for example, A flag type data and B flag type data are both the same type of flag data, wherein the amount of data of A flag type data is less than 50, or the amount of data of A flag type data is less than 1 / 10 of the amount of data of B flag type data. In the embodiment of the present application, the target flag type is a flag type with small sample data.
[0120] The small sample sign type represents an uncommon sign type, so the data volume is small. In some embodiments of the present application, the target sign type can also be a pre-set type of sign. For example, a sign that is very rare in daily life can be pre-set as an uncommon target sign type. For example, the signs of this type collected by image acquisition terminals (such as vehicle-mounted cameras) are very rare. For example, warning signs in traffic signs that are only found on the edge of a cliff are rarely collected, so they can be set to a target sign type that meets the small sample sign type.
[0121] Due to the naturally small amount of data in the detection of some traffic signs, such as warning signs that are only available on the edge of a cliff, it is difficult for deep learning algorithms to learn and accurately identify these categories with small sample sizes. Considering that traditional methods such as support vector machines (SVM) are suitable for small sample data sets but the recognition accuracy is not high enough, therefore, if the signs in the embodiments of the present application specifically refer to traffic signs, due to the particularity of traffic signs, there must be uncommon traffic signs. The sign feature extraction model training method in this application is more suitable for the training of traffic sign feature extraction models.
[0122] In the embodiment of the present application, the first sample data set may include image sample data of multiple mark types, or the data in the first sample data set may all be image sample data of the target mark type. The specific setting can be based on actual conditions and is not limited here.
[0123] 202. Input the first sample data set into a preset network model for training to generate a first landmark feature extraction model.
[0124] Among them, the preset network model can be a convolutional neural network model (Convolutional Neural Network, CNN), specifically, for example, an EfficientNet network model.
[0125] 203. Obtain a second sample data set of the mark.
[0126] The second sample data set and the first sample data set are sample data sets of the same type, all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different.
[0127] It should be noted that, in the embodiment of the present application, the type refers to the sign type, and the sign type refers to the specific sign type under the sign type. Specifically, the second sample data set and the first sample data set are sample data sets of the same type, that is, the second sample data set and the first sample data set belong to sample data of the same type of signs, for example, both belong to sample data of traffic signs, or both belong to sample data of logistics signs, or both belong to sample data of bank signs.
[0128] The first sample data set and the second sample data set are different, which means that although the first sample data set and the second sample data set are sample data sets with the same type of mark, the sample data in the sample data sets are different. In the embodiment of the present application, the first sample data set and the second sample data set are different, which can exist in two ways:
[0129] (1) The first sample data set and the second sample data set include sample data of the same type, but the sample data in the first sample data set and the second sample data set are different.
[0130] For example, all the data in the first sample data set are image sample data of the target mark type, and all the data in the second sample data set may also be image sample data of the target mark type. However, compared with the image sample data of the target mark type in the first sample data set and the image sample data of the target mark type in the second sample data set, the amount of image sample data in the first sample data set and the second sample data set may be different, or / and there may be different image samples in the image sample data in the first sample data set and the second sample data set.
[0131] (2) The first sample data set and the second sample data set include sample data of different types
[0132] For example, the first sample data set includes image sample data of multiple mark types, including a target mark type, and the data in the second sample data set are all image sample data of the target mark type.
[0133] In an embodiment of the present invention, the image sample data of the target mark type included in the second sample data set may be the same as or different from the image sample data of the target mark type included in the first sample data set. Since the target mark type itself is uncommon and the amount of sample data itself is small, it is obviously not practical for all of them to be different. They may be partially different (for example, the amount of image sample data of the target mark type in one of the data sets is smaller), and the specific details are not limited here.
[0134] As described above, since the target mark type itself is uncommon and the amount of sample data itself is small, the image data of the target mark type in the first sample data set and the second sample data set can be obtained by performing augmentation processing on a small amount of image data of the target mark type to obtain more image sample data of the target mark type. Specifically, one or more augmentation methods including translation, scaling, flipping, rotation, and cropping can be used to obtain more sample data.
[0135] In the embodiment of the present application, there are many ways to obtain the second sample data set of the mark, for example:
[0136] (1) Filtered from the first sample data set.
[0137] Specifically, the acquiring of the second sample data set with the mark includes: filtering sample data of a target mark type from the first sample data set to obtain the sample data set.
[0138] (2) Augmentation processing to obtain
[0139] Specifically, the method of obtaining the second sample data set of the mark includes: obtaining small sample data of the target mark type; augmenting the small sample data to obtain a second sample data set, wherein at least 300 images of the target mark type exist in the second sample data set.
[0140] 204. Perform small sample training according to the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0141] Among them, small sample training refers to model training for small sample marker types. In the embodiment of the present application, the second marker feature extraction model obtained by small sample training is a model for targeted feature extraction of uncommon target marker types, which can effectively improve the accuracy and efficiency of feature extraction of uncommon target marker types.
[0142] Based on the low accuracy and poor effect of the prior art in classifying and identifying uncommon signs, the present application uses two different sample data sets including image sample data of uncommon target sign types to perform model training in sequence to generate a sign feature extraction model. Without increasing the cost of the data set, the application makes full use of the small sample training set for training to achieve the purpose of higher classification and recognition accuracy and rapid response. Due to the second sign feature extraction model, other types of sign sample data can also be trained to obtain feature extraction models of different sign types, thereby taking into account the scalability of the model.
[0143] In the embodiment of the present application, there are multiple ways to perform small sample training according to the second sample data set and the first signature feature extraction model in step 204 to obtain the second signature feature extraction model, which are applicable to multiple situations, specifically the following ways (1), (2), and (3):
[0144] (1) The first landmark feature extraction model is a universal landmark feature extraction model. The universal landmark feature extraction model is first trained to obtain the universal landmark feature extraction model, and then the second landmark feature extraction model for the target landmark type is trained to obtain the second landmark feature extraction model.
[0145] At this time, the preset network model is a CNN model, the first sample data set includes image sample data of multiple sign types, and the multiple sign types include a target sign type; the loss function in the first sign feature extraction model is a cross entropy loss function;
[0146] Therefore, the image sample data of various logo types in the first sample data set constitutes a universal image sample data set of various logo types. In step 202, the first sample data set is input into a preset network model for training. The generated first logo feature extraction model is a universal logo feature extraction model, which can recognize the various logo types. However, since the target logo type is uncommon, the feature extraction accuracy of the first logo feature extraction model for the target logo type is not high.
[0147] In this embodiment, Figure 3 As shown, the step 204 described above performs small sample training according to the second sample data set and the first marker feature extraction model to obtain the second marker feature extraction model, which may further include:
[0148] 301. Adjust the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model.
[0149] Since the preset network model is a CNN model, such as an EfficientNet network model, in which the loss function is a cross entropy loss function (cross entropy loss), the loss function in the first landmark feature extraction model trained in step 202 is also a cross entropy loss function.
[0150] In this step, the loss function in the first landmark feature extraction model is adjusted from the cross entropy loss function to the triplet loss function.
[0151] Furthermore, in the embodiment of the present application, the triplet loss is improved by introducing smoothness augmentation (image augmentation by overlapping images of different categories). In a general CNN network, the triplet loss is determined by randomly selecting two images to compare with the current image and calculating the corresponding feature distance.
[0152] In this application, by introducing a weight vector representing the overlap ratio of two images into triplet loss, the classification accuracy of the trained model on the test sample set is improved by 2%. The loss value calculated by the improved triplet loss is a set of two types of loss values. For example, the calculation method of the loss value calculated by the improved triplet loss is as follows: take two traffic sign pictures, one is a speed limit of 40 and the other is no honking, randomly select a number between 0 and 1, assuming 0.9, multiply the picture of the speed limit of 40 by 0.9, and multiply the other by 0.1, and add the two numbers after multiplication to get an "overlapping picture". At this time, the picture label is neither a speed limit of 40 nor a prohibition of honking, but 0.9 speed limit 40 + 0.1 no honking.
[0153] 302. Input the second sample data set into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
[0154] During the generation process of the second landmark feature extraction model, two different sample data and two types of loss functions are used for training, that is, the general landmark feature extraction model is first trained, and then the loss function in the general landmark feature extraction model is improved to obtain the third landmark feature extraction model. Finally, the third landmark feature extraction model is trained through the second sample data set (all image sample data of the target landmark type) to obtain the second landmark feature extraction model. The second landmark feature extraction model has higher feature extraction accuracy and extraction efficiency for the target landmark type image.
[0155] (2) Both the first landmark feature extraction model and the second landmark feature extraction model are trained using image sample data of uncommon target landmark types.
[0156] At this time, all the data in the first sample data set are image sample data of the target mark type; the loss function in the first mark feature extraction model is a cross entropy loss function. The second mark feature extraction model is obtained by performing small sample training based on the second sample data set and the first mark feature extraction model, including: adjusting the loss function in the first mark feature extraction model to a triplet loss function to obtain a fifth mark feature extraction model; inputting the second sample data set into the fifth mark feature extraction model for training to generate a second mark feature extraction model.
[0157] In this embodiment, it is equivalent to first inputting a first sample data set of image sample data of the target mark type into a preset network model whose loss function is a cross entropy loss function, thereby obtaining a first mark feature extraction model whose loss function is a cross entropy loss function; and then inputting a second sample data set of image sample data of the target mark type into a fifth mark feature extraction model whose loss function is a triplet loss function, thereby generating a second mark feature extraction model.
[0158] In this embodiment, sample data of the target mark type is input into models with different loss functions for training twice to improve the feature extraction accuracy and efficiency of the second mark feature extraction model for uncommon target mark types.
[0159] (3) The preset network model includes a first detector for detecting the mark area, and the first mark feature extraction model includes a second detector for detecting the mark area.
[0160] In view of the fact that some types of signs (such as traffic signs) need to be detected before being identified, the sign area is determined, and then features are extracted within the sign area, the preset network model may include a first detector for detecting the sign area. The first sign feature extraction model obtained after training the preset network model includes a second detector for detecting the sign area. Since the trained model has better adaptive detection capabilities for detecting the sign area than the pre-trained model, it can better detect the sign area. Therefore, the second detector is different from the first detector. At the same time, since the detector and the feature extraction part in the model in this embodiment are two relatively independent parts, they can be replaced separately.
[0161] At this time, if Figure 4 As shown, performing small sample training according to the second sample data set and the first marker feature extraction model to obtain the second marker feature extraction model may further include:
[0162] 401. Replace the first detector in the preset network model with the second detector in the first landmark feature extraction model to obtain a fourth landmark feature extraction model.
[0163] 402. Input the second sample data set into the fourth landmark feature extraction model for training to generate a second landmark feature extraction model.
[0164] For the first marker feature extraction model trained in either method (1) or method (2), the second detector in the first marker feature extraction model obtained after training replaces the first detector in the preset network model to obtain a fourth marker feature extraction model. The fourth marker feature extraction model is then trained using image sample data that are all of the target marker type to obtain image sample data for uncommon target marker types. This can also improve the accuracy and efficiency of feature extraction of the second marker feature extraction model for uncommon target marker types.
[0165] After a small sample-based second signature feature extraction model is constructed through pre-training, the second signature feature extraction model is used to extract features on the original small sample training set. Specifically, Figure 5 As shown, in some embodiments of the present application, the landmark feature extraction model training method can further provide a model use method, which is applied to a model use device, and the method includes:
[0166] 501. Obtain a logo image to be recognized.
[0167] The logo image to be identified may be a logo image collected in real time by an image acquisition terminal. In addition, the logo image to be identified may be an image of a target logo type.
[0168] 502. Input the logo image to be recognized into the second logo feature extraction model to obtain a first classification feature of the logo image to be recognized.
[0169] like Figure 6 As shown, in some embodiments of the present application, the landmark feature extraction model training method may further include:
[0170] 601. Establish multiple feature libraries, where the multiple feature libraries correspond to multiple marker types.
[0171] Specifically, multiple feature libraries may correspond to multiple marker types one by one. For example, the first feature library corresponds to the first marker type, and the first feature library may be used to store classification features corresponding to the first marker type.
[0172] 602. After acquiring the first classification feature of the logo image to be identified, determine the logo type of the logo image to be identified.
[0173] 603. If the logo type of the logo image to be identified is a first logo type, save the first classification feature in a first feature library corresponding to the first logo type.
[0174] In some embodiments, the extracted classification features can be stored in an external folder in the form of hdf5 according to different categories for later use.
[0175] Theoretically, when the landmark feature extraction model in the above embodiment is well trained, the classification feature distance of each category of pictures in the feature library should be clearly separated. However, if the number of training is too small, there may be a situation where the classification feature distance of some pictures cannot be well distinguished. In order to make the classification features of each category of logos extracted accurate enough, in the embodiment of the present application, the vectors of different pictures in the same category can be dot-multiplied in pairs, and the distance between classes can be judged by the dot product value and a first preset threshold can be set to select objects below the first preset threshold, so as to find different classification features representing the same category to the greatest extent. Then these objects will be dot-multiplied in pairs with the classification features of the pictures of different categories of logos, and the features of all targets below this threshold can be selected and put into the feature library based on the first preset threshold. Through such a screening strategy, it is finally guaranteed that the features in the feature library are representative within the class, and at the same time, the distance between classes is far enough to ensure the classification accuracy.
[0176] Specifically, the feature library corresponding to each logo type may include at least one second classification feature corresponding to the classified logo image; at this time, determining the logo type of the logo image to be identified includes: respectively calculating the similarity value between the first classification feature and a second classification feature stored in the feature library of each logo type; if the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0177] The first preset threshold and the second preset threshold can be set according to actual conditions. For example, the first preset threshold is 0.6 and the second preset threshold is 0.4, which are not specifically limited here.
[0178] It should be noted that the at least one second classification feature corresponding to the classified sign image included in the feature library corresponding to each sign type can be the at least one second classification feature corresponding to the classified sign image stored in the feature library corresponding to the sign type. Specifically, the method includes: obtaining an image of each of the multiple sign types (the sign type of each image is known), performing feature extraction on the image of each of the multiple sign types through a second sign feature extraction model to obtain a second classification feature of the multiple sign types, and saving the feature library corresponding to each sign type.
[0179] Based on the landmark feature extraction model training method described in the above embodiment, a second landmark feature extraction model can be obtained, so landmark recognition can be performed based on the second landmark feature extraction model, such as Figure 7 As shown, an embodiment of the present application also provides a method for identifying a sign, the method comprising:
[0180] 701. Obtain a logo image to be recognized.
[0181] The logo image to be identified may be an image of a target logo type.
[0182] 702. Input the logo image to be identified into a second logo feature extraction model to obtain classification features of the logo image to be identified.
[0183] 703. After obtaining the classification features of the logo image to be identified, determine the logo type of the logo image to be identified.
[0184] Based on the low accuracy and poor effect of the prior art in classifying and recognizing uncommon signs, the present application uses two different sample data sets including image sample data of uncommon target sign types, sequentially performs model training, generates a second sign feature extraction model, and performs sign recognition through the second sign feature extraction model, thereby improving the accuracy and efficiency of sign recognition.
[0185] In some embodiments of the present application, multiple sign types correspond to multiple feature libraries, and the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0186] The determining the logo type of the logo image to be identified may further include:
[0187] respectively calculating the similarity value between the classification feature of the logo image to be identified and a second classification feature stored in the feature library of each logo type;
[0188] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0189] In order to better implement the landmark feature extraction model training method in the embodiment of the present invention, based on the landmark feature extraction model training method, the embodiment of the present invention also provides a landmark feature extraction model training device, such as Figure 8 As shown, the landmark feature extraction model training device 800 includes:
[0190] A first acquisition unit 801 is used to acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type;
[0191] A generating unit 802 is used to input the first sample data set into a preset network model for training to generate a first landmark feature extraction model;
[0192] A second acquisition unit 803 is used to acquire a second sample data set of the mark, the second sample data set is a sample data set of the same type as the first sample data set, all data in the second sample data set are image sample data of the target mark type, and the first sample data set is different from the second sample data set;
[0193] The training unit 804 is used to perform small sample training according to the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0194] Based on the low accuracy and poor effect of the existing technology in classifying and recognizing uncommon signs, this application uses two different sample data sets including image sample data of uncommon target sign types to perform model training in sequence to generate a sign feature extraction model. Without increasing the cost of the data set, the application fully utilizes the small sample training set for training to achieve the purpose of higher classification and recognition accuracy and rapid response, while taking into account the scalability of the model.
[0195] In some embodiments of the present application, the first sample data set includes image sample data of multiple sign types, and the multiple sign types include target sign types; the loss function in the first sign feature extraction model is a cross entropy loss function;
[0196] The training unit 804 is specifically used for:
[0197] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model;
[0198] The second sample data set is input into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
[0199] In some embodiments of the present application, the second acquiring unit 803 is specifically used to:
[0200] The sample data set is obtained by filtering the sample data of the target marker type from the first sample data set.
[0201] In some embodiments of the present application, the second acquiring unit 803 is specifically used to:
[0202] Acquire small sample data of the target marker type;
[0203] The small sample data is augmented to obtain a second sample data set, wherein the second sample data set contains at least 300 target mark type images.
[0204] In some embodiments of the present application, all the data in the first sample data set are image sample data of the target mark type; the loss function in the first mark feature extraction model is a cross entropy loss function;
[0205] The training unit 804 is specifically used for:
[0206] Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a fifth landmark feature extraction model;
[0207] The second sample data set is input into the fifth landmark feature extraction model for training to generate a second landmark feature extraction model.
[0208] In some embodiments of the present application, the preset network model includes the first detector for detecting the mark area; the first mark feature extraction model includes the second detector for detecting the mark area;
[0209] The training unit 804 is specifically used for:
[0210] Replacing the first detector in the preset network model with the second detector in the first marker feature extraction model to obtain a fourth marker feature extraction model;
[0211] The second sample data set is input into the fourth marker feature extraction model for training to generate a second marker feature extraction model.
[0212] On the other hand, the present application also provides a model using device, the device comprising:
[0213] An acquisition unit, used for acquiring a logo image to be recognized, wherein the logo image to be recognized is an image of a target logo type;
[0214] The classification unit is used to input the logo image to be identified into the second logo feature extraction model to obtain the first classification feature of the logo image to be identified.
[0215] In some embodiments of the present application, the device further includes:
[0216] An establishing unit, used for establishing a plurality of feature libraries, wherein the plurality of feature libraries correspond to a plurality of marker types;
[0217] a determination unit, configured to determine the logo type of the logo image to be identified after acquiring the first classification feature of the logo image to be identified;
[0218] A storage unit is configured to store the first classification feature in a first feature library corresponding to the first mark type if the mark type of the mark image to be identified is a first mark type.
[0219] In some embodiments of the present application, the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0220] The determining unit is specifically used for:
[0221] respectively calculating similarity values between the first classification feature and a second classification feature stored in a feature library of each mark type;
[0222] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0223] In order to better implement the mark recognition method in the embodiment of the present invention, based on the mark recognition method, the embodiment of the present invention also provides a mark recognition device, such as Fig. 9 As shown, the sign recognition device 900 includes:
[0224] An acquisition unit 901 is used to acquire a logo image to be recognized;
[0225] The recognition unit 902 is used to input the logo image to be recognized into a second logo feature extraction model to obtain classification features of the logo image to be recognized;
[0226] The determination unit 903 is used to determine the logo type of the logo image to be identified after obtaining the classification features of the logo image to be identified.
[0227] Based on the low accuracy and poor effect of the prior art in classifying and recognizing uncommon signs, the present application uses two different sample data sets including image sample data of uncommon target sign types, sequentially performs model training, generates a second sign feature extraction model, and performs sign recognition through the second sign feature extraction model, thereby improving the accuracy and efficiency of sign recognition.
[0228] In some embodiments of the present application, multiple sign types correspond to multiple feature libraries, and the feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image;
[0229] The determining unit is specifically used for:
[0230] respectively calculating the similarity value between the classification feature of the logo image to be identified and a second classification feature stored in the feature library of each logo type;
[0231] If the first similarity value is lower than a first preset threshold, and the second similarity value is lower than a second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
[0232] The embodiment of the present invention further provides a computer device, which integrates any one of the landmark feature extraction model training devices provided in the embodiment of the present invention, and the computer device includes:
[0233] one or more processors;
[0234] Memory; and
[0235] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor in the steps of the landmark feature extraction model training method described in any of the above landmark feature extraction model training method embodiments.
[0236] The embodiment of the present invention also provides another computer device, which integrates any one of the sign recognition devices provided by the embodiment of the present invention, and the computer device includes:
[0237] one or more processors;
[0238] Memory; and
[0239] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to perform the steps of the landmark recognition method described in any of the above-mentioned model usage method or landmark recognition method embodiments.
[0240] The embodiment of the present invention further provides a computer device, which integrates any one of the landmark feature extraction model training devices or landmark recognition devices provided by the embodiment of the present invention. Fig.10 As shown, it shows a schematic diagram of the structure of a computer device involved in an embodiment of the present invention, specifically:
[0241] The computer device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Fig.10 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:
[0242] The processor 1001 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.
[0243] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.
[0244] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.
[0245] The computer device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0246] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 1002 according to the following instructions, and the processor 1001 will run the application programs stored in the memory 1002, thereby realizing various functions, as follows:
[0247] Acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type;
[0248] Inputting the first sample data set into a preset network model for training to generate a first landmark feature extraction model;
[0249] Acquire a second sample data set of the mark, the second sample data set and the first sample data set are sample data sets of the same type, all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different;
[0250] A small sample training is performed based on the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0251] Alternatively, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 1002 according to the following instructions, and the processor 1001 will run the application programs stored in the memory 1002, thereby realizing various functions, as follows:
[0252] Obtaining a logo image to be recognized;
[0253] Inputting the logo image to be identified into a second logo feature extraction model to obtain classification features of the logo image to be identified;
[0254] After obtaining the classification features of the logo image to be recognized, the logo type of the logo image to be recognized is determined.
[0255] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0256] To this end, an embodiment of the present invention provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the landmark feature extraction model training methods provided in the embodiment of the present invention. For example, the computer program can be loaded by a processor to execute the following steps:
[0257] Acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type;
[0258] Inputting the first sample data set into a preset network model for training to generate a first landmark feature extraction model;
[0259] Acquire a second sample data set of the mark, the second sample data set and the first sample data set are sample data sets of the same type, all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different;
[0260] A small sample training is performed based on the second sample data set and the first landmark feature extraction model to obtain a second landmark feature extraction model.
[0261] Alternatively, the computer program may be loaded by a processor and may execute the following steps:
[0262] Obtaining a logo image to be recognized;
[0263] The logo image to be recognized is input into a second logo feature extraction model to obtain classification features of the logo image to be recognized.
[0264] Alternatively, the computer program may be loaded by a processor and may execute the following steps:
[0265] Obtaining a logo image to be recognized;
[0266] Inputting the logo image to be identified into a second logo feature extraction model to obtain classification features of the logo image to be identified;
[0267] After obtaining the classification features of the logo image to be recognized, the logo type of the logo image to be recognized is determined.
[0268] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.
[0269] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.
[0270] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0271] The above is a detailed introduction to a model training method, a method of use, related devices and a storage medium provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A landmark feature extraction model training method, characterized in that: The landmark feature extraction model training method comprises: Acquire a first sample data set of a mark, wherein the first sample data set includes image sample data of a target mark type, and the target mark type is a small sample mark type; the first sample data set includes image sample data of multiple mark types, and the multiple mark types include the target mark type; Inputting the first sample data set into a preset network model for training to generate a first marker feature extraction model; the loss function in the first marker feature extraction model is a cross entropy loss function; Acquire a second sample data set of the mark, the second sample data set and the first sample data set are sample data sets of the same type, all data in the second sample data set are image sample data of the target mark type, and the first sample data set and the second sample data set are different; Performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model; The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises: Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model; The second sample data set is input into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
2. The landmark feature extraction model training method according to claim 1, characterized in that: The second sample data set of the acquisition mark includes: Filtering sample data of a target marker type from the first sample data set to obtain the sample data set; Alternatively, small sample data of the target mark type is obtained; and augmentation processing is performed on the small sample data to obtain a second sample data set, wherein the second sample data set contains at least 300 images of the target mark type.
3. The landmark feature extraction model training method according to claim 1, characterized in that: The preset network model includes a first detector for detecting a mark area; the first mark feature extraction model includes a second detector for detecting a mark area; The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises: Replacing the first detector in the preset network model with the second detector in the first marker feature extraction model to obtain a fourth marker feature extraction model; The second sample data set is input into the fourth marker feature extraction model for training to generate a second marker feature extraction model.
4. A method for using a model, characterized in that: The method further comprises: Obtaining a logo image to be recognized; Inputting the logo image to be identified into a second logo feature extraction model to obtain a first classification feature of the logo image to be identified; Establishing a plurality of feature libraries, wherein the plurality of feature libraries correspond to a plurality of marker types; After acquiring the first classification feature of the logo image to be identified, determining the logo type of the logo image to be identified; If the logo type of the logo image to be identified is the first logo type, the first classification feature is stored in a first feature library corresponding to the first logo type.
5. The method for using the model according to claim 4, characterized in that: The feature library corresponding to each sign type includes at least one second classification feature corresponding to the classified sign image; The determining the logo type of the logo image to be identified includes: respectively calculating similarity values between the first classification feature and a second classification feature stored in a feature library of each mark type; If the first similarity value is lower than the first preset threshold, and the second similarity value is lower than the second preset threshold, it is determined that the logo type of the logo image to be identified is the first logo type, the first similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the first feature library, and the second similarity value is the similarity value between the classification feature of the logo image to be identified and a second classification feature in the target feature library.
6. A landmark feature extraction model training device, characterized in that: The landmark feature extraction model training device comprises: A first acquisition unit is used to acquire a first sample data set of marks, wherein the first sample data set includes image sample data of an uncommon target mark type; the first sample data set includes image sample data of multiple mark types, wherein the multiple mark types include a target mark type; A generating unit, configured to input the first sample data set into a preset network model for training, and generate a first marker feature extraction model; the loss function in the first marker feature extraction model is a cross entropy loss function; A second acquisition unit is used to acquire a second sample data set of the mark, where the second sample data set is a sample data set of the same type as the first sample data set, and the second sample data set only includes image sample data of the target mark type; A training unit, configured to perform small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model; The step of performing small sample training according to the second sample data set and the first marker feature extraction model to obtain a second marker feature extraction model comprises: Adjusting the loss function in the first landmark feature extraction model to a triplet loss function to obtain a third landmark feature extraction model; The second sample data set is input into the third landmark feature extraction model for training to generate a second landmark feature extraction model.
7. A model using device, characterized in that: The model using device comprises: An acquisition unit, used for acquiring a logo image to be recognized; a recognition unit, configured to input the logo image to be recognized into a second logo feature extraction model to obtain classification features of the logo image to be recognized; Establishing a plurality of feature libraries, wherein the plurality of feature libraries correspond to a plurality of marker types; After acquiring the first classification feature of the logo image to be identified, determining the logo type of the logo image to be identified; If the logo type of the logo image to be identified is the first logo type, the first classification feature is stored in a first feature library corresponding to the first logo type.
8. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the landmark feature extraction model training method described in any one of claims 1 to 3, or to implement the model usage method described in any one of claims 4 to 5.
9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the landmark feature extraction model training method described in any one of claims 1 to 3, or to implement the steps in the model use method described in any one of claims 4 to 5.
