Image recognition method, device, electronic device and storage medium

Abstract

The disclosed embodiments provide an image recognition method, device, electronic device and storage medium. The method includes: obtaining an image to be recognized; and processing the image to be recognized using a target identification recognition model to obtain a recognition result for characterizing whether the image to be recognized includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each sample image group includes a positive sample and a negative sample, the positive sample in each sample image group is an image obtained by setting a first identification on the original image, and the negative sample in each sample image group is the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification have an associated relationship.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more specifically, to an image recognition method, device, electronic device, and storage medium. Background Art

[0002] In the related art, some objects are usually provided with logos, wherein the logos are used to convey information or attract attention, and the logos can be changed according to actual conditions.

[0003] For example, the object is a commodity, and the commodity can be a commodity sold on an online shopping platform. In order to increase sales, online shopping platforms or merchants stationed on the online platform will carry out multiple rounds of promotional activities, such as New Year's Day promotions, May Day promotions, National Day promotions, Double Eleven promotions, and Double Twelve promotions. In order to enable users to identify the commodities participating in the promotional activities, promotional activity logos will be set on the commodities participating in the promotional activities. Usually, the promotional activity logos of different promotional activities are different. In this process, merchants may illegally mark for the purpose of promotion, or, after the promotional activities are over, the normal operation of the online shopping platform may be affected due to the failure to promptly remove the logos. Whether it is during the promotional activities, it is necessary to identify the promotional commodities for supervision, or after the promotional activities are over, it is necessary to remove the promotional logos so as not to affect the normal operation of the online shopping platform, the logos need to be identified.

[0004] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the cost of using the related art to identify the logo in the image is high and the recognition efficiency is low. Summary of the invention

[0005] In view of this, embodiments of the present disclosure provide an image recognition method, device, electronic device, and storage medium.

[0006] One aspect of an embodiment of the present disclosure provides an image recognition method, the method comprising: acquiring an image to be recognized; and processing the image to be recognized using a target identification recognition model to obtain a recognition result for characterizing whether the image to be recognized includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples from multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each of the sample image groups comprises a positive sample and a negative sample, the positive sample in each of the sample image groups is an image obtained by setting a first identification on the original image, and the negative sample in each of the sample image groups is the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification have an associated relationship.

[0007] According to an embodiment of the present disclosure, each of the above sample image groups has a corresponding original image;

[0008] The original image corresponding to each of the above sample image groups is obtained as follows:

[0009] At least one original image corresponding to each of the above categories in multiple categories is obtained, wherein the number of original images under different categories meets a preset condition.

[0010] According to an embodiment of the present disclosure, the above-mentioned first identifier and the above-mentioned second identifier have an associated relationship, including: the similarity between the above-mentioned first identifier and the above-mentioned second identifier is greater than or equal to a similarity threshold and the above-mentioned first identifier and the above-mentioned second identifier are different identifiers; or, the above-mentioned second identifier is an identifier obtained after covering part of the above-mentioned first identifier.

[0011] According to an embodiment of the present disclosure, the above-mentioned target identification recognition model is obtained by alternately inputting positive samples and negative samples from multiple sample image groups into the convolutional neural network model and training the above-mentioned convolutional neural network model, including: obtaining multiple sample image groups, wherein each of the above-mentioned sample image groups includes a positive sample and a negative sample, and the above-mentioned positive samples and the above-mentioned negative samples in each of the above-mentioned sample image groups have an associated relationship; obtaining the true annotation information of each of the above-mentioned positive samples and the true annotation information of each of the above-mentioned negative samples; alternately inputting the above-mentioned positive samples and the above-mentioned negative samples from the multiple above-mentioned sample image groups into the above-mentioned convolutional neural network model, and outputting the predicted annotation information of each of the above-mentioned positive samples and the predicted annotation information of each of the above-mentioned negative samples respectively; and, training the above-mentioned convolutional neural network model according to the true annotation information of the multiple above-mentioned positive samples, the true annotation information of the multiple above-mentioned negative samples, the predicted annotation information of the multiple above-mentioned positive samples and the predicted annotation information of the multiple above-mentioned negative samples, to obtain the above-mentioned target identification recognition model.

[0012] According to an embodiment of the present disclosure, the above-mentioned obtaining of the true labeling information of each of the above-mentioned positive samples and the true labeling information of each of the above-mentioned negative samples includes: obtaining the initial labeling information of each of the above-mentioned positive samples and the initial labeling information of each of the above-mentioned negative samples; and performing label smoothing processing on the initial labeling information of the above-mentioned positive samples and the initial labeling information of the above-mentioned negative samples in each of the above-mentioned sample image groups to obtain the true labeling information of the above-mentioned positive samples and the true labeling information of the above-mentioned negative samples in each of the above-mentioned sample image groups.

[0013] According to an embodiment of the present disclosure, the above-mentioned convolutional neural network model is trained according to the real labeling information of the plurality of positive samples, the real labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples and the predicted labeling information of the plurality of negative samples to obtain the above-mentioned target identification recognition model, including: determining the wrong labeling information according to the real labeling information of the plurality of positive samples, the real labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples and the predicted labeling information of the plurality of negative samples; determining the sample image corresponding to the above-mentioned wrong labeling information; determining the new negative sample and the new positive sample corresponding to the new negative sample according to the sample image corresponding to the above-mentioned wrong labeling information; obtaining the real labeling information of the new positive sample and the real labeling information of the new negative sample; alternately inputting the new positive sample and the new negative sample into the above-mentioned convolutional neural network model, and outputting the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample respectively; and training the above-mentioned convolutional neural network model according to the real labeling information of the new positive sample, the real labeling information of the new negative sample, the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample to obtain the above-mentioned target identification recognition model.

[0014] According to an embodiment of the present disclosure, the above-mentioned sample image group includes images obtained after at least one of translation, rotation, cropping, non-rigid transformation, noise disturbance and color transformation is processed.

[0015] Another aspect of the disclosed embodiments provides an image recognition device, which includes: an acquisition module for acquiring an image to be recognized; and a processing module for processing the image to be recognized using a target identification recognition model to obtain a recognition result for characterizing whether the image to be recognized includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples from multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each of the sample image groups includes a positive sample and a negative sample, and the positive sample in each of the sample image groups is an image obtained by setting a first identification on the original image, and the negative sample in each of the sample image groups is the original image or an image obtained by setting a second identification on the original image, and wherein the first identification and the second identification have an associated relationship.

[0016] Another aspect of an embodiment of the present disclosure provides an electronic device, which includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0017] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method described above.

[0018] Another aspect of an embodiment of the present disclosure provides a computer program, wherein the computer program includes computer executable instructions, and the instructions are used to implement the method described above when executed.

[0019] According to an embodiment of the present disclosure, by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model, a target identification model obtained by training the convolutional neural network model is used to process an image to be identified, and an identification result for characterizing whether the image to be identified includes a target identification is obtained, each sample image group includes a positive sample and a negative sample, and the positive sample and the negative sample in each sample image group have an associated relationship. Since the target identification identification model generated by training the convolutional neural network model is used to identify whether the image to be identified includes a target identification, the recognition efficiency of the target identification identification model is high, and in this process, no manual participation in the review is required, thus reducing the cost of identification identification and improving the recognition efficiency, thus at least partially overcoming the technical problem of high cost and low recognition efficiency of using related technologies to identify identification in images. Since each sample image group includes a positive sample and a negative sample, the balance of positive and negative samples is guaranteed. On this basis, since the positive samples and negative samples in a plurality of sample image groups are alternately input into the convolutional neural network model to train the convolutional neural network model, the convolutional neural network model can learn the features of positive samples and negative samples by comparison. Since the features of positive samples and negative samples with associated relationships can be compared and learned, the positive samples can be effectively identified, thereby ensuring the accuracy of the target identification model identification. In addition, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, and the training of the convolutional neural network model also speeds up the model training speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0021] Figure 1 An exemplary system architecture to which an image recognition method according to an embodiment of the present disclosure can be applied is schematically shown;

[0022] Figure 2 A flowchart of an image recognition method according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 3A schematic diagram schematically shows a first identification according to an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram schematically shows a second identification according to an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram schematically shows a positive sample generation process according to an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram schematically shows a negative sample generation process according to an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram schematically shows a schematic diagram of alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional audit network model according to an embodiment of the present disclosure;

[0028] Figure 8 A schematic diagram schematically shows another second identification according to an embodiment of the present disclosure;

[0029] Fig. 9 A flowchart schematically shows another image recognition method according to an embodiment of the present disclosure;

[0030] Fig.10 A schematic diagram schematically shows a target identification model generation process according to an embodiment of the present disclosure;

[0031] Fig.11 A schematic diagram schematically shows an image recognition method according to an embodiment of the present disclosure;

[0032] Fig.12 A block diagram schematically shows an image recognition device according to an embodiment of the present disclosure; and

[0033] Fig.13 A block diagram of an electronic device suitable for implementing an image recognition method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0037] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the process of implementing the present disclosure, the inventors found that usually a large number of objects need to be marked. In addition, the update cycle of the mark is short, the size is small, and there are many styles. At the same time, there are also many interference elements related to the mark, and the interference elements are constantly changing. Usually the number of interference elements is much larger than the number of marks. Among them, the relatively small size of the mark is reflected in the fact that the mark usually accounts for a very small proportion in the image, generally not exceeding 1 / 9 of the total area of ​​the image. There are many styles, which can be reflected in the different positions, shapes, colors, and occlusions with other elements on the object of the same mark on different objects. Interference elements refer to elements that affect the identification of the mark to be identified, such as the similarity between the interference element and the mark to be identified is greater than or equal to the similarity threshold.

[0039] In the process of realizing the concept of the present disclosure, the inventors discovered that the problem of identifying the identification with the above-mentioned characteristics is essentially a data-free and boundary-free identification problem. Among them, no data means that there is no labeled data set that can be used. Boundaryless means that the data set is an open set. Specifically, since the number of objects that need to be identified is relatively large, the update cycle of the identification is short, and there are many interference elements related to the identification, and data labeling requires a lot of manpower and material resources and takes a long time, it is difficult to generate a large number of sample images with real labeling information, among which the difficulty of labeling is reflected in the difficulty of labeling the identification and the difficulty of labeling the interference elements. Since the interference elements related to the identification are constantly changing, the data set is an open set.

[0040] In the related art, manual review is usually used to identify the logo. Since there are a large number of objects with logos, a lot of manpower is needed for review, which in turn requires high manpower and material costs. In addition, manual review is not efficient and is difficult to meet business needs.

[0041] In order to solve the problems existing in the related art, the inventors found that logo recognition can be understood as an image classification problem, and it is a binary classification problem. A convolutional neural network model can be used to process image classification problems. However, if a convolutional neural network model is used to process image classification problems, it is necessary to obtain a large number of sample images with real annotation information as training samples to train the model in order to obtain a logo recognition model with higher prediction accuracy. As mentioned above, the identification problem with the above characteristics is essentially a data-free and boundary-free recognition problem. In addition, due to the large amount of images to be identified, the model is also required to have a high prediction speed. It can be seen that the above problems need to be solved in order to realize logo recognition based on a convolutional neural network model. This will be explained in conjunction with specific embodiments below.

[0042] The embodiments of the present disclosure provide an image recognition method, device, and electronic device to which the method can be applied. The method includes a training process and a prediction process. In the training process, positive samples and negative samples in multiple sample image groups are alternately input into a convolutional neural network model, and the convolutional neural network model is trained to obtain a target identification recognition module, wherein each sample image group includes a positive sample and a negative sample, and the positive samples and negative samples in each sample image group have an associated relationship. In the prediction process, an image to be identified is obtained, and the image to be identified is processed using a target identification recognition model to obtain a recognition result for characterizing whether the image to be identified includes a target identification.

[0043] Figure 1 The exemplary system architecture 100 to which the image recognition method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0044] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0045] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0046] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0047] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0048] It should be noted that the image recognition method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image recognition device provided in the embodiment of the present disclosure can generally be set in the server 105. The image recognition method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image recognition device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the image recognition method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the image recognition device provided in the embodiment of the present disclosure can also be set in the terminal devices 101, 102, or 103, or can be set in other terminal devices different from the terminal devices 101, 102, or 103.

[0049] For example, the image to be identified and the multiple sample image groups may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the image recognition method provided by the embodiment of the present disclosure, or send the image to be identified and the multiple sample image groups to other terminal devices, servers, or server clusters, and the image recognition method provided by the embodiment of the present disclosure may be executed by the other terminal devices, servers, or server clusters that receive the image to be identified and the multiple sample image groups.

[0050] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0051] Figure 2 The flowchart of an image recognition method according to an embodiment of the present disclosure is schematically shown.

[0052] like Figure 2 As shown, the method includes operations S210 to S220.

[0053] In operation S210, an image to be recognized is acquired.

[0054] In the embodiments of the present disclosure, the image to be identified may refer to an image that needs to be identified. The image to be identified may be a commodity image. For example, the image to be identified is an image of a certain model of a smartphone.

[0055] In operation S220, the image to be identified is processed using a target identification recognition model to obtain a recognition result for characterizing whether the image to be identified includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples in multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each sample image group includes a positive sample and a negative sample, the positive sample in each sample image group is an image obtained by setting a first identification on the original image, and the negative sample in each sample image group is the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification are associated with each other.

[0056] In the embodiments of the present disclosure, in order to reduce the cost of identification and improve the efficiency of identification, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, and the target identification recognition model obtained by training the convolutional neural network model processes the image to be identified. Each sample image group may include a positive sample and a negative sample of a specific association relationship.

[0057] According to the embodiments of the present disclosure, the above-mentioned method of training the convolutional neural network model can be called contrast training. Contrast training includes two aspects. First, the positive samples and negative samples in each sample image group have an associated relationship; second, the input method is to input positive samples and negative samples alternately. Among them, the first aspect is the basis of the second aspect. That is, since the positive samples and negative samples in each sample image group have an associated relationship, the prediction accuracy of the target identification recognition model trained by the input method of alternating positive and negative samples is improved, thereby improving the accuracy of identification recognition.

[0058] According to the embodiments of the present disclosure, the target identification model obtained by the above-mentioned comparative training method can ensure a high identification accuracy. The reason is that identification recognition is a binary classification problem. The target identification model generated by the convolutional neural network model training has a class imbalance problem caused by the imbalance of positive and negative samples. The above problem makes the prediction accuracy of the target identification model not high. This is because: if the number of positive samples is too small, the convolutional neural network model cannot extract effective features and cannot effectively identify the positive samples well, resulting in overfitting, that is, the convolutional neural network model is easy to fit to the part that is not related to the identification.

[0059] In order to achieve effective recognition of positive samples, it is necessary to balance positive and negative samples. On this basis, if the positive and negative samples have an associated relationship, the way in which the samples are input into the convolutional neural network model can be changed so that the convolutional neural network model can better learn the features. By setting a positive sample and a negative sample with an associated relationship to form a sample image group, the positive and negative samples are balanced, that is, the number of positive samples and the number of negative samples are kept consistent, which increases the possibility of effective recognition of positive samples. Since the positive samples and negative samples in the sample image group have an associated relationship, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, so that the convolutional neural network model can learn the features of positive samples and negative samples by comparison. Since the features of positive samples and negative samples with an associated relationship can be compared and learned, the effective recognition of positive samples is achieved, thereby ensuring the accuracy of the target identification recognition model identification recognition. In addition, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, and the convolutional neural network model is trained, which also speeds up the model training speed.

[0060] According to the embodiments of the present disclosure, the input method of alternately inputting positive samples and negative samples into the convolutional neural network model adopted in the embodiments of the present disclosure changes the way the Shuffle method used in the related art processes positive samples and negative samples.

[0061] According to an embodiment of the present disclosure, the positive sample and the negative sample in each sample image group have an associated relationship, which may refer to the positive sample and the negative sample in each sample image group including the same image part and the different image part. The same image part and the different image part may be reflected in the same main elements of the image, and the difference is that the positive sample may include an identifier, and the negative sample may not include an identifier or the identifier included is different from the identifier included in the positive sample. The positive sample and the negative sample in each sample image group may be generated in the following manner: at least one original image under each category in multiple categories is obtained, and the number of original images under different categories meets a preset condition. For each original image, a first identifier is set on the original image to obtain a positive sample in the sample image group. The original image is used as a negative sample or a second identifier is set on the original image to obtain a negative sample. Each sample image group is obtained based on different processing of the same original image. The number of original images under different categories meeting the preset condition can be understood as the number of original images under different categories being the same, or the absolute value of the difference between the number of original images under any two categories is less than or equal to a preset threshold, and the preset threshold is an integer greater than zero. Exemplary, the preset threshold is 10. Category can refer to product category. Product categories can include mobile phones, computers, home appliances, maternal and child clothing, and luggage and handbags.

[0062] According to an embodiment of the present disclosure, a sample sampling method may be used to obtain at least one original image corresponding to each category from multiple categories, wherein the sample sampling method may include a probability sampling method or a non-probability sampling method, wherein the probability sampling method may include a simple random sampling method, a systematic sampling method, a stratified sampling method, or a cluster sampling method. The non-probability sampling method may include a convenience sampling method, a quota sampling method, a judgment sampling method, or a snowball sampling method. The sample sampling method may be selected according to actual conditions, and the embodiment of the present disclosure is not limited to this.

[0063] It should be noted that the original images corresponding to different sample image groups may be the same or different. The positive samples in different sample image groups may be the same or different. The negative samples in different sample image groups are usually different. If the original images corresponding to different sample image groups are the same, the positive samples in different sample image groups are the same, but the negative samples are different. If the original images corresponding to different sample image groups are different, the positive samples in different sample image groups are different, and the negative samples are different.

[0064] According to the embodiments of the present disclosure, part of the original image is usually selected with a preset probability, and a negative sample is generated on the selected original image by setting a second identifier on the original image. That is, the negative sample may not be generated for all the original images by setting the second identifier on the original image, but only part of the original images are selected.

[0065] As mentioned above, the identification problem to be solved by the embodiments of the present disclosure is essentially a data-free and boundary-free identification problem. The model training in the related art is aimed at the identification of a fixed closed set and requires a large amount of labeled data. That is, in order to identify whether the image includes the target identification, it is necessary to obtain the data of all easily confused images to train the model, so as to correctly identify whether the image to be identified includes the target identification. For example, the ImageNet data set includes several easily confused categories, such as cats, dogs, lions and tigers. However, since the training samples are closed sets, the model can easily learn the difference between these categories through training. Even if another animal similar to a cat appears, the model can still perform effective identification.

[0066] In view of the problem of data-free and boundary-free identification recognition to be solved by the embodiments of the present disclosure, a set of data generation methods is proposed. The main idea is to balance the sampled images from multiple categories to be processed, and use them as the original images for generating sample images, that is, to obtain at least one original image under each category of multiple categories, and the number of original images under different categories meets the preset conditions. After obtaining the original image, for each original image, a first identifier can be set on the original image to obtain a positive sample, and a second identifier can be set on the same original image to obtain a negative sample, or the original image can be not processed and the original image can be used as a negative sample. Among them, the first identifier can be the identifier to be identified. The second identifier can refer to an interference identifier that affects the identification of the first identifier. The positive samples and negative samples obtained in the above manner are positive samples and negative samples with an associated relationship.

[0067] According to an embodiment of the present disclosure, the reason for setting the second identifier is: in order to improve the prediction accuracy of the target identifier recognition model and to improve the accuracy of identifier recognition, it is necessary to ensure that the target identifier recognition model can also effectively identify interference identifiers that are different from the first identifier but will affect the recognition of the first identifier. Based on this, the second identifier can be set.

[0068] Figure 3 A schematic diagram of a first identification according to an embodiment of the present disclosure is schematically shown. Figure 4 A schematic diagram of a second identifier according to an embodiment of the present disclosure is schematically shown. Figure 5 A schematic diagram of a positive sample generation process according to an embodiment of the present disclosure is schematically shown. Figure 6 A schematic diagram of a negative sample generation process according to an embodiment of the present disclosure is schematically shown. Figure 5 The positive samples and Figure 6 The negative samples in belong to a sample image group. Figure 5 The positive samples in are set on the original image Figure 3 The first identifier in the resulting image. Figure 6 The negative samples in are set on the same original image Figure 4 The second identifier in the image obtained.

[0069] According to an embodiment of the present disclosure, in order to improve the prediction accuracy of the target identification model and to improve the accuracy of identification, a second identification associated with the first identification can be set. This is because the first identification can be an identification to be identified. In order to improve the prediction accuracy of the target identification model and to improve the accuracy of identification, it is necessary to ensure that the target identification model can also effectively identify the interference identification that affects the identification of the first identification.

[0070] Since the interference identifier will affect the identification of the first identifier, it can be explained that there is an association relationship between the interference identifier and the first identifier. The association relationship between the interference identifier and the first identifier mentioned here can include two aspects. First, the interference identifier does not come from the first identifier, that is, the interference identifier and the first identifier are independent, and the similarity between the interference identifier and the first identifier is greater than or equal to the similarity threshold; second, the interference identifier comes from the first identifier, that is, the interference identifier is part of the first identifier. At this time, the similarity between the interference identifier and the first identifier may be higher or lower, such as the similarity between the interference identifier and the first identifier is greater than or equal to the first similarity threshold, or the similarity between the interference identifier and the first identifier is less than or equal to the second similarity threshold, and the first similarity threshold is greater than the second similarity threshold. The interference identifier mentioned above can be called the second identifier.

[0071] According to an embodiment of the present disclosure, positive samples and negative samples in a plurality of sample image groups are alternately input into a convolutional neural network model, which can be understood as including N sample image groups, namely sample image group 1, sample image group 2, ..., sample image group N-1 and sample image group N. Sample image group i includes positive sample i and negative sample i, where i∈{1, 2, ..., N-1, N}, and N is an integer greater than 1. The input order of the N sample image groups can be determined, that is, the N sample image groups can be sorted in the order of sample image group 1, sample image group 2, ..., sample image group N-1 and sample image group N, and the sorting order is the input order. The input order of the positive samples and negative samples in each sample image group can be determined, that is, each sample image group can be sorted in the order of positive samples and negative samples, and the sorting order is the input order. The positive samples and negative samples in the N sample image groups are input into the convolutional neural network model alternately, that is, the positive samples and negative samples in each sample image group are input into the convolutional neural network model in sequence according to the order of the N sample image groups. It should be noted that the N sample image groups can also be sorted in other orders, which is not specifically limited here.

[0072] Figure 7 A schematic diagram schematically shows a method of alternately inputting positive samples and negative samples from a plurality of sample image groups into a convolutional review network model according to an embodiment of the present disclosure.

[0073] like Figure 7 As shown, Figure 7 The N sample image groups are input into the convolutional neural network model alternately in the order shown by the arrows. That is, they are input into the convolutional neural network model alternately in the order of sample image 1, sample image group 2, ..., sample image group N-1 and sample image group N, wherein each sample image group is input alternately in the order of positive samples and negative samples.

[0074] According to the technical solution of the embodiment of the present disclosure, by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model, the target identification model obtained by training the convolutional neural network model processes the image to be identified, and obtains an identification result for characterizing whether the image to be identified includes the target identification, each sample image group includes a positive sample and a negative sample, the positive sample in each sample image group is an image obtained by setting a first identification on the original image, and the negative sample in each sample image group is the original image or an image obtained by setting a second identification on the original image, and the first identification and the second identification have an associated relationship. Since the target identification identification model generated by training the convolutional neural network model is used to identify whether the image to be identified includes the target identification, the recognition efficiency of the target identification identification model is high, and in this process, no manual participation in the review is required, thereby reducing the cost of identification identification and improving the recognition efficiency, thereby at least partially overcoming the technical problem that the cost of identifying the identification in the image using the related technology is high and the recognition efficiency is low. Since each sample image group includes a positive sample and a negative sample, the balance of positive and negative samples is guaranteed. On this basis, since positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model to train the convolutional neural network model, the convolutional neural network model can learn the features of positive samples and negative samples by comparison. Since the features of positive samples and negative samples with an associated relationship can be learned by comparison, the positive samples are effectively identified, thereby ensuring the accuracy of the target identification model identification. In addition, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, and the convolutional neural network model is trained to speed up the model training speed.

[0075] Optionally, based on the above technical solution, each sample image group has a corresponding original image. The original image corresponding to each sample image group can be obtained in the following manner: obtaining at least one original image corresponding to each category in multiple categories, wherein the number of original images under different categories meets a preset condition.

[0076] In the embodiments of the present disclosure, in order to more fully obtain data information and interference information under different categories, balanced sampling images can be used from multiple categories to be processed to serve as original images for generating sample images, that is, to obtain relatively balanced original images.

[0077] According to an embodiment of the present disclosure, at least one original image corresponding to each category in multiple categories can be obtained, and a sample image group corresponding to each original image can be generated based on each original image. The number of original images under different categories meeting the preset condition can be understood as the number of original images under different categories being the same, or the absolute value of the difference between the numbers of original images under any two categories being less than or equal to a preset threshold, and the preset threshold is an integer greater than zero.

[0078] Optionally, based on the above technical solution, obtaining at least one original image corresponding to each category in multiple categories may include the following operations.

[0079] A sample sampling method is adopted to obtain at least one original image corresponding to each category from multiple categories, wherein the sample sampling method may include a probability sampling method or a non-probability sampling method.

[0080] In the embodiments of the present disclosure, probability sampling methods may include simple random sampling methods, systematic sampling methods, stratified sampling methods, or cluster sampling methods. Non-probability sampling methods may include convenience sampling methods, quota sampling methods, judgment sampling methods, or snowball sampling methods. The sampling method may be selected according to actual conditions, and the embodiments of the present disclosure do not limit this.

[0081] Optionally, based on the above technical solution, the first identifier and the second identifier have an association relationship, which may include: the similarity between the first identifier and the second identifier is greater than or equal to a similarity threshold and the first identifier and the second identifier are different identifiers; or, the second identifier is an identifier obtained by covering part of the first identifier.

[0082] In an embodiment of the present disclosure, the second identifier is an identifier that is different from the first identifier and whose similarity to the first identifier is greater than or equal to a similarity threshold, or the second identifier may be a part of the first identifier.

[0083] According to the embodiments of the present disclosure, the second identifier can be a part of the first identifier, which can be understood as a reverse data enhancement method, that is, the second identifier is obtained by covering part of the first identifier, and a negative sample is generated according to the second identifier, thereby achieving data enhancement of the negative sample.

[0084] Figure 8 A schematic diagram of another second identification according to an embodiment of the present disclosure is schematically shown. Figure 8 The second mark in the Figure 4 The obtained identifier is the part after the first identifier.

[0085] Optionally, based on the above technical solution, the target identification recognition model is obtained by alternately inputting positive samples and negative samples in multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, which may include the following operations.

[0086] A plurality of sample image groups are obtained, wherein each sample image group includes a positive sample and a negative sample, and the positive samples and negative samples in each sample image group are associated with each other. The true labeling information of each positive sample and the true labeling information of each negative sample are obtained. The positive samples and negative samples in the plurality of sample image groups are alternately input into a convolutional neural network model, and the predicted labeling information of each positive sample and the predicted labeling information of each negative sample are output respectively. The convolutional neural network model is trained according to the true labeling information of the plurality of positive samples, the true labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples, and the predicted labeling information of the plurality of negative samples to obtain a target identification recognition model.

[0087] In an embodiment of the present disclosure, a convolutional neural network model may be used to train multiple sample image groups to obtain a target identification model.

[0088] According to an embodiment of the present disclosure, obtaining multiple sample image groups may include: obtaining multiple original images under each category in multiple categories. For each original image, a first identifier may be set on the original image to obtain a positive sample, and a second identifier may be set on the original image to obtain a negative sample or the original image may be used as a negative sample. Among them, the first identifier is an identifier to be identified. The setting method of the first identifier may be determined according to the characteristics of the original image. The setting method of the second identifier may adopt a minimum clone (mix-clone) method. A positive sample and a negative sample corresponding to the original image are formed into a sample image group corresponding to the original image. Based on this, multiple sample image groups can be obtained, wherein each sample image group has a corresponding original image. The same original image can correspond to different sample image groups.

[0089] According to an embodiment of the present disclosure, the real annotation information of multiple sample image groups is obtained. That is, the real annotation information of the positive sample and the real annotation information of the negative sample in each sample image group is obtained. The real annotation information may include the identifier to be identified (i.e., the first identifier) ​​or not include the identifier to be identified.

[0090] According to an embodiment of the present disclosure, the positive samples and negative samples in a plurality of sample image groups are alternately input into a convolutional neural network model, and the predicted labeling information of each positive sample and the predicted labeling information of each negative sample are outputted respectively, which may include: determining the input order between the plurality of sample image groups. Determining the input order of the positive samples and the negative samples in each sample image group. According to the input order between the plurality of sample image groups and the input order of the positive samples and the negative samples in each sample image group, the positive samples and the negative samples are alternately input into the convolutional neural network model, and the predicted labeling information of each positive sample and the predicted labeling information of each negative sample are outputted respectively. The predicted labeling information may include including the identification to be identified (i.e., the first identification) or excluding the identification to be identified.

[0091] According to an embodiment of the present disclosure, the training process of the convolutional neural network model is to calculate the loss function of the convolutional neural network model through forward propagation, that is, the real labeling information of multiple positive samples, the real labeling information of multiple negative samples, the predicted labeling information of multiple positive samples and the predicted labeling information of multiple negative samples are input into the loss function of the convolutional neural network model to obtain the loss result, and determine the partial derivative of the loss function with respect to the network parameters, and use the reverse gradient propagation method to adjust the network parameters of the convolutional neural network according to the loss result until the loss function of the convolutional neural network model converges. When the loss function of the convolutional neural network model converges, it means that the convolutional neural network model has been trained, and at this time, the network parameters of the convolutional neural network model are also determined. On this basis, the trained convolutional neural network model can be used as a target identification recognition model.

[0092] According to the embodiments of the present disclosure, since identification recognition is a binary classification problem, the probability threshold of the target identification recognition model finally obtained can be adjusted to obtain a better recognition result.

[0093] In the embodiments of the present disclosure, due to the complexity of logo recognition and the requirement for high computational efficiency, it is necessary to select a suitable convolutional neural network model on the basis of ensuring the accuracy of logo recognition to meet the above requirements.

[0094] In order to meet the above requirements, the convolutional neural network model of the embodiment of the present disclosure may be EfficientNet. Among them, Efficientnet is a fast and high-precision model proposed by Google in 2019. It uses the joint adjustment technology of depth, width and input image resolution. According to the order of model complexity, it can be divided into 11 versions including EfficientNet-B0~EfficientNet-B8, EfficientNet-L2 and Noisy Student.

[0095] The disclosed embodiments may use TensorFlow to implement EfficientNet, where TensorFlow is an open source Python-based machine learning framework.

[0096] Optionally, based on the above technical solution, obtaining the true labeling information of each positive sample and the true labeling information of each negative sample may include the following operations.

[0097] Obtain initial labeling information of each positive sample and initial labeling information of each negative sample. Perform label smoothing on the initial labeling information of the positive sample and initial labeling information of the negative sample in each sample image group to obtain the true labeling information of the positive sample and the true labeling information of the negative sample in each sample image group.

[0098] In the embodiments of the present disclosure, in order to further improve the accuracy of identification, a label smoothing method can be used to process the initial labeling information of the sample image group to obtain the real labeling information. Label smoothing (LS) is a regularization method in machine learning. Label smoothing is used to make the real labeling information have a certain error tolerance probability.

[0099] The reason why the label smoothing method is used to process the initial labeling information of the sample image group to obtain the real labeling information can further improve the accuracy of the identification recognition is that: on the one hand, due to the relatively complex feature space, errors may occur when determining the initial labeling information of the sample image group, such as a certain original image itself includes the first identification (i.e., the identification to be identified), and the original image is not labeled in the data generation part. If the original image is not labeled, the original image will be used as a negative sample, but in fact the original image should be used as a positive sample, and the label smoothing method can reduce the adverse effects of labeling errors. Therefore, the label smoothing method can improve the prediction accuracy of the target identification model, and thus improve the accuracy of identification recognition. On the other hand, since identification recognition is a binary classification problem, it is easy to overfit, and label smoothing can suppress overfitting. Therefore, the label smoothing method can improve the prediction accuracy of the target identification model, and thus improve the accuracy of identification recognition.

[0100] According to an embodiment of the present disclosure, label smoothing is performed on the initial labeling information of the positive sample in each sample image group and the initial labeling information of each negative sample to obtain the true labeling information of the positive sample and the true labeling information label of the negative sample in each sample image group, which may include: the sample image group is represented as (x i ,y i ), where x i represents the i-th sample image group, y irepresents the initial annotation information of the i-th sample image group, i∈{1, 2, ..., N, N-1}, N represents the number of sample image groups. Since the initial annotation information of the positive sample is 1 and the initial annotation information of the negative sample is 0, y i =[0,1] or y i = [1, 0]. Set the hyperparameter to ε. Use the label smoothing method to process the initial annotation information y of the i-th sample image group. i , get the true annotation information y′ of the i-th sample image group i =[ε, 1-ε] or y′ i =[1-ε,ε]. That is, the initial labeling information 1 of the positive sample is modified to the true labeling information 1-ε of the positive sample, and the initial labeling information 0 of the negative sample is modified to the true labeling information ε of the negative sample.

[0101] For example, if ε=0.1, the initial annotation information y of the i-th sample image group is i =[0, 1], then the true annotation information y′ of the i-th sample image group i =[0.1, 0.9].

[0102] Optionally, based on the above technical solution, a convolutional neural network model is trained according to the real labeling information of multiple positive samples, the real labeling information of multiple negative samples, the predicted labeling information of multiple positive samples and the predicted labeling information of multiple negative samples to obtain a target identification recognition model, which may include the following operations.

[0103] Determine the wrong labeling information based on the true labeling information of multiple positive samples, the true labeling information of multiple negative samples, the predicted labeling information of multiple positive samples, and the predicted labeling information of multiple negative samples. Determine the sample image corresponding to the wrong labeling information. Determine the new negative sample and the new positive sample corresponding to the new negative sample based on the sample image corresponding to the wrong labeling information. Obtain the true labeling information of the new positive sample and the true labeling information of the new negative sample. Alternately input the new positive sample and the new negative sample into the convolutional neural network model, and output the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample respectively. Train the convolutional neural network model based on the true labeling information of the new positive sample, the true labeling information of the new negative sample, the predicted labeling information of the new positive sample, and the predicted labeling information of the new negative sample to obtain the target identification recognition model.

[0104] In an embodiment of the present disclosure, in order to further improve the accuracy of identification recognition, it is possible to determine the erroneous labeling information from the predicted labeling information obtained based on the initial identification recognition model, and continue to train the convolutional neural network model based on the erroneous labeling information to obtain a target identification recognition model.

[0105] After obtaining the true labeling information of multiple positive samples, the true labeling information of multiple negative samples, the predicted labeling information of multiple positive samples, and the predicted labeling information of multiple negative samples, the predicted labeling information of each positive sample can be compared with the true labeling information of the positive sample. If the predicted labeling information of the positive sample is inconsistent with the true labeling information of the positive sample, the inconsistent predicted labeling information can be determined to be erroneous labeling information. The predicted labeling information of each negative sample is compared with the true labeling information of the negative sample. If the predicted labeling information of the negative sample is inconsistent with the true labeling information of the negative sample, the inconsistent predicted labeling information can be determined to be erroneous labeling information.

[0106] After obtaining the erroneous labeling information, the sample image corresponding to the erroneous labeling information can be determined according to the erroneous labeling information. According to the sample image corresponding to the erroneous labeling information, the newly added second identifier is determined. According to the second identifier and the newly added second identifier, a new negative sample is determined, and the first second identifier mentioned here refers to the original second identifier. Determine a new positive sample corresponding to the new negative sample. Obtain the initial labeling information of the new negative sample and the initial labeling information of the new positive sample. Perform label smoothing on the initial labeling information of the new positive sample and the initial labeling information of the new negative sample to obtain the true labeling information of the new positive sample and the true labeling information of the new negative sample. The number of sample images with erroneous labeling information can be at least one. The number of newly added second identifiers can be at least one. Correspondingly, the number of new positive samples and new negative samples can be at least one. The number of new positive samples and new negative samples is the same.

[0107] After obtaining new positive samples and new negative samples, the new positive samples and new negative samples can be alternately input into the convolutional neural network model to output the predicted labeling information of the new positive samples and the predicted labeling information of the new negative samples, respectively. The method of alternately inputting the new positive samples and the new negative samples into the convolutional neural network model is the same as the method of alternately inputting the positive samples and the negative samples into the convolutional neural network described above, and will not be described in detail here.

[0108] After obtaining the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample, the real labeling information of the new positive sample, the real labeling information of the new negative sample, the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample can be input into the loss function of the convolutional neural network model to obtain the loss result, and the network parameters of the convolutional neural network are adjusted according to the loss result until the loss function of the convolutional neural network model converges. When the loss function value of the convolutional neural network model converges, it means that the convolutional neural network model has been trained, and at this time, the network parameters of the convolutional neural network model are also determined. On this basis, the trained convolutional neural network model can be used as a target identification recognition model.

[0109] According to the embodiments of the present disclosure, by determining the wrong labeling information based on the true labeling information of the positive sample, the true labeling information of the negative sample, the predicted labeling information of the positive sample and the predicted labeling information of the negative sample, and forming a new negative sample based on the wrong labeling information, the data enhancement processing of the negative sample is realized, that is, a reverse data enhancement method is adopted to realize the data enhancement of the negative sample. Since the data enhancement of the negative sample is realized, the prediction accuracy and robustness of the target identification model are improved, thereby improving the accuracy of identification recognition.

[0110] According to the embodiments of the present disclosure, the above-mentioned reverse data enhancement method proposed in the embodiments of the present disclosure can be used to solve the problem that the accuracy of identification is not high due to the existence of a large number of unknown and constantly changing interference elements in identification.

[0111] According to the embodiment of the present disclosure, after obtaining the newly added second identifier, the original image will be processed with a preset probability to obtain a new negative sample. That is, the newly added second identifier may not be set for all original images, but for some of the original images.

[0112] Optionally, based on the above technical solution, the sample image group includes images obtained after at least one of translation, rotation, cropping, non-rigid transformation, noise disturbance and color transformation.

[0113] In the embodiments of the present disclosure, the target identification model is generated based on the training of the convolutional neural network model. One of the major advantages of the convolutional neural network model is its ability to absorb data and convert it into continuous learning and updating of parameters, thereby obtaining a model with good predictive performance and generalization ability. In order to obtain a model with good predictive performance and generalization ability, it puts forward requirements on the quantity and quality of training samples. In other words, the quantity and quality of training samples have an important influence on the predictive performance and generalization ability of the model. Based on this, data enhancement methods can be used to process the training samples to increase the number of training samples and improve the quality of training samples, thereby improving the predictive performance and generalization ability of the model.

[0114] According to the embodiments of the present disclosure, the training samples mentioned here refer to multiple sample image groups. That is, the embodiments of the present disclosure use the data enhancement method to process multiple sample image groups. In other words, the sample image groups are images obtained after data enhancement processing.

[0115] According to an embodiment of the present disclosure, at least one original image under each category of multiple categories is obtained. Each original image is processed by a data enhancement method to obtain each processed original image. The above can be understood as balancing the sampled images from the multiple categories to be processed, so as to use them as processed original images for generating sample images. For each processed original image, a first identifier can be set on the processed original image to obtain a positive sample, and a second identifier can be set on the processed original image to obtain a negative sample or the processed original image can be used as a negative sample. Among them, the first identifier is the identifier to be identified. A positive sample and a negative sample corresponding to the processed original image are formed into a sample image group corresponding to the processed original image. The first identifier and the second identifier can also be identifiers processed by data enhancement. Based on this, multiple sample image groups can be obtained, wherein each sample image group has a corresponding processed original image. The same processed original image can correspond to different sample image groups.

[0116] According to an embodiment of the present disclosure, the data enhancement method may include at least one of translation, rotation, cropping, non-rigid transformation, noise perturbation and color transformation. Among them, a rigid transformation may refer to a transformation in which only the position and orientation of the image change, while the shape remains unchanged. A non-rigid transformation is a more complex transformation than a rigid transformation. Non-rigid transformations may include beveling, distortion and perspective. Noise perturbation may include Gaussian noise, and color transformation may include saturation enhancement, brightness enhancement and contrast enhancement. It should be noted that the data enhancement method may be selected according to actual conditions and is not specifically limited here.

[0117] For example, for each original image, the original image is cropped to obtain a processed original image. Since the logo is usually set at the upper left corner, lower left corner, upper right corner and lower right corner of the original image, the cropping position can be selected from the above four corner positions. At the same time, the shapes of the first logo and the second logo can also be changed. The first logo can be randomly set at any position on the processed original image.

[0118] Fig. 9 The flowchart schematically shows another image recognition method according to an embodiment of the present disclosure.

[0119] like Fig. 9 As shown, the method includes operations S901 to S913.

[0120] In operation S901, at least one original image corresponding to each category in a plurality of categories is acquired, wherein the number of original images in different categories satisfies a preset condition.

[0121] In operation S902, a sample image group corresponding to each original image is generated according to each original image, wherein each sample image group includes a positive sample and a negative sample, and the positive sample and the negative sample in each sample image have an associated relationship.

[0122] In an embodiment of the present disclosure, the positive sample in each sample image group is an image obtained by setting a first identifier on the original image, and the negative sample in each sample image group is the original image or an image obtained by setting a second identifier on the original image. The similarity between the first identifier and the second identifier is greater than or equal to a similarity threshold and the first identifier and the second identifier are different identifiers. Alternatively, the second identifier is an identifier obtained by covering part of the first identifier. Each sample image group has a corresponding original image. The original image corresponding to each sample image group can be obtained in the following manner: obtaining at least one original image corresponding to each category in multiple categories, wherein the number of original images under different categories meets a preset condition.

[0123] In operation S903 , initial labeling information of each positive sample and initial labeling information of each negative sample are obtained.

[0124] In operation S904, label smoothing is performed on the initial labeling information of the positive samples and the initial labeling information of the negative samples in each sample image group to obtain the true labeling information of the positive samples and the true labeling information of the negative samples in each sample image group.

[0125] In operation S905, positive samples and negative samples in the plurality of sample image groups are alternately input into a convolutional neural network model, and predicted labeling information of each positive sample and predicted labeling information of each negative sample are output respectively.

[0126] In operation S906 , incorrect labeling information is determined according to the true labeling information of the plurality of positive samples, the true labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples, and the predicted labeling information of the plurality of negative samples.

[0127] In operation S907 , a sample image corresponding to the erroneous labeling information is determined.

[0128] In operation S908 , a new negative sample and a new positive sample corresponding to the new negative sample are determined according to the sample image corresponding to the incorrectly labeled information.

[0129] In operation S909 , initial labeling information of a new positive sample and initial labeling information of a new negative sample are obtained.

[0130] In operation S910 , label smoothing is performed on the initial labeling information of the new positive sample and the initial labeling information of the new negative sample to obtain the real labeling information of the new positive sample and the real labeling information of the new negative sample.

[0131] In operation S911, new positive samples and new negative samples are alternately input into the convolutional neural network model, and predicted labeling information of the new positive samples and predicted labeling information of the new negative samples are output respectively.

[0132] In operation S912, a convolutional neural network model is trained according to the real labeling information of the new positive sample, the real labeling information of the new negative sample, the predicted labeling information of the new positive sample, and the predicted labeling information of the new negative sample to obtain a target identification recognition model.

[0133] In operation S913, an image to be recognized is acquired.

[0134] In operation S914, the image to be identified is processed using the target identification model to obtain a recognition result indicating whether the image to be identified includes the target identification.

[0135] In the embodiments of the present disclosure, in order to better understand the technical solutions provided by the embodiments of the present disclosure, specific examples are provided below for illustration.

[0136] The sample image group and the image to be identified are commodity images. In order to realize logo recognition, the embodiment of the present disclosure is designed from the aspects of data generation, model and reverse data enhancement.

[0137] In terms of data generation, operations S901 to S903 are used to generate the required training samples in the absence of labeled data and in the presence of a large number of unknown interference elements.

[0138] In terms of the model, label smoothing such as operations S903 to S904 and operations S909 to S910 is used to further improve the prediction accuracy of the target identification model, thereby further improving the accuracy of identification. Operations S902 and S908 are used to obtain a sample image group consisting of a positive sample and a negative sample with an associated relationship, and operations S905 and S911 are used to train the convolutional neural network model in a comparative training manner, which speeds up the model training speed, improves the prediction accuracy of the target identification model, and thereby improves the accuracy of identification.

[0139] In terms of reverse data enhancement, operations S906 to S912 are used, and the second identifier is an identifier obtained by covering part of the first identifier. Negative samples are generated based on the second identifier, which solves the problem of many unknown and constantly changing interference elements in identifier recognition and improves the prediction accuracy and robustness of the target identifier recognition model.

[0140] It is assumed that there are N sample image groups, namely sample image group 1, sample image group 2, ..., sample image group N-1 and sample image group N. Sample image group i includes positive sample i and negative sample i, where i∈{1, 2, ..., N-1, N}, and N is an integer greater than 1. The positive sample i and the negative sample i in each sample image group i are both product images.

[0141] Fig.10 A schematic diagram schematically shows a target identification model generation process according to an embodiment of the present disclosure. Fig.10 The N sample image groups are input into the convolutional neural network model alternately in the order shown by the arrows. That is, the sample image 1, sample image group 2, ..., sample image group N-1 and sample image group N are input into the convolutional neural network model alternately in the order of the positive sample and the negative sample, and each sample image group is input alternately in the order of the positive sample and the negative sample, and the predicted labeling information of the positive sample i and the predicted labeling information of the negative sample i (i.e. the predicted labeling information of the sample image group i) are output respectively. The convolutional neural network model is trained according to the real labeling information of the sample image group i (i.e. the real labeling information of the positive sample i and the real labeling information of the negative sample i) and the predicted labeling information of the sample image group i (i.e. the predicted labeling information of the positive sample i and the predicted labeling information of the negative sample i), and the target identification recognition model is obtained.

[0142] Fig.11 The following is a schematic diagram of an image recognition method according to an embodiment of the present disclosure. Fig.11 The image to be recognized in Fig.10 In the obtained target identification recognition model, the image to be identified includes the recognition result of the target identification.

[0143] According to the technical solution of the embodiment of the present disclosure, since the target identification recognition model generated by the training of the convolutional neural network model is used to identify whether the image to be identified includes the target identification, the recognition efficiency of the target identification recognition model is high, and in this process, no manual participation in the review is required, so the cost of identification recognition is reduced and the recognition efficiency is improved, thus at least partially overcoming the technical problem that the cost of identifying the identification in the image using the related technology is high and the recognition efficiency is low. Since each sample image group includes a positive sample and a negative sample, the balance of positive and negative samples is guaranteed. On this basis, since the positive samples and negative samples in the multiple sample image groups are alternately input into the convolutional neural network model to train the convolutional neural network model, the convolutional neural network model can learn the features of the positive samples and negative samples by comparison. Since the features of the positive samples and negative samples with an associated relationship can be compared and learned, the effective recognition of the positive samples is achieved, thereby ensuring the accuracy of the identification recognition of the target identification recognition model. In addition, the positive samples and negative samples in the multiple sample image groups are alternately input into the convolutional neural network model, and the convolutional neural network model is trained to speed up the model training speed.

[0144] Since the label smoothing method can reduce the adverse effects of labeling errors, the label smoothing method can improve the prediction accuracy of the target identification model, and thus improve the accuracy of identification. Since identification is a binary classification problem, it is prone to overfitting, and label smoothing can suppress overfitting. Therefore, the label smoothing method can improve the prediction accuracy of the target identification model, and thus improve the accuracy of identification. By determining the wrong labeling information based on the true labeling information of the positive sample, the true labeling information of the negative sample, the predicted labeling information of the positive sample, and the predicted labeling information of the negative sample, and forming a new negative sample based on the wrong labeling information, the data enhancement processing of the negative sample is realized. Since the data enhancement of the negative sample is realized, the prediction accuracy and robustness of the target identification model are improved, and thus the accuracy of identification is improved.

[0145] Fig.12 The block diagram schematically shows an image recognition device according to an embodiment of the present disclosure.

[0146] like Fig.12 As shown, the image recognition device 1200 may include an acquisition module 1210 and a processing module 1220 .

[0147] The acquisition module 1210 and the processing module 1220 are communicatively connected.

[0148] The acquisition module 1210 is used to acquire the image to be recognized.

[0149] The processing module 1220 is used to process the image to be identified by using the target identification recognition model to obtain a recognition result for characterizing whether the image to be identified includes the target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples in multiple sample image groups into the convolutional neural network model and training the convolutional neural network model, wherein the positive sample in each of the sample image groups is an image obtained by setting a first identification on the original image, and the negative sample in each of the sample image groups is the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification have an associated relationship.

[0150] According to the technical solution of the embodiment of the present disclosure, by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model, the target identification model obtained by training the convolutional neural network model processes the image to be identified, and obtains an identification result for characterizing whether the image to be identified includes the target identification, each sample image group includes a positive sample and a negative sample, and the positive sample and the negative sample in each sample image group have an associated relationship. Since the target identification identification model generated by training the convolutional neural network model is used to identify whether the image to be identified includes the target identification, the recognition efficiency of the target identification identification model is high, and in this process, no manual participation in the review is required, thus reducing the cost of identification identification and improving the recognition efficiency, thus at least partially overcoming the technical problem that the cost of identifying the identification in the image using related technologies is high and the recognition efficiency is low. Since each sample image group includes a positive sample and a negative sample, the balance of positive and negative samples is guaranteed. On this basis, since the positive samples and negative samples in a plurality of sample image groups are alternately input into the convolutional neural network model to train the convolutional neural network model, the convolutional neural network model can learn the features of the positive samples and the negative samples by comparison. Since the features of positive samples and negative samples with associated relationships can be compared and learned, the positive samples can be effectively identified, thereby ensuring the accuracy of the target identification model identification. In addition, the positive samples and negative samples in multiple sample image groups are alternately input into the convolutional neural network model, and the training of the convolutional neural network model also speeds up the model training speed.

[0151] Optionally, based on the above technical solution, each sample image group has a corresponding original image;

[0152] The original image corresponding to each sample image group is obtained as follows:

[0153] At least one original image corresponding to each category in a plurality of categories is obtained, wherein the number of original images under different categories meets a preset condition.

[0154] Optionally, based on the above technical solution, the first identifier and the second identifier have an association relationship, which may include: the similarity between the first identifier and the second identifier is greater than or equal to a similarity threshold and the first identifier and the second identifier are different identifiers. Or, the second identifier is an identifier obtained by covering part of the first identifier.

[0155] Optionally, based on the above technical solution, the target identification recognition model is obtained by alternately inputting positive samples and negative samples in multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, which may include the following operations.

[0156] A plurality of sample image groups are obtained, wherein each sample image group includes a positive sample and a negative sample, and the positive samples and negative samples in each sample image group are associated with each other. The true labeling information of each positive sample and the true labeling information of each negative sample are obtained. The positive samples and negative samples in the plurality of sample image groups are alternately input into a convolutional neural network model, and the predicted labeling information of each positive sample and the predicted labeling information of each negative sample are output respectively. The convolutional neural network model is trained according to the true labeling information of the plurality of positive samples, the true labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples, and the predicted labeling information of the plurality of negative samples to obtain a target identification recognition model.

[0157] Optionally, based on the above technical solution, obtaining the true labeling information of each positive sample and the true labeling information of each negative sample may include the following operations.

[0158] Obtain initial labeling information of each positive sample and initial labeling information of each negative sample. Perform label smoothing on the initial labeling information of the positive sample and initial labeling information of the negative sample in each sample image group to obtain the true labeling information of the positive sample and the true labeling information of the negative sample in each sample image group.

[0159] Optionally, based on the above technical solution, a convolutional neural network model is trained according to the real labeling information of multiple positive samples, the real labeling information of multiple negative samples, the predicted labeling information of multiple positive samples and the predicted labeling information of multiple negative samples to obtain a target identification recognition model, which may include the following operations.

[0160] Determine the wrong labeling information based on the true labeling information of multiple positive samples, the true labeling information of multiple negative samples, the predicted labeling information of multiple positive samples, and the predicted labeling information of multiple negative samples. Determine the sample image corresponding to the wrong labeling information. Determine the new negative sample and the new positive sample corresponding to the new negative sample based on the sample image corresponding to the wrong labeling information. Obtain the true labeling information of the new positive sample and the true labeling information of the new negative sample. Alternately input the new positive sample and the new negative sample into the convolutional neural network model, and output the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample respectively. Train the convolutional neural network model based on the true labeling information of the new positive sample, the true labeling information of the new negative sample, the predicted labeling information of the new positive sample, and the predicted labeling information of the new negative sample to obtain the target identification recognition model.

[0161] Optionally, based on the above technical solution, the sample image group includes images obtained after at least one of translation, rotation, cropping, non-rigid transformation, noise disturbance and color transformation.

[0162] According to any one or more of the modules of the embodiments of the present disclosure, or at least part of the functions of any one of them, can be implemented in one module. According to any one or more of the modules of the embodiments of the present disclosure, it can be split into multiple modules to implement. According to any one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, according to one or more of the modules of the embodiments of the present disclosure, it can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0163] For example, any multiple of the acquisition module 1210 and the processing module 1220 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 1210 and the processing module 1220 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 1210 and the processing module 1220 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0164] According to the embodiments of the present disclosure, the image recognition device part in the embodiments of the present disclosure corresponds to the image recognition method part in the embodiments of the present disclosure. The description of the image recognition device part specifically refers to the image recognition method part and will not be repeated here.

[0165] Fig.13 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0166] like Fig.13 As shown, the electronic device 1300 according to the embodiment of the present disclosure includes a processor 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303. The processor 1301 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1301 may also include an onboard memory for caching purposes. The processor 1301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0167] In RAM 1303, various programs and data required for the operation of electronic device 1300 are stored. Processor 1301, ROM 1302 and RAM 1303 are connected to each other through bus 1304. Processor 1301 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1302 and / or RAM 1303. It should be noted that the program can also be stored in one or more memories other than ROM 1302 and RAM 1303. Processor 1301 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0168] According to an embodiment of the present disclosure, the system 1300 may further include an input / output (I / O) interface 1305, which is also connected to the bus 1304. The system 1300 may further include one or more of the following components connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0169] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the processor 1301, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0170] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0171] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM (Erasable Programmable Read Only Memory) or flash memory), a portable compact disk read-only memory (Computer Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0172] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1302 and / or the RAM 1303 described above and / or one or more memories other than the ROM 1302 and the RAM 1303 .

[0173] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0174] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image recognition method, comprising: Obtain an image to be recognized; as well as The image to be identified is processed using a target identification recognition model to obtain a recognition result for characterizing whether the image to be identified includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each of the sample image groups includes a positive sample and a negative sample, the positive sample in each of the sample image groups is an image obtained by setting a first identification on the original image, and the negative sample in each of the sample image groups is the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification are associated with each other, the first identification is used to indicate the identification to be identified, and the second identification is used to indicate an interference identification that affects the identification of the first identification.

2. The method according to claim 1, wherein: Each of the sample image groups has a corresponding original image; The original image corresponding to each sample image group is obtained by: At least one original image corresponding to each of the categories in the plurality of categories is obtained, wherein the number of original images under different categories meets a preset condition.

3. The method according to claim 1, wherein: The first identifier and the second identifier have an associated relationship, including: The similarity between the first identifier and the second identifier is greater than or equal to a similarity threshold, and the first identifier and the second identifier are different identifiers; or The second identifier is an identifier obtained by covering part of the first identifier.

4. The method according to claim 1, wherein: The target identification model is obtained by alternately inputting positive samples and negative samples in a plurality of sample image groups into a convolutional neural network model and training the convolutional neural network model, including: Acquire a plurality of the sample image groups, wherein each of the sample image groups includes a positive sample and a negative sample, and the positive sample and the negative sample in each of the sample image groups are associated with each other; Obtaining true labeling information of each positive sample and true labeling information of each negative sample; Alternately inputting the positive samples and the negative samples in the plurality of sample image groups into the convolutional neural network model, and outputting predicted labeling information of each positive sample and predicted labeling information of each negative sample respectively; and The convolutional neural network model is trained according to the real labeling information of the multiple positive samples, the real labeling information of the multiple negative samples, the predicted labeling information of the multiple positive samples and the predicted labeling information of the multiple negative samples to obtain the target identification recognition model.

5. The method according to claim 4, wherein: The obtaining of the true labeling information of each positive sample and the true labeling information of each negative sample includes: Obtaining initial labeling information of each of the positive samples and initial labeling information of each of the negative samples; and Label smoothing is performed on the initial labeling information of the positive sample and the initial labeling information of the negative sample in each of the sample image groups to obtain the true labeling information of the positive sample and the true labeling information of the negative sample in each of the sample image groups.

6. The method according to claim 4, wherein: The method of training the convolutional neural network model according to the real labeling information of the plurality of positive samples, the real labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples and the predicted labeling information of the plurality of negative samples to obtain the target identification model includes: Determine the error labeling information according to the true labeling information of the plurality of positive samples, the true labeling information of the plurality of negative samples, the predicted labeling information of the plurality of positive samples, and the predicted labeling information of the plurality of negative samples; Determining a sample image corresponding to the erroneous labeling information; Determining, according to the sample image corresponding to the erroneous labeling information, a new negative sample and a new positive sample corresponding to the new negative sample; Obtaining true labeling information of the new positive sample and true labeling information of the new negative sample; Alternately inputting the new positive sample and the new negative sample into the convolutional neural network model, and outputting the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample respectively; and The convolutional neural network model is trained according to the true labeling information of the new positive sample, the true labeling information of the new negative sample, the predicted labeling information of the new positive sample and the predicted labeling information of the new negative sample to obtain the target identification recognition model.

7. The method according to claim 1, wherein: The sample image group includes images obtained after at least one of translation, rotation, cropping, non-rigid transformation, noise disturbance and color transformation.

8. An image recognition device, comprising: An acquisition module, used for acquiring an image to be recognized; as well as A processing module is used to process the image to be identified by using a target identification recognition model to obtain a recognition result for characterizing whether the image to be identified includes a target identification, wherein the target identification recognition model is obtained by alternately inputting positive samples and negative samples in multiple sample image groups into a convolutional neural network model and training the convolutional neural network model, wherein each of the sample image groups includes a positive sample and a negative sample, and the positive samples and the negative samples in each of the sample image groups are associated with each other, and the positive samples in each of the sample image groups are images obtained by setting a first identification on an original image, and the negative samples in each of the sample image groups are the original image or an image obtained by setting a second identification on the original image, wherein the first identification and the second identification are associated with each other, and the first identification is used to indicate a to-be-identified identification, and the second identification is used to indicate an interference identification that affects the identification of the first identification.

9. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 7.

11. A computer program product, comprising computer executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed.

Images