Image recognition method, device and equipment

Abstract

The present disclosure relates to an image recognition method, device, and equipment, and relates to the field of image recognition technology. The method includes: obtaining an image to be recognized; identifying the scene category corresponding to the image to be recognized; detecting the facial area contained in the image to be recognized, and identifying facial attribute information corresponding to the facial area; performing object detection on the image to be recognized to obtain the object area in the image to be recognized; if the body area of ​​the target object exists in the object area, and the body area and the facial area correspond to the same target object, then associating the facial attribute information with the target object to obtain target object information; if a non-target object area exists in the object area, then identifying the object category corresponding to the non-target object area to obtain non-target object category information; and outputting scene category information, target object information, and non-target object category information. After obtaining the image to be recognized, the present disclosure can simultaneously obtain multiple attribute information corresponding to the image to be recognized, and has good versatility.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to an image recognition method, apparatus, and device. Background Art

[0002] Image recognition is a crucial step in automated scene analysis, automatically determining the category of objects or scenes contained in an image. However, related image recognition technologies typically input the image to be identified into a deep learning network model related to the object or scene, and then output a recognition result. However, these image recognition results often only contain a single attribute, such as the object or scene. For example, for a single image, only object attributes or scene attributes can be output, resulting in limited versatility. Summary of the Invention

[0003] The present disclosure provides an image recognition method, apparatus, and device to at least address the problem in related technologies that only object attributes can be output for a single image, resulting in poor versatility. The technical solutions of the present disclosure are as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image recognition method, comprising:

[0005] Obtain the image to be recognized;

[0006] Identifying the scene category corresponding to the image to be identified, and obtaining scene category information;

[0007] Detecting a facial region contained in the image to be recognized, and identifying facial attribute information corresponding to the facial region;

[0008] Performing object detection on the image to be identified to obtain an object region in the image to be identified;

[0009] If a body region of a target object exists in the object region, and the body region and the facial region correspond to the same target object, the facial attribute information is associated with the target object and marked to obtain target object information; if a non-target object region exists in the object region, the object category corresponding to the non-target object region is identified to obtain non-target object category information;

[0010] The scene category information, the target object information, and the non-target object category information are output.

[0011] In some embodiments, identifying the scene category corresponding to the image to be identified includes:

[0012] Identify a first scene category corresponding to the image to be identified using a first scene recognition model;

[0013] A second scene recognition model corresponding to the first scene category is used to identify a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

[0014] In some embodiments, detecting a facial region contained in the image to be recognized and identifying facial attribute information corresponding to the facial region includes:

[0015] Detecting facial regions and facial key points in the image to be identified using a facial key point detection model;

[0016] Using a facial recognition model, facial attribute information corresponding to the facial area is identified based on the facial key points.

[0017] In some embodiments, after obtaining the object area of ​​the packet in the image to be identified, the method further includes:

[0018] Using a first object recognition model, respectively identifying first object categories corresponding to the object areas, the first object categories including target object categories and non-target object categories;

[0019] If a non-target object area exists in the object area, identifying an object category corresponding to the non-target object area includes:

[0020] If it is determined that the object area includes a non-target object area according to the first object categories corresponding to the object areas, a second object recognition model is used to identify a second object category corresponding to the non-target object area, where the second object category is a subcategory corresponding to the non-target object category.

[0021] In some embodiments, the steps of training the object recognition model used in different recognition processes include:

[0022] Obtaining a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image;

[0023] Obtaining multiple teacher models pre-trained using the training sample images;

[0024] Using the target recognition model as a student model, inputting the training sample images into the multiple teacher models and the target recognition model respectively, obtaining classification prediction vectors output by the multiple teacher models and the classification prediction vector output by the target recognition model;

[0025] Determine a mean vector corresponding to the supervision vector and the classification prediction vectors outputted by the plurality of teacher models respectively;

[0026] Determining a first KL divergence loss between the mean vector and the classification prediction vector output by the object recognition model;

[0027] Inputting the mean vector and the classification prediction vector output by the target recognition model into a preset discriminator model, and determining the discrimination loss corresponding to the discriminator model according to the output result of the discriminator model;

[0028] Adjusting parameters of the object recognition model according to the first KL divergence loss and the discrimination loss.

[0029] In some embodiments, the target recognition model training step further includes:

[0030] Performing different data enhancement processing on the training sample images to obtain a first training sample image and a second training sample image;

[0031] Inputting the first training sample image into the target recognition model to obtain a first classification prediction result, and inputting the second training sample image into the target recognition model to obtain a second classification prediction result;

[0032] Determining a first cross entropy loss based on the first classification prediction result and the supervisory information corresponding to the first training sample image; determining a second cross entropy loss based on the second classification prediction result and the supervisory information corresponding to the second training sample image;

[0033] determining feature similarity losses between convolutional layers of the same level of the target recognition model after the first training sample image and the second training sample image are respectively input into the target recognition model;

[0034] Determine a second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and determine a third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model;

[0035] According to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss, the parameters of the target recognition model are adjusted to obtain a final target recognition model.

[0036] In some embodiments, determining the first classification prediction result as the output result of the teacher model and the second classification prediction result as the output result of the student model when the second KL divergence loss is used, and determining the first classification prediction result as the output result of the student model and the third KL divergence loss when the second classification prediction result is used as the output result of the teacher model includes:

[0037] When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0038] Determine the second KL divergence loss according to the first weight coefficient, the first classification prediction result, and the second classification prediction result;

[0039] When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0040] The third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

[0041] In some embodiments, the process of determining the first weight coefficient and the second weight coefficient includes:

[0042] If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero;

[0043] If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one;

[0044] If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

[0045] In some embodiments, the target recognition model training step further includes:

[0046] Acquire an image segmentation model, wherein the image segmentation model is used to distinguish the foreground and background of the training sample image;

[0047] Inputting the training sample image into the target recognition model to obtain a classification prediction vector output by the target recognition model;

[0048] Determining a third cross entropy loss corresponding to the target recognition model according to the classification prediction vector and the supervision vector;

[0049] Inputting the feature maps extracted by different convolutional layers in the target recognition model into different upsampling layers in the image segmentation model accordingly, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by the different upsampling layers, and determining the segmentation loss corresponding to the image segmentation model according to the segmentation processing result;

[0050] Parameters of the object recognition model are adjusted according to the third cross entropy loss and the segmentation loss.

[0051] According to a second aspect of an embodiment of the present disclosure, there is provided an image recognition device, including:

[0052] An acquisition unit, configured to acquire an image to be recognized;

[0053] a first recognition unit configured to identify a scene category corresponding to the image to be recognized and obtain scene category information;

[0054] A first detection unit is configured to detect a facial region contained in the image to be recognized and identify facial attribute information corresponding to the facial region;

[0055] a second detection unit, configured to perform object detection on the image to be identified, and obtain an object region in the image to be identified;

[0056] The judgment unit is configured to: if there is a body region of a target object in the object region, and the body region and the facial region correspond to the same target object, associate the facial attribute information with the target object to obtain target object information; if there is a non-target object region in the object region, identify the object category corresponding to the non-target object region to obtain non-target object category information;

[0057] An output unit is configured to output the scene category information, the target object information, and the non-target object category information.

[0058] In some embodiments, the first recognition unit is further configured to perform:

[0059] Identify a first scene category corresponding to the image to be identified using a first scene recognition model;

[0060] A second scene recognition model corresponding to the first scene category is used to identify a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

[0061] In some embodiments, the first detection unit is further configured to perform:

[0062] Detecting facial regions and facial key points in the image to be identified using a facial key point detection model;

[0063] Using a facial recognition model, facial attribute information corresponding to the facial area is identified based on the facial key points.

[0064] In some embodiments, the apparatus further comprises:

[0065] A second recognition unit is configured to use a first object recognition model to respectively identify first object categories corresponding to the object areas, where the first object categories include target object categories and non-target object categories;

[0066] The third recognition unit is configured to execute, if it is determined that the object area includes a non-target object area based on the first object categories corresponding to the object areas, then use a second object recognition model to identify a second object category corresponding to the non-target object area, where the second object category is a subcategory corresponding to the non-target object category.

[0067] In some embodiments, the system further includes: a training unit configured to train target recognition models used in different recognition processes, wherein the training steps of the target recognition models used in different recognition processes include:

[0068] Obtaining a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image;

[0069] Obtaining multiple teacher models pre-trained using the training sample images;

[0070] Using the target recognition model as a student model, inputting the training sample images into the multiple teacher models and the target recognition model respectively, obtaining classification prediction vectors output by the multiple teacher models and the classification prediction vector output by the target recognition model;

[0071] Determine a mean vector corresponding to the supervision vector and the classification prediction vectors outputted by the plurality of teacher models respectively;

[0072] Determining a first KL divergence loss between the mean vector and the classification prediction vector output by the object recognition model;

[0073] Inputting the mean vector and the classification prediction vector output by the target recognition model into a preset discriminator model, and determining the discrimination loss corresponding to the discriminator model according to the output result of the discriminator model;

[0074] Adjusting parameters of the object recognition model according to the first KL divergence loss and the discrimination loss.

[0075] In some embodiments, the training unit is further configured to perform:

[0076] Performing different data enhancement processing on the training sample images to obtain a first training sample image and a second training sample image;

[0077] Inputting the first training sample image into the target recognition model to obtain a first classification prediction result, and inputting the second training sample image into the target recognition model to obtain a second classification prediction result;

[0078] Determining a first cross entropy loss based on the first classification prediction result and the supervisory information corresponding to the first training sample image; determining a second cross entropy loss based on the second classification prediction result and the supervisory information corresponding to the second training sample image;

[0079] determining feature similarity losses between convolutional layers of the same level of the target recognition model after the first training sample image and the second training sample image are respectively input into the target recognition model;

[0080] Determine a second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and determine a third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model;

[0081] According to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss, the parameters of the target recognition model are adjusted to obtain a final target recognition model.

[0082] In some embodiments, the training unit is further configured to perform:

[0083] When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0084] Determine the second KL divergence loss according to the first weight coefficient, the first classification prediction result, and the second classification prediction result;

[0085] When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0086] The third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

[0087] In some embodiments, the training unit is further configured to perform:

[0088] If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero;

[0089] If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one;

[0090] If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

[0091] In some embodiments, the training unit is further configured to perform:

[0092] Acquire an image segmentation model, wherein the image segmentation model is used to distinguish the foreground and background of the training sample image;

[0093] Inputting the training sample image into the target recognition model to obtain a classification prediction vector output by the target recognition model;

[0094] Determining a third cross entropy loss corresponding to the target recognition model according to the classification prediction vector and the supervision vector;

[0095] Inputting the feature maps extracted by different convolutional layers in the target recognition model into different upsampling layers in the image segmentation model accordingly, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by the different upsampling layers, and determining the segmentation loss corresponding to the image segmentation model according to the segmentation processing result;

[0096] Parameters of the object recognition model are adjusted according to the third cross entropy loss and the segmentation loss.

[0097] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory and a processor; wherein executable code is stored on the memory, and when the executable code is executed by the processor, the processor executes the above-mentioned image recognition method.

[0098] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned image recognition method.

[0099] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above-mentioned image recognition method when executed by a processor.

[0100] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0101] The embodiments of the present disclosure can be applied to mobile terminals. After acquiring the image to be identified, the scene category corresponding to the image to be identified can be identified to obtain scene category information. And by detecting the facial area contained in the image to be identified, the facial attribute information corresponding to the facial area can be identified. Then, by performing object detection on the image to be identified, the object area in the image to be identified can be obtained. If the body area of ​​the target object is identified in the object area, and the body area and the facial area correspond to the same target object, the facial attribute information is associated with the target object to obtain target object information. If a non-target object area is identified in the object area, the object category corresponding to the non-target object area is identified to obtain non-target object category information. It can be seen that after acquiring the image to be identified, the present disclosure can simultaneously obtain multiple attribute information corresponding to the image to be identified, specifically including: scene category information, target object information and non-target object category information, and has good versatility.

[0102] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0104] Figure 1 is a flow chart showing an image recognition method according to an exemplary embodiment;

[0105] Figure 2 is a flowchart of the first training of the target recognition model according to an exemplary embodiment;

[0106] Figure 3 is a flowchart of the second training of the target recognition model according to an exemplary embodiment;

[0107] Figure 4 is a flowchart of the third training of the target recognition model according to an exemplary embodiment;

[0108] Figure 5 is a schematic diagram illustrating an application of a first training of an object recognition model according to an exemplary embodiment;

[0109] Figure 6 is a schematic diagram showing an application of the second training of the target recognition model according to an exemplary embodiment;

[0110] Figure 7 is a schematic diagram showing an application of the third training of the target recognition model according to an exemplary embodiment;

[0111] Figure 8 is a schematic diagram illustrating an application of an image recognition method according to an exemplary embodiment;

[0112] Figure 9 is a block diagram of an image recognition device according to an exemplary embodiment;

[0113] Figure 10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0114] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0115] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0116] Image recognition is a crucial step in automated scene analysis, automatically determining the category of objects or scenes contained in an image. However, related image recognition technologies typically input the image to be identified into a deep learning network model related to the object or scene, and then output a recognition result. However, these image recognition results often only contain a single attribute, such as the object or scene. For example, for a single image, only object attributes or scene attributes can be output, resulting in limited versatility.

[0117] In view of this, the image recognition method provided by the embodiment of the present disclosure can identify the scene category corresponding to the image to be recognized after obtaining the image to be recognized, and obtain scene category information. And by detecting the facial area contained in the image to be recognized, the facial attribute information corresponding to the facial area can be identified, and then by performing object detection on the image to be recognized, the object area in the image to be recognized can be obtained. If the body area of ​​the target object is recognized in the object area, and the body area and the facial area correspond to the same target object, the facial attribute information is associated with the target object to obtain target object information. If a non-target object area is recognized in the object area, the object category corresponding to the non-target object area is identified to obtain non-target object category information. It can be seen that after obtaining the image to be recognized, the present disclosure can simultaneously obtain multiple attribute information corresponding to the image to be recognized, specifically including: scene category information, target object information and non-target object category information, and has good versatility.

[0118] It should be noted that the image recognition method provided in the embodiments of the present disclosure can be applied to mobile terminals, such as tablets, laptops, and smartphones. To ensure that the image recognition method provided in the embodiments of the present disclosure can be successfully applied to mobile terminals while maintaining a small storage footprint and real-time performance on mobile terminals, the target recognition model used in the various recognition processes described below in the embodiments of the present disclosure can be a model trained on a convolutional neural network that employs depthwise separable convolution. Specifically, for example, the model can be a MobileNet model. MobileNet is a model based on depthwise separable convolution, which decomposes a standard convolution into a depthwise convolution and a 1x1 convolution, or pointwise convolution. For MobileNet, depthwise convolution applies a single filter to each input channel, followed by a pointwise convolution operation using a 1x1 convolution to combine the outputs of all depthwise convolutions. Standard convolution combines all inputs in a single step to produce a new set of outputs. Depthwise separable convolution breaks this down into two steps: filtering each individual layer and then combining them. This decomposition can significantly reduce computational complexity and model size.

[0119] Figure 1 A flowchart of an image recognition method provided in an embodiment of the present disclosure is applied to a mobile terminal, such as Figure 1 As shown, the method includes:

[0120] In step S11, an image to be recognized is obtained.

[0121] In the disclosed embodiment, the image to be identified may be a picture taken of a person, object, or scenery, or a picture searched on the Internet, or a multi-frame image obtained by framing a video, without limitation.

[0122] In step S12, the scene category corresponding to the image to be identified is identified to obtain scene category information.

[0123] In the embodiment of the present disclosure, a scene recognition model is used to identify the scene category corresponding to the image to be recognized. The specific steps are as follows:

[0124] Step S1211: Use a first scene recognition model to identify a first scene category corresponding to the image to be recognized.

[0125] Step S1212: Use a second scene recognition model corresponding to the first scene category to identify a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

[0126] In practical applications, the first scene category may include indoor, outdoor, etc., and the second scene category is a subcategory of the first scene category. Specifically, if the recognition result of the first scene category is indoor, the second scene category may be classroom, office, bedroom, etc. If the recognition result of the first scene category is outdoor, the second scene category may be natural scenery, architecture, etc.

[0127] By using the first scene recognition model and the second scene recognition model to perform secondary recognition on the image to be recognized, the scene category corresponding to the image to be recognized can be accurately identified.

[0128] In step S13, the facial region contained in the image to be recognized is detected, and facial attribute information corresponding to the facial region is recognized.

[0129] In the embodiment of the present disclosure, in order to accurately detect the facial region contained in the image to be recognized and identify the facial attribute information corresponding to the facial region, a facial landmark detection model and a facial recognition model are used. The specific steps are as follows:

[0130] Step S1311: Use a facial key point detection model to detect facial areas and facial key points contained in the image to be recognized.

[0131] Step S1312: Using the facial recognition model, identify facial attribute information corresponding to the facial area based on the facial key points.

[0132] In practical applications, the facial landmark detection model is used to detect the facial locations and landmarks contained in the current input image. These facial locations and landmarks are then used as input to the facial recognition model, which outputs the predicted facial attribute information corresponding to the facial region. Facial landmarks include eyes, nose, and mouth, while facial attribute information includes gender and age.

[0133] In step S14, object detection is performed on the image to be identified to obtain the object region in the image to be identified.

[0134] In the embodiments of the present disclosure, object detection refers to object detection. In practical applications, a target detection model can be used to perform object detection on the image to be identified to obtain the object regions contained in the image to be identified. Specifically, object detection can be performed on the image to be identified based on the SSD (Single Shot MultiBox Detector) algorithm. To ensure that the embodiments of the present disclosure can be successfully applied to mobile terminals, the object detection model can also be a MobileNet model.

[0135] In specific implementation, for example, the image to be identified is preprocessed into a pixel matrix and then input into the target detection model. In the intermediate convolutional layer of the target detection model, n (n can be 4, 5, 6, etc.) feature maps of different scales are extracted, and the feature maps of the n feature maps are summed. The summation result is then input into the SENet structure border prediction network (this border prediction network has only one simple convolutional layer, and the SENet structure can improve the prediction accuracy of the border prediction network), and four coordinate values ​​(center x offset value, center y offset value, width offset value, height offset value) are output to obtain the four-point coordinates of the frame corresponding to the object area. Similarly, the summation result is input into the SE structure classification network (two-classification model), and 0 or 1 is output (0 means it is not an object, 1 means it is an object), to determine whether there is an object in the object area.

[0136] In step S15, if there is a body area of ​​the target object in the object area, and the body area and the facial area correspond to the same target object, the facial attribute information is associated with the target object and marked to obtain the target object information; if there is a non-target object area in the object area, the object category corresponding to the non-target object area is identified to obtain the non-target object category information.

[0137] In step S16, scene category information, target object information, and non-target object category information are output.

[0138] In order to accurately obtain the specific category corresponding to the object region, after obtaining the object region in the image to be identified, the method provided by the embodiment of the present disclosure further includes:

[0139] Using a first object recognition model, respectively identifying first object categories corresponding to the object regions, the first object categories including target object categories and non-target object categories;

[0140] If there is a non-target object area in the object area, the object category corresponding to the non-target object area is identified, including:

[0141] If the object regions include non-target object regions according to the first object categories corresponding to the object regions, a second object recognition model is used to identify a second object category corresponding to the non-target object regions, where the second object category is a subcategory of the non-target object category.

[0142] In practical applications, after obtaining each object region contained in the image to be identified, the first object recognition model will be used to identify whether each object region is a target object category or a non-target object category (such as a human category or a non-human category).

[0143] If the body area of ​​the target object is identified in the object area, and the body area and the facial area correspond to the same target object, the facial attribute information is associated with the target object and marked to perform identity matching of the facial and body areas, and the facial attribute recognition result is assigned to the target object, that is, the complete person in the image to be identified is found, and the target object information is output.

[0144] If it is a non-target object category, the specific object category of the object area is identified through the second object recognition model. For example, an image is input into the first object recognition model, and the output structure is the target object (human body) category, then the process ends directly. If the output result is a non-target object category such as an animal or a digital product, the image is further input into the second object recognition model. If the result output by the first object recognition model is an animal, the specific category of animals such as cats, dogs, and swans can be output through the second object recognition model. If the previous output result is a digital product, the second object recognition model can output mobile phones, tablet computers, etc. Other categories are not listed here one by one.

[0145] It should be noted here that if the same image is input into the second object recognition model and the second scene recognition model respectively, and both generate recognition results, its specific output category can be determined according to the preset classification threshold. For example, when an image is input into the second scene recognition model, the output recognition result is a classroom, and when the image is input into the second object recognition model, the output recognition result is a table. Then, it is determined whether the recognition result threshold output by the second object recognition model is greater than the preset classification threshold. If it is greater, the output recognition result is a table, and the scene recognition result is ignored. Otherwise, the output recognition result is a classroom, and the object recognition result is ignored. In order to facilitate understanding of the threshold, the following example further illustrates the above situation: If an image shows a table in a classroom, but this table already occupies 80% of the entire pixel area of ​​the image, then it is obviously unreasonable to output the scene recognition result (classroom) at this time. Therefore, the scene recognition result is ignored at this time, and the object recognition result (table) is directly output.

[0146] The following describes the training steps of the target recognition model used in the above different recognition processes, such as Figure 2 and Figure 5 As shown, the details are as follows:

[0147] In step S21, a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image are obtained.

[0148] In the embodiment of the present disclosure, the target recognition model can be the above-mentioned first scene recognition model, second scene recognition model, facial recognition model, first object recognition model, second object recognition model, etc. The training sample image is an image of the object area in the image to be recognized, which can be a sample image with a secondary label. Specifically, due to the large recognition category system, if the secondary category of the target in the image is directly marked, it is difficult to find the corresponding category that meets the requirements. Therefore, the present disclosure performs two-step labeling:

[0149] First, the objects in the image are annotated at the first level. For example, if the image is a photo of a cat, the cat in the image is labeled as "animal". The specific annotation methods can be classification annotation and frame annotation.

[0150] Then, the second-level annotation is performed based on the first-level annotation results. For example, if the first-level annotation result is "animal", the second-level annotation result is now "cat". At the same time, during the second-level annotation process, it is also possible to further check whether there are any annotation errors in the first-level annotation.

[0151] In step S22, a plurality of teacher models pre-trained using training sample images are obtained.

[0152] In the disclosed embodiment, the training process of multiple teacher models is the same as the training process of the target recognition model, and is also trained based on training sample images and supervision vectors corresponding to the training sample images.

[0153] In step S23, the target recognition model is used as the student model, and the training sample images are input into multiple teacher models and the target recognition model respectively to obtain the classification prediction vectors output by the multiple teacher models and the classification prediction vector output by the target recognition model.

[0154] In step S24, the mean vector corresponding to the supervision vector and the classification prediction vectors output by the multiple teacher models is determined.

[0155] For example, let's assume we need to identify three categories: cat, dog, and snake. If the current image is of a dog, the dog classification label is 1, and the supervision vector is 010. If the current image is of a cat, the cat classification label is 1, and the supervision vector is 100. If the current image is of a snake, the snake classification label is 1, and the supervision vector is 001. After determining the supervision vector, we sum the supervision vector with the classification prediction vectors output by multiple teacher models and take the mean to obtain the mean vector.

[0156] In step S25 , a first KL divergence loss between the mean vector and the classification prediction vector output by the target recognition model is determined.

[0157] It should be understood that the teacher model is used to guide the training of the student model, and a loss will occur between the mean vector obtained based on the teacher model and the classification prediction vector obtained based on the student model (target recognition model), that is, the first KL divergence loss in the embodiment of the present disclosure.

[0158] In step S26, the mean vector and the classification prediction vector output by the target recognition model are input into a preset discriminator model, and the discrimination loss corresponding to the discriminator model is determined according to the output result of the discriminator model.

[0159] In the disclosed embodiment, the role of the discriminator is to distinguish the classification prediction vectors output by the teacher model and the student model. After the mean vector and the classification prediction vector output by the target recognition model are input into the discriminator model, the discriminator model will compare the mean vector and the classification prediction vector output by the target recognition model with a pre-set threshold. If it is higher than this threshold, it is determined that the vector is output by the teacher model, and if it is lower than this threshold, it is determined that the vector is output by the student model. Specifically, the teacher model and the student model can be represented by labels 0 and 1, and the output 1 represents the teacher model, and the output 0 represents the student model. The discriminant loss corresponding to the discriminant model can be determined based on the output result. By determining the discriminant loss, the outputs of the teacher model and the student model can be made clearly separable, thereby ensuring the smooth progress of subsequent student model training.

[0160] In step S27, the parameters of the target recognition model are adjusted according to the first KL divergence loss and the discrimination loss.

[0161] After obtaining the first KL divergence loss and the discriminant loss, the parameters of the target recognition model can be updated according to the first KL divergence loss and the discriminant loss, thereby completing the training of the target recognition model.

[0162] In order to further optimize the training of the target recognition model, such as Figure 3 and Figure 6 As shown, the training steps of the target recognition model also include:

[0163] In step S31, an image segmentation model is obtained, where the image segmentation model is used to distinguish the foreground and background of a training sample image.

[0164] In step S32, the training sample image is input into the target recognition model to obtain a classification prediction vector output by the target recognition model.

[0165] In step S33, a third cross entropy loss corresponding to the target recognition model is determined according to the classification prediction vector and the supervision vector.

[0166] In step S34, the feature maps extracted by different convolutional layers in the target recognition model are correspondingly input into different upsampling layers in the image segmentation model, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by different upsampling layers, and determines the segmentation loss corresponding to the image segmentation model according to the segmentation processing results.

[0167] In step S35 , the parameters of the object recognition model are adjusted according to the third cross entropy loss and the segmentation loss.

[0168] In the disclosed embodiment, the essential function of the image segmentation model is binary classification, which is used to distinguish the foreground and background of the training sample image. Its output determines its corresponding loss. After determining its corresponding loss, it is gradient-backed together with the third cross entropy loss obtained by training the target recognition model through the classification prediction vector and the supervision vector to update the parameters in the target recognition model and realize the training of the target recognition model. Specifically, the different convolutional layers in the target recognition model (see Figure 6 The feature maps extracted by Conv1-Conv5 in the image segmentation model are correspondingly input into different upsampling layers (see Figure 6The image segmentation model uses the feature maps generated by the different upsampling layers (UpConv2-UpConv4) to segment the training sample images, identifying features corresponding to foreground objects that are more important for classification prediction. In summary, setting up the segmentation model helps the student model (object recognition model) identify features corresponding to foreground objects that are more important for classification prediction, while reducing the influence of background.

[0169] In order to successfully obtain the final target recognition model after completing the training and optimization of the target recognition model, Figure 4 and Figure 7 As shown, the training steps of the target recognition model also include:

[0170] In step S41 , different data enhancement processes are performed on the training sample images to obtain a first training sample image and a second training sample image.

[0171] In the embodiment of the present disclosure, data enhancement processing refers to performing processing such as rotation, translation, and color change on the training sample image.

[0172] In step S42, the first training sample image is input into the target recognition model to obtain a first classification prediction result, and the second training sample image is input into the target recognition model to obtain a second classification prediction result.

[0173] In practical applications, see Figure 7 After obtaining the first training sample image and the second training sample image, the first training sample image and the second training sample image are successively input into the same target recognition model to obtain the first classification prediction result and the second classification prediction result respectively. It should be noted that Figure 7 The two target recognition models are shown in the figure only for the convenience of understanding. In fact, the first training sample image and the second training sample image are input into the same target recognition model one after another.

[0174] In step S43, the first cross entropy loss is determined according to the first classification prediction result and the supervision information corresponding to the first training sample image, that is, Figure 7 The CE loss of the final output of the first training sample image in . According to the second classification prediction result and the supervision information corresponding to the second training sample image, the second cross entropy loss is determined, that is, Figure 7 The CE loss of the final output of the second training sample image.

[0175] It should be understood that there will be a certain loss between the first classification prediction result and the supervisory information corresponding to the first training sample image, that is, the first cross-entropy loss in the embodiment of the present disclosure. Similarly, there will be a certain loss between the second classification prediction result and the supervisory information corresponding to the second training sample image, that is, the second cross-entropy loss in the embodiment of the present disclosure. The calculation formulas for the first cross-entropy loss and the second cross-entropy loss are as follows:

[0176]

[0177] Among them, loss ce is the first cross entropy loss or the second cross entropy loss, is the first classification prediction result or the second classification prediction result, y i It is the supervisory information corresponding to the first training sample image or the supervisory information corresponding to the second training sample image.

[0178] In step S44, the feature similarity loss between the convolution layers of the same level in the target recognition model after the first training sample image and the second training sample image are respectively input into the target recognition model is determined. The convolution layers in the target recognition model can be found in Figure 7 Conv1-Conv5 in.

[0179] In practical applications, the calculation formula for the feature similarity loss between the same-level convolutional layers of the target recognition model is as follows:

[0180]

[0181] Among them, loss ssim is the feature similarity loss, x and y represent the feature values ​​corresponding to the two same-level convolutional layers, σ is the feature mean corresponding to the convolutional layer, and μ is the feature standard deviation corresponding to the convolutional layer.

[0182] In step S45, the second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model is determined, that is, Figure 7 KLloss(outA, outB) in, and the third KL divergence loss when determining the first classification prediction result as the output result of the student model and the second classification prediction result as the output result of the teacher model, that is, Figure 7 KLloss(outB, outA) in.

[0183] In the embodiment of the present disclosure, the target recognition model for the first training sample image and the second training sample image input is a student model, but during training, the first classification prediction result is regarded as the output result of the teacher model, and the second classification prediction result is regarded as the output result of the student model to obtain a second KL divergence loss, or the first classification prediction result is regarded as the output result of the student model, and the second classification prediction result is regarded as the output result of the teacher model to obtain a third KL divergence loss.

[0184] The specific calculation process of the second KL divergence loss and the third KL divergence loss is as follows:

[0185] When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0186] A second KL divergence loss is determined according to the first weight coefficient, the first classification prediction result, and the second classification prediction result.

[0187] When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0188] A third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

[0189] Specifically, the calculation formula of the second KL divergence loss or the third KL divergence loss is as follows:

[0190]

[0191] Among them, p(Xi) is the output result of the teacher model, q(Xi) is the output result of the student model, and weight is the weight coefficient.

[0192] The specific process of determining the first weight coefficient and the second weight coefficient is as follows:

[0193] If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero;

[0194] If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one;

[0195] If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

[0196] The set value greater than one may be 1.5, 2, etc., which is not limited here.

[0197] In step S46, the parameters of the target recognition model are adjusted according to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss to obtain the final target recognition model.

[0198] In practical applications, after obtaining the above-mentioned first cross entropy loss, second cross entropy loss, feature similarity loss, second KL divergence loss and third KL divergence loss, the parameters of the target recognition model can be updated according to the first cross entropy loss, second cross entropy loss, feature similarity loss, second KL divergence loss and third KL divergence loss, thereby completing the training of the target recognition model and obtaining the final target recognition model. It should be noted that if there are n pairs of training samples, then after all training samples are tested, the total KL divergence loss (composed of n KL p-q and n KL q-p Add them together), and the total feature similarity loss and the total cross entropy loss, then add the total KL divergence loss, the total feature similarity loss and the total cross entropy loss, and return the gradient.

[0199] The following is an exemplary description of the solution of the present disclosure based on a specific application scenario:

[0200] like Figure 8 As shown, the image to be recognized is input into the facial key point detection model, the object detection model and the first scene recognition model respectively.

[0201] Among them, the facial key point detection model will output the facial area and facial key points contained in the image to be identified. Through the facial recognition model, the facial attribute information corresponding to the facial area is identified based on the facial key points.

[0202] The object detection model will output the object regions contained in the image to be identified. Since the detection accuracy of general object detection models is not high, the image to be identified is input into a binary classifier to further determine whether there is actually an object in the object regions contained in the image to be identified. If not, the object recognition operation ends. If it does, the object region in the image to be identified is extracted (i.e., cutout). Subsequently, the extracted object region is input into the first object recognition model for primary recognition, and the first object category is output, i.e., whether the object is a human category or a non-human category. If it is a human category, the object region (in this case, the human region) is matched with the facial region detected by the facial key point model. If it is the same person, the facial attribute information corresponding to the facial region is associated with the human region. If it is a non-human category, the extracted object region image is input into the second object recognition model for secondary recognition, and the second object category corresponding to the object region (in this case, the non-human region) is output. This second object category is a subcategory of the non-human category.

[0203] The first scene recognition model outputs a first scene category corresponding to the image to be recognized, and then inputs the image to be recognized into a second scene recognition model corresponding to the first scene recognition model, outputting a second scene category, which is a subcategory of the first scene category.

[0204] In summary, after acquiring an image to be recognized, the present disclosure can identify the scene category corresponding to the image to be recognized and obtain scene category information. Furthermore, by detecting the facial region contained in the image to be recognized, facial attribute information corresponding to the facial region can be identified. Object detection can then be performed on the image to be recognized to obtain the object region in the image to be recognized. If a body region of a target object is identified within the object region, and the body region and the facial region correspond to the same target object, the facial attribute information is associated with the target object and labeled to obtain target object information. If a non-target object region is identified within the object region, the object category corresponding to the non-target object region is identified to obtain non-target object category information. Thus, after acquiring an image to be recognized, the present disclosure can simultaneously obtain multiple attribute information corresponding to the image to be recognized, specifically including scene category information, target object information, and non-target object category information, thus achieving high versatility. Furthermore, the present disclosure can also be applied to mobile terminals, offering wide applicability. For large companies with large data processing volumes, this solves the problem of requiring image recognition to be uploaded to a server, which infringes on user privacy. For ordinary users, it allows for quick access without lengthy waiting times.

[0205] Figure 9 FIG. 1 is a block diagram of an image recognition device according to an exemplary embodiment. Figure 9The device includes an acquisition unit 901, a first recognition unit 902, a first detection unit 903, a second detection unit 904, a judgment unit 905 and an output unit 906.

[0206] The acquisition unit 901 is configured to acquire an image to be recognized;

[0207] The first recognition unit 902 is configured to identify the scene category corresponding to the image to be recognized and obtain scene category information;

[0208] The first detection unit 903 is configured to detect a facial region contained in the image to be recognized and identify facial attribute information corresponding to the facial region;

[0209] The second detection unit 904 is configured to perform object detection on the image to be identified, and obtain an object region in the image to be identified;

[0210] The judgment unit 905 is configured to perform, if there is a body region of the target object in the object region, and the body region and the facial region correspond to the same target object, then associate the facial attribute information with the target object and mark it, thereby obtaining target object information; if there is a non-target object region in the object region, then identify the object category corresponding to the non-target object region, thereby obtaining non-target object category information;

[0211] The output unit 906 is configured to output scene category information, target object information, and non-target object category information.

[0212] In an exemplary embodiment of the present disclosure, the first identification unit 902 is further configured to perform:

[0213] Identify a first scene category corresponding to the image to be identified using a first scene recognition model;

[0214] A second scene recognition model corresponding to the first scene category is used to recognize a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

[0215] In an exemplary embodiment of the present disclosure, the first detection unit 903 is further configured to perform:

[0216] Use the facial key point detection model to detect the facial area and facial key points contained in the image to be recognized;

[0217] Using the facial recognition model, the facial attribute information corresponding to the facial area is identified based on the facial key points.

[0218] In an exemplary embodiment of the present disclosure, the apparatus further includes:

[0219] A second recognition unit is configured to use the first object recognition model to respectively identify first object categories corresponding to the object areas, where the first object categories include target object categories and non-target object categories;

[0220] The third recognition unit is configured to execute, if it is determined that the object area includes a non-target object area according to the first object categories corresponding to the object areas, then use the second object recognition model to identify the second object category corresponding to the non-target object area, where the second object category is a subcategory corresponding to the non-target object category.

[0221] In an exemplary embodiment of the present disclosure, the apparatus further includes: a training unit configured to perform training on target recognition models used in different recognition processes, wherein the training steps of the target recognition models used in different recognition processes include:

[0222] Obtaining a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image;

[0223] Get multiple teacher models pre-trained using training sample images;

[0224] The target recognition model is used as the student model, and the training sample images are input into multiple teacher models and the target recognition model respectively, to obtain the classification prediction vectors output by the multiple teacher models and the classification prediction vector output by the target recognition model;

[0225] Determine the mean vector corresponding to the supervision vector and the classification prediction vectors output by multiple teacher models;

[0226] Determine the first KL divergence loss between the mean vector and the classification prediction vector output by the target recognition model;

[0227] Input the mean vector and the classification prediction vector output by the target recognition model into the preset discriminator model, and determine the discrimination loss corresponding to the discriminator model according to the output result of the discriminator model;

[0228] According to the first KL divergence loss and the discriminant loss, the parameters of the target recognition model are adjusted.

[0229] In an exemplary embodiment of the present disclosure, the training unit is further configured to perform:

[0230] Performing different data enhancement processing on the training sample images to obtain a first training sample image and a second training sample image;

[0231] Inputting the first training sample image into the target recognition model to obtain a first classification prediction result, and inputting the second training sample image into the target recognition model to obtain a second classification prediction result;

[0232] Determine a first cross entropy loss based on the first classification prediction result and the supervisory information corresponding to the first training sample image; determine a second cross entropy loss based on the second classification prediction result and the supervisory information corresponding to the second training sample image;

[0233] Determining the feature similarity loss between convolutional layers of the same level of the target recognition model after the first training sample image and the second training sample image are respectively input into the target recognition model;

[0234] Determine a second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and determine a third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model;

[0235] According to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss, the parameters of the target recognition model are adjusted to obtain the final target recognition model.

[0236] In an exemplary embodiment of the present disclosure, the training unit is further configured to perform:

[0237] When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0238] Determine a second KL divergence loss according to the first weight coefficient, the first classification prediction result, and the second classification prediction result;

[0239] When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result;

[0240] A third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

[0241] In an exemplary embodiment of the present disclosure, the training unit is further configured to perform:

[0242] If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero;

[0243] If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one;

[0244] If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

[0245] In an exemplary embodiment of the present disclosure, the training unit is further configured to perform:

[0246] Obtain an image segmentation model, which is used to distinguish the foreground and background of a training sample image;

[0247] Input the training sample image into the target recognition model to obtain the classification prediction vector output by the target recognition model;

[0248] Determine the third cross entropy loss corresponding to the target recognition model based on the classification prediction vector and the supervision vector;

[0249] The feature maps extracted by different convolutional layers in the target recognition model are correspondingly input into different upsampling layers in the image segmentation model, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by different upsampling layers, and determines the segmentation loss corresponding to the image segmentation model according to the segmentation processing results;

[0250] The parameters of the object recognition model are adjusted based on the third cross entropy loss and the segmentation loss.

[0251] In an exemplary embodiment of the present disclosure, the above Figure 9 The structure of the image recognition device shown can be implemented as an electronic device, such as Figure 10 As shown, the electronic device may include: a processor 1001, a memory 1002, and a display 1003; wherein the memory 1002 stores an executable code, and when the executable code is executed by the processor 1001, the processor 1001 executes the above-mentioned image recognition method. The processor 1001 may also execute other related steps in the image recognition method provided in the aforementioned related embodiments, which are not repeated here. Optionally, the storage medium may be a non-temporary storage medium, for example, the non-temporary storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0252] In an exemplary embodiment, the present disclosure further provides a computer-readable storage medium including instructions, such as a memory including instructions, wherein the instructions can be executed by the processor 1001 of the electronic device to perform the above-mentioned image recognition method. Optionally, the storage medium can be a non-transitory storage medium, for example, a non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The present disclosure further provides a computer program product, including a computer program, which implements the above-mentioned image recognition method when executed by the processor 1001.

[0253] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0254] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image recognition method, characterized in that: Applied to a mobile terminal, the method includes: Obtain the image to be recognized; Identifying the scene category corresponding to the image to be identified, and obtaining scene category information; Detecting a facial region contained in the image to be recognized, and identifying facial attribute information corresponding to the facial region; Performing object detection on the image to be identified to obtain an object region in the image to be identified; If a body region of a target object exists in the object region, and the body region and the facial region correspond to the same target object, the facial attribute information is associated with the target object and marked to obtain target object information; if a non-target object region exists in the object region, the object category corresponding to the non-target object region is identified to obtain non-target object category information; outputting the scene category information, the target object information, and the non-target object category information; The training steps of the target recognition model used in different recognition processing include: obtaining a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image; using the target recognition model as a student model, inputting the training sample image into a plurality of teacher models pre-trained using the training sample image and the target recognition model respectively, and obtaining classification prediction vectors output by the plurality of teacher models and the classification prediction vector output by the target recognition model; determining the mean vector corresponding to the supervision vector and the classification prediction vectors output by the plurality of teacher models respectively; determining the first KL divergence loss between the mean vector and the classification prediction vector output by the target recognition model; inputting the mean vector and the classification prediction vector output by the target recognition model into a discriminator model to determine the discrimination loss corresponding to the discriminator model; adjusting the parameters of the target recognition model according to the first KL divergence loss and the discrimination loss; Perform different data enhancement processing on the training sample image to obtain a first training sample image and a second training sample image; determine a first cross entropy loss based on a first classification prediction result obtained after the first training sample image is input into the target recognition model and the supervisory information corresponding to the first training sample image; determine a second cross entropy loss based on a second classification prediction result obtained after the second training sample image is input into the target recognition model and the supervisory information corresponding to the second training sample image; determine the feature similarity loss between each convolutional layer of the same level after the first training sample image and the second training sample image are respectively input into the target recognition model; determine a second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and determine a third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model; adjust the parameters of the target recognition model according to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss to obtain a final target recognition model.

2. The method according to claim 1, characterized in that The identifying the scene category corresponding to the image to be identified includes: Identify a first scene category corresponding to the image to be identified using a first scene recognition model; A second scene recognition model corresponding to the first scene category is used to identify a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

3. The method according to claim 1, characterized in that The detecting of the facial region contained in the image to be recognized and identifying facial attribute information corresponding to the facial region includes: Detecting facial regions and facial key points in the image to be identified using a facial key point detection model; Using a facial recognition model, facial attribute information corresponding to the facial area is identified based on the facial key points.

4. The method according to claim 1, wherein After obtaining the object area in the image to be identified, the method further includes: Using a first object recognition model, respectively identifying first object categories corresponding to the object areas, the first object categories including target object categories and non-target object categories; If a non-target object area exists in the object area, identifying an object category corresponding to the non-target object area includes: If it is determined that the object area includes a non-target object area according to the first object categories corresponding to the object areas, a second object recognition model is used to identify a second object category corresponding to the non-target object area, where the second object category is a subcategory corresponding to the non-target object category.

5. The method according to claim 1, characterized in that The determining of the second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and the determining of the third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, including: When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result; Determine the second KL divergence loss according to the first weight coefficient, the first classification prediction result, and the second classification prediction result; When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result; The third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

6. The method according to claim 5, characterized in that The process of determining the first weight coefficient and the second weight coefficient includes: If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero; If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one; If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

7. The method according to claim 1, characterized in that The training step of the target recognition model also includes: Acquire an image segmentation model, wherein the image segmentation model is used to distinguish the foreground and background of the training sample image; Inputting the training sample image into the target recognition model to obtain a classification prediction vector output by the target recognition model; Determining a third cross entropy loss corresponding to the target recognition model according to the classification prediction vector and the supervision vector; Inputting the feature maps extracted by different convolutional layers in the target recognition model into different upsampling layers in the image segmentation model accordingly, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by the different upsampling layers, and determining the segmentation loss corresponding to the image segmentation model according to the segmentation processing result; Parameters of the object recognition model are adjusted according to the third cross entropy loss and the segmentation loss.

8. An image recognition device, characterized in that: Applied to a mobile terminal, the device includes: An acquisition unit, configured to acquire an image to be recognized; a first recognition unit configured to identify a scene category corresponding to the image to be recognized and obtain scene category information; A first detection unit is configured to detect a facial region contained in the image to be recognized and identify facial attribute information corresponding to the facial region; a second detection unit, configured to perform object detection on the image to be identified, and obtain an object region in the image to be identified; The judgment unit is configured to: if there is a body region of a target object in the object region, and the body region and the facial region correspond to the same target object, associate the facial attribute information with the target object to obtain target object information; if there is a non-target object region in the object region, identify the object category corresponding to the non-target object region to obtain non-target object category information; an output unit, configured to output the scene category information, the target object information, and the non-target object category information; A training unit is configured to perform training on a target recognition model used for different recognition processing, and the training steps of the target recognition model used for different recognition processing include: obtaining a training sample image corresponding to the target recognition model and a supervision vector corresponding to the training sample image; using the target recognition model as a student model, inputting the training sample image into a plurality of teacher models pre-trained using the training sample image and the target recognition model, respectively, to obtain classification prediction vectors output by a plurality of teacher models and a classification prediction vector output by the target recognition model; determining a mean vector corresponding to the supervision vector and the classification prediction vectors output by the plurality of teacher models; determining a first KL divergence loss between the mean vector and the classification prediction vector output by the target recognition model; inputting the mean vector and the classification prediction vector output by the target recognition model into a discriminator model to determine the discrimination loss corresponding to the discriminator model; adjusting the parameters of the target recognition model according to the first KL divergence loss and the discrimination loss; and, Perform different data enhancement processing on the training sample image to obtain a first training sample image and a second training sample image; determine a first cross entropy loss based on a first classification prediction result obtained after the first training sample image is input into the target recognition model and the supervisory information corresponding to the first training sample image; determine a second cross entropy loss based on a second classification prediction result obtained after the second training sample image is input into the target recognition model and the supervisory information corresponding to the second training sample image; determine the feature similarity loss between each convolutional layer of the same level after the first training sample image and the second training sample image are respectively input into the target recognition model; determine a second KL divergence loss when the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, and determine a third KL divergence loss when the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model; adjust the parameters of the target recognition model according to the first cross entropy loss, the second cross entropy loss, the feature similarity loss, the second KL divergence loss and the third KL divergence loss to obtain a final target recognition model.

9. The device according to claim 8, characterized in that The first recognition unit is further configured to perform: Identify a first scene category corresponding to the image to be identified using a first scene recognition model; A second scene recognition model corresponding to the first scene category is used to identify a second scene category corresponding to the image to be recognized, where the second scene category is a subcategory of the first scene category.

10. The device according to claim 8, characterized in that The first detection unit is further configured to perform: Detecting facial regions and facial key points in the image to be identified using a facial key point detection model; Using a facial recognition model, facial attribute information corresponding to the facial area is identified based on the facial key points.

11. The device according to claim 8, characterized in that The device further comprises: A second recognition unit is configured to use a first object recognition model to respectively identify first object categories corresponding to the object areas, where the first object categories include target object categories and non-target object categories; The third recognition unit is configured to execute, if it is determined that the object area includes a non-target object area based on the first object categories corresponding to the object areas, then use a second object recognition model to identify a second object category corresponding to the non-target object area, where the second object category is a subcategory corresponding to the non-target object category.

12. The device according to claim 8, characterized in that The training unit is further configured to perform: When the first classification prediction result is used as the output result of the teacher model and the second classification prediction result is used as the output result of the student model, a first weight coefficient corresponding to the first classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result; Determine the second KL divergence loss according to the first weight coefficient, the first classification prediction result, and the second classification prediction result; When the first classification prediction result is used as the output result of the student model and the second classification prediction result is used as the output result of the teacher model, a second weight coefficient corresponding to the second classification prediction result is determined according to the correctness of the first classification prediction result and the second classification prediction result; The third KL divergence loss is determined according to the second weight coefficient, the first classification prediction result, and the second classification prediction result.

13. The device according to claim 12, characterized in that The training unit is further configured to perform: If the classification prediction result output by the teacher model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is zero; If the classification prediction result output by the teacher model is correct, and the classification prediction result output by the student model is correct, then the weight coefficient corresponding to the classification prediction result output by the teacher model is one; If the classification prediction result output by the teacher model is correct and the classification prediction result output by the student model is wrong, the weight coefficient corresponding to the classification prediction result output by the teacher model is a set value greater than one.

14. The device according to claim 8, characterized in that The training unit is further configured to perform: Acquire an image segmentation model, wherein the image segmentation model is used to distinguish the foreground and background of the training sample image; Inputting the training sample image into the target recognition model to obtain a classification prediction vector output by the target recognition model; Determining a third cross entropy loss corresponding to the target recognition model according to the classification prediction vector and the supervision vector; Inputting the feature maps extracted by different convolutional layers in the target recognition model into different upsampling layers in the image segmentation model accordingly, so that the image segmentation model performs segmentation processing on the training sample image based on the feature maps generated by the different upsampling layers, and determining the segmentation loss corresponding to the image segmentation model according to the segmentation processing result; Parameters of the object recognition model are adjusted according to the third cross entropy loss and the segmentation loss.

15. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the image recognition method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image recognition method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 7 is implemented.

Images