Image recognition method, system, recognition model training method and storage medium

Abstract

The present application relates to an image recognition method, system, recognition model training method and storage medium, wherein the method comprises: extracting features from an original image using a first recognition model to obtain a first feature vector; obtaining a second feature vector sent by a target node device; the second feature vector is a feature vector obtained by the target node device extracting features from an image to be recognized using a second recognition model, and the second recognition model is trained based on scene information corresponding to the target node device and the feature vector extracted by the first recognition model; comparing the first feature vector and the second feature vector to generate a recognition result of the image to be recognized. Through the present application, the problem in the prior art that the basic face image recognition model has a poor recognition effect on face images in specific scenes due to the large differences in face snapshots in different scenes is solved, and the recognition effect of face images in specific scenes is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition method, system, recognition model training method and storage medium. Background Art

[0002] Face image recognition is a common biometric recognition technology. The general process of biometric recognition includes image preprocessing, image feature extraction, and image feature matching. Image preprocessing will crop, scale, and align the biometric image to obtain the standard input of the feature extraction model; image feature extraction will use the feature extraction model to convert the image containing biometric information into a fixed-dimensional feature vector; image feature matching will calculate the similarity between the feature vector corresponding to the image to be identified and the known label feature vector in the base library. If the similarity is greater than the specified threshold, it is determined that the image to be identified has the same identity label as the corresponding feature vector in the base library.

[0003] Facial attribute analysis is a technology that obtains facial attribute information from facial images. Common facial attributes include gender, age, expression, head posture, whether the face is blocked, whether glasses are worn, etc. Different facial attributes correspond to different application scenarios of facial image recognition. In different application scenarios of facial image recognition, facial images vary greatly. Common low-quality scenarios include face occlusion, low-resolution faces, faces at large angles, and elderly scenes. Since the facial features extracted by different facial image recognition models are generally not cross-comparable, in order to facilitate the deployment of facial image recognition algorithms, the most commonly used facial image recognition solution is to train a basic facial image recognition model with good generalization ability, while giving priority to ensuring its recognition performance for standard facial images, and taking into account the recognition performance of different low-quality facial image recognition scenarios as much as possible. However, due to the large differences in facial snapshots in different scenarios, the basic facial image recognition model does not have a good recognition effect on facial images in specific scenarios. Summary of the invention

[0004] The embodiments of the present application provide an image recognition method, system, recognition model training method and storage medium to solve the problem that the basic face image recognition model in the prior art has poor recognition effect on face images in specific scenarios.

[0005] In a first aspect, an embodiment of the present application provides an image recognition method, the method comprising:

[0006] Extracting features from the original image using the first recognition model to obtain a first feature vector;

[0007] Acquire a second feature vector sent by the target node device; the second feature vector is a feature vector obtained by the target node device by extracting features of the image to be recognized using a second recognition model, and the second recognition model is trained according to scene information corresponding to the target node device and the feature vector extracted by the first recognition model;

[0008] The first feature vector and the second feature vector are compared to generate a recognition result of the image to be recognized.

[0009] In some embodiments, comparing the first feature vector and the second feature vector to generate a recognition result of the image to be recognized includes:

[0010] Calculating similarity between the first feature vector and the second feature vector;

[0011] According to the similarity calculation result and the preset recognition requirement, the recognition result of the image to be recognized is generated.

[0012] In some embodiments, the second recognition model is also trained based on a third recognition model, and the third recognition model includes a low-order feature extraction module and a scene information extraction module.

[0013] In a second aspect, an embodiment of the present application provides a recognition model training method for training the second recognition model described in the first aspect, the method comprising:

[0014] Constructing a first recognition model, a second recognition model and a third recognition model; the first recognition model includes a low-order feature extraction module and a first high-order feature extraction module, the first recognition model, the second recognition model and the third recognition model share the low-order feature extraction module, the second recognition model also includes a second high-order feature extraction module, and the third recognition model also includes a scene information extraction module;

[0015] Constructing a target loss function; the target loss function includes scene information recognized by the third recognition model and feature vector information extracted by the first recognition model;

[0016] The second recognition model is trained according to the training sample set and the target loss function.

[0017] In some embodiments, the method further comprises:

[0018] Construct classification and recognition module;

[0019] The feature vector extracted by the second recognition model is input into the classification recognition module to obtain a predicted label vector of the training sample.

[0020] In some embodiments, constructing the target loss function includes:

[0021] Determining a first weight value according to a scene recognition result of the training sample set by the third recognition model;

[0022] Determine a first function according to the first weight value and a recognition result of the second feature vector by the classification recognition module;

[0023] The objective loss function is determined according to the first function.

[0024] In some embodiments, constructing the target loss function includes:

[0025] Determining a second weight value according to a scene recognition result of the training sample set by the third recognition model;

[0026] Obtaining a first feature vector after the first recognition model performs feature extraction on the training sample set; obtaining a second feature vector after the second recognition model performs feature extraction on the training sample set;

[0027] Determine a second function according to the first eigenvector and the second eigenvector;

[0028] The target loss function is determined according to the second function; or, the target loss function is determined according to the first function and the second function.

[0029] In some embodiments, the scene recognition result of the third recognition model for the training sample set includes a scene recognition probability, the first weight value is directly proportional to the scene recognition probability, and the second weight value is inversely proportional to the scene recognition probability.

[0030] In a third aspect, an embodiment of the present application provides an image recognition system, the system comprising a training server, a cloud server and an end node device;

[0031] The training server is used to train the first recognition model, the second recognition model and the third recognition model;

[0032] The end node device is used to deploy a trained second recognition model corresponding to the scene information of the end node device, and use the second recognition model to extract features of the image to be recognized to obtain a second feature vector;

[0033] The cloud server is used to deploy the trained first recognition model, and use the first recognition model to extract features of the image to be recognized to obtain a first feature vector, compare the first feature vector with the second feature vector, and generate a recognition result of the image to be recognized.

[0034] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the image recognition method described in the first aspect, or the processor is configured to run the computer program to execute the recognition model training method described in the second aspect.

[0035] In a fifth aspect, a computer-readable storage medium is provided in an embodiment of the present application, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image recognition method described in the first aspect are implemented, or when the computer program is executed by a processor, the steps of the recognition model training method described in the second aspect are implemented.

[0036] Compared with the prior art, an image recognition method, system, recognition model training method, electronic device and storage medium provided in the embodiment of the present application extract features from the original image using the first recognition model to obtain a first feature vector; obtain a second feature vector sent by the target end node device; the second feature vector is a feature vector obtained by the target end node device using the second recognition model to extract features from the image to be recognized, and the second recognition model is trained according to the scene information corresponding to the target end node device and the feature vector extracted by the first recognition model; the first feature vector and the second feature vector are compared to generate a recognition result of the image to be recognized. The second recognition model deployed by the end node device uses the scene information corresponding to the end node device and the feature vector extracted by the first recognition model during training, so that the second recognition model is suitable for a specific scene, that is, a specific face attribute, thereby improving the recognition effect of the face image in the specific scene, and solving the problem in the prior art that the basic face image recognition model has a poor recognition effect on the face image in the specific scene due to the large difference in the face snapshots in different scenes.

[0037] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 It is a hardware structure block diagram of a terminal for executing an image recognition method or a recognition model training method according to an embodiment of the present application;

[0040] Figure 2 is a structural schematic diagram of an image recognition system according to an embodiment of the present application;

[0041] Figure 3 is a flow chart of an image recognition method according to an embodiment of the present application;

[0042] Figure 4 is a flow chart of a recognition model training method according to an embodiment of the present application;

[0043] Figure 5 It is a schematic diagram of a training network structure of a specific sub-model in an embodiment of the present application;

[0044] Figure 6 This is an architecture diagram of a face recognition system or image recognition system with a terminal-cloud architecture according to an embodiment of the present application;

[0045] Figure 7 is a flow chart of an image recognition preprocessing method according to an embodiment of the present application;

[0046] Figure 8 is a flow chart of a model training method in an embodiment of the present application;

[0047] Fig. 9 It is a flow chart of an image recognition method according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiments provided in the embodiments of the present application can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 1 is a hardware structure block diagram of a terminal for executing an image recognition method or a recognition model training method according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to an image recognition method or a recognition model training method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] An image recognition system is provided in an embodiment of the present application. Figure 2 2 is a structural diagram of an image recognition system according to an embodiment of the present application, and the system includes a training server 210, a cloud server 220, and an end node device 230. The training server 210 is used to train the first recognition model, the second recognition model, and the third recognition model; the end node device 230 is used to deploy the trained second recognition model corresponding to the scene information of the end node device 230, and use the second recognition model to extract features of the image to be recognized to obtain a second feature vector; the cloud server 220 is used to deploy the trained first recognition model, and use the first recognition model to extract features of the image to be recognized to obtain a first feature vector, compare the first feature vector and the second feature vector, and generate a recognition result of the image to be recognized.

[0054] The first recognition model, the second recognition model and the third recognition model are trained on the training server 210, wherein the second recognition model and the third recognition model share or reuse the low-order feature extraction module of the first recognition model, and the low-order feature extraction module is used to extract the low-order features of the input image, and the low-order feature extraction module can be implemented by a deep learning network model with a smaller network depth or fewer network layers. The first recognition model includes a low-order feature extraction module and a first high-order feature extraction module, the second recognition model includes a low-order feature extraction module and a second high-order feature extraction module, and the third recognition model includes a low-order feature extraction module and a scene information extraction module. As a non-limiting example, the training server 210 can first perform model training on the first recognition model to obtain the trained first recognition model, fix the network weights of the low-order feature extraction module and the first high-order feature extraction module, and then train the third recognition model to obtain the trained third recognition model, fix the network weights of the scene information extraction module, and finally train the second recognition model, and construct a corresponding loss function according to the output results of the first recognition model and the third recognition model to train the second recognition model to obtain the trained second recognition model, and the loss function includes scene information, so that the trained second recognition model is suitable for specific scenes, that is, specific facial attributes, thereby improving the recognition effect of facial images in specific scenes. As another non-limiting example, the training server 210 may first perform model training on the first recognition model to obtain the trained first recognition model, fix the network weights of the low-order feature extraction module and the first high-order feature extraction module, and then jointly train the second recognition model and the third recognition model, and construct a corresponding loss function according to the output results of the first recognition model and the third recognition model to obtain the trained second recognition model, wherein the loss function includes scene information, so that the trained second recognition model is suitable for specific scenes, i.e., specific facial attributes, thereby improving the recognition effect of facial images in specific scenes.

[0055] The training server 210 trains the second recognition model to obtain a trained second recognition model corresponding to a specific scene, and generates a scene identifier corresponding to the specific scene. The training server 210 sends the trained second recognition model of the specific scene corresponding to the scene identifier to the end node device of the specific scene based on the scene identifier. The end node device deploys the second recognition model and performs image recognition on the images collected by the end node device, thereby improving the recognition effect of facial images in specific scenes.

[0056] It should be noted that the recognition models or corresponding modules in the embodiments of the present application, such as the first recognition model, the second recognition model, the third recognition model, the first high-order feature extraction module, the second high-order feature extraction module, the scene information extraction module, the classification recognition module, etc., can all be implemented using a deep learning network model. The deep learning network models here include a fully connected network (FC) model, a convolutional neural network (CNN) model, a recurrent neural network (RNN) model, a generative adversarial network (GAN) model, etc., which are not specifically limited here.

[0057] The present application also provides an image recognition method, Figure 3 is a flow chart of an image recognition method according to an embodiment of the present application, which is applied to a cloud server, such as Figure 3 As shown, the process includes the following steps:

[0058] Step S310: extract features from the original image using a first recognition model to obtain a first feature vector.

[0059] In this step, a first recognition model is deployed on the cloud server, and the cloud server uses the first recognition model to extract features from the original image to obtain a first feature vector. The first recognition model here can be a general face recognition model, which is used to analyze the original image, such as the face base library image, such as feature extraction, to obtain the first feature vector.

[0060] Step S320, obtaining a second feature vector sent by the target node device; the second feature vector is a feature vector obtained by the target node device using a second recognition model to extract features from the image to be recognized, and the second recognition model is trained based on the scene information corresponding to the target node device and the feature vector extracted by the first recognition model.

[0061] In this step, the cloud server obtains the second feature vector sent by the target node device. A second recognition model corresponding to the target node scene information is deployed on the target node device. After the target node device collects the image to be recognized, the image to be recognized is input into the second recognition model for feature extraction to obtain a second feature vector. Among them, the second recognition model is a recognition model corresponding to the scene information of the target node device. The second recognition model is trained based on the scene information corresponding to the target node device and the feature vector extracted by the first recognition model.

[0062] Step S330: compare the first feature vector and the second feature vector to generate a recognition result of the image to be recognized.

[0063] In this step, the cloud server compares the first feature vector and the second feature vector, and generates a recognition result of the image to be recognized based on the comparison result. The recognition result here can be set according to the recognition requirement. For example, when the recognition requirement is control warning, the recognition result outputs the corresponding warning information. When the recognition requirement is identity verification, the recognition result outputs the corresponding identity verification result information. When the recognition requirement is image search, the recognition result outputs the corresponding search result.

[0064] In this embodiment, the first feature vector is obtained by extracting features from the original image using the first recognition model; the second feature vector sent by the target end node device is obtained; the second feature vector is the feature vector obtained by the target end node device using the second recognition model to extract features from the image to be recognized, and the second recognition model is trained based on the scene information corresponding to the target end node device and the feature vector extracted by the first recognition model; the first feature vector and the second feature vector are compared to generate the recognition result of the image to be recognized. The second recognition model deployed by the end node device uses the scene information corresponding to the end node device and the feature vector extracted by the first recognition model during training, so that the second recognition model is suitable for specific scenes, that is, specific facial attributes, thereby improving the recognition effect of facial images in specific scenes.

[0065] In some of the embodiments, the first feature vector and the second feature vector are compared to generate a recognition result of the image to be recognized, including: calculating the similarity between the first feature vector and the second feature vector; and generating the recognition result of the image to be recognized based on the similarity calculation result and preset recognition requirements.

[0066] In some of the embodiments, the second recognition model is also trained based on a third recognition model, and the third recognition model includes a low-order feature extraction module and a scene information extraction module.

[0067] The present application also provides a recognition model training method, Figure 4 is a flow chart of a recognition model training method according to an embodiment of the present application, which is applied to a training server, such as Figure 4 As shown, the process includes the following steps:

[0068] Step S410, construct a first recognition model, a second recognition model and a third recognition model; the first recognition model includes a low-order feature extraction module and a first high-order feature extraction module, the first recognition model, the second recognition model and the third recognition model share the low-order feature extraction module, the second recognition model also includes a second high-order feature extraction module, and the third recognition model also includes a scene information extraction module.

[0069] In this step, the training server constructs a first recognition model, a second recognition model, and a third recognition model. The first recognition model includes a low-order feature extraction module and a first high-order feature extraction module. The first recognition model, the second recognition model, and the third recognition model share or reuse the low-order feature extraction module. The second recognition model also includes a second high-order feature extraction module. The third recognition model also includes a scene information extraction module. During training, the training sample set is input into the low-order feature extraction module for feature extraction. After the low-order feature vector is extracted, three branches are extended after the low-order feature matrix. That is, the low-order feature vector is respectively input into the first high-order feature extraction module, the second high-order feature extraction module, and the scene information extraction module to perform corresponding feature extraction, and the first high-order feature vector, the second high-order feature vector, and the scene feature vector are respectively obtained. The scene feature vector is used to characterize scene information or face attribute information.

[0070] Step S420, constructing a target loss function; the target loss function includes scene information recognized by the third recognition model and feature vector information extracted by the first recognition model.

[0071] In this step, the training server constructs a classification recognition module, inputs the second high-order feature vector into the classification recognition module for feature extraction, and obtains the label vector predicted by the training sample set. The target loss function is constructed based on the label vector predicted by the training sample set, the first high-order feature vector, and the scene feature vector.

[0072] Step S430: training the second recognition model according to the training sample set and the target loss function.

[0073] In this step, the second recognition model is trained according to the training sample set and the target loss function to obtain a trained second recognition model.

[0074] In this embodiment, a target loss function is constructed according to the label vector predicted by the training sample set, the first high-order feature vector and the scene feature vector, and the second recognition model is trained according to the target loss function to obtain a trained second recognition model, so that the trained second recognition model is suitable for specific scenes, that is, specific facial attributes, thereby improving the recognition effect of facial images in specific scenes.

[0075] In some of the embodiments, constructing the target loss function includes: determining a first weight value based on the scene recognition result of the training sample set by the third recognition model; determining a first function based on the first weight value and the recognition result of the second feature vector by the classification recognition module; and determining the target loss function based on the first function.

[0076] In some of the embodiments, constructing the target loss function includes: determining a second weight value based on the scene recognition results of the training sample set by the third recognition model; obtaining a first feature vector after the first recognition model performs feature extraction on the training sample set; obtaining a second feature vector after the second recognition model performs feature extraction on the training sample set; determining a second function based on the first feature vector and the second feature vector; determining the target loss function based on the second function; or, determining the target loss function based on the first function and the second function.

[0077] In some of the embodiments, the scene recognition result of the third recognition model for the training sample set includes a scene recognition probability, the first weight value is proportional to the scene recognition probability, and the second weight value is inversely proportional to the scene recognition probability. In this embodiment, when the facial attributes are more consistent with the specified specific face, that is, when the scene is a specified specific scene, the first weight value is large, and when the facial attributes are more consistent with the general face, that is, the scene is a general scene, the second weight value is large, so that the trained second recognition model is suitable for specific scenes, that is, specific facial attributes, thereby improving the recognition effect of facial images in specific scenes.

[0078] The following describes an embodiment of the present application by taking the first recognition model as a general face recognition model, the second recognition model as a specific sub-model, and the third recognition model as a face attribute recognition model as an example.

[0079] In the process of training the specific sub-model, the features of the general face recognition model are introduced as supervision, so that each sub-model and the general face recognition model still maintain a certain feature comparison ability. The training network structure diagram of the specific sub-model is shown in the figure below. Figure 5 As shown. On this basis, a face recognition system or image recognition system with end-to-end architecture can be realized, that is, a specific sub-model is deployed on each end node to obtain the best recognition effect for a specific scene, and the model is lightweight, saving end node computing resources; a general face recognition large model is deployed on the cloud server to analyze the face base library images. The facial features analyzed by the end node are uploaded to the cloud server for unified comparison and recognition services. The specific architecture diagram is shown in Figure 6 .

[0080] The original general face recognition model includes two parts: low-order general feature extraction and high-order general feature extraction. Each part is composed of several stacked neural network layers. The low-order general feature extraction part is used to extract general features from the input face image; the high-order general feature extraction target is used to extract high-order abstract features from the general features. In this part, the correlation of face features in different scenes will be explored to achieve the generalization ability of the model in different scenes, and finally output the feature vector representing the input face image. Among them, the low-order general feature extraction is equivalent to the low-order feature extraction module in the aforementioned embodiment, and the high-order general feature extraction is equivalent to the first high-order feature extraction module in the aforementioned embodiment. The general face recognition model can be described as:

[0081] Y c =f c (X);

[0082] Y g =f g (Y c );

[0083] Among them, X is the input face image matrix, Corresponding to the general feature extraction operation, Y c The low-order general feature matrix corresponding to the face, Corresponding to the high-order feature extraction part, Y g The high-order abstract feature vector corresponding to the output of the face recognition model. The network weights of the general face recognition model are fixed and do not participate in training. They only provide basic feature extraction capabilities and supervision information.

[0084] Continue to refer Figure 5 , the low-order general feature extraction part of the general face recognition model extends into three branches, one branch corresponds to the original general face recognition model, followed by the high-order general feature extraction part; another branch corresponds to face attribute extraction, which is used to extract face attributes, that is, scene information, from the low-order general features of the face image; the third branch corresponds to the specific sub-model to be trained, including high-order specific feature extraction and classification recognition. Among them, the face attribute extraction here is equivalent to the scene information extraction module in the aforementioned embodiment, the high-order specific feature extraction is equivalent to the second high-order feature extraction module in the aforementioned embodiment, and the classification recognition is equivalent to the classification recognition module in the aforementioned embodiment.

[0085] The output of the facial attribute feature extraction branch corresponds to the facial attribute feature. The facial attribute feature is the concatenation of multiple facial attribute feature codes. Common facial attributes include whether the face is wearing a mask, whether the face is small, whether the face is at a large angle, whether the face is old, etc. Each dimension in the feature vector corresponds to a feature, and the value range is [0-1], 0 corresponds to no, 1 corresponds to yes, and the predicted output of the facial attribute recognition model is a floating point value between 0 and 1, which can be understood as a predicted probability. The facial attribute feature extraction branch can be a pre-trained feature extraction model, which can be trained separately. After the training is completed, the weight is fixed and is only used to extract the facial attribute feature vector Y for the general features of the face image. f , can also be trained together with the specific sub-model branches.

[0086] The specific sub-model branch is divided into two parts: specific high-order feature extraction and classification recognition. The specific high-order feature extraction branch extracts high-order specific features based on low-order universal features of the face, and the classification recognition part is used to output face identity label prediction based on high-order specific feature vectors. Network structure of high-order specific feature extraction Compared with high-order general feature extraction To be simpler, ensure that the specific sub-model is lighter than the general face recognition model. The specific sub-model can be described as:

[0087] Y s =f s (Y c );

[0088] Y l =f l (Y s );

[0089] Among them, Y s corresponds to the high-order specific eigenvector, Represents the low-order general feature matrix Y c Extract high-order specific feature vectors, Corresponding to the identity label prediction based on the high-order specific features output, Y l The identity label vector corresponding to the model prediction.

[0090] During the training process of specific feature extraction analysis, the low-order general feature matrix Y of the image c In addition to the input-specific feature extraction branch, the identity label vector Y for identity prediction is obtained l In addition, it will be sent to the face attribute extraction branch and the original general model feature extraction branch. Get the face attribute feature vector Y f and the general model feature output , the cross entropy loss L between the identity prediction and the true identity label is calculated using cross entropy during model training crossSince the specific model needs to pay more attention to the recognition of a certain type of face, the system will predict the face based on the face attributes. The cross entropy loss is weighted; on the other hand, since the specific sub-model features need to maintain a certain feature compatibility with the general face recognition model features, another mean square error loss Used to calculate the general face model feature Y g and the specific sub-model feature Y s In summary, the overall loss function L of the training model can be described as:

[0091] ;

[0092] ;

[0093] ;

[0094] in, is the cross entropy loss weight mapping function, is the mean square error loss weight mapping function. During the training process, if the face attribute Y f Closer to the specific face in a specific scene, the cross entropy loss weight f f1 (Y f ) is larger, the better the recognition effect of the model on specific faces. f Closer to the face in the general scene, the mean square error loss weight f f2 (Y f ) is larger, ensuring the feature compatibility of the model in general scenarios. is the cross entropy loss, correspond The value of a specific dimension in , y l Corresponding to Y l The value of a specific dimension in . is the mean square error loss, n is the feature dimension, Corresponding to Y g The value of a specific dimension in Corresponding to Y s The loss weight mapping function is adjusted to train specific sub-models for different scenarios based on the general face recognition model. The specific sub-models for different scenarios focus on different specific faces, but at the same time maintain the compatibility of facial features in general scenarios. Therefore, a face recognition system with an end-cloud architecture can be built, and the corresponding specific sub-models can be deployed on the end nodes to ensure the lightweight and recognition performance of the model in specific scenarios. The general face recognition model is deployed on the cloud server, which is responsible for extracting features from high-quality face base images. The specific face features are uploaded from the end nodes to the cloud server for feature comparison.

[0095] This is different from the traditional method of using a general face recognition large model to take into account the face recognition effects in different scenarios. The model training method in the embodiment of the present application is an attribute-based face recognition large model fine-tuning training method. This method can quickly fine-tune a specific sub-model for a specific scene based on the general face recognition large model. The specific sub-model has better recognition effect in specific scenes and is more lightweight. The end-cloud architecture of the present application has the following advantages over traditional face recognition systems: better specific face recognition capabilities in specific scenes; the end node model is lighter and has lower computational cost than the general face recognition large model; feature extraction is performed at the end node, and feature comparison is performed on a unified cloud server, making the system more efficient and secure.

[0096] The attribute-based face recognition large model fine-tuning training method is divided into a training phase and a testing phase. The training phase includes data preprocessing, network construction, weight mapping function design, and network training. The testing phase includes data preprocessing, specific model construction, edge deployment, and feature extraction.

[0097] Data preprocessing: Use the facial feature point detection algorithm to extract facial feature points from the facial images in the training set, and then perform affine transformation on the facial images based on the facial feature points to obtain a standard facial image.

[0098] Network construction, build a training network with general face feature extraction branch, specific face feature extraction branch, and face attribute extraction branch. The general face feature extraction branch corresponds to a pre-trained face recognition large model, and its network weight is fixed; the specific face feature extraction branch corresponds to a specific sub-model, and its network weight participates in training; the face attribute extraction branch is used for face attribute extraction. This branch can be a pre-trained face attribute recognition model, or it can be trained together with the specific face feature extraction branch. At this time, the total loss function L needs to add the attribute recognition loss :

[0099] .

[0100] The weight mapping function is designed. The input of the weight mapping function is a face attribute vector. When the face attributes are more consistent with the specified specific face, the weight value output by the cross entropy loss weight mapping function is larger. When the face attributes are more consistent with the general face, the weight value output by the mean square error loss weight mapping function is larger.

[0101] Network training: input a standard face image with a known identity label into a three-branch network, obtain face attribute predictions in the attribute branch, obtain universal face features in the universal face feature extraction branch, and obtain face identity label predictions in the specific face feature extraction branch; calculate the network loss based on the network output and the true identity label, and adjust the network weights of the specific face feature extraction branch through the back propagation algorithm to reduce the network loss until the network loss is less than the preset value or reaches a specific number of iterations.

[0102] Data preprocessing: Use the facial feature point detection algorithm to extract facial feature points from the face image to be identified, and then perform affine transformation on the face image based on the facial feature points to obtain a standard face image.

[0103] Specificity model construction: the low-order general feature extraction part of the general face recognition model is spliced ​​with the specific high-order feature extraction part to form a specific sub-model.

[0104] In the end-edge deployment, the end node corresponds to the specific face recognition scenario, and deploys the corresponding specific sub-model to extract specific face features for specific faces; the cloud server deploys a general face recognition large model to extract features from standard face base library images.

[0105] Feature extraction: Use different specific sub-models to extract features from specific faces in different scenarios, and use the face recognition model to extract features from standard face database images. Specific face features are transmitted to the cloud server for feature comparison to achieve specific face recognition services such as image search, control and alarm.

[0106] Figure 7 is a flowchart of an image recognition preprocessing method according to an embodiment of the present application. Figure 7 As shown, the image recognition preprocessing method includes the following steps:

[0107] Step S710: image preprocessing.

[0108] In this step, a facial feature point detection algorithm is used to obtain the key points of facial features and contours of the face image to be identified; an affine transformation is performed on the face image to be identified according to the coordinates of the key points to obtain a standard face image to be identified of standard size.

[0109] Step S720, determining the facial attribute feature coding.

[0110] In this step, according to the specific face recognition scenario, taking five specific face scenarios as an example, select whether it is a large-angle face (whether the face heading angle is greater than the preset angle, such as 45 degrees, 60 degrees, etc.), whether it is a small face (whether the face width is less than the preset width, such as 20 cm, 18 cm, etc.), whether wearing a mask, whether it is a child (age is younger than the preset age, such as 15 years old, 12 years old, etc.), whether it is an elderly person (such as age is greater than the preset value, such as 65 years old, 60 years old, etc.) to construct a face feature vector with a feature dimension of 5. Each dimension corresponds to a face attribute, and the value range is [0,1], 0 corresponds to no, 1 corresponds to yes, and each dimension can be regarded as the predicted probability of the attribute.

[0111] Step S730, constructing a model to be trained.

[0112] In this step, a model to be trained is constructed, which has three branches: a general face feature extraction branch, a specific face feature extraction branch, and a face attribute extraction branch. Figure 5 The training network of the specific sub-model shown is the model to be trained. The general face feature extraction branch and the face attribute extraction branch both correspond to the pre-trained model, with fixed weights, and the low-order general feature extraction part is reused by the three branches.

[0113] Step S740, determining a loss weight mapping function.

[0114] In this step, the following mapping function is used:

[0115] f f1 (Y f )=y f,i ;

[0116] f f2 (Y f )=1-y f,i ;

[0117] Among them, y f,i Represents the i-th dimension in the face attribute vector. The value on this dimension corresponds to the predicted probability of face attributes in a specific scenario. For example, the face attributes of a masked face in a mask scenario. Then the value on the i-th dimension is the probability that the face is wearing a mask. The higher the probability of the face wearing a mask, the greater the cross entropy loss weight, and the smaller the feature mean square error loss weight.

[0118] Figure 8 is a flow chart of a model training method in an embodiment of the present application, such as Figure 8 As shown, the model training method includes the following steps:

[0119] Step S810: model training.

[0120] In this step, the preprocessed training set face images are sent to the input layer of the training model in batches, and the face attribute output, common face features and face label prediction are obtained from the three branches respectively; the model loss is calculated based on the output of the three branches according to the above loss function, and the loss value is minimized through the back propagation algorithm.

[0121] Step S820, completing the training of a single specific sub-model.

[0122] In this step, the iteration step number t is set, and step S810 is repeated until the training step number reaches t, completing the training of the specific sub-model. The classification and recognition parts of the facial attribute extraction branch, the general facial feature extraction branch, and the specific facial feature extraction branch are removed to obtain the final specific sub-model.

[0123] Step S830, completing the training of multiple specific sub-models.

[0124] In this step, steps S810 and S820 are repeated to train specific sub-models suitable for different scenarios based on the facial attributes in different scenarios.

[0125] Fig. 9 is a flowchart of an image recognition method according to an embodiment of the present application, and the method is applied to Figure 2 or Figure 6 The system shown, Fig. 9 As shown, the image recognition method includes the following steps

[0126] Step S910: model deployment.

[0127] In this step, according to the usage scenario, the corresponding specific sub-model is deployed on the peer node, and the corresponding general face recognition large model is deployed on the cloud server.

[0128] Step S920: feature extraction.

[0129] In this step, the general face recognition large model on the cloud server extracts features from the standard face base image; the specific sub-model on each end node extracts features from the specific face;

[0130] Step S930: feature comparison.

[0131] In this step, the specific facial features extracted by each end node are transmitted to the cloud server, and feature comparison is performed uniformly on the cloud server.

[0132] It should be noted that the above embodiment is only one way to implement the present application, and the areas that can be flexibly configured according to actual conditions include but are not limited to: the selection of facial attributes is not limited to the mask face, small face, large-angle face, elderly face and child face. The face loss function weight mapping function can have other designs. When the face attributes of specific scenes are met, the classification cross entropy loss weight is high; when the face attributes of general scenes are met, the feature mean square error loss weight is high. The facial attribute feature extraction branch can choose a pre-trained model or train the branch together with the specific feature extraction branch. There are many different options for the network structure, and the current network structures in deep learning can be used for the face recognition model in this proposal.

[0133] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0134] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0135] Optionally, in the embodiment of the present application, the processor may be configured to perform the following steps through a computer program:

[0136] S1, extracting features from the original image using a first recognition model to obtain a first feature vector;

[0137] S2, obtaining a second feature vector sent by the target node device; the second feature vector is a feature vector obtained by the target node device by extracting features of the image to be recognized using a second recognition model, and the second recognition model is trained according to scene information corresponding to the target node device and the feature vector extracted by the first recognition model;

[0138] S3: Compare the first feature vector and the second feature vector to generate a recognition result of the image to be recognized.

[0139] Optionally, in the embodiment of the present application, the processor may also be configured to perform the following steps through a computer program:

[0140] Q1, construct a first recognition model, a second recognition model and a third recognition model; the first recognition model includes a low-order feature extraction module and a first high-order feature extraction module, the first recognition model, the second recognition model and the third recognition model share the low-order feature extraction module, the second recognition model also includes a second high-order feature extraction module, and the third recognition model also includes a scene information extraction module;

[0141] Q2, constructing a target loss function; the target loss function includes the scene information recognized by the third recognition model and the feature vector information extracted by the first recognition model;

[0142] Q3, training the second recognition model according to the training sample set and the target loss function.

[0143] It should be noted that the specific examples in the embodiments of the present application can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in the embodiments of the present application.

[0144] In addition, in combination with an image recognition method provided in the above embodiment, a storage medium may be provided in the embodiment of the present application to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any image recognition method in the above embodiment is implemented.

[0145] In addition, in combination with a recognition model training method provided in the above embodiment, a storage medium can also be provided in the embodiment of the present application to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any recognition model training method in the above embodiment is implemented.

[0146] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0147] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0148] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0149] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An image recognition method, applied to a cloud server, characterized in that: The method comprises: Using the first recognition model to extract features from the original image, to obtain a first feature vector; Acquire a second feature vector sent by the target node device; the second feature vector is a feature vector obtained by the target node device using a second recognition model to extract features from the image to be recognized, the second recognition model is trained based on scene information corresponding to the target node device and the feature vector extracted by the first recognition model; the second recognition model is also trained based on a third recognition model, the first recognition model, the second recognition model and the third recognition model share a low-order feature extraction module, and the third recognition model also includes a scene information extraction module; The first feature vector and the second feature vector are compared to generate a recognition result of the image to be recognized.

2. The image recognition method according to claim 1, characterized in that: The comparing the first feature vector and the second feature vector to generate a recognition result of the image to be recognized includes: Calculating similarity between the first feature vector and the second feature vector; According to the similarity calculation result and the preset recognition requirement, the recognition result of the image to be recognized is generated.

3. A recognition model training method, used to train the second recognition model according to any one of claims 1 to 2, characterized in that: The method comprises: Constructing a first recognition model, a second recognition model and a third recognition model; the first recognition model includes a low-order feature extraction module and a first high-order feature extraction module, the first recognition model, the second recognition model and the third recognition model share the low-order feature extraction module, the second recognition model also includes a second high-order feature extraction module, and the third recognition model also includes a scene information extraction module; Constructing a target loss function; the target loss function includes scene information recognized by the third recognition model and feature vector information extracted by the first recognition model; The second recognition model is trained according to the training sample set and the target loss function.

4. The recognition model training method according to claim 3, characterized in that: The method further comprises: Construct classification and recognition module; The feature vector extracted by the second recognition model is input into the classification recognition module to obtain a predicted label vector of the training sample.

5. The recognition model training method according to claim 4, characterized in that: The constructing target loss function comprises: Determining a first weight value according to a scene recognition result of the training sample set by the third recognition model; Determine a first function according to the first weight value and a recognition result of the second feature vector by the classification recognition module; The objective loss function is determined according to the first function.

6. The recognition model training method according to claim 5, characterized in that: The constructing target loss function comprises: Determining a second weight value according to a scene recognition result of the training sample set by the third recognition model; Obtaining a first feature vector after the first recognition model performs feature extraction on the training sample set; obtaining a second feature vector after the second recognition model performs feature extraction on the training sample set; Determine a second function according to the first eigenvector and the second eigenvector; The target loss function is determined according to the second function; or, the target loss function is determined according to the first function and the second function.

7. The recognition model training method according to claim 6, characterized in that: The scene recognition result of the third recognition model for the training sample set includes a scene recognition probability, the first weight value is directly proportional to the scene recognition probability, and the second weight value is inversely proportional to the scene recognition probability.

8. An image recognition system, characterized in that: The system includes a training server, a cloud server and an end node device; The training server is used to train a first recognition model, a second recognition model and a third recognition model; the second recognition model is trained based on the first recognition model and the third recognition model, the first recognition model, the second recognition model and the third recognition model share a low-order feature extraction module, and the third recognition model also includes a scene information extraction module; The end node device is used to deploy a trained second recognition model corresponding to the scene information of the end node device, and use the second recognition model to extract features of the image to be recognized to obtain a second feature vector; The cloud server is used to deploy the trained first recognition model, and use the first recognition model to extract features of the image to be recognized to obtain a first feature vector, compare the first feature vector with the second feature vector, and generate a recognition result of the image to be recognized.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the image recognition method described in any one of claims 1 to 2, or the processor is configured to run the computer program to execute the recognition model training method described in any one of claims 3 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image recognition method described in any one of claims 1 to 2 are implemented, or when the computer program is executed by a processor, the steps of the recognition model training method described in any one of claims 3 to 7 are implemented.

Images