A certification method, training method, device and equipment

Abstract

The embodiments of this specification disclose an authentication method, a training method, a device and an apparatus. The authentication method includes obtaining an original image, performing random mosaic mask processing on the original image to obtain a mask image, and sending the mask image to a server, wherein the mask image is used by the server to extract image features from the mask image, authenticate the image features and obtain an authentication result.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to an authentication method, training method, device and equipment. Background Art

[0002] In various scenarios such as access control and payment, images captured by terminal devices need to be uploaded to the server for identification. During this process, private information is uploaded to the server, which poses a risk of leakage and makes users feel insecure. Summary of the invention

[0003] The embodiments of this specification provide an authentication method, a training method, an apparatus and a device for solving the following technical problem: how to reduce the risk of leakage of privacy information during the authentication process of a terminal device.

[0004] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0005] The present invention provides an authentication method, including:

[0006] Get the original image;

[0007] Performing random mosaic mask processing on the original image to obtain a mask image;

[0008] The mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image, and authenticate the image features to obtain an authentication result.

[0009] The embodiment of this specification also provides an authentication method, including:

[0010] receiving a mask image, wherein the mask image is generated by performing a random mosaic masking process on the original image;

[0011] extracting image features from the mask image;

[0012] The image feature is authenticated to obtain an authentication result.

[0013] The embodiment of this specification also provides a model training method, including:

[0014] Obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image;

[0015] Processing the sample image through an encoder to obtain image features;

[0016] Processing the image features through a decoder to obtain a restored image, and using the label image for supervision to train encoder parameters and decoder parameters;

[0017] The image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

[0018] The embodiment of this specification also provides an authentication device, including:

[0019] A first acquisition module is configured to obtain an original image;

[0020] A mask module is configured to perform random mosaic mask processing on the original image to obtain a mask image;

[0021] The sending module is configured to send the mask image to the server, and the mask image is used by the server to extract image features from the mask image and authenticate the image features to obtain an authentication result.

[0022] The embodiment of this specification also provides an authentication device, including:

[0023] A receiving module is configured to receive a mask image, where the mask image is generated by performing a random mosaic masking process on the original image;

[0024] A feature extraction module, configured to extract image features from the mask image;

[0025] The authentication module is configured to authenticate the image features and obtain an authentication result.

[0026] The embodiment of this specification also provides an authentication device, including:

[0027] A second acquisition module is configured to obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image;

[0028] An encoding module, configured to process the sample image through an encoder to obtain image features;

[0029] A first training module is configured to process the image features through a decoder to obtain a restored image, and use the label image for supervision to train encoder parameters and decoder parameters;

[0030] The second training module is configured to process the image features through a classifier to obtain a classification result, and use the label image for supervision to train the encoder parameters and the classifier parameters.

[0031] The embodiment of the present specification also provides an electronic device, including:

[0032] at least one processor; and,

[0033] a memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0035] Get the original image;

[0036] Performing random mosaic masking processing on the original image to obtain a masked image; and

[0037] The mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image, and authenticate the image features to obtain an authentication result.

[0038] The embodiment of the present specification also provides an electronic device, including:

[0039] at least one processor; and,

[0040] a memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0042] receiving a mask image, wherein the mask image is generated by performing a random mosaic masking process on the original image;

[0043] extracting image features from the mask image; and

[0044] The image feature is authenticated to obtain an authentication result.

[0045] The embodiment of the present specification also provides an electronic device, including:

[0046] at least one processor; and,

[0047] a memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0049] Obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image;

[0050] Processing the sample image through an encoder to obtain image features;

[0051] Processing the image features through a decoder to obtain a restored image, and using the label image for supervision to train encoder parameters and decoder parameters;

[0052] The image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

[0053] The embodiments of this specification provide an authentication method, which performs random mosaic processing on images captured by a terminal device. The images uploaded to the server are random mosaic images with a certain proportion. The server can combine mask image reconstruction and feature extraction algorithms to perform large-scale image retrieval and recognition. The original image is not transmitted in the entire communication link, which can effectively protect user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 A schematic diagram of the system architecture of the authentication method using the embodiments of this specification is shown;

[0056] Figure 2 A flowchart illustrating an authentication method according to an embodiment of the present specification;

[0057] Figure 3 A flowchart of obtaining a mask image according to an embodiment of this specification is shown;

[0058] Figure 4A A schematic diagram illustrating an original image of an embodiment of this specification;

[0059] Figure 4B A schematic diagram illustrating an original image after the size is scaled according to an embodiment of the present specification;

[0060] Figure 4C A schematic diagram illustrating a mask image of an original image according to an embodiment of the present specification;

[0061] Figure 4D A schematic diagram illustrating a portion of an original image according to an embodiment of the present specification;

[0062] Figure 4EA schematic diagram illustrating a partial mask image of an original image according to an embodiment of the present specification;

[0063] Figure 5 A flowchart of obtaining a mask image according to another embodiment of the present specification is shown;

[0064] Figure 6 A flowchart illustrating an authentication method according to another embodiment of the present specification;

[0065] Figure 7 A flowchart illustrating a training method according to an embodiment of the present specification;

[0066] Figure 8 A schematic diagram illustrating the model structure of an embodiment of this specification;

[0067] Fig. 9 A flowchart illustrating an authentication method according to another embodiment of the present specification;

[0068] Fig.10 A schematic diagram illustrating an authentication device according to an embodiment of the present specification;

[0069] Fig.11 A schematic diagram illustrating an authentication device according to another embodiment of the present specification; and

[0070] Fig.12 A schematic diagram illustrating a training device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0072] The authentication method provided in the embodiments of this specification performs random mosaic processing on the images collected by the terminal device. The images uploaded to the server are random mosaic images with a certain proportion. The server can combine the mask image reconstruction and feature extraction algorithms to perform large-scale image retrieval and recognition. The original image is not transmitted in the entire communication link, which can effectively protect user privacy.

[0073] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0074] Figure 1 A schematic diagram of the system architecture of the authentication method using the embodiments of this specification is shown.

[0075] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0076] The terminal devices 101, 102, 103 interact with the server 105 via the network 104 to receive or send messages, etc. Various client applications may be installed on the terminal devices 101, 102, 103, such as browser applications, search applications, instant messaging tools, etc.

[0077] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various special or general electronic devices, including but not limited to smart access control, payment or cash register equipment, smart phones, tablet computers, e-book readers, laptop portable computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, multiple software or software modules used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0078] The server 105 may be a server that provides various services, such as a backend server that tests client applications installed on the terminal devices 101 , 102 , 103 .

[0079] The server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0080] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0081] Figure 2 The flowchart of the authentication method of the embodiment of this specification is shown.

[0082] like Figure 2 As shown, the authentication method includes operations S202, S204 and S206. The authentication method can be, for example, Figure 1 The process is performed by the terminal devices 101 , 102 , and 103 shown, and may also be performed by other devices different from the terminal devices 101 , 102 , and 103 .

[0083] In operation S202, an original image is obtained.

[0084] In operation S204, random mosaic mask processing is performed on the original image to obtain a mask image.

[0085] In operation S206, the mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image and authenticate the image features to obtain an authentication result.

[0086] The authentication method provided in the embodiments of this specification performs random mosaic processing on the images collected by the terminal device. The images uploaded to the server are random mosaic images with a certain proportion. The server can combine mask image reconstruction and feature extraction algorithms to perform large-scale image retrieval and recognition. The original image is not transmitted in the entire communication link, which can effectively protect user privacy.

[0087] According to the embodiments of this specification, the original image may be an image captured by a terminal device, or an image selected and provided by a user. The original image is an image used for authentication and contains private content. For example, the original image may include a face, so that the user's identity can be authenticated by face recognition technology. In other embodiments of the present disclosure, in addition to the face, the original image may include any one of a variety of feature information such as human body posture, iris, fingerprint, signature, etc. For ease of description, the following embodiments are all described by taking the original image containing the face as an example.

[0088] Figure 3 The flowchart of obtaining the mask image according to the embodiment of this specification is illustrated.

[0089] like Figure 3 As shown, the method includes operations S302 and S304.

[0090] In operation S302, an image to be processed is determined based on the original image, wherein the image to be processed includes the original image, or the image to be processed includes a partial image of the original image, or the image to be processed includes the original image and a partial image of the original image.

[0091] In operation S304, the image to be processed is divided into a plurality of image blocks, and some of the image blocks are randomly masked to obtain a mask image.

[0092] According to the embodiments of the present specification, mask processing is performed on the image to be processed. Therefore, obtaining a mask image based on the original image can be implemented in at least three situations, which are respectively introduced below.

[0093] In the first implementation, the image to be processed is the original image, and the original image is directly divided into multiple image blocks, and some of the image blocks are randomly masked to obtain a mask image. If the algorithm is robust enough, this implementation can be adopted. However, since the feature information used for authentication is usually concentrated in one or more parts of the original image, only masking the entire original image will result in a large loss of details, increasing the difficulty of authentication. Therefore, this specification also provides a second and a third implementation.

[0094] In a second embodiment, the image to be processed is a partial image of the original image, and the partial image contains feature information for authentication. According to an embodiment of the present specification, the determination of the image to be processed based on the original image includes identifying a local area containing feature information for authentication from the original image, extracting the local area, obtaining a partial image of the original image, and determining the image to be processed based on the partial image of the original image. Taking an original image containing a face as an example, since the feature information for authentication is concentrated in the face area of ​​the original image, the face area in the original image can be determined by a face detection algorithm, and the area can be cropped out as the image to be processed.

[0095] After obtaining a partial image of the original image, the partial image (for example, the face area in the original image) is divided into a plurality of image blocks, and some of the image blocks are randomly masked to obtain a mask image.

[0096] Although in most cases, there is only one interesting part in an original image, in this embodiment, the original image may also have multiple partial images. For example, when multiple faces are recognized at one time, multiple partial images may be intercepted and mask processing may be performed separately.

[0097] Through the second implementation, less local feature information (such as the face area) is lost, which is beneficial for the server to authenticate the image.

[0098] According to the embodiments of the present specification, facial images can be unified into one size (for example, 96*122 size), and a special model can be made to mask and extract features, so that the features of the face can be better restored.

[0099] Some related technologies have emerged in this field, which use convolutional feature maps of the face area instead of the original image for uploading to avoid privacy leakage. The inventors found that there are at least two problems with this approach. First, the recognition model must be at least partially deployed on the end side in order to generate a convolutional feature map on the end side. Once the model is leaked, the risk of image restoration is very high. Second, in some cases, if the user claims that the image acquisition process was not initiated or agreed to by himself (for example, in a scene where many people are queuing, the person in front may use the face of the person behind to complete the payment, and the person being photographed may not look at the screen at all), even if the convolutional feature map is restored, there is only the face part, and it is impossible to judge what the scene was at the time, which is not conducive to resolving disputes. This situation is called willingness risk.

[0100] The various implementations of this specification can solve the first problem well. By uploading the mask image, the mask process is completely independent and does not rely on the subsequent machine learning model for feature extraction, and there is no risk of model leakage. However, the second implementation method mentioned above also cannot solve the second problem well. Therefore, the embodiments of this specification also provide a third implementation method.

[0101] In the third implementation, the images to be processed are two or more images, which are the original image and one or more partial images of the original image, that is, both the first implementation and the second implementation are performed. The final mask images are also two or more images, the image obtained by masking the original image is called the global mask image, and the image obtained by masking the partial image of the original image is called the local mask image.

[0102] In this implementation, the server can not only extract and identify features of the face area, but also effectively restore the scene image, so as to effectively check the risk of willingness. For example, if the user indicates that he has no intention to scan his face, the staff can retrieve the original image from the server to judge whether there is any violation of the intention in the scene at that time.

[0103] According to the embodiments of this specification, an algorithm for intentional risk identification can also be deployed on the terminal device side to detect situations where intentional risk may exist on site. In the case of possible intentional risk, authentication can be rejected and images can be required to be recaptured. In the case of low intentional risk, authentication is allowed, but global mask images and local mask images are generated and uploaded to the server side to avoid privacy leakage.

[0104] Figure 4A-4E Schematic diagrams illustrating various images of the embodiments of this specification. Specifically, Figure 4A Schematic diagram of the original image of the embodiment of this specification is shown. Optionally, the terminal device can standardize the size of the original image, such as Figure 4BAs shown, Figure 4B Schematic diagram of the original image after the size is scaled according to the embodiment of the present specification. In the first or third implementation manner, the original image is directly subjected to mask processing, for example, Figure 4C In the second or third implementation manner, a local area can be selected and cropped by algorithms such as face detection to obtain the following: Figure 4D The local image of the original image is shown in FIG. 1 , and then the local image is masked to obtain the following Figure 4E The mask image shown is the local mask image.

[0105] During the masking process, the image to be processed is divided into multiple image blocks, for example, it can be divided into 32*32 image blocks. Then, some of the image blocks in the multiple image blocks are randomly masked, and the proportion of these image blocks to all the image blocks can be called the mosaic ratio, and the value can be freely defined. For example, a mosaic ratio of 0.5 means that half of the image blocks are randomly masked, and a mosaic ratio of 0.75 means that 75% of the image blocks are randomly masked. Alternatively, the mosaic ratio can be defined in other ways as long as it reflects the proportion of the masked image blocks. For example, the mosaic ratio can be defined as 0.5, which means that when the image is divided into 32*32, a random number of 16*16 image blocks are randomly masked; a mosaic ratio of 0.75 means that when the image is divided into 32*32, a random number of 24*24 image blocks are randomly masked.

[0106] According to the embodiments of the present specification, when the original image and the part of the original image are masked respectively, the mosaic ratios of the two may be the same or different.

[0107] According to the embodiments of this specification, the mosaic ratio can be determined in an adaptive manner. The mosaic ratio can be controllable on demand and can be configured according to different scenes, users, and regulatory requirements. It can be adjusted and customized at any time, such as Figure 5 shown.

[0108] Figure 5 A flowchart of obtaining a mask image according to another embodiment of the present specification is illustrated.

[0109] like Figure 5 As shown, the method includes operations S502 and S504.

[0110] In operation S502 , a mosaic ratio is adaptively determined.

[0111] In operation S504, based on the mosaic ratio, some blocks in the plurality of blocks are randomly masked to obtain a mask image.

[0112] According to the embodiments of the present specification, when the original image and the part of the original image are masked respectively, the same mosaic ratio may be adaptively determined for both, or the mosaic ratios of the two may be determined separately.

[0113] In some embodiments, the mosaic ratio can be determined based on the scene. The scene may include, for example, a payment scene, a general identity authentication scene, a scenic spot access control, a room access control, etc., which can be roughly or finely divided as needed. For example, according to the different requirements of different scenes, a lower mosaic ratio can be set for the payment scene than for the access control scene. The scene can be determined according to the function of the terminal device itself. For example, the scene of application of the smart access control device is the access control scene, the scene of application of the payment device is the payment scene, and general devices such as smart phones need to determine the application scene according to the function being used.

[0114] That is, the adaptively determining the mosaic ratio includes adaptively determining the mosaic ratio based on a scene, wherein there is a first scene and a second scene different from the first scene, and the mosaic ratio determined based on the first scene is different from the mosaic ratio determined based on the second scene.

[0115] In some embodiments, the mosaic ratio can be determined based on the user. For example, an option can be provided to the user so that the user can select the mosaic ratio that is suitable for him / her. The higher the mosaic ratio, the higher the privacy, but it will affect the authentication success rate; conversely, the lower the mosaic ratio, the higher the authentication success rate, but the privacy becomes weaker. In some embodiments, the mosaic ratio suitable for the user can be determined based on the user's characteristic data, for example, the mosaic ratio of a new user is low, and the mosaic ratio of an old user is high.

[0116] That is, the adaptively determining the mosaic ratio includes adaptively determining the mosaic ratio based on a user, wherein there is a first user and a second user different from the first user, and the mosaic ratio determined based on the first user is different from the mosaic ratio determined based on the second user.

[0117] In some embodiments of the present specification, the adaptively determining the mosaic ratio may include determining the mosaic ratio based on regulatory requirements. For example, if the regulatory requirements on privacy are strict, the mosaic ratio is high; if the regulatory requirements on privacy are loose, the mosaic ratio is low.

[0118] The embodiments of this specification implement a hierarchical random mosaic scheme for images collected during the face scanning process. The images uploaded to the server can be random mosaic images of different proportions. Random mosaics of different proportions can be masked according to the security level, privacy tolerance, regulatory requirements, etc. of different users.

[0119] Return to reference Figure 2In operation S206, the mask image is sent to the server, so that the server extracts image features from the mask image and authenticates the image features to obtain an authentication result. Figures 6 to 9 Describes the methods performed by the server, including authentication methods and training methods.

[0120] Figure 6 A flow chart illustrating an authentication method according to another embodiment of the present specification.

[0121] like Figure 6 As shown, the authentication method includes operations S602, S604 and S606. The authentication method can be, for example, Figure 1 The process is executed by the server 105 shown, but may also be executed by other devices other than the server 105.

[0122] In operation S602, a mask image is received.

[0123] In operation S604, image features are extracted from the mask image.

[0124] In operation S606, the image feature is authenticated to obtain an authentication result.

[0125] According to an embodiment of the present specification, the mask image received in operation S602 may be generated by performing random mosaic mask processing on the original image, and may include at least one of the global mask image or the local mask image described above. A trained encoder may be used to extract image features from the mask image, and finally the extracted image features may be authenticated to obtain an authentication result.

[0126] The authentication method provided in the embodiments of this specification can directly extract image features from the mask image, avoiding the transmission of the original image in the communication link and reducing the risk of leakage of user privacy data.

[0127] In order to facilitate understanding of the method of extracting image features from a mask image, the training method of the embodiment of this specification is first introduced below.

[0128] Figure 7 A flow chart illustrating the training method according to an embodiment of the present specification.

[0129] like Figure 7 As shown, the training method includes operations S702, S704, S706 and S708. The training method can be, for example, Figure 1 The process is executed by the server 105 shown, but may also be executed by other devices other than the server 105.

[0130] In operation S702 , training data is obtained, where the training data includes a sample image and a label image, where the sample image is a mask image generated by randomly mosaicking the label image.

[0131] In operation S704, the sample image is processed by an encoder to obtain image features.

[0132] In operation S706, the image features are processed by a decoder to obtain a restored image, and the label image is used for supervision to train encoder parameters and decoder parameters.

[0133] In operation S708, the image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

[0134] According to the embodiments of this specification, in the training data, each sample image has a corresponding label image, and the sample image is a mask image generated by randomly mosaicking the label image. Due to the randomness of the mosaic, the same label image can be used to generate multiple sample images, which is conducive to constructing more training data. As an example, Figure 4D and Figure 4E The images shown can constitute a set of training data, where Figure 4E For the sample image, Figure 4D is the label image.

[0135] According to the embodiments of this specification, since only the label image can be input, the machine automatically generates the corresponding sample image without manual annotation, and self-supervised training can be achieved. In the training, a self-attention mechanism is also introduced to better utilize the position relationship of the image blocks in the image and improve the feature extraction effect.

[0136] According to the embodiments of this specification, the model structure used in training is as follows Figure 8 As shown, it includes two data processing links, the first data processing link includes an encoder (encoder 1) and a decoder, and the second data processing link includes an encoder (encoder 2) and a classifier. Among them, encoder 1 and encoder 2 share parameters and can be understood as the same encoder.

[0137] On the one hand, in the first data processing link, the encoder encodes the sample image into a feature vector (for example, a 1024-dimensional feature vector), and then restores it pixel by pixel through the decoder to obtain a restored image, and the result is supervised by the label image to train the encoder parameters and the decoder parameters to make the restored image closer to the label image. The encoder and the decoder can form a set of adversarial networks.

[0138] On the other hand, in the second data processing link, the encoder encodes the sample image into a feature vector, and then classifies it through the classifier to obtain the classification result, and supervises the result with the label image to train the encoder parameters and the classifier parameters so that the classification result of the sample image matches the label image. In this way, the encoder can pay more attention to the personalized features in the sample image rather than the common features.

[0139] After training in the above two links, the features extracted by the encoder can not only be used to restore the original image well, but also have a large enough difference from other images, thus effectively improving the accuracy of authentication.

[0140] According to an embodiment of the present specification, operations S704, S706 and S708 may be repeatedly performed, and the training of the two links may be performed simultaneously or alternately until the training is finally completed.

[0141] After introducing the training method of the embodiment of this specification, return to continue to refer to Figure 6 .

[0142] In operation S604, the mask image can be processed using an encoder trained by the above method to obtain image features. The image features contain the feature information of the original image. In operation S606, the image features are authenticated to obtain an authentication result. For example, authentication features of multiple authentication images in the authentication library can be obtained; when the image features successfully match at least one of the authentication features, the authentication is determined to be successful. Among them, a large number of images can be stored in the authentication library, and features are extracted separately for matching during matching; or, the authentication library only stores feature information of a large number of images, that is, authentication features, and the image features are directly matched with the authentication features during matching. Classic algorithms can be used for feature matching, which will not be repeated here. According to the method of the embodiments of this specification, operations S604 and S606 are independent algorithms, which reduce the risk of data leakage and improve data security.

[0143] As described above, according to the embodiment of the present specification, the mask image may include at least one of a local mask image and a global mask image. In the case where the mask image has only a local mask image or a global mask image, authentication may be performed directly according to the above method. In the case where the mask image includes a local mask image and a global mask image, the two types of images need to be processed separately.

[0144] According to an embodiment of the present specification, the training method and the authentication method as described above can be applied to process a local mask image. In the process of processing the local mask image, the encoder used is called a local encoder. That is, the mask image includes a local mask image and a global mask image, and the extracting of image features from the mask image includes processing the local mask image using a local encoder to obtain image features.

[0145] The global mask image can be directly stored and restored when a dispute occurs; or it can be restored first and the restored image can be stored for query when a dispute occurs.

[0146] According to an embodiment of the present specification, the training process of the model used by the global mask image can be similar to the training process of the model used by the local mask image, and the model structure in the training stage may include a global encoder, a global decoder, and a global classifier. However, the two are trained independently, and their respective model parameters are trained separately, which can better meet their respective needs.

[0147] According to an embodiment of the present specification, in the prediction stage, the processing of the global mask image can retain the link of the global encoder and the global decoder to obtain the global restored image. That is, the method also includes processing the global mask image using the global encoder and the global decoder to obtain the global restored image.

[0148] Fig. 9 The flowchart of the authentication method according to another embodiment of the present specification is shown.

[0149] like Fig. 9 As shown, the method may include operations S902, S904, S906, S908, and S910.

[0150] In operation S902, a mask image is received, wherein the mask image includes a global mask image and a local mask image.

[0151] In operation S904, the global mask image is processed using a global encoder and a global decoder to obtain a global restored image.

[0152] In operation S906, the local mask image is processed using a local encoder to obtain image features.

[0153] In operation S908, authentication features of a plurality of authentication images in the authentication library are obtained.

[0154] In operation S910, if the image feature successfully matches at least one of the authentication features, it is determined that the authentication is successful.

[0155] According to the embodiments of the present specification, the server can not only extract and identify features of the face area, but also effectively restore the scene image, thereby effectively checking the intention risk.

[0156] The authentication method provided in the embodiments of this specification can combine the random scale mask image reconstruction and feature extraction algorithms to perform large-scale face retrieval and recognition during the server-side identification process. In the entire authentication communication link, the original scene image and the original face image cannot be seen, which can effectively protect user privacy.

[0157] The above content describes in detail the authentication method and training method of the embodiment of the present disclosure. Correspondingly, this specification also provides an authentication device and a training device, such as Figure 10-12 shown.

[0158] Fig.10 A schematic diagram illustrating an authentication device according to an embodiment of the present specification, the device comprising:

[0159] A first obtaining module 1002 is configured to obtain an original image;

[0160] The mask module 1004 is configured to perform random mosaic mask processing on the original image to obtain a mask image;

[0161] The sending module 1006 is configured to send the mask image to the server, where the mask image is used by the server to extract image features from the mask image and authenticate the image features to obtain an authentication result.

[0162] Fig.11 A schematic diagram illustrating another authentication device according to an embodiment of the present specification, the device comprising:

[0163] The receiving module 1102 is configured to receive a mask image, where the mask image is generated by performing a random mosaic masking process on the original image;

[0164] A feature extraction module 1104 is configured to extract image features from the mask image;

[0165] The authentication module 1106 is configured to authenticate the image features and obtain an authentication result.

[0166] Fig.12 A schematic diagram of a training device according to an embodiment of the present specification is shown, the device comprising:

[0167] The second obtaining module 1202 is configured to obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image;

[0168] The encoding module 1204 is configured to process the sample image through an encoder to obtain image features;

[0169] A first training module 1206 is configured to process the image features through a decoder to obtain a restored image, and use the label image for supervision to train encoder parameters and decoder parameters;

[0170] The second training module 1208 is configured to process the image features through a classifier to obtain a classification result, and use the label image for supervision to train the encoder parameters and the classifier parameters.

[0171] The present specification also provides an electronic device for executing the authentication method and training method based on the above-described embodiments of the present disclosure.

[0172] The embodiment of the present specification also provides an electronic device, including:

[0173] at least one processor; and,

[0174] a memory communicatively connected to the at least one processor; wherein,

[0175] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0176] Get the original image;

[0177] Performing random mosaic masking processing on the original image to obtain a masked image; and

[0178] The mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image, and authenticate the image features to obtain an authentication result.

[0179] The embodiment of the present specification also provides an electronic device, including:

[0180] at least one processor; and,

[0181] a memory communicatively connected to the at least one processor; wherein,

[0182] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0183] receiving a mask image, wherein the mask image is generated by performing a random mosaic masking process on the original image;

[0184] extracting image features from the mask image; and

[0185] The image feature is authenticated to obtain an authentication result.

[0186] The embodiment of the present specification also provides an electronic device, including:

[0187] at least one processor; and,

[0188] a memory communicatively connected to the at least one processor; wherein,

[0189] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0190] Obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image;

[0191] Processing the sample image through an encoder to obtain image features;

[0192] Processing the image features through a decoder to obtain a restored image, and using the label image for supervision to train encoder parameters and decoder parameters;

[0193] The image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

[0194] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0195] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0196] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0197] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0198] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0199] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0200] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0201] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0202] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0205] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0206] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0207] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0208] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0209] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0210] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0211] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An authentication method, comprising: Get the original image; Determine an image to be processed based on the original image; Adaptively determine a mosaic ratio based on one of a scenario, a user, and a regulatory requirement; The image to be processed is divided into a plurality of image blocks, and some of the image blocks in the plurality of image blocks are randomly masked based on the mosaic ratio to obtain a mask image; The mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image, and authenticate the image features to obtain an authentication result.

2. According to the method of claim 1, the image to be processed includes the original image, or the image to be processed includes a partial image of the original image, or the image to be processed includes the original image and a partial image of the original image.

3. The method according to claim 2, wherein determining the image to be processed based on the original image comprises: Identifying a local area containing feature information for authentication from the original image; Extracting the local area to obtain a local image of the original image; An image to be processed is determined based on the partial image of the original image.

4. The method according to claim 1, wherein the adaptively determining the mosaic ratio based on one of a scenario, a user, and a regulatory requirement comprises: The mosaic ratio is adaptively determined based on a scene, wherein there is a first scene and a second scene different from the first scene, and the mosaic ratio determined based on the first scene is different from the mosaic ratio determined based on the second scene.

5. The method according to claim 1, wherein the adaptively determining the mosaic ratio based on one of a scenario, a user, and a regulatory requirement comprises: The mosaic ratio is adaptively determined based on users, wherein there is a first user and a second user different from the first user, and the mosaic ratio determined based on the first user is different from the mosaic ratio determined based on the second user.

6. The method according to claim 1, wherein the adaptively determining the mosaic ratio based on one of a scenario, a user, and a regulatory requirement comprises: Determine the mosaic ratio based on regulatory requirements.

7. An authentication method, comprising: receiving a mask image; The mask image is obtained by randomly masking some image blocks among the plurality of image blocks based on a mosaic ratio; The mosaic ratio is adaptively determined based on one of a scene, a user, and a regulatory requirement; the plurality of image blocks are obtained by segmenting the image to be processed; the image to be processed is determined based on the original image; extracting image features from the mask image; The image feature is authenticated to obtain an authentication result.

8. The method according to claim 7, wherein extracting image features from the mask image comprises: The mask image is processed using an encoder to obtain image features.

9. The method according to claim 7, wherein the mask image comprises a local mask image and a global mask image, and the extracting image features from the mask image comprises: The local mask image is processed using a local encoder to obtain image features.

10. The method according to claim 9, further comprising: The global mask image is processed using a global encoder and a global decoder to obtain a global restored image.

11. According to the method according to any one of claims 7 to 10, authenticating the image feature to obtain an authentication result comprises: Obtain authentication features of multiple authentication images in an authentication library; When the image feature successfully matches at least one of the authentication features, it is determined that the authentication is successful.

12. A model training method, comprising: Obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image; The mask image is obtained by randomly masking some image blocks in a plurality of image blocks based on a mosaic ratio; the mosaic ratio is adaptively determined based on one of a scene, a user and a regulatory requirement; the plurality of image blocks are obtained by segmenting the image to be processed; the image to be processed is determined based on a label image; Processing the sample image through an encoder to obtain image features; Processing the image features through a decoder to obtain a restored image, and using the label image for supervision to train encoder parameters and decoder parameters; The image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

13. An authentication device, comprising: A first acquisition module is configured to obtain an original image; A first determining module is configured to determine an image to be processed based on the original image; A second determination module is configured to adaptively determine a mosaic ratio based on one of a scenario, a user, and a regulatory requirement; a mask module configured to divide the image to be processed into a plurality of image blocks, and randomly mask some of the image blocks in the plurality of image blocks based on the mosaic ratio to obtain a mask image; The sending module is configured to send the mask image to the server, and the mask image is used by the server to extract image features from the mask image and authenticate the image features to obtain an authentication result.

14. An authentication device, comprising: A receiving module, configured to receive a mask image; The mask image is obtained by randomly masking some image blocks among the plurality of image blocks based on a mosaic ratio; The mosaic ratio is adaptively determined based on one of a scene, a user, and a regulatory requirement; the plurality of image blocks are obtained by segmenting the image to be processed; the image to be processed is determined based on the original image; A feature extraction module, configured to extract image features from the mask image; The authentication module is configured to authenticate the image features and obtain an authentication result.

15. A model training device, comprising: The second acquisition module is configured to obtain training data, wherein the training data includes a sample image and a label image, wherein the sample image is a mask image generated by randomly mosaicking the label image; the mask image is obtained by randomly masking part of a plurality of image blocks based on a mosaic ratio; the mosaic ratio is adaptively determined based on one of a scene, a user, and a regulatory requirement; the plurality of image blocks are obtained by segmenting an image to be processed; and the image to be processed is determined based on the label image; An encoding module, configured to process the sample image through an encoder to obtain image features; A first training module is configured to process the image features through a decoder to obtain a restored image, and use the label image for supervision to train encoder parameters and decoder parameters; The second training module is configured to process the image features through a classifier to obtain a classification result, and use the label image for supervision to train the encoder parameters and the classifier parameters.

16. An electronic device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Get the original image; Determine an image to be processed based on the original image; Adaptively determine a mosaic ratio based on one of a scenario, a user, and a regulatory requirement; The image to be processed is divided into a plurality of image blocks, and some of the image blocks in the plurality of image blocks are randomly masked based on the mosaic ratio to obtain a mask image; The mask image is sent to the server, and the mask image is used by the server to extract image features from the mask image, and authenticate the image features to obtain an authentication result.

17. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: receiving a mask image; the mask image is obtained by randomly masking some image blocks of a plurality of image blocks based on a mosaic ratio; the mosaic ratio is adaptively determined based on one of a scene, a user, and a regulatory requirement; the plurality of image blocks are obtained by segmenting an image to be processed; the image to be processed is determined based on an original image; extracting image features from the mask image; and The image feature is authenticated to obtain an authentication result.

18. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Obtain training data, the training data including a sample image and a label image, the sample image being a mask image generated by randomly mosaicking the label image; the mask image being obtained by randomly masking a portion of a plurality of image blocks based on a mosaic ratio; the mosaic ratio being adaptively determined based on one of a scene, a user, and a regulatory requirement; the plurality of image blocks being obtained by segmenting an image to be processed; the image to be processed being determined based on the label image; Processing the sample image through an encoder to obtain image features; Processing the image features through a decoder to obtain a restored image, and using the label image for supervision to train encoder parameters and decoder parameters; The image features are processed by a classifier to obtain a classification result, and the label image is used for supervision to train the encoder parameters and the classifier parameters.

Images