Image detection method and device
The neural network model enhances liveness detection accuracy by extracting local and global features, including border recognition, to effectively counter paper and screen attacks, addressing the limitations of existing liveness detection technologies.
Patent Information
- Application Number
- CN202411173753.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-26
AI Technical Summary
Current liveness detection technologies struggle with low accuracy in distinguishing between real faces and forged faces, particularly in attacks involving paper printing and mobile phone screen displays, as they primarily focus on facial texture and global features without effectively addressing non-face-based attacks.
A neural network model is trained using a sample image set to extract both local and global features, with a detection head recognizing borders and a classifier determining whether an image is living, enhancing the model's ability to detect paper and screen attacks by fusing these features.
The proposed method significantly improves the accuracy of liveness recognition by effectively identifying and preventing counterfeiting effects from paper printing and screen attacks, leveraging both detailed facial and global image information.
Smart Images

Figure CN118692155B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of image recognition, and in particular, to a method and device for image detection. Background Art
[0002] Target recognition technology is currently widely used in many fields such as attendance clocking in, face payment, access control, etc., providing convenience for people's lives. However, there are news reports of using paper, mobile phone screens and other devices to display the target face to achieve successful access. In order to provide people with a convenient life experience while ensuring the security of facial information, liveness detection technology came into being. Liveness detection technology is mainly used to defend against simple flat attacks such as paper printing and mobile phone screen display, as well as three-dimensional attacks such as complex 3D face head models, and distinguish these forged faces from real faces.
[0003] Current liveness detection technologies generally focus on detailed texture features and global features on the face. However, there are many attack features for liveness detection, such as paper and screen attacks, where the forged features exist not only on the face, but outside the face. At this time, the global information obtained from the scaled full-image input image will greatly enhance the model's defense against such attacks.
[0004] There is currently no effective solution to the above problems. Summary of the invention
[0005] The embodiments of the present invention provide a method and device for image detection, so as to at least solve the problem of low accuracy of living body recognition in the related art.
[0006] According to one embodiment of the present invention, there is provided a method for image detection, comprising: inputting an image to be detected into a target neural network model, performing feature extraction on the image to be detected by a first encoder of the target neural network model to obtain target local features, wherein the target neural network model is obtained by training an initial neural network model using a sample image set; performing feature extraction on the image to be detected by a second encoder of the target neural network model to obtain target global features; performing border recognition on the target global features by a detection head of the target neural network model to obtain border features of the image to be detected; fusing the target local features with the target global features to obtain target fused features; inputting the border features and the target fused features into a classifier of the target neural network model to obtain an output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living image.
[0007] In an exemplary embodiment, the target neural network model is obtained by training an initial neural network model using a sample image set, including: acquiring the sample image set, wherein the sample images in the sample image set include living sample images and non-living sample images; training the initial neural network model using the sample image set, and terminating the training when a target convergence condition is met to obtain the target neural network model, wherein the initial neural network model includes: the detection head, a domain discriminator, a classifier, the first encoder, and the second encoder, the detection head is used to perform border detection on the non-living sample image, the domain discriminator is used to discriminate the presentation mode of the sample image, the classifier is used to identify the label of the sample image, the label includes: a label for indicating that the sample image is the living sample image or a label for indicating that the sample image is the non-living sample image, the first encoder is used to extract local features of the sample image, and the second encoder is used to extract global features of the sample image.
[0008] In an exemplary embodiment, the target convergence condition includes at least a first loss value and a second loss value, wherein the loss value between the predicted border coordinates of the non-living sample image output by the detection head and the actual border coordinates of the non-living sample image is determined as the first loss value; the loss value between the value of the first predicted label of the sample image output by the classifier and the value of the actual label of the sample image is determined as the second loss value, wherein the first predicted label is obtained by inputting a fused feature after fusing the local features output by the first encoder and the global features output by the second encoder into the classifier.
[0009] In an exemplary embodiment, the target convergence condition also includes at least one of the following: a third loss value, the third loss value is a loss value between the value of the second predicted label of the sample image output by the classifier and the value of the actual label of the sample image, and the value of the second predicted label is obtained by inputting the local features output by the first encoder into the classifier; an optimization value, the optimization value is the value obtained by inputting the discrimination result output by the domain discriminator into the optimization function.
[0010] In an exemplary embodiment, the training is terminated when the target convergence condition is met to obtain the target neural network model, including: when the target convergence condition includes at least the first loss value and the second loss value, the weighted sum of the first loss value and the second loss value is determined as the first target loss value, and when the first target loss value is less than or equal to a first preset threshold, it is determined that the target convergence condition is met; when the target convergence condition includes the first loss value, the second loss value and the third loss value, the weighted sum of the first loss value, the second loss value and the third loss value is determined as the second target loss value, and when the second target loss value is less than or equal to the second preset threshold, it is determined that the target convergence condition is met; when the target convergence condition includes the first loss value, the second loss value, the third loss value and the optimization value, the weighted sum of the first loss value, the second loss value, the third loss value and the optimization value is determined as the third target loss value, and when the third target loss value is less than or equal to the third preset threshold, it is determined that the target convergence condition is met.
[0011] In an exemplary embodiment, obtaining a set of sample images includes: performing the following operations on each image acquired by an image acquisition device, wherein the image performing the following operations is determined as the current image: obtaining a face frame for selecting a face area in the current image; cropping the current image with the long side of the face frame as the side length to obtain a first image, wherein the first image contains the face frame; cropping the current image with a first preset multiple of the long side of the face frame and a second preset multiple of the wide side of the face frame as the side length to obtain a second image, wherein the second image contains the face frame; and obtaining the sample image through the first image and the second image.
[0012] In an exemplary embodiment, the sample image is obtained by using the first image and the second image, including: compressing the first image and the second image according to a first size respectively to obtain a first target image and a second target image, wherein the first target image and the second target image are used to input the first encoder when training the initial neural network model; compressing the current image according to a second size to obtain a current target image, and the current target image is used to input the second encoder when training the initial neural network model; and determining the first target image, the second target image and the current target image as the sample images included in the sample image set.
[0013] According to another embodiment of the present invention, there is provided an image detection device, comprising: a first extraction module, used to input an image to be detected into a target neural network model, and perform feature extraction on the image to be detected through a first encoder of the target neural network model to obtain target local features, wherein the target neural network model is obtained by training an initial neural network model using a sample image set; a second extraction module, used to perform feature extraction on the image to be detected through a second encoder of the target neural network model to obtain target global features; an identification module, used to perform border recognition on the target global features through a detection head of the target neural network model to obtain border features of the image to be detected; a fusion module, used to fuse the target local features with the target global features to obtain target fused features; a classification module, used to input the border features and the target fused features into a classifier of the target neural network model to obtain an output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living image.
[0014] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of any of the above methods when executed by a processor.
[0015] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0016] According to yet another embodiment of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0017] According to the present invention, the sample image set is used to train the initial neural network model to obtain the target neural network model, and the target neural network model is used to identify whether the image to be detected is a living body image.
[0018] Since the target neural network model includes a first encoder and a second encoder, the first encoder is used to extract local features of the image to be detected, and the second encoder is used to extract global features of the image to be detected. In this way, the global information of the image and the detailed information of the face can be extracted respectively by the first encoder and the second encoder. In addition, the target neural network model also includes a detection head, which can extract the border features of the image to be detected, so that the borders of the screen and paper of the non-living image can be auxiliary detected, thereby helping the branch network to extract more forged information about paper and screen attacks from the whole image, and the target neural network model identifies whether the image to be detected is a living image based on the fusion features of the border features, the target local features and the target global features. The network not only focuses on the global and detailed features in the face, but also greatly improves the model's ability to prevent counterfeiting effects on paper printing and screen attacks. Therefore, the problem of low accuracy of live recognition in the related technology can be solved, and the effect of improving the accuracy of live recognition can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a hardware structure block diagram of a mobile terminal according to an image detection method of an embodiment of the present invention;
[0020] Figure 2 is a flowchart of an image detection method according to an embodiment of the present invention;
[0021] Figure 3 is a flowchart of image preprocessing according to an embodiment of the present invention;
[0022] Figure 4 is a training flow chart of a target neural network model according to an embodiment of the present invention;
[0023] Figure 5 is a structural block diagram of an image detection device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.
[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0026] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of an image detection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method of image detection in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0028] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0029] In this embodiment, a method running on the above mobile terminal is provided. Figure 2 is a flow chart of an image detection method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:
[0030] Step S202, inputting the image to be detected into a target neural network model, performing feature extraction on the image to be detected through a first encoder of the target neural network model to obtain target local features, wherein the target neural network model is obtained by training an initial neural network model using a sample image set;
[0031] The image to be detected can be a real face, a 3D head model, or a printed face image, etc.; the first encoder is used to extract local features of the image to be detected, such as detailed texture of the face, etc. Optionally, by extracting features from the image to be detected, detailed information on the face can be obtained, such as some small differences between the features of a real face under conditions of reflection, shadow, noise, etc. and the features of paper printing, screen and 3D head model under the same conditions; or, by extracting features from the image to be detected, background information near the face can be obtained, such as the face covering the entire printed paper, or the entire display screen, and some small differences in the fit between the head model and the face in the 3D head model. Using the first encoder to extract local features will have a better tolerance rate for possible inaccurate face frames, and can more stably extract detailed features that can correctly classify living and non-living objects. Optionally, the target neural network model is obtained by training an initial neural network model using a sample image set, including: obtaining the sample image set, wherein the sample images in the sample image set include living sample images and non-living sample images; using the sample image set to train the initial neural network model, and terminating the training when a target convergence condition is met to obtain the target neural network model, wherein the initial neural network model includes: the detection head, the domain discriminator, the classifier, the first encoder and the second encoder, the detection head is used to perform border detection on the non-living sample image, the domain discriminator is used to discriminate the presentation mode of the sample image, the classifier is used to identify the label of the sample image, the label includes: a label for indicating that the sample image is the living sample image or a label for indicating that the sample image is the non-living sample image, the first encoder is used to extract local features of the sample image, and the second encoder is used to extract global features of the sample image.
[0032] The above-mentioned sample image set can be an image captured by an image acquisition device, and a set of images obtained after preprocessing, and the preprocessing can be cropping, compression, border marking, etc.; the above-mentioned living samples can be real face images, and the above-mentioned non-living samples can be disguised face images, such as faces printed on paper, faces displayed on mobile phones or tablet screens, or simulated face images such as 3D head models; the sample image set is obtained by collecting living and non-living images and performing a series of preprocessing on the collected images.
[0033] The second encoder is used to extract global features of the image, and the detection head is used to perform border detection on non-living sample images containing borders such as paper or screens based on the global features extracted by the second encoder. The detection head is used to detect the borders of the screen and paper to extract global information from the full image input for the auxiliary branch network, which improves the network's ability to extract forged features other than faces, and can greatly improve the model's anti-counterfeiting effect on paper printing and screen attacks.
[0034] The domain discriminator is used to discriminate the presentation mode of the sample image based on the fused features, such as real face, paper attack, 3D head model, etc. The classifier is used to discriminate the label of the sample image based on the fused features, such as living body, non-living body. By using different encoders to extract global information and facial detail information, the corresponding encoder's ability to extract specific information can be improved, and the corresponding information can be extracted when the network parameters are small, so as to achieve better anti-counterfeiting effect under small network parameters, reduce the overall size of the network, and improve the inference speed, so as to be deployed in a low-cost and efficient manner, reducing the difficulty of deployment in actual scenarios.
[0035] Step S204, extracting features of the to-be-detected graph through the second encoder of the target neural network model to obtain target global features;
[0036] Step S206, performing border recognition on the target global features through the detection head of the target neural network model to obtain the border features of the image to be detected;
[0037] The second encoder extracts global features and identifies the borders of paper, screen, etc. based on the global features, so that when learning features, only screen and paper attacks containing faces are learned, thereby reducing the interference of paper or screen devices that do not contain faces, greatly improving the model's defense effect against screen and printed face attacks while reducing the probability of misclassification. At the same time, since the features extracted from real faces that do not contain obvious borders and other attack methods such as 3D head models are not sent to the detection head to identify borders, the second encoder can learn more global features and learn features that can distinguish between living and non-living objects on a larger scale.
[0038] Step S208, fusing the target local feature and the target global feature to obtain a target fused feature;
[0039] Step S210, inputting the border feature and the target fusion feature into the classifier of the target neural network model to obtain the output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living image.
[0040] Optionally, the executor of the above steps may be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least an image acquisition device and a data processing device, wherein the image acquisition device may include a graphics acquisition module such as a camera, and the data processing device may include a computer, a mobile phone or other terminal, but is not limited thereto.
[0041] Through the above steps, the image to be detected is input into multiple encoders to extract the global information of the image and the detailed information of the face respectively, and the detection head is used to perform auxiliary detection on the borders of the screen and paper at the same time, so as to help the branch network extract more forged information about paper and screen attacks from the whole image, so that the network not only focuses on the global and detailed features of the face, but also greatly improves the model's ability to prevent counterfeiting effects on paper printing and screen attacks. The problem of low accuracy of liveness recognition in related technologies is solved, and the accuracy of liveness recognition is improved.
[0042] As an optional embodiment, the target convergence condition includes at least a first loss value and a second loss value, wherein the loss value between the predicted border coordinates of the non-living sample image output by the detection head and the actual border coordinates of the non-living sample image is determined as the first loss value; the loss value between the value of the first predicted label of the sample image output by the classifier and the value of the actual label of the sample image is determined as the second loss value, wherein the first predicted label is obtained by inputting a fused feature after fusing the local features output by the first encoder and the global features output by the second encoder into the classifier.
[0043] The first loss value can be used to constrain the border detected by the detection head, and the second loss value can be used to constrain the label value predicted by the classifier. By using the borders of the screen and paper to extract global information from the full image input for the auxiliary branch network, and using the detection frame (detection head) of the paper and screen to ensure that the network extracts the correct object information (paper, screen), it helps the branch network extract more forged information about paper and screen attacks from the full image, so as to greatly improve the model's ability to prevent counterfeiting of paper printing and screen attacks.
[0044] Optionally, the first loss value is obtained by the following formula: :
[0045] ;
[0046] in, is the actual bounding box coordinate of the non-living sample image, are the predicted bounding box coordinates of the non-living sample image.
[0047] Optionally, the second loss value is obtained by the following formula: :
[0048] ;
[0049] in, is the number of sample images, is an integer greater than 1, , is the value of the actual label of the i-th sample image, is the value of the first predicted label of the i-th sample image.
[0050] As an optional embodiment, the target convergence condition also includes at least one of the following: a third loss value, the third loss value is a loss value between the value of the second predicted label of the sample image output by the classifier and the value of the actual label of the sample image, and the value of the second predicted label is obtained by inputting the local features output by the first encoder into the classifier; an optimization value, the optimization value is the value obtained by inputting the discrimination result output by the domain discriminator into the optimization function.
[0051] The third loss value is used to constrain the local features extracted by the first encoder, and by comparing the predicted label value based on the local features with the actual label value, it is possible to more stably extract detailed features that can correctly classify living and non-living objects.
[0052] The above optimization value is used to reduce the feature distance between real faces, and to increase the feature difference between different attack types (and non-living images) by constraining the fusion features through the domain discriminator, thereby improving the accuracy of recognition between features. Optionally, the third loss value is obtained by the following formula: :
[0053] ;
[0054] The optimized value is obtained by the following formula :
[0055] ;
[0056] in, is the number of sample images, is an integer greater than 1, , is the value of the actual label of the i-th sample image, is the value of the second predicted label of the i-th sample image, is the fusion feature of the sample image, is the discrimination result output by the domain discriminator.
[0057] As an optional implementation, the training is terminated when the target convergence condition is met to obtain the target neural network model, including:
[0058] In a case where the target convergence condition includes at least the first loss value and the second loss value, a weighted sum of the first loss value and the second loss value is determined as a first target loss value, and in a case where the first target loss value is less than or equal to a first preset threshold, it is determined that the target convergence condition is satisfied;
[0059] In a case where the target convergence condition includes the first loss value, the second loss value, and the third loss value, a weighted sum of the first loss value, the second loss value, and the third loss value is determined as a second target loss value, and in a case where the second target loss value is less than or equal to a second preset threshold, it is determined that the target convergence condition is satisfied;
[0060] When the target convergence condition includes the first loss value, the second loss value, the third loss value and the optimization value, the weighted sum of the first loss value, the second loss value, the third loss value and the optimization value is determined as the third target loss value. When the third target loss value is less than or equal to a third preset threshold, it is determined that the target convergence condition is met.
[0061] As an optional implementation, when the target convergence condition includes the first loss value, the second loss value, the third loss value and the optimization value, the third target loss value is determined by the following formula:
[0062] ;
[0063] in, is the second loss value, is the third loss value, is the optimized value, is the first loss value, is the target loss value, λ 1 , 2 , 3 , 4 is a preset weight value; when the third target loss value is less than or equal to a third preset threshold, it is determined that the initial neural network model meets the target convergence condition, and the target neural network model is obtained.
[0064] As an optional implementation, obtaining a set of sample images includes: performing the following operations on each image acquired by an image acquisition device, wherein the image performing the following operations is determined as the current image: obtaining a face frame in the current image for selecting a face area; cropping the current image with the long side of the face frame as the side length to obtain a first image, wherein the first image contains the face frame; cropping the current image with a first preset multiple of the long side of the face frame and a second preset multiple of the wide side of the face frame as the side length to obtain a second image, wherein the second image contains the face frame; and obtaining the sample image through the first image and the second image.
[0065] Optionally, the sample image is obtained through the first image and the second image, including: compressing the first image and the second image according to a first size respectively to obtain a first target image and a second target image, wherein the first target image and the second target image are used to input the first encoder when training the initial neural network model; compressing the current image according to a second size to obtain a current target image, and the current target image is used to input the second encoder when training the initial neural network model; and determining the first target image, the second target image and the current target image as the sample images included in the sample image set.
[0066] For example, Figure 3 is a flowchart of image preprocessing according to an embodiment of the present invention, such as Figure 3 As shown, the specific processing method of the image collected by the acquisition device is as follows:
[0067] S1, cropping. The image is cropped in three ways:
[0068] First, the original size of the face frame is taken as the reference, and the short side is expanded to make the entire cropping frame into a square, and then a square area is cropped to obtain a square image; second, the original size of the face frame is taken as the reference, the length and width are expanded outward by one time to form a rectangular area, and then the rectangular area is cropped to obtain a rectangular image; third, the input image is obtained without any cropping.
[0069] S2, compression. After the cropping is completed, the square image and the rectangular image are scaled to 112x112 size to obtain the first target image and the second target image; the input image is scaled to 224x224 size to obtain the current target image.
[0070] S3, normalization, normalizes the scaled image. At the same time, during training, different domain labels are given to the images in the sample image set according to real faces and different attack types, such as screen, paper, 3D mask and other attack methods; for each image, it contains the position coordinates of the face and the corresponding face frame coordinates, and for images of attack types such as screen and paper, it also contains the position coordinates of the corresponding screen and paper frame and other borders.
[0071] By cropping the acquired images in various ways, multi-scale information, such as global information and facial details, can be extracted based on the cropped images of different sizes, so that not only the forged features on the face are focused on, but also the global information outside the face is focused on, which greatly improves the model's defense against attacks and improves the accuracy of model recognition. As an optional implementation, Figure 4 is a training flow chart of a target neural network model according to an embodiment of the present invention, such as Figure 4 As shown, the specific process is as follows:
[0072] Step S401, the image collected from the acquisition device is processed in three ways, and the three cropping methods are: 1. Based on the original size of the face frame, the short side is expanded to make the entire cropping frame a square and then a square area is cropped; 2. Based on the original size of the face frame, the length and width are doubled outward to form a rectangular area, and then the rectangular area is cropped; 3. No cropping is performed on the input image. After cropping, the square face and the doubled face are scaled to 112x112 size, and the uncropped image is scaled to 224x224 size, and then these scaled images are normalized; during training, these input images are given different domain labels according to real faces and different attack types, such as screen, paper, 3D head model mask and other attack methods. For each image, it contains the face and the corresponding face frame coordinate position. At the same time, for screen and paper attacks, there are corresponding screen and paper frames to mark the position of the screen and paper.
[0073] Step S402, the fully scaled image (input image) after preprocessing is sent to the second encoder for extracting global features. For the global features, different processing is performed according to the category to which the input image belongs during the training phase. For example, the image categories are divided into two major categories, one is images with obvious borders such as paper print attacks, mobile phone attacks, tablet attacks, etc.; the other is images without obvious borders such as real faces and 3D head models. The set of sample images sent to the network during training is composed of a random mixture of real faces and various attack types. After these images undergo the second feature extraction, according to the above-mentioned image classification, the extracted features are also divided into two categories: features F21 extracted from input data with obvious borders, and features F22 extracted from input data such as real faces and 3D head models without obvious borders;
[0074] For feature F22, no operation is performed. For feature F21, it is sent to the detection head to predict the paper or screen border, and the SmoothL1 loss function is used to constrain the predicted paper or screen border, so that the second encoder can learn global information from the complete image input, and extract special features belonging to attack types such as screen devices and paper from a global perspective, and merge them into the fusion features in the subsequent step S404 to help the classifier classify paper and screen devices more accurately. The calculation formula of the SmoothL1 loss function is as follows:
[0075] ;
[0076] in, is the actual bounding box coordinate of the non-living sample image, are the predicted bounding box coordinates of the non-living sample image.
[0077] Step S403, the square crop (square image) and the doubled image (rectangular image) after preprocessing are respectively sent to the shared first encoder to extract local features such as face detail texture. These detail texture features will learn more face detail features. The square face and the doubled face share the first decoder, and the first decoder will learn face information input at multiple scales. The square cropped face pays more attention to detailed information on the face, such as reflection, shadow, noise and other detail problem features of the real face for all attack methods. There are generally some small differences between the print, screen and 3D head model under the same conditions. The square cropped face mainly allows the decoder to extract these detail information. For the doubled input, the background information near the face will be extracted, mainly for the face covering the entire printed paper, or the entire display screen and the common head model and head fit in the 3D head model. This complements the features in the second decoder of the full image zoom input to cope with attacks of different face proportions. At the same time, for these detailed features, they are sent to the live body classifier, the live body results are input and the cross entropy loss is used to constrain these features. The calculation formula for the binary cross entropy is as follows:
[0078] ;
[0079] in, is the number of sample images, is an integer greater than 1, , is the value of the actual label of the i-th sample image, is the value of the second predicted label of the i-th sample image.
[0080] Step S404, integrate the features extracted by the first decoder and the second decoder into a new fusion feature, and finally send the obtained fusion feature to the classifier. The classifier uses the local detail texture and global image information contained in the fusion feature to determine whether the input image belongs to a living body. For this fusion feature, it is also sent to the living body classifier for classification. By cropping the original image in different ways at the input level, using the decoder to extract living body detection classification features from inputs of different scales, and using the features extracted at different scales to complement each other, the overall accuracy of the model is improved. At the same time, during training, the image is divided into different domains according to the real type and different attack types, such as paper printing, screen attack, 3D mask, etc., and then the domain discriminator is used to constrain the fusion feature to reduce the feature distance between real faces and increase the feature difference between different attack types. The domain discriminant optimization function is as follows:
[0081] ;
[0082] in, is the number of sample images, is an integer greater than 1, , is the fusion feature of the sample image, is the discrimination result output by the domain discriminator.
[0083] After the fusion features are sent to the living body classifier, the final output result of the entire network will be output. For this classification result, the binary cross entropy loss function is also used. Constraints are imposed, and the loss function is calculated with Similarly, the loss function is as follows:
[0084] ;
[0085] in, is the number of sample images, is an integer greater than 1, , is the value of the actual label of the i-th sample image, is the value of the first predicted label of the i-th sample image.
[0086] Combining the above steps, the loss function of the whole process is as follows:
[0087] ;
[0088] in, is the second loss value, is the third loss value, is the optimized value, is the first loss value, is the target loss value, λ 1 =0.5, λ 2 =1,λ 3 =0.5, λ 4 =0.5.
[0089] Step S405, in the loss function When the value is less than or equal to the preset threshold, the training of the neural network model is determined to be completed, and the target neural network model is obtained.
[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0091] In the present embodiment, a device for image detection is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0092] Figure 5 is a structural block diagram of an image detection device according to an embodiment of the present invention, such as Figure 5 As shown, the device includes: a first extraction module 502, used to input the image to be detected into the target neural network model, and extract the features of the image to be detected through the first encoder of the target neural network model to obtain the target local features, wherein the target neural network model is obtained by training the initial neural network model using a sample image set; a second extraction module 504, used to extract the features of the image to be detected through the second encoder of the target neural network model to obtain the target global features; an identification module 506, used to perform border recognition on the target global features through the detection head of the target neural network model to obtain the border features of the image to be detected; a fusion module 508, used to fuse the target local features and the target global features to obtain the target fusion features; a classification module 510, used to input the border features and the target fusion features into the classifier of the target neural network model to obtain the output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living image.
[0093] In an exemplary embodiment, the device is also used to obtain the sample image set, wherein the sample images in the sample image set include living sample images and non-living sample images; use the sample image set to train the initial neural network model, terminate the training when the target convergence condition is met, and obtain the target neural network model, wherein the initial neural network model includes: the detection head, the domain discriminator, the classifier, the first decoder and the second decoder, the detection head is used to perform border detection on the non-living sample image, the domain discriminator is used to discriminate the presentation method of the sample image, the classifier is used to identify the label of the sample image, and the label includes: a label for indicating that the sample image is the living sample image or a label for indicating that the sample image is the non-living sample image, the first decoder is used to extract local features of the sample image, and the second decoder is used to extract global features of the sample image.
[0094] In an exemplary embodiment, the target convergence condition includes at least a first loss value and a second loss value, wherein the loss value between the predicted border coordinates of the non-living sample image output by the detection head and the actual border coordinates of the non-living sample image is determined as the first loss value; the loss value between the value of the first predicted label of the sample image output by the classifier and the value of the actual label of the sample image is determined as the second loss value, wherein the first predicted label is obtained by inputting a fused feature after fusing the local features output by the first decoder and the global features output by the second decoder into the classifier.
[0095] In an exemplary embodiment, the device is also used to include at least one of the following when the target convergence condition includes: a third loss value, the third loss value is a loss value between the value of the second predicted label of the sample image output by the classifier and the value of the actual label of the sample image, and the value of the second predicted label is obtained by inputting the local features output by the first encoder into the classifier; an optimization value, the optimization value is the value obtained by inputting the discrimination result output by the domain discriminator into the optimization function.
[0096] In an exemplary embodiment, the device is also used to, when the target convergence condition includes at least the first loss value and the second loss value, determine the weighted sum of the first loss value and the second loss value as the first target loss value, and determine that the target convergence condition is satisfied when the first target loss value is less than or equal to a first preset threshold; when the target convergence condition includes the first loss value, the second loss value and the third loss value, determine the weighted sum of the first loss value, the second loss value and the third loss value as the second target loss value, and determine that the target convergence condition is satisfied when the second target loss value is less than or equal to a second preset threshold; when the target convergence condition includes the first loss value, the second loss value, the third loss value and the optimization value, determine the weighted sum of the first loss value, the second loss value, the third loss value and the optimization value as the third target loss value, and determine that the target convergence condition is satisfied when the third target loss value is less than or equal to a third preset threshold.
[0097] In an exemplary embodiment, the apparatus is also used to perform the following operations on each image captured by an image acquisition device, wherein the image performing the following operations is determined as the current image: obtaining a face frame in the current image for selecting a face area; cropping the current image with the long side of the face frame as the side length to obtain a first image, wherein the first image contains the face frame; cropping the current image with a first preset multiple of the long side of the face frame and a second preset multiple of the wide side of the face frame as the side length to obtain a second image, wherein the second image contains the face frame; and obtaining the sample image through the first image and the second image.
[0098] In an exemplary embodiment, the device is also used to compress the first image and the second image according to a first size, respectively, to obtain a first target image and a second target image, wherein the first target image and the second target image are used to input the first encoder when training the initial neural network model; compress the current image according to a second size to obtain a current target image, and the current target image is used to input the second encoder when training the initial neural network model; and determine the first target image, the second target image and the current target image as the sample images included in the sample image set.
[0099] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0100] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0101] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0102] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0103] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0104] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method described in each embodiment of the present application when executed by a processor.
[0105] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0106] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for image detection, characterized in that: include: Inputting the image to be detected into the target neural network model, extracting features of the image to be detected by a first encoder of the target neural network model to obtain target local features, wherein the target neural network model is obtained by training an initial neural network model using a sample image set; Extracting features of the image to be detected by a second encoder of the target neural network model to obtain target global features; Performing border recognition on the target global features through the detection head of the target neural network model to obtain the border features of the image to be detected; Fusing the target local features and the target global features to obtain target fused features; Inputting the frame feature and the target fusion feature into the classifier of the target neural network model to obtain an output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living body image; The method of using the sample image set to train the initial neural network model to obtain the target neural network model includes: Acquire the sample image set, wherein the sample images in the sample image set include living sample images and non-living sample images; The initial neural network model is trained using the sample image set, and the training is terminated when the target convergence condition is met to obtain the target neural network model, wherein the initial neural network model includes: the detection head, the classifier, The target convergence condition includes at least a first loss value and a second loss value, wherein: Determine a loss value between the predicted frame coordinates of the non-living sample image output by the detection head and the actual frame coordinates of the non-living sample image as the first loss value; The loss value between the value of the first predicted label of the sample image output by the classifier and the value of the actual label of the sample image is determined as the second loss value, wherein the first predicted label is obtained by inputting a fused feature after fusing the local features output by the first encoder and the global features output by the second encoder into the classifier.
2. The method according to claim 1, characterized in that The initial neural network model includes: the detection head, the domain discriminator, the classifier, the first encoder and the second encoder, the detection head is used to perform border detection on the non-living sample image, the domain discriminator is used to discriminate the presentation mode of the sample image, the classifier is used to identify the label of the sample image, and the label includes: a label indicating that the sample image is the living sample image or a label indicating that the sample image is the non-living sample image, the first encoder is used to extract local features of the sample image, and the second encoder is used to extract global features of the sample image.
3. The method according to claim 1, characterized in that The target convergence condition also includes at least one of the following: a third loss value, wherein the third loss value is a loss value between a value of a second predicted label of the sample image output by the classifier and a value of an actual label of the sample image, wherein the value of the second predicted label is obtained by inputting the local features output by the first encoder into the classifier; The optimized value is a value obtained by inputting the discrimination result output by the domain discriminator into the optimization function.
4. The method according to claim 3, characterized in that The training is terminated when the target convergence condition is met, and the target neural network model is obtained, including: In a case where the target convergence condition includes at least the first loss value and the second loss value, a weighted sum of the first loss value and the second loss value is determined as a first target loss value, and in a case where the first target loss value is less than or equal to a first preset threshold, it is determined that the target convergence condition is satisfied; In a case where the target convergence condition includes the first loss value, the second loss value, and the third loss value, a weighted sum of the first loss value, the second loss value, and the third loss value is determined as a second target loss value, and in a case where the second target loss value is less than or equal to a second preset threshold, it is determined that the target convergence condition is satisfied; When the target convergence condition includes the first loss value, the second loss value, the third loss value and the optimization value, the weighted sum of the first loss value, the second loss value, the third loss value and the optimization value is determined as the third target loss value. When the third target loss value is less than or equal to a third preset threshold, it is determined that the target convergence condition is met.
5. The method according to claim 1, characterized in that Get the sample image collection including: Perform the following operations on each image acquired by the image acquisition device, wherein the image on which the following operations are performed is determined as the current image: Get the face frame used to select the face area in the current image; Cropping the current image according to the side length of the face frame to obtain a first image, wherein the first image contains the face frame; The current image is cropped using a first preset multiple of a long side of the face frame and a second preset multiple of a wide side of the face frame as side lengths to obtain a second image, wherein the second image contains the face frame; The sample image is obtained through the first image and the second image.
6. The method according to claim 5, characterized in that Obtaining the sample image through the first image and the second image includes: Compressing the first image and the second image according to a first size respectively to obtain a first target image and a second target image, wherein the first target image and the second target image are used to input the first encoder when training the initial neural network model; Compressing the current image according to a second size to obtain a current target image, wherein the current target image is used to input the second encoder when training the initial neural network model; The first target image, the second target image and the current target image are all determined as the sample images included in the sample image set.
7. An image detection device, characterized in that: include: A first extraction module is used to input the image to be detected into a target neural network model, and extract features of the image to be detected through a first encoder of the target neural network model to obtain target local features, wherein the target neural network model is obtained by training an initial neural network model using a sample image set; A second extraction module is used to extract features of the image to be detected through a second encoder of the target neural network model to obtain target global features; A recognition module, used for performing border recognition on the target global features through the detection head of the target neural network model to obtain the border features of the image to be detected; A fusion module, used for fusing the target local features with the target global features to obtain target fusion features; A classification module, used for inputting the frame feature and the target fusion feature into the classifier of the target neural network model to obtain an output result of the target neural network model, wherein the output result is used to indicate whether the image to be detected is a living image; The device is further used to obtain the sample image set, wherein the sample images in the sample image set include living sample images and non-living sample images; The initial neural network model is trained using the sample image set, and the training is terminated when the target convergence condition is met to obtain the target neural network model, wherein the initial neural network model includes: the detection head, the classifier, The target convergence condition includes at least a first loss value and a second loss value, wherein: Determine a loss value between the predicted frame coordinates of the non-living sample image output by the detection head and the actual frame coordinates of the non-living sample image as the first loss value; The loss value between the value of the first predicted label of the sample image output by the classifier and the value of the actual label of the sample image is determined as the second loss value, wherein the first predicted label is obtained by inputting a fused feature after fusing the local features output by the first encoder and the global features output by the second encoder into the classifier.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 6 when executed by a processor.
9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.
