Liveness detection and data processing method, device, system and storage medium
The judging of live detection by lighting information matching degree has solved the problem of the existing problems of long and low accuracy of live detection, and achieved rapid and accurate live recognition and improved safety.
Patent Information
- Application Number
- CN201910542696.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-21
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-06-21
AI Technical Summary
The existing live detection methods take a long time and have low accuracy, so they cannot effectively prevent non-living attacks, such as using photos to simulate permission theft caused by facial recognition.
By judging the matching degree between the lighting information in the image of the object to be detected and the current ambient lighting information, identifying whether the object to be detected is a living body, reducing dependence on image texture details, and improving detection efficiency and accuracy.
Fast and accurate live detection is achieved, the image quality requirements are reduced, the stability and safety of live detection are improved, and the in vivo detection is prevented from in vivo attacks.
Smart Images

Figure CN112115747B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, system and storage medium for liveness detection and data processing. Background Art
[0002] With the continuous development of artificial intelligence technology, face recognition technology is widely used to identify users. For example, in mobile payment and Internet finance, face recognition technology can be used for face-swiping login and face-swiping payment. For another example, in access control gate equipment, face recognition technology can be used to control the gate.
[0003] However, current face recognition technology is often subject to non-live attacks, that is, artificial props (photos) are used to simulate the biometric features required for face recognition, resulting in the theft of normal user permissions. To solve the above problems, liveness detection technology has emerged, but existing liveness detection methods either require the user to cooperate with actions such as shaking the head and blinking, which takes a long time, or rely on texture details in the face image, which has high requirements on the quality of the face image and low accuracy of liveness detection. Summary of the invention
[0004] Multiple aspects of the present application provide a liveness detection and data processing method, device, system and storage medium to reduce the time consumption of liveness detection and improve the efficiency and accuracy of liveness detection.
[0005] An embodiment of the present application provides a living body detection method, comprising: in a current environment, collecting an image containing an object to be detected; performing illumination analysis on the image to obtain illumination information in the image; if the illumination information in the image matches the illumination information in the current environment, determining that the object to be detected is a living body.
[0006] An embodiment of the present application also provides a data processing method, including: in the current environment, collecting a facial image of an object to be detected; performing lighting analysis on the facial image to obtain lighting information in the facial image; if the lighting information in the facial image matches the lighting information in the current environment, and the facial image is a facial image of a known object, determining that the object to be detected passes face recognition.
[0007] An embodiment of the present application also provides a data processing method, including: obtaining first lighting information characterizing ambient lighting; obtaining an image containing an object to be detected; performing lighting analysis on the image to obtain second lighting information characterizing the lighting in the image; and determining whether the object to be detected passes the detection based on the similarity between the first lighting information and the second lighting information.
[0008] An embodiment of the present application also provides a gate system, including: a gate, an image acquisition device and a computing device; the computing device is respectively connected to the gate and the image acquisition device; the image acquisition device is used to acquire a facial image of an object to be detected located around the entrance and exit of the gate, and transmit the facial image to the computing device; the computing device is used to perform lighting analysis on the facial image to obtain lighting information in the facial image, and when the lighting information in the facial image matches the lighting information in the current environment of the gate, and the object to be detected is a legal object, control the gate to open to allow the object to be detected to pass.
[0009] An embodiment of the present application also provides a living body detection device, comprising: a visual sensor, a memory and a processor; wherein the memory is used to store a computer program; the visual sensor is used to collect an image containing an object to be detected in a current environment; the processor is coupled to the memory, and is used to execute the computer program for: performing illumination analysis on the image to obtain illumination information in the image; if the illumination information in the image matches the illumination information in the current environment, determining that the object to be detected is a living body.
[0010] An embodiment of the present application also provides a gate, comprising: a gate body, and a camera, a memory and a processor arranged on the gate body; wherein the memory is used to store a computer program; the camera is used to: collect facial images of objects to be detected located around the entrance and exit of the gate body; the processor is coupled to the memory, and is used to execute the computer program for: performing lighting analysis on the facial image of the object to be detected, obtaining lighting information in the facial image, and when the lighting information in the facial image matches the lighting information in the current environment, and when the object to be detected is a legal object, controlling the gate body to open to allow the object to be detected to pass.
[0011] An embodiment of the present application also provides a face recognition device, characterized in that it includes: a visual sensor, a memory and a processor; wherein the memory is used to store a computer program; the visual sensor is used to collect a facial image containing an object to be detected in a current environment; the processor is coupled to the memory, and is used to execute the computer program for: performing lighting analysis on the facial image to obtain lighting information in the facial image; if the lighting information in the facial image matches the lighting information in the current environment, and the facial image is a facial image of a known object, it is determined that the object to be detected passes the face recognition.
[0012] An embodiment of the present application also provides a computer device, comprising: a memory and a processor; wherein the memory is used to store a computer program; the processor is coupled to the memory, and is used to execute the computer program for: obtaining first lighting information characterizing ambient lighting; obtaining an image containing an object to be detected; performing lighting analysis on the image to obtain second lighting information characterizing the lighting in the image; and determining whether the object to be detected passes the detection based on the similarity between the first lighting information and the second lighting information.
[0013] An embodiment of the present application also provides a readable storage medium storing computer instructions, characterized in that when the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned living body detection method.
[0014] An embodiment of the present application also provides a readable storage medium storing computer instructions, characterized in that when the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data processing method.
[0015] In the embodiment of the present application, by judging whether the illumination information in the image containing the object to be detected matches the illumination information in the current environment, it is determined whether the object to be detected is alive. This method of liveness detection, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0017] Figure 1a A schematic diagram of the structure of a gate system provided in an embodiment of the present application;
[0018] Figure 1b A schematic diagram of the structure of another gate system provided in an embodiment of the present application;
[0019] Figure 2 A schematic diagram of a flow chart of a liveness detection method provided in an embodiment of the present application;
[0020] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application;
[0021] Figure 4A schematic diagram of the structure of a living body detection device provided in an embodiment of the present application;
[0022] Figure 5 A schematic diagram of the structure of a face recognition device provided in an embodiment of the present application;
[0023] Figure 6 A schematic diagram of the structure of a gate provided in an embodiment of the present application;
[0024] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0026] In view of the technical problems of the prior art in liveness detection, such as long time consumption and low accuracy, in some embodiments of the present application, by judging whether the illumination information in the image containing the object to be detected matches the illumination information in the current environment, it is determined whether the object to be detected is alive. This liveness detection method, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0027] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0028] Figure 1a This is a schematic diagram of the structure of a gate system provided in an embodiment of the present application. Figure 1a As shown, the system includes: a gate machine 101, an image acquisition device 102 and a computing device 103. Among them, Figure 1a The implementation forms of the gate machine 101, the image acquisition device 102 and the computing device 103 shown in the figure are all exemplary descriptions and do not constitute limitations thereto.
[0029] In this embodiment, according to the different blocking bodies and blocking methods, the gate 101 can be a three-roller gate, a swing gate, a wing gate, a translation gate, a turnstile, a straight gate, etc., but not limited thereto. In addition to being implemented in the form of a gate, the gate 101 can also be implemented in the form of a door with access control function, a baffle, etc., but not limited thereto. This embodiment does not limit the number of gates 101, which can be one or more. Figure 1a In the schematic diagrams of other corresponding embodiments, for the convenience of illustration, one gate 101 is taken as an example, but this does not limit the number of gates 101.
[0030] In this embodiment, the image acquisition device 102 may be a visual sensor such as a camera, a laser sensor or an infrared sensor. For example, the image acquisition device 102 may be a binocular camera, a monocular camera or a depth of field camera, but is not limited thereto. The image acquisition device 102 may be arranged on the gate 101, or may be arranged near the entrance and exit of the gate 101. In this embodiment, the acquisition angle of the image acquisition device 102 covers the area around the entrance and exit of the gate 101, and images around the entrance and exit of the gate 101 may be acquired. When the object to be detected is located in the area around the entrance and exit of the gate 101, the image acquisition device 102 may acquire a facial image of the object to be detected. In one scenario, the image acquisition device 102 is arranged at the entrance of the gate 101, and the image acquisition angle of the image acquisition device 102 is toward the entrance of the gate 101. When the object to be detected enters through the entrance of the gate 101, the face of the object to be detected is opposite to the acquisition angle of the image acquisition device 102, and the image acquisition device 102 may acquire a facial image of the object to be detected.
[0031] In this embodiment, the computing device 103 is respectively connected to the gate 101 and the image acquisition device 102 in communication, and is mainly responsible for processing the image acquired by the image acquisition device 102 and controlling the gate 101 to open or close. This embodiment does not limit the implementation form of the computing device 103, and it can be any computer device, processor or processing chip with certain computing capabilities. Taking a computer device as an example, the computing device 103 can be a terminal device such as a desktop computer, a laptop computer or a smart phone, or a server-side device such as a conventional server, a cloud server or a server array. Of course, the computing device 103 can also be a processor with certain processing capabilities.
[0032] The computing device 103 may be located locally at the gate 101 or in the cloud. In the case where the computing device 103 is located locally at the gate 101, the computing device 103 may be a processor disposed on the gate 101, or may be a terminal device such as a desktop computer, a laptop computer, or a smart phone disposed in the local space where the gate 101 is located. Of course, it may also be a conventional server or server array disposed in the local space where the gate 101 is located. In the case where the computing device 103 is located in the cloud, the computing device 103 may be a server-side device such as a conventional server, a cloud server, or a server array.
[0033] Further, in the case where the computing device 103 is a terminal device or a server device, no matter whether the computing device 103 is located in the gate 101 or in the cloud, the computing device 103 can be connected to the gate 101 by wireless or wired communication. Similarly, it can also be connected to the image acquisition device 102 by wireless or wired communication. In the case where the computing device 103 is connected to the gate 101 and the image acquisition device 102 by wireless communication, the computing device 103 can be connected to the gate 101 and the image acquisition device 102 by mobile network. The network standard of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G and other new standards that may appear in the future. Optionally, the computing device 103 can also be connected to the gate 101 and the image acquisition device 102 by wireless communication, such as Bluetooth, WiFi, infrared, etc.
[0034] Further, in this embodiment, the image acquisition device 102 may provide the acquired facial image of the object to be detected to the computing device 103. Optionally, the computing device 103 may read the facial image of the object to be detected from the storage space of the image acquisition device 102, or the image acquisition device 102 may actively transmit the facial image of the object to be detected to the computing device 103.
[0035] Furthermore, in practical applications, Figure 1aAs shown, paper photos or electronic photos of legal users are often used to generate biometric features required for face recognition, resulting in the theft of legal users' rights. Since paper photos or electronic photos carry the lighting information when the photos were taken, even if the illegal user takes the paper photos or electronic photos of the legal user to the image acquisition device 102 of the gate system for the image acquisition device 102 to collect the facial image in the photo, the lighting in the current environment around the gate 101 cannot overwhelm the original lighting information in the photo. Therefore, if the object to be detected uses props such as photos for the image acquisition device 102 to collect, the facial image of the object to be detected collected by the image acquisition device 102 includes not only the lighting information in the current environment, but also the lighting information carried in the photo; if the object to be detected is a real person, the facial image of the object to be detected collected by the image acquisition device 102 only carries the lighting information in the current environment. Moreover, the performance of light phenomena such as refraction, reflection, and diffuse reflection of light by real faces and photos is also different.
[0036] Based on the above analysis, in this embodiment, the computing device 103 can perform lighting analysis on the facial image of the object to be detected, and then obtain the lighting information in the facial image. Further, the lighting information in the facial image of the object to be detected is matched with the lighting information in the current environment; if the lighting information in the facial image of the object to be detected matches the lighting information in the current environment, the computing device 103 can determine that the object to be detected is a living body; otherwise, it can be determined that the object to be detected is not a living body. Optionally, if the computing device 103 determines that the object to be detected is not a living body, a control instruction prohibiting passage can be sent to the gate 101. Accordingly, the gate 101 can respond to the control instruction and display a prohibition mark information. For example, Figure 1a As shown, an “×” mark is displayed on the main body of the gate 101.
[0037] The illumination information in the current environment includes illumination information generated by various light source devices in the environment space where the gate 101 is located. Figure 1b As shown, the gate system also includes a light source device 104. The light source device 104 is responsible for providing lighting for the environment space where the gate 101 is located. On the other hand, it can also provide the image acquisition device 102 with the lighting information required for acquiring the facial image of the object to be detected.
[0038] Furthermore, the light source device 104 may also be different depending on the environment in which the gate 101 is located. For example, when the gate system is an access control system for a community or a company, the light source device 104 may include street lights, or natural light sources, etc., but is not limited thereto. For another example, when the gate system is used in indoor places such as subway stations, railway stations, and airports that require customs clearance verification, the light source device 104 may include indoor lighting, indoor billboards with light sources, or natural light sources that can provide natural light to the room, etc., but is not limited thereto. In addition, if the image acquisition device 102 has a built-in fill light, then the light source device 104 may also be the built-in fill light of the image acquisition device 102.
[0039] Further, for security reasons, in practical applications, the object to be detected needs to be not only a living body, but also a legal object. Among them, a legal object refers to a user who has the authority to pass through the gate 101. For example, in places such as subway stations, train stations or airports where tickets are required to enter the station, a legal object refers to a user who has purchased a ticket or an air ticket from this station. For another example, in places such as a community or a company, a legal object refers to a resident of the community or an employee of the company. For another example, in places such as museums and commercial buildings, a legal object refers to a person holding his or her valid ID. Further, in this embodiment, when the illumination information in the facial image of the object to be detected matches the illumination information in the current environment of the gate 101, and the object to be detected is a legal object, the computing device 103 can control the gate 101 to open to allow the detected object to pass through the gate 101.
[0040] Optionally, the computing device 103 may send a control instruction or a control signal to the gate control component of the gate 101, and the gate control component performs a corresponding action according to the received control instruction or control signal. For example, the computing device 103 may send a gate opening instruction or a gate opening control signal to the gate control component of the gate 101 when the object to be detected passes the verification set by the gate system, and the gate control component controls the gate of the gate 101 to open according to the received gate opening instruction or gate opening control signal. For another example, the computing device 103 may send a gate prohibition opening instruction or a gate prohibition opening control signal to the gate control component of the gate 101 when the object to be detected fails to pass the verification set by the gate system, and the gate control component keeps the gate 101 in a closed state according to the received instruction or control signal to prohibit the object to be detected from passing.
[0041] In the gate system provided by this embodiment, the computing device can identify whether the object to be detected is alive by judging whether the lighting information in the facial image of the object to be detected collected by the image acquisition device matches the lighting information in the current environment. This liveness detection method, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency, thereby helping to shorten the time the object to be detected stays at the gate; on the other hand, it can reduce the dependence on texture details in the facial image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0042] Furthermore, in this embodiment, the liveness detection process has relatively low requirements on image quality and the accuracy requirements on the image acquisition equipment. Based on this, some low-precision image acquisition equipment can be used to capture the facial image of the object to be detected, such as a monocular camera, which is conducive to cost saving.
[0043] It is worth noting that the gate system provided in the embodiment of the present application can be applied to a variety of places that require customs clearance, such as railway stations, airports, art venues, cinemas, sports arenas, toll booths, commercial buildings, etc. On the other hand, the gate system provided in the embodiment of the present application can also be applied to the access control system of places such as residential areas, companies, dormitory buildings, and apartment buildings, especially unmanned access control systems, which can improve the security of the access control system.
[0044] In an embodiment of the present application, the computing device 103 may perform illumination analysis on the facial image of the object to be detected to obtain illumination information in the facial image. In an embodiment of the present application, the computing device 103 may adopt a variety of methods to obtain illumination information in the facial image of the object to be detected. Among them, the illumination information includes but is not limited to: information such as the angle, color temperature, and strength of the light source. For example, the computing device 103 may use the SAIFS (shape, albedo, and illumination from shading) algorithm, the SIRFS (shape, illumination, and reflectance from shading) algorithm, or the Scene-SIRFS algorithm to solve the variable parameters of the facial image of the object to be detected, and recover the shape (depth) information, reflectivity, and illumination from the facial image. Taking the SIRFS algorithm as an example, the implementation method of the computing device 103 obtaining illumination information in the facial image of the object to be detected is exemplified below.
[0045] When using the SIFRS algorithm to obtain the illumination information in the facial image of the object to be detected, the spherical harmonics (SH) illumination model can be used to express the light source model corresponding to the facial image, wherein the coefficient matrix of the spherical harmonic illumination model represents the illumination information in the facial image. Further, the computing device 103 can solve the variable parameters of the facial image according to the prior information of the reflected image, depth information and illumination information corresponding to the facial image of the object to be detected, and then obtain the coefficient matrix of the spherical harmonic illumination model. Since the coefficient matrix of the spherical harmonic illumination model can reflect the illumination information in the facial image, the illumination information in the facial image of the object to be detected can be obtained. For ease of description, in the following embodiments, the coefficient matrix of the spherical harmonic illumination model is referred to as the illumination system matrix.
[0046] Furthermore, the SIFRS algorithm can recover illumination information, depth information, and reflectivity from a single color image. In the SIFRS algorithm, the problem of obtaining illumination information in the facial image of the object to be detected can be expressed as:
[0047]
[0048] Among them, R is the logarithmic reflectance image, Z is the depth image, L is the illumination coefficient matrix, and I is the RGB image containing the object to be detected.
[0049] R and Z are two images with the same dimension as I. P(R), P(Z) and P(L) are the prior information of reflectivity, depth information and illumination information respectively; S(Z,L) is the rendering engine, which can obtain light and shadow information according to the parameters Z and L. Formula (1) requires the solution of the optimal solution of shape (depth), reflectivity and illumination information that maximizes the probability of prior information under the condition that the RGB image is equal to the rendering model R+S(Z,L).
[0050] Furthermore, according to formula (1), we can obtain: R = IS (Z, L), and by performing a negative logarithmic transformation on P (R) P (Z) P (L), the above problem of finding the maximum likelihood can be transformed into a problem of finding the minimum sum of the loss function:
[0051]
[0052] Among them, g(R), f(Z) and h(L) are the loss functions of the reflected image, depth information and illumination information respectively. Among them, the loss function of the reflected image can be expressed as:
[0053] g(R)=λ s g s (R)+λ e g e (R)+λ a g a(R) (3)
[0054] Among them, g s (R) is the prior condition for smoothness, g e (R) is the prior condition of the entropy of reflectivity, g a (R) is the prior condition for color consistency. s , e and λ a The specific expressions of the prior conditions for smoothness, reflectivity and color consistency are prior art and will not be described in detail here.
[0055] Furthermore, the loss function of depth information can be expressed as:
[0056] f(Z)=λ k f k (Z)+λ i g i (R)+λ c g c (R) (4)
[0057] Among them, f k (Z) represents the prior condition of smoothness, f i (Z) represents the prior condition of orientation, f c (Z) represents the prior condition of the appearance contour. k , i and λ c The specific expressions of the prior conditions of smoothness, orientation and appearance profile are prior art and will not be described in detail here.
[0058] Furthermore, the prior information of the illumination information can be represented by matching the illumination information in the training library using a multivariate Gaussian model. Based on this, the loss function of the illumination information can be expressed as:
[0059]
[0060] Among them, μ L and∑ L is the parameter of the Gaussian function, λ L is the coefficient of prior information. L is the illumination coefficient matrix.
[0061] Furthermore, the rendering engine S(Z,L) is used to calculate the brightness S of a point on the surface of an object, where the surface normal direction is n i . The rendering engine can be expressed as:
[0062] S(n i ,L)=[n i ; 1] T M[n i; 1] (6)
[0063] in, c1=0.429043, c2=0.511664, c3=0.743125, c4=0.886227, c5=0.247708; L1-L9 represent the illumination coefficient matrix of each color channel (R channel, G channel and B channel in RGB channel) respectively. Among them, the representation of M in formula (6) is only exemplified by using the 2nd-order spherical harmonic basis to construct the SH illumination model. If the SH function is constructed using other higher-order spherical harmonic basis, the representation of M is changed accordingly, which is not shown here one by one.
[0064] Furthermore, an unconstrained optimization method can be used to solve the minimum value of the loss function shown in formula (2), and the illumination coefficient matrix corresponding to the minimum value of the loss function is used as the illumination information in the facial image of the object to be detected. For the convenience of description and distinction, in the embodiment of the present application, the illumination coefficient matrix corresponding to the minimum value of the loss function is represented as L t . The unconstrained optimization method may be Newton's method or quasi-Newton's method. Further, quasi-Newton's method includes: DFS algorithm, DFGS algorithm, LBFGS algorithm, etc., but is not limited thereto. The specific process of using Newton's method or quasi-Newton's method to solve the minimum value of formula (2) is a prior art and will not be described in detail here.
[0065] Optionally, considering that diffuse radiance is a low-frequency function, the diffuse radiance can be simulated using a second-order spherical harmonic basis, that is, a SH illumination model is constructed using a second-order spherical harmonic basis to express the light source model corresponding to the facial image of the object to be detected. Further, if a second-order spherical harmonic basis is used to construct the SH illumination model, when calculating the matrix coefficients of the SH illumination model, each color channel (three channels of RGB) corresponds to 9 illumination coefficients, that is, the coefficient matrix of the SH illumination model includes 27 parameters, and these 27 parameters are used to describe the illumination information.
[0066] Further, after the computing device 103 obtains the lighting information in the facial image of the object to be detected, the lighting information in the facial image can be matched with the lighting information in the current environment where the gate 101 is located. Optionally, the computing device 103 can calculate the similarity between the lighting information in the facial image of the object to be detected and the lighting information in the current environment where the gate 101 is located. Further, if the calculated similarity is greater than or equal to a preset lighting similarity threshold, the computing device 103 determines that the lighting information in the facial image of the object to be detected matches the lighting information in the current environment, and can also determine that the facial image of the object to be detected is a living image. Correspondingly, if the calculated similarity is less than the preset lighting similarity threshold, the computing device 103 determines that the lighting information in the facial image of the object to be detected does not match the lighting information in the current environment where the gate 101 is located, and can determine that the facial image of the object to be detected is not a living image.
[0067] Optionally, the computing device 103 may use a deep learning model to calculate the similarity between the illumination information in the facial image of the object to be detected and the illumination information in the current environment where the gate 101 is located. For example, an Alex network model, a VGG network model, a Google network model, a Res network model, or a CNN network model may be used to calculate the similarity between the illumination information in the facial image of the object to be detected and the illumination information in the current environment where the gate 101 is located, but is not limited thereto.
[0068] Furthermore, before the gate system performs the liveness detection work, the illumination information of the current environment in which the gate 101 is located can also be estimated and calculated. For the convenience of description and distinction, the illumination information in the current environment is represented as L e The following is an exemplary description of an implementation method for obtaining the lighting information in the current environment where the gate machine 101 is located.
[0069] In the initial working stage of the gate system, since the gate system does not know the lighting information in its current environment, the computing device 103 needs to pre-calculate the lighting information in the current environment where the gate 101 is located. Optionally, the image acquisition device 102 can acquire the initial environment image corresponding to the environmental space to which the current environment of the gate 101 belongs, and provide the initial environment image to the computing device 103. Accordingly, the computing device 103 can perform lighting analysis on the initial environment image to obtain initial lighting information L0. Among them, the computing device 103 can use the above-mentioned implementation method of performing lighting analysis on the image to be detected to perform lighting analysis on the initial environment image. The specific implementation method can be found in the above-mentioned related content and will not be repeated here. Based on this, in the initial working stage of the gate system, the computing device 103 can use the pre-obtained initial lighting information L0 in the environmental space to which the current environment of the gate 101 belongs as the lighting information in the current environment.
[0070] Of course, in some scenarios, the lighting information in the environment space where the gate 101 is located is relatively stable and will not change over time. In other working stages of the gate system, the initial lighting information can also be used as the lighting information in the current environment where the gate 101 is located.
[0071] In some other application scenarios, the light source in the environment space where the gate 101 is located will change to a certain extent. In order to improve the accuracy of the lighting information in the current environment, an update cycle can be preset in the computing device 103, and a timer or counter can be started to time the update cycle. Whenever the update cycle arrives, the computing device 103 controls the image acquisition device 102 to acquire the environment image of the environment space to which the gate 101 belongs as the initial environment image within the cycle, and provides the initial environment image to the computing device 103. Accordingly, the computing device 103 can perform lighting analysis on the initial environment image within each cycle to obtain the initial lighting information L0. Accordingly, the computing device 103 can use the initial lighting information L0 in the environment space to which the gate 101 belongs obtained in the current cycle as the lighting information in the current environment where the image acquisition device 102 is located in the cycle.
[0072] In order to further improve the accuracy of the illumination information in the current environment where the image acquisition device 102 is located, the illumination information in the N historical images in which the detected object is determined to be a living body can also be used to calculate the illumination information in the current environment where the image acquisition device 102 is located, where N is a positive integer. Furthermore, if N ≥ 2, the average of the illumination information in the N historical images can be used as the illumination information in the current environment, that is, Among them, L i They are respectively the lighting information in N historical images. Preferably, the N historical images are the most recently acquired N historical images, that is, the N historical images that are closest to the current moment and contain the object to be detected that is determined to be alive. Because the acquisition time of these N historical images is close to the acquisition time of the current moment, the lighting information in them is closer to the lighting information in the current environment. In this way, when the computing device 103 determines whether the lighting information in the facial image of the object to be detected is consistent with the lighting information in the current environment, the lighting information in the current environment used by the computing device 103 is consistent with the lighting information in the actual environment space where the gate 101 is located, which helps to improve the accuracy of subsequent liveness detection.
[0073] In practical applications, for security reasons, the object to be detected needs to be not only a living body, but also a legal object. Only legal objects that are living bodies have the authority to pass through the gate 101. Based on this, the computing device 103 also needs to determine whether the facial image of the object to be detected matches the facial image of a known object, and if the lighting information in the facial image of the object to be detected matches the lighting information in the current environment of the gate 101, and the object to be detected is a legal object, the computing device 103 can control the gate 101 to open to allow the detected object to pass through the gate 101. Optionally, the computing device 103 can match the facial image of the object to be detected in the facial image of a known object. If the facial image of the object to be detected is matched in the facial image of the known object, it is determined that the object to be detected is a legal object.
[0074] Furthermore, the computing device 103 may calculate the similarity between the facial image of the object to be detected and the facial image of the known object. If there is a facial object in the facial image of the known object whose similarity to the facial image of the object to be detected is greater than or equal to a preset image similarity threshold, it means that the facial image of the object to be detected is matched in the facial image of the known object, and it can be determined that the object to be detected is a legal object.
[0075] It is worth noting that the liveness detection method and the legal object verification method provided in the embodiment of the present application can be executed sequentially or in parallel. When the two are executed sequentially, the order of execution is not limited.
[0076] Further, in the embodiment of the present application, the facial image of the known object may be a facial image of a legal object acquired in advance and stored in the computing device 103; or it may be a facial image of a valid identification document provided by the object to be detected acquired on-site by the gate system. The valid identification document may be an identification card, a passport, a student ID card, a work card, etc., but is not limited thereto.
[0077] In practical applications, if the gate system is an access control system for a company, a community, or a unit building, since the personnel in these application places are relatively fixed, the facial images of the personnel in these places can be acquired in advance as facial images of known objects and stored in the computing device 103. If the gate system is a passage system for application places with relatively mobile personnel such as railway stations, bus stations, airports, museums, and commercial buildings, it is difficult to collect facial images of all personnel in advance due to the high mobility of personnel in these places. Therefore, in these application places, the facial images in the valid identity documents provided by the objects to be detected can be acquired on site.
[0078] Furthermore, for the case where the gate system obtains the facial image of the valid identity document provided by the object to be detected on site, such as Figure 1bAs shown, the gate system also includes: an identity information collection device 105. Among them, the identity collection device 105 is connected to the computing device 103, and its connection method can refer to the connection method between the above-mentioned image acquisition device 102 and the computing device 103, which will not be repeated here. Further, the identity information collection device 105 can be a camera, a scanner, an identifier based on OCR (Optical Character Recognition) technology, a card reader, etc., but is not limited to this. Optionally, the identity information collection device 105 can be set on the gate 101, or it can be a peripheral device set at the entrance and exit of the gate 101. Figure 1b In the figure, only the identity information collection device 105 is arranged on the gate 101 for illustration, but this is not limited to the above.
[0079] In this embodiment, the identity information collection device 105 can obtain the facial image in the identity document provided by the object to be detected as the facial image of the known object, and provide the facial image of the known object to the computing device. Accordingly, the computing device 103 can calculate the similarity between the facial image of the object to be detected and the facial image of the known object. If the calculated similarity is greater than or equal to the preset image similarity threshold, the object to be detected is determined to be a legal object.
[0080] In addition, in some application scenarios, for example, at railway stations, bus stations, airports, etc., tickets and valid identity documents are required to enter the waiting hall or terminal hall. The object to be detected not only needs to provide a valid identity document, but also needs to provide a ticket or air ticket. Accordingly, the identity information acquisition device 105 can also obtain the associated information of the ticket or air ticket provided by the object to be detected, and provide the acquired associated information of the ticket or air ticket to the computing device 103. Accordingly, the computing device 103 can determine whether the ticket or air ticket information provided by the object to be detected is valid according to the current time and the physical space location of the gate 101. For example, the gate system is set at the entrance of a certain railway station, and the identity information acquisition device 105 can obtain the train number, departure time, starting station and other information in the train ticket provided by the object to be detected, and provide this information to the computing device 103. The computing device 103 can determine whether the ticket provided by the object to be detected meets the entry time according to the current time and the departure time in the train ticket; further, the computing device 103 can determine whether the station is the boarding point of the object to be detected according to the station information and the train number and starting station provided by the object to be detected. Furthermore, if the ticket provided by the subject to be detected meets the entry time, and the station is the corresponding boarding point to be detected, it can be determined that the ticket provided by the subject to be detected is valid. Among them, the train ticket is valid means that the subject to be detected can enter the station at the current time with this ticket.
[0081] It is worth noting that in some application scenarios, on the basis of the above-mentioned liveness detection and facial recognition of the object to be detected, other verification conditions can be added to determine whether the object to be detected has the authority to pass through the gate 101. For example, in application scenarios where special groups of people can enjoy discounts with valid certificates that they can enjoy discounts. For example, when students buy train tickets, they can enjoy a half-price discount with their ID cards; for example, in some scenic spots, the elderly can visit for free with their ID cards or soldiers can visit for free with their military certificates, etc. In these application scenarios, the gate system can also verify the valid certificates that can enjoy discounts provided by the object to be detected, etc. In these application scenarios, in order to improve security, the computing device 103 can control the gate 101 to open when the object to be detected passes all the verifications set by the gate system to allow the object to be detected to pass.
[0082] In addition to the gate system provided in the embodiment of the present application, the embodiment of the present application also provides a liveness detection method and a data processing method. The liveness detection method and the data processing method provided in the embodiment of the present application are respectively described as examples below.
[0083] Figure 2 A flow chart of a liveness detection method provided in an embodiment of the present application. Figure 2 As shown, the method includes:
[0084] 201. In the current environment, an image containing an object to be detected is collected.
[0085] 202. Perform illumination analysis on the collected image to obtain illumination information in the image.
[0086] 203. If the illumination information in the image matches the illumination information in the current environment, it is determined that the object to be detected is a living body.
[0087] In this embodiment, for step 201, an image acquisition device such as a camera, a laser sensor or an infrared sensor can be used to collect images of the object to be detected in the current environment. Among them, the camera can be a binocular camera, a monocular camera or a depth of field camera, etc., but is not limited thereto. Depending on the application scenario, the setting position of the image acquisition device varies. For example, in a gate system, the image acquisition device can be set on the gate, or set near the entrance and exit of the gate. For another example, when a user uses a terminal device to log in or pay by face scanning, or when a user uses a terminal device in a bank business hall to handle business independently, the image acquisition device can be a camera on the terminal device, and so on. Optionally, if the image acquisition device is a monocular camera, the monocular camera in the current environment can be used to collect images containing the object to be detected. Among them, the use of a monocular camera can reduce costs.
[0088] Furthermore, in practical applications, paper photos or electronic photos of legal users are often used to generate biometric features required for face recognition, resulting in the theft of legal users' rights. Since paper photos or electronic photos carry the lighting information when the photos were taken, even if the illegal user takes the paper photos or electronic photos of legal users to the image acquisition device for the image acquisition device to collect the facial image in the photo, the lighting in the current environment cannot overwhelm the original lighting information in the photo. Therefore, if the object to be detected uses props such as photos for the image acquisition device to collect, the image of the object to be detected collected by the image acquisition device includes not only the lighting information in the current environment, but also the lighting information carried in the photo; if the object to be detected is a real person, the facial image of the object to be detected collected by the image acquisition device only carries the lighting information in the current environment. Moreover, the performance of light phenomena such as refraction, reflection, and diffuse reflection of light by real people and photos is also different. Among them, the lighting information in the current environment is the lighting information generated by the light source device in the environment space where the living body detection device is located. Among them, the description of the lighting device can refer to the relevant content of the above-mentioned gate system embodiment, which will not be repeated here.
[0089] Based on the above analysis, in step 202, the facial image of the object to be detected can be subjected to illumination analysis to obtain illumination information in the facial image. Further, if the illumination information in the facial image of the object to be detected matches the illumination information in the current environment, the object to be detected is determined to be a living body.
[0090] In this embodiment, it is possible to identify whether the object to be detected is alive by judging whether the illumination information in the image containing the object to be detected matches the illumination information in the current environment. This method of liveness detection, on the one hand, does not require the object to be detected to cooperate with actions, which can reduce the time of liveness detection and improve detection efficiency; on the other hand, it can reduce the dependence on texture details in facial images, which not only reduces the quality requirements for images, but also has high stability, which helps to improve the accuracy of liveness detection.
[0091] Furthermore, the present application has relatively low requirements for image quality and the accuracy of the image acquisition equipment, so some low-precision image acquisition equipment can be used to capture the facial image of the object to be detected, such as a monocular camera, which is conducive to cost saving.
[0092] In the embodiment of the present application, a variety of methods can be used to obtain the illumination information in the image of the object to be detected. For example, the SAIFS algorithm, SIRFS algorithm or Scene-SIRFS algorithm can be used to solve the variable parameters of the image of the object to be detected, and the shape (depth information), reflectivity and illumination can be restored from the image of the object to be detected. When using the SIFRS algorithm to obtain the illumination information in the image of the object to be detected, the spherical harmonic illumination model can be used to express the light source model corresponding to the facial image, wherein the coefficient matrix of the spherical harmonic illumination model represents the illumination information in the facial image. Further, the facial image can be solved for the variable parameters according to the prior information of the reflected image, depth information and illumination information corresponding to the image of the object to be detected, and then the matrix coefficients of the spherical harmonic function are obtained. Since the coefficient matrix of the spherical harmonic illumination model can reflect the illumination information in the image, the illumination information in the image of the object to be detected is obtained. Among them, the specific implementation method of using the SIFRS algorithm to perform illumination analysis on the image containing the object to be detected can refer to the relevant content in the above-mentioned gate system, which will not be repeated here.
[0093] Further, after obtaining the illumination information in the image of the object to be detected, the illumination information in the image can be matched with the illumination information in the current environment. Optionally, the similarity between the illumination information in the image of the object to be detected and the illumination information in the current environment can be calculated. Further, if the calculated similarity is greater than or equal to a preset illumination similarity threshold, it is determined that the illumination information in the image of the object to be detected matches the illumination information in the current environment, and it can also be determined that the object to be detected is a living body. Correspondingly, if the calculated similarity is less than the preset illumination similarity threshold, it is determined that the illumination information in the image of the object to be detected does not match the illumination information in the current environment, and it can be determined that the object to be detected is not a living body.
[0094] Optionally, a deep learning model may be used to calculate the similarity between the illumination information in the image of the object to be detected and the illumination information in the current environment. For example, an Alex network model, a VGG network model, a Google network model, a Res network model, or a CNN network model may be used to calculate the similarity between the illumination information in the image of the object to be detected and the illumination information in the current environment, but is not limited thereto.
[0095] It is worth noting that in the embodiment of the present application, in order to improve the accuracy of liveness detection, after step 201, facial feature recognition can also be performed on the image containing the object to be detected, and the facial area image in the image can be determined. Afterwards, the facial area image is subjected to illumination analysis to obtain illumination information in the facial area image. Further, if the illumination information in the facial area image matches the illumination information in the current environment, it is determined that the object to be detected is a living body. Using the illumination information of the facial area image of the object to be detected for liveness detection can, on the one hand, improve the accuracy of liveness detection, and on the other hand, can also reduce the amount of calculation, increase the processing speed, and thus help improve the detection efficiency. Among them, the implementation method of illumination analysis of the facial area image can refer to the above-mentioned relevant content of illumination analysis of the image of the object to be detected, which will not be repeated here.
[0096] Furthermore, before performing the liveness detection work, the light information of the current environment in which the liveness detection device is located can also be estimated and calculated. For the convenience of description and distinction, the light information in the current environment is represented as L e The following is an exemplary description of the implementation of the illumination information in the current environment where the living body detection device is located.
[0097] In the initial working stage of the living body detection device, since the living body detection device does not know the lighting information in its current environment, it is necessary to pre-calculate the lighting information in the current environment where the living body detection device is located. Optionally, the initial environment image corresponding to the environmental space to which the current environment of the living body detection device belongs can be collected, and the initial environment image can be subjected to lighting analysis to obtain the initial lighting information L0. Among them, the above-mentioned implementation method of performing lighting analysis on the image to be detected can be adopted to perform lighting analysis on the initial environment image. The specific implementation method can be referred to the above-mentioned related content, which will not be repeated here. Based on this, in the initial working stage of the living body detection device, the initial lighting information L0 in the environmental space to which the current environment of the living body detection device belongs, which is obtained in advance, can be used as the lighting information in the current environment.
[0098] Of course, in some scenarios, the lighting information in the environment space where the liveness detection device is located is relatively stable and will not change over time. In other working stages of the liveness detection device, the initial lighting information can also be used as the lighting information in the current environment where the liveness detection device is located.
[0099] In some other application scenarios, for example, users use their terminal devices to make face-scanning payments or log in, etc. Since the environment in which the terminal device is located may vary greatly each time the user makes face-scanning payments or logs in, each time the object to be detected makes face-scanning payments or logs in, an initial environment image corresponding to the environmental space to which the current environment of the terminal device belongs can be collected, and lighting analysis is performed on the initial environment image to obtain initial lighting information as the lighting information of the current environment in which the terminal device is located.
[0100] In some other application scenarios, the light source in the environment space where the liveness detection device is located will change to a certain extent. In order to improve the accuracy of the lighting information in the environment space to which the current environment belongs, an update cycle can be preset, and a timer or counter can be started to time the update cycle. Whenever the update cycle arrives, the initial environment image corresponding to the environment space to which the current environment of the liveness detection device belongs is collected, and the initial environment image is analyzed for lighting to obtain the initial lighting information L0. Accordingly, the initial lighting information L0 in the environment space to which the current environment of the liveness detection device belongs obtained in the current cycle can be used as the lighting information in the current environment within the cycle.
[0101] In order to further improve the accuracy of the lighting information in the environment space to which the current environment belongs, the lighting information in N historical images containing objects to be detected and determined to be living can be used to calculate the lighting information in the current environment, where N is a positive integer. Furthermore, if N ≥ 2, the average of the lighting information in these N historical images can be used as the lighting information in the current environment, that is, Among them, L i are respectively the illumination information in N historical images. Preferably, the N historical images are the most recently acquired N historical images, that is, the N historical images in which the object to be detected closest to the current moment is determined to be alive. Because the acquisition time of these N historical images is close to the acquisition time of the current moment, the illumination information in them is closer to the illumination information in the current environment. In this way, when the liveness detection device determines whether the illumination information in the facial image of the object to be detected is consistent with the illumination information in the current environment, the illumination information in the current environment used is consistent with the illumination information in the environmental space where the liveness detection device is actually located, which helps to improve the accuracy of subsequent liveness detection.
[0102] Furthermore, for security reasons, in actual applications, the object to be detected must not only be a living body, but also a legal object. The application scenarios of liveness detection technology are different, and the legal objects are also different. For example, if liveness detection technology is applied to a gate system, the legal object refers to a user who has the authority to pass through the gate. For another example, if liveness detection technology is applied to the application scenario of face-swiping login or face-swiping payment, the legal object is the user who has registered an account. In the case that the object to be detected is a living body and a legal object, subsequent services can be provided to the object to be detected.
[0103] Furthermore, when detecting whether the object to be detected is a legitimate object, a facial region image of the object to be detected can be obtained, and the facial region image can be matched with the facial region image of a known object; if the facial region image of the object to be detected is matched with the facial region image of the known object, and the object to be detected is determined to be a living body, subsequent services are provided for the object to be detected. Figure 2 The liveness detection provided in the method can be executed sequentially or in parallel. When the two are executed sequentially, the order of execution is not limited.
[0104] Furthermore, the similarity between the facial area image of the object to be detected and the facial area image of the known object can be calculated. If there is a facial object in the facial area image of the known object whose similarity to the facial area image of the object to be detected is greater than or equal to a preset image similarity threshold, it means that the facial area image of the object to be detected is matched in the facial image of the known object, and the object to be detected can be determined to be a legal object.
[0105] Furthermore, in different application scenarios, different subsequent services can be provided for the object to be detected. The following is an exemplary description in combination with several common application scenarios.
[0106] Application scenario 1: When the liveness detection method provided in this embodiment is applied to a gate system, when the facial region image of the object to be detected is matched with the facial region image of the known object, and the object to be detected is determined to be a live object, the gate of the gate can be controlled to open to allow the object to be detected to pass. For the definition of the known object in this application scenario, please refer to the relevant content of the gate system embodiment above, which will not be repeated here.
[0107] Application scenario 2: When the liveness detection method provided in this embodiment is applied to face payment, if the facial area image of the object to be detected is matched in the facial area image of the known object, and the object to be detected is determined to be alive, the payment service can be provided to the object to be detected in combination with the payment account bound to the object to be detected. For example, the amount set by the object to be detected can be deducted from the payment account bound to the object to be detected, or the amount set by the object to be detected can be transferred from the payment account bound to the object to be detected to the collection account set by the object to be detected, or the payment page can be displayed to the object to be detected, etc.
[0108] In application scenario 2, the facial image of the known object can be the facial image of the user bound to the payment account set for the object to be detected. The facial image can be a facial image in a valid identification document provided by the user when registering the account, or it can be a facial image of the user collected by the bank that opens the payment account; or it can be a facial image in a valid identification document provided by the object to be detected collected on site. Furthermore, if the facial image of the known object is a facial image in a valid identification document provided by the object to be detected collected on site, the facial image in the collected valid identification document can also be matched with the facial image of the user bound to the payment account. If the calculated similarity between the two is greater than or equal to the set similarity threshold, the facial image in the collected valid identification document can be used as the facial image of the known object.
[0109] Application scenario 3: The liveness detection method provided in this embodiment is applied to application scenarios such as face recognition authentication, face recognition login, or face recognition password modification. When the liveness verification is performed on the object to be detected, if the facial area image of the object to be detected is matched in the facial area image of the known object, and the object to be detected is determined to be alive, a relevant information interface can be provided for the object to be detected to obtain relevant information or perform relevant operations. For example, when the object to be detected modifies the password, the password modification page can be displayed to the object to be detected; for another example, when the object to be detected inquires about the balance, the balance information can be displayed to the object to be detected, etc., but it is not limited to this.
[0110] In application scenario 3, the facial image of the known object can be a facial image of a user bound to the user name or account input by the object to be detected. The facial image can be a facial image in a valid identification document provided by the user when registering an account, or it can be a facial image of the user collected by the background device of the user name or account input by the object to be detected.
[0111] Accordingly, an embodiment of the present application also provides a readable storage medium storing computer instructions. When these computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned living body detection method.
[0112] Figure 3 The following is a flow chart of a data processing method provided in an embodiment of the present application. The data processing method can be used for face recognition. Figure 3 As shown, the method includes:
[0113] 301. Under the current environment, collect a facial image of an object to be detected.
[0114] 302. Perform illumination analysis on the facial image to obtain illumination information in the facial image.
[0115] 303. If the illumination information in the facial image matches the illumination information in the current environment, and the facial image is a facial image of a known object, it is determined that the object to be detected passes the face recognition.
[0116] In this embodiment, for step 301, an image acquisition device such as a camera, a laser sensor or an infrared sensor can be used to acquire a facial image of the object to be detected in the current environment. The camera can be a binocular camera, a monocular camera or a depth of field camera, etc., but is not limited thereto. The setting position of the image acquisition device can refer to the relevant content of the above embodiment. Optionally, if the image acquisition device is a monocular camera, the monocular camera in the current environment can be used to acquire the facial image of the object to be detected. The use of a monocular camera can reduce costs.
[0117] Furthermore, in practical applications, paper photos or electronic photos of legal users are often used to generate biometric features required for face recognition, resulting in the theft of legal users' rights. Since paper photos or electronic photos carry the lighting information when the photos were taken, even if the illegal user takes the paper photos or electronic photos of legal users to the image acquisition device for the image acquisition device to collect the facial image in the photo, the lighting in the current environment cannot overwhelm the original lighting information in the photo. Therefore, if the object to be detected uses props such as photos for the image acquisition device to collect, the image of the object to be detected collected by the image acquisition device includes not only the lighting information in the current environment, but also the lighting information carried in the photo; if the object to be detected is a real person, the facial image of the object to be detected collected by the image acquisition device only carries the lighting information in the current environment. Moreover, the performance of light phenomena such as refraction, reflection, and diffuse reflection of light by real faces and photos is also different. Among them, the lighting information in the current environment is the lighting information generated by the light source device in the environment space where the living body detection device is located. Among them, the description of the lighting device can refer to the relevant content of the above-mentioned gate system embodiment, which will not be repeated here.
[0118] Based on the above analysis, in step 302, the facial image of the object to be detected can be subjected to illumination analysis, thereby obtaining illumination information in the facial image. Further, if the illumination information in the facial image of the object to be detected matches the illumination information in the current environment, the object to be detected is determined to be a living body. The specific implementation method of performing illumination analysis on the facial image of the object to be detected and determining whether the illumination information in the facial image of the object to be detected matches the illumination information in the current environment can be referred to the relevant contents of the above embodiments, which will not be repeated here.
[0119] Further, for security reasons, in practical applications, the object to be detected not only needs to be a living body, but also needs to be a legal object. Wherein, the description of the legal object can refer to the relevant content of the above embodiment, which will not be repeated here. Further, in the present embodiment, if the facial image of the object to be detected belongs to the facial image of a known object, the object to be detected is determined to be a legal object. In other words, if the illumination information in the facial image of the object to be detected matches the illumination information in the current environment, and the facial image of the object to be detected belongs to the facial image of a known object, it is determined that the object to be detected passes face recognition. Wherein, the specific implementation method of the facial image of a known object and determining whether the facial image of the object to be detected belongs to the facial image of a known object can refer to the relevant content of the above embodiment, which will not be repeated here.
[0120] In this embodiment, face recognition is combined with liveness detection. During the face recognition process, by judging whether the illumination information in the face image of the object to be detected acquired by the image acquisition device matches the illumination information in the current environment, it is determined whether the object to be detected is alive, which is conducive to improving the accuracy of face recognition and thus improving the security of face recognition technology. Among them, the liveness detection method based on illumination information, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the facial image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0121] Furthermore, the present application has relatively low requirements for image quality and the accuracy of the image acquisition equipment, so some low-precision image acquisition equipment can be used to capture the facial image of the object to be detected, such as a monocular camera, which is conducive to cost saving.
[0122] Further, in the present embodiment, when the object to be detected passes face recognition, subsequent services can be provided to the object to be detected. Wherein, the subsequent services provided to the object to be detected are different in different application scenarios. For example, in the above-mentioned application scenario 1, if the object to be detected passes face recognition, the gate of the gate machine can be controlled to open to allow the object to be detected to pass. For another example, in the above-mentioned application scenario 2, if the object to be detected passes face recognition, payment services can be provided for the object to be detected in combination with the payment account bound to the object to be detected. For example, the amount set by the object to be detected can be deducted from the payment account bound to the object to be detected, or the amount set by the object to be detected can be transferred from the payment account bound to the object to be detected to the collection account set by the detection object, or the payment page can be displayed to the object to be detected, etc.
[0123] For another example, in the above-mentioned application scenario 3, if the object to be detected passes face recognition, a relevant information interface can be provided for the object to be detected, so that the object to be detected can obtain relevant information or perform relevant operations. For example, when the object to be detected changes the password, the password change page can be displayed to the object to be detected; for another example, when the object to be detected checks the balance, the balance information can be displayed to the object to be detected, etc., but it is not limited thereto.
[0124] Accordingly, an embodiment of the present application also provides a readable storage medium storing computer instructions. When these computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data processing method.
[0125] In addition to the above data processing method, the embodiment of the present application also provides another data processing method. The data processing method is not only applicable to online real-time face recognition or liveness detection, but also applicable to offline face recognition or liveness detection. The data processing method includes the following steps:
[0126] S1: Acquire first illumination information representing ambient illumination.
[0127] S2: Acquire an image containing the object to be detected.
[0128] S3: Performing illumination analysis on the image containing the object to be detected to obtain second illumination information representing the illumination in the image.
[0129] S4: Determine whether the object to be detected passes the detection based on the similarity between the first illumination information and the second illumination information.
[0130] In this embodiment, in step S1, the first illumination information may be illumination information in an environment image obtained by performing illumination analysis on an environment image of the current environment captured by the image processing device by the computer device, or may be illumination information obtained by other means. For example, the first illumination information may be illumination information sent by other devices, or illumination information read from a storage medium, but is not limited thereto. The image acquisition device may be disposed on the computer device, or may be disposed in the same physical space as the computer device.
[0131] Similarly, in step S2, the image containing the object to be detected acquired by the computer device may be an image of the object to be detected collected by the image processing device on the computer device, or may be an image acquired by other means. For example, the image containing the object to be detected may be an image sent by other devices, or the computer device may be an image read from a storage medium, but is not limited thereto.
[0132] In step S3, the computer device performs illumination analysis on the image containing the object to be detected to obtain second illumination information in the image containing the object to be detected. Next, in step S4, based on the similarity between the first illumination information and the second illumination information, it is determined whether the object to be detected passes the detection.
[0133] The data processing method provided in this embodiment determines whether the object to be detected has passed the detection based on the similarity between the illumination information in the image containing the object to be detected and the illumination information in the environment. This data processing method, on the one hand, does not require the object to be detected to cooperate in action, which can reduce the time of liveness detection and improve detection efficiency; on the other hand, it can reduce the dependence on texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0134] The data processing method provided in this embodiment can be applied to gate machines or online payment environments that require real-time online liveness detection or face recognition; it is also applicable to some application scenarios that require offline liveness detection. For example, in some application scenarios, the police can use the data processing method provided in this embodiment to assist in solving cases. For example, if someone reports that his bank deposits have been stolen, the police can use the data processing method provided in this embodiment to retrieve relevant surveillance videos to determine whether the person’s report is true, that is, to determine whether he is the one who withdraws the money or someone else impersonates him to withdraw the money, etc., but is not limited to this.
[0135] Optionally, in step S3, a spherical harmonic illumination model can be used to express a light source model corresponding to the image, and the coefficient matrix of the spherical harmonic illumination model represents the illumination in the image, which can be used as the second illumination information; the image is solved for variable parameters according to the prior information of the reflected image, depth information, and illumination information corresponding to the image to obtain the coefficient matrix of the spherical harmonic illumination model. For a detailed description, please refer to the above embodiment, which will not be repeated here.
[0136] Optionally, for the application scenario of real-time online detection, the first illumination information in step S1 is the illumination information of the current environment in which the computer device is located. The specific implementation method of the computer device obtaining the illumination information in the current environment can refer to the relevant contents of the above embodiments, which will not be repeated here.
[0137] Optionally, for the application scenario of offline detection, the first illumination information in step S1 is the illumination information of the environment in which the device that collects the image containing the object to be detected is located during the process of collecting the image. Based on this, the first illumination information can perform illumination analysis on the environmental image collected by the device that collects the image containing the object to be detected to obtain the first illumination information. It is worth noting that the specific implementation process of the computer device performing illumination analysis on the environmental image and the illumination analysis on the image containing the object to be detected in step S3 can refer to the relevant content in the above embodiments, which will not be repeated here.
[0138] Regardless of which of the above application scenarios, when determining whether the object to be detected has passed the detection based on the similarity between the first illumination information and the second illumination information, the similarity between the second illumination information and the first illumination information can be calculated; if the calculated similarity is greater than or equal to the preset illumination similarity threshold, it is determined that the object to be detected has passed the detection; if the calculated similarity is less than the preset illumination similarity threshold, it is determined that the object to be detected has not passed the detection.
[0139] Accordingly, an embodiment of the present application also provides a readable storage medium storing computer instructions. When these computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned computer device executing the data processing method.
[0140] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201, 202 and 203 can be device A; for another example, the execution subject of step 201 can be device A, and the execution subject of steps 202 and 203 can be device B; and so on.
[0141] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the sequence numbers of the operations, such as 301, 302, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.
[0142] Figure 4 This is a schematic diagram of the structure of a living body detection device provided in an embodiment of the present application. Figure 4 As shown, the device includes: a visual sensor 40a, a memory 40b and a processor 40c.
[0143] The memory 40b is used to store computer programs.
[0144] The visual sensor 40a is used to collect images containing the object to be detected in the current environment.
[0145] The processor is coupled to the memory 40b and is used to execute a computer program for: performing illumination analysis on an image containing the object to be detected to obtain illumination information in the image; if the illumination information in the image containing the object to be detected matches the illumination information in the current environment, determining that the object to be detected is a living body.
[0146] Optionally, the visual sensor 40a may be a visual sensor such as a camera, a laser sensor, or an infrared sensor. For example, the visual sensor 40a may be a binocular camera, a monocular camera, or a depth-of-field camera, but is not limited thereto.
[0147] In one embodiment, when the processor 40b performs illumination analysis on an image containing an object to be detected, it is specifically used to: use a spherical harmonic illumination model to express a light source model corresponding to the image containing the object to be detected, wherein the coefficient matrix of the spherical harmonic illumination model represents the illumination information in the image containing the object to be detected; solve variable parameters of the image containing the object to be detected based on prior information of the reflected image, depth information, and illumination information corresponding to the image containing the object to be detected, to obtain a coefficient matrix of the spherical harmonic illumination model.
[0148] In another embodiment, the processor 40b is also used to: calculate the similarity between the lighting information in the image containing the object to be detected and the lighting information in the current environment; if the similarity is greater than or equal to a preset lighting similarity threshold, determine that the lighting information in the image containing the object to be detected matches the lighting information in the current environment; if the similarity is less than the preset lighting similarity threshold, determine that the lighting information in the image containing the object to be detected does not match the lighting information in the current environment.
[0149] In another embodiment, before determining that the illumination information in the image containing the object to be detected matches the illumination information in the current environment, the processor 40b is further used to: use the initial illumination information in the environment space to which the current environment belongs, which is obtained in advance, as the illumination information in the current environment; or, using the illumination information in N historical images containing the object to be detected and determined to be a living body, calculate the illumination information in the current environment, where N is a positive integer. Preferably, the N historical images are the most recently acquired N historical images.
[0150] Furthermore, when the processor 40b obtains the initial lighting information in the environmental space to which the current environment belongs in advance, it is specifically used to: collect the initial environmental image corresponding to the environmental space to which the current environment belongs; and perform lighting analysis on the initial environmental image to obtain the initial lighting information.
[0151] On the other hand, when calculating the illumination information in the current environment, the processor 40b is specifically used to calculate the average of the illumination information in N historical images as the illumination information in the current environment.
[0152] In some further embodiments, the image containing the object to be detected includes: a facial region image of the object to be detected. The processor 40b is further configured to: match the facial region image of the object to be detected in the facial region image of a known object; if the facial region image of the object to be detected is matched in the facial region image of the known object, and the object to be detected is determined to be a living body, then provide subsequent services for the object to be detected.
[0153] Furthermore, if the liveness detection device is used in a gate system, the processor 40b is specifically used to control the gate of the gate to open to allow the object to be detected to pass through when providing subsequent services for the object to be detected. If the liveness detection device is used in face payment, the processor 40b is specifically used to provide payment services for the object to be detected in combination with the payment account bound to the object to be detected. If the liveness detection device is used in other systems that can be authenticated by face brushing, the processor 40b is specifically used to display the relevant information interface for the object to be detected so that the object to be detected can obtain relevant information or perform related operations.
[0154] In some embodiments, the liveness detection device further includes: an identity information collection device 40d. The identity information collection device 40d may be a camera, a scanner, an identifier based on OCR technology, a card reader, etc., but is not limited thereto. In this embodiment, the identity information collection device 40d may obtain a facial image in an identity document provided by the object to be detected as a facial image of a known object, and provide the facial image of the known object to the processor 40c. Accordingly, the processor 40c may calculate the similarity between the facial image of the object to be detected and the facial image of the known object. If the calculated similarity is greater than or equal to a preset image similarity threshold, it is determined that the facial area image of the known object matches the facial area image of the object to be detected.
[0155] In some optional embodiments, such as Figure 4 As shown, the living body detection device may also include: a communication component 40e, a power component 40f, an audio component 40g, a display 40h and other components. Figure 4 Only some components are shown schematically, which does not mean that the liveness detection device must include Figure 4 All components shown do not mean that the liveness detection device can only include Figure 4 As shown in the components. Figure 4 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the liveness detection device. The liveness detection device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server device such as a conventional server, a cloud server, or a server array. If the liveness detection device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 4 If the liveness detection device of this embodiment is implemented as a server-side device such as a conventional server, a cloud server or a server array, it may not include Figure 4 Components within the dashed box.
[0156] The liveness detection device provided in this embodiment can identify whether the object to be detected is alive by judging whether the illumination information in the image containing the object to be detected matches the illumination information in the current environment. This liveness detection method based on illumination information, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0157] Figure 5 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of the present application. Figure 5As shown, the device comprises: a visual sensor 50a, a memory 50b and a processor 50c. The memory 50a is used to store computer programs.
[0158] The processor 50c is coupled to the memory 50b, and is used to execute a computer program for: performing lighting analysis on the facial image to obtain lighting information in the facial image; if the lighting information in the facial image matches the lighting information in the current environment, and the facial image belongs to a facial image of a known object, it is determined that the object to be detected passes face recognition.
[0159] Furthermore, if the object to be detected passes the face recognition, the processor 40b can also provide subsequent services for the object to be detected. For example, if the face recognition device is used in a gate system, the processor 40b is specifically used to control the gate of the gate to open to allow the object to be detected to pass. If the face recognition device is used in face payment, the processor 40b is specifically used to provide subsequent services for the object to be detected. It is used to provide payment services for the object to be detected in combination with the payment account bound to the object to be detected. If the face recognition device is used in other systems that can be authenticated by face brushing, the processor 40b is specifically used to provide subsequent services for the object to be detected. Display the relevant information interface for the object to be detected, so that the object to be detected can obtain relevant information or perform related operations.
[0160] In some embodiments, the face recognition device further includes: an identity information collection device 50d. The identity information collection device 50d may be a camera, a scanner, an identifier based on OCR technology, a card reader, etc., but is not limited thereto. In this embodiment, the identity information collection device 50d may obtain a facial image in an identity document provided by the object to be detected as a facial image of a known object, and provide the facial image of the known object to the processor 50c. Accordingly, the processor 50c may calculate the similarity between the facial image of the object to be detected and the facial image of the known object. If the calculated similarity is greater than or equal to a preset image similarity threshold, it is determined that the facial image of the object to be detected belongs to the facial image of the known object.
[0161] In some optional embodiments, such as Figure 5 As shown, the face recognition device may also include optional components such as a communication component 50e, a power component 50f, an audio component 50g, and a display 50h. Figure 5 Only some components are shown schematically, which does not mean that the face recognition device must include Figure 5 All components shown do not mean that the face recognition device can only include Figure 5 As shown in the components. Figure 5The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the face recognition device. The face recognition device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server device such as a conventional server, a cloud server, or a server array. If the face recognition device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 5 If the face recognition device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 5 Components within the dashed box.
[0162] The face recognition device provided by the present embodiment combines face recognition with liveness detection, which is beneficial to improving the accuracy of face recognition, can prevent illegal users from using paper photos or electronic photos of legal users to impersonate, and is beneficial to improving the accuracy of face recognition and improving the security based on face recognition; when performing liveness detection, it is determined whether the object to be detected is alive by judging whether the illumination information in the image containing the object to be detected matches the illumination information in the current environment. The liveness detection method based on illumination information, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of face recognition.
[0163] Figure 6 This is a schematic diagram of the structure of a gate provided in an embodiment of the present application. Figure 6 As shown, the gate includes: a gate body 60a and a camera 60b, a memory 60c and a processor 60d arranged on the gate body 60a.
[0164] The memory 60c is used to store computer programs.
[0165] The camera 60b is used to collect facial images of objects to be detected around the entrance and exit of the gate body.
[0166] The processor 60d is coupled to the memory 60c, and is used to execute a computer program for: performing illumination analysis on the facial image of the object to be detected, obtaining illumination information in the facial image, and controlling the gate body 60a to open to allow the object to be detected to pass if the illumination information in the facial image of the object to be detected matches the illumination information in the current environment and the object to be detected is a legal object. The illumination information in the current environment refers to the illumination information in the current environment where the gate is located.
[0167] Optionally, the camera 60b can be a binocular camera, a monocular camera or a depth of field camera, but is not limited thereto. Further, the camera 60b also includes: a fill light 60e, wherein the fill light 60e is used to provide lighting conditions for the camera 60b to collect the facial image of the object to be detected when the lighting conditions of the gate cycle are insufficient.
[0168] In some embodiments, when the processor 60bd performs illumination analysis on a facial image, it is specifically used to: use a spherical harmonic illumination model to express a light source model corresponding to the facial image of the object to be detected, wherein the coefficient matrix of the spherical harmonic illumination model represents the illumination information in the facial image; solve variable parameters of the facial image according to the prior information of the reflected image, depth information, and illumination information corresponding to the image of the object to be detected, and obtain the coefficient matrix of the spherical harmonic illumination model.
[0169] In other embodiments, the processor 60d is also used to: calculate the similarity between the lighting information in the facial image of the object to be detected and the lighting information in the current environment; if the similarity is greater than or equal to a preset lighting similarity threshold, determine that the lighting information in the facial image of the object to be detected matches the lighting information in the current environment; if the similarity is less than the preset lighting similarity threshold, determine that the lighting information in the facial image does not match the lighting information in the current environment.
[0170] In some other embodiments, before determining that the lighting information in the image matches the lighting information in the current environment, the processor 60d is further used to: use the pre-acquired initial lighting information in the environmental space to which the current environment belongs as the lighting information in the current environment; or use the lighting information in N historical images in which the objects to be detected are determined to be living bodies to calculate the lighting information in the current environment, where N is a positive integer.
[0171] Further, when the processor 60d obtains the initial lighting information in the environmental space to which the current environment belongs in advance, it is specifically used to: control the camera 60b to collect the initial environmental image corresponding to the environmental space to which the current environment belongs; and perform lighting analysis on the initial environmental image to obtain initial lighting information.
[0172] On the other hand, when the processor 60d calculates the illumination information in the current environment, it is specifically used to: calculate the average of the illumination information in N historical images as the illumination information in the current environment. Preferably, the N historical images are the most recently acquired N historical images. Further, the historical images can be historical facial images.
[0173] In some further embodiments, Figure 6As shown, the gate also includes: a gate control component 60f. Accordingly, the processor 60d is also used to: match the facial image of the object to be detected in the facial image of the known object; if the facial image of the object to be detected is matched in the facial image of the known object, and the object to be detected is determined to be a living body, then the gate control component 60f is controlled to operate, and then the gate control component 60f drives the gate body 60a to open to allow the object to be detected to pass.
[0174] Alternatively, if Figure 6 As shown, the gate also includes: an identity information collection device 60g arranged on the gate body 60a. The identity information collection device 60g can be a camera, a scanner, an identifier based on OCR technology, a card reader, etc., but is not limited thereto. In this embodiment, the identity information collection device 60g can obtain the facial image in the identity document provided by the object to be detected as the facial image of the known object, and provide the facial image of the known object to the processor 60d. Accordingly, the processor 60d can calculate the similarity between the facial image of the object to be detected and the facial image of the known object. If the calculated similarity is greater than or equal to the preset image similarity threshold, it is determined that the facial image of the object to be detected belongs to the facial image of the known object.
[0175] In some optional embodiments, such as Figure 6 As shown, the gate machine may also include: a power supply component 60h. Further, the gate machine may also include: a communication component, an audio component, a display screen and other optional components ( Figure 6 not shown). Figure 6 Only some components are shown schematically, which does not mean that the gate must include Figure 6 All components shown do not mean that the gate can only include Figure 6 Components shown.
[0176] The gate provided in this embodiment can identify whether the object to be detected is alive by judging whether the illumination information in the facial image containing the object to be detected matches the illumination information in the current environment, and then further judge whether the object to be detected is a legal object on the basis of being alive, allowing legal living objects to pass, which is conducive to improving security. Among them, the liveness detection method based on illumination information, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0177] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 7 As shown, the device includes: a memory 70a and a processor 70b.
[0178] The memory 70a is used to store computer programs.
[0179] The processor 70b is coupled to the memory 70a, and is used to execute a computer program for: obtaining first lighting information characterizing ambient lighting; obtaining an image containing an object to be detected; performing lighting analysis on the image containing the object to be detected, and obtaining second lighting information characterizing the lighting in the image; and determining whether the object to be detected passes the detection based on the similarity between the first lighting information and the second lighting information.
[0180] In some embodiments, when determining whether the object to be detected has passed the detection, the processor 70b is specifically used to: calculate the similarity between the second illumination information and the first illumination information; if the calculated similarity is greater than or equal to a preset illumination similarity threshold, determine that the object to be detected has passed the detection; if the calculated similarity is less than the preset illumination similarity threshold, determine that the object to be detected has not passed the detection.
[0181] In other embodiments, when the processor 70b obtains the first lighting information characterizing the ambient lighting, it is specifically used to: use the pre-acquired initial lighting information in the ambient space to which the current environment belongs as the first lighting information; or, use the lighting information in N historical images in which the object to be detected is determined to be a living body, calculate the lighting information in the current environment as the first lighting information, where N is a positive integer.
[0182] Furthermore, the computer device further includes: a camera 70c. Accordingly, when the processor 70b obtains the initial illumination information in the environment space to which the current environment belongs in advance, it is specifically used to: control the camera 70c to collect the initial environment image corresponding to the environment space to which the current environment belongs; and perform illumination analysis on the initial environment image to obtain the initial illumination information.
[0183] Optionally, when calculating the illumination information in the current environment, the processor 70b is specifically configured to: calculate the average of the illumination information in N historical images as the illumination information in the current environment. Preferably, the N historical images are the most recently acquired N historical images.
[0184] It is worth noting that, in this embodiment, the specific implementation methods of the processor 70b performing illumination analysis on the image containing the object to be detected and the illumination analysis on the environment image can refer to the relevant contents of the above embodiments, which will not be repeated here. Figure 7 As shown, the computer device may also include optional components such as an identity information collection device 70d, a communication component 70e, a power supply component 70f, an audio component 70g, and a display 70h. The implementation and function of the identity information collection device 70d can refer to the relevant contents of the above embodiment, and will not be repeated here. Figure 7The components are shown only schematically, and do not mean that the computer equipment must include Figure 7 The components shown do not necessarily mean that the computer equipment can only include Figure 7 As shown in the components. Figure 7 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the computer device. The computer device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server device such as a conventional server, a cloud server, or a server array. If the computer device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 7 If the computer device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 7 Components within the dashed box.
[0185] The computer device provided in this embodiment can detect the object to be detected based on the similarity between the illumination information in the image containing the object to be detected and the illumination information in the process of collecting the image. The detection method based on illumination information, on the one hand, does not require the object to be detected to cooperate with the action, which can reduce the time of liveness detection and improve the detection efficiency; on the other hand, it can reduce the dependence on the texture details in the image, which not only reduces the quality requirements for the image, but also has high stability, which helps to improve the accuracy of liveness detection.
[0186] In various embodiments of the present application, the memory is used to store computer programs and can be configured to store various other data to support operations on the device where it is located. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0187] Among them, the communication component is configured to facilitate wired or wireless communication between the device where it is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology or other technologies.
[0188] Among them, the display screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0189] The power supply component is configured to provide power to various components of the device in which it is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.
[0190] Among them, the audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, if the device where the audio component is located has a language interaction function, voice interaction with the user can be achieved through the audio component.
[0191] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.
[0192] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0193] The present invention is described with reference to the flowchart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0194] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0195] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0196] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0197] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0198] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0199] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0200] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for detecting a living body, characterized in that: include: In the current environment, an image containing an object to be detected is acquired; A spherical harmonic illumination model is used to express a light source model corresponding to the image, wherein a coefficient matrix of the spherical harmonic illumination model represents illumination information in the image; The variable parameters of the image are solved according to the prior information of the reflected image, depth information and illumination information corresponding to the image to obtain the coefficient matrix of the spherical harmonic illumination model; if the illumination information in the image matches the illumination information in the current environment, it is determined that the object to be detected is a living body.
2. The method according to claim 1, characterized in that: In the current environment, images containing objects to be detected are collected, including: Use a monocular camera in the current environment to collect images containing the object to be detected.
3. The method according to claim 1, characterized in that Also includes: Calculating the similarity between the illumination information in the image and the illumination information in the current environment; If the similarity is greater than or equal to a preset illumination similarity threshold, determining that the illumination information in the image matches the illumination information in the current environment; If the similarity is less than a preset illumination similarity threshold, it is determined that the illumination information in the image does not match the illumination information in the current environment.
4. The method according to claim 1, characterized in that: Before determining that the illumination information in the image matches the illumination information in the current environment, the method further includes: Using the pre-obtained initial illumination information in the environment space to which the current environment belongs as the illumination information in the current environment; or The illumination information in the current environment is calculated by using the illumination information in N historical images in which the object to be detected is determined to be a living body, where N is a positive integer.
5. The method according to claim 4, characterized in that Pre-obtain the initial lighting information in the environment space of the current environment, including: Collect the initial environment image corresponding to the environment space to which the current environment belongs; Performing illumination analysis on the initial environment image to obtain the initial illumination information.
6. The method according to claim 4, characterized in that Using the illumination information in N historical images in which the object to be detected is determined to be alive, the illumination information in the current environment is calculated, including: The average value of the illumination information in the N historical images is calculated as the illumination information in the current environment.
7. The method according to claim 4 or 6, characterized in that: The N historical images are the most recently acquired N historical images.
8. The method according to any one of claims 1 to 6, characterized in that: The image includes a facial region image of the object to be detected; the method further comprises: Matching the facial region image of the object to be detected with the facial region image of a known object; If the facial region image of the object to be detected is matched in the facial region image of the known object, and the object to be detected is determined to be a living body, subsequent services are provided for the object to be detected.
9. The method according to claim 8, characterized in that Providing subsequent services for the object to be detected, including at least one of the following: Controlling the gate of the gate machine to open to allow the object to be detected to pass; Provide payment services for the object to be detected in combination with the payment account bound to the object to be detected; A relevant information interface is displayed for the object to be detected so that the object to be detected can obtain relevant information or perform relevant operations.
10. A data processing method, characterized in that: include: Under the current environment, a facial image of the object to be detected is collected; Using a spherical harmonic illumination model to express a light source model corresponding to the facial image, wherein a coefficient matrix of the spherical harmonic illumination model represents illumination information in the facial image; Solving the variable parameters of the facial image according to the prior information of the reflected image, depth information and illumination information corresponding to the facial image to obtain the coefficient matrix of the spherical harmonic illumination model; If the illumination information in the facial image matches the illumination information in the current environment, and the facial image is a facial image of a known object, it is determined that the object to be detected passes the face recognition.
11. The method according to claim 10, characterized in that In the case where the object to be detected is recognized by face recognition, the method further includes at least one of the following operations: Controlling the gate to open to allow the object to be detected to pass; Provide payment services for the object to be detected in combination with the payment account bound to the object to be detected; A relevant information interface is displayed for the object to be detected so that the object to be detected can obtain relevant information or perform relevant operations.
12. A data processing method, characterized in that: include: Acquire first illumination information representing ambient illumination; Acquire an image containing an object to be detected; A spherical harmonic illumination model is used to express a light source model corresponding to the image, wherein a coefficient matrix of the spherical harmonic illumination model represents second illumination information of the illumination in the image; Solving the variable parameters of the image according to the prior information of the reflection image, depth information and illumination information corresponding to the image to obtain the coefficient matrix of the spherical harmonic illumination model; Based on the similarity between the first illumination information and the second illumination information, it is determined whether the object to be detected passes the detection.
13. The method according to claim 12, characterized in that The step of obtaining first illumination information representing ambient illumination includes: The first illumination information representing the ambient illumination is calculated by using the second illumination information in the N historical images that have been detected most recently.
14. A gate system, characterized in that: include: A gate machine, an image acquisition device and a computing device; the computing device is respectively connected to the gate machine and the image acquisition device; The image acquisition device is used to acquire facial images of objects to be detected located around the entrance and exit of the gate, and transmit the facial images to the computing device; The computing device is used to express a light source model corresponding to the facial image using a spherical harmonic illumination model, wherein a coefficient matrix of the spherical harmonic illumination model represents illumination information in the facial image; The variable parameters of the facial image are solved according to the prior information of the reflected image, depth information and illumination information corresponding to the facial image to obtain the coefficient matrix of the spherical harmonic illumination model; and when the illumination information in the facial image matches the illumination information in the current environment of the gate and the object to be detected is a legal object, the gate is controlled to open to allow the object to be detected to pass.
15. The system according to claim 14, characterized in that The computing device is also used to: Calculating the similarity between the illumination information in the facial image and the illumination information in the current environment; If the similarity is greater than or equal to a preset illumination similarity threshold, determining that the illumination information in the facial image matches the illumination information in the current environment; If the similarity is less than a preset illumination similarity threshold, it is determined that the illumination information in the facial image does not match the illumination information in the current environment.
16. The system according to claim 14, characterized in that The computing device is also used to: Matching the facial image of the object to be detected with the facial image of a known object; If the facial image of the object to be detected is matched in the facial image of the known object, the object to be detected is a legal object.
17. The system according to claim 16, characterized in that Also includes: Identity information collection device; the identity collection device is connected to the computing device; The identity information collection device is used to obtain a facial image in the identity document provided by the object to be detected as the facial image of the known object, and provide the facial image of the known object to the computing device.
18. The system according to claim 17, characterized in that The identity information collection device is arranged on the gate.
19. The system according to claim 14, characterized in that Also includes: Light source equipment; the light source equipment is used to provide the image acquisition device with the lighting information required for acquiring images.
20. The system according to any one of claims 14 to 19, characterized in that: The image acquisition device is a monocular camera.
21. The system according to any one of claims 14 to 19, characterized in that: The image acquisition device is arranged on the gate, and the acquisition viewing angle of the image acquisition device covers the area around the entrance and exit of the gate.
22. The system according to any one of claims 14 to 19, characterized in that: The computing device is a processor disposed on the gate, or the computing device is a server device located in the cloud.
23. A living body detection device, characterized in that: include: A visual sensor, a memory and a processor; wherein the memory is used to store a computer program; The visual sensor is used to collect images containing the object to be detected in the current environment; The processor is coupled to the memory and is configured to execute the computer program for: A spherical harmonic illumination model is used to express a light source model corresponding to the image, and a coefficient matrix of the spherical harmonic illumination model represents illumination information in the image; variable parameters of the image are solved according to a priori information of a reflected image, depth information, and illumination information corresponding to the image to obtain a coefficient matrix of the spherical harmonic illumination model; if the illumination information in the image matches the illumination information in the current environment, it is determined that the object to be detected is a living body.
24. A gate machine, characterized in that: include: A gate body, and a camera, a memory and a processor arranged on the gate body; wherein the memory is used to store a computer program; The camera is used to: collect facial images of objects to be detected located around the entrance and exit of the gate body; The processor is coupled to the memory and is used to execute the computer program for: expressing the light source model corresponding to the facial image using a spherical harmonic illumination model, wherein the coefficient matrix of the spherical harmonic illumination model represents the illumination information in the facial image; solving variable parameters of the facial image according to the prior information of the reflected image, depth information and illumination information corresponding to the facial image to obtain the coefficient matrix of the spherical harmonic illumination model; and when the illumination information in the facial image matches the illumination information in the current environment and the object to be detected is a legal object, controlling the gate body to open to allow the object to be detected to pass.
25. A face recognition device, characterized in that: include: A visual sensor, a memory and a processor; wherein the memory is used to store a computer program; The visual sensor is used to collect a facial image containing an object to be detected in the current environment; The processor is coupled to the memory and is configured to execute the computer program for: A spherical harmonic illumination model is used to express a light source model corresponding to the facial image, wherein a coefficient matrix of the spherical harmonic illumination model represents illumination information in the facial image; variable parameters of the facial image are solved according to a priori information of a reflected image, depth information, and illumination information corresponding to the facial image to obtain a coefficient matrix of the spherical harmonic illumination model; If the illumination information in the facial image matches the illumination information in the current environment, and the facial image is a facial image of a known object, it is determined that the object to be detected passes the face recognition.
26. A computer device, characterized in that: include: Memory and processor; Wherein, the memory is used to store computer programs; The processor is coupled to the memory and is configured to execute the computer program for: Acquire first illumination information characterizing ambient illumination; acquire an image containing an object to be detected; use a spherical harmonic illumination model to express a light source model corresponding to the image, wherein a coefficient matrix of the spherical harmonic illumination model represents second illumination information of illumination in the image; solve variable parameters of the image according to a priori information of a reflected image, depth information, and illumination information corresponding to the image to obtain a coefficient matrix of the spherical harmonic illumination model; and determine whether the object to be detected passes the detection based on a similarity between the first illumination information and the second illumination information.
27. A readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 13.
