Face data processing method and device, and storage medium

Abstract

The application provides a face data processing method and device and a storage medium, which can be applied to an electronic device with a shooting function. The front camera of the electronic device is placed below a liquid crystal display screen, the top polarizing plate, the bottom polarizing plate and the backlight layer of the liquid crystal display screen are provided with openings, and the opening area of the top polarizing plate is filled with a material without a polarization effect. Based on this, the electronic device controls the front camera to collect multiple polarization images, and the multiple polarization images all include a target person's face. The electronic device constructs three-dimensional face data based on the multiple polarization images, which is used for face recognition. Compared with configuring a three-dimensional recognition module, the device cost is reduced, the physical space of the device is reduced, a better face recognition effect can be achieved, and the face recognition security can be improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method, device and storage medium for processing facial data. Background Technology

[0002] Facial recognition technology is an important branch of artificial intelligence. Its core lies in analyzing images or videos of faces to identify their features. With the development of computer vision technology, facial recognition technology has been widely used in smart terminals such as mobile phones and tablets, for example, in facial unlocking and facial payment functions. These functions not only improve the security of mobile phones but also provide users with a more convenient operating experience.

[0003] Active 3D recognition modules, such as structured light or TOF (time of flight) modules, have significant advantages in the field of facial recognition. Their main principle is to construct a 3D image of the object to be recognized (such as a face) by actively projecting light and measuring changes in the reflected light.

[0004] Considering hardware costs, some mobile phones do not have the aforementioned 3D recognition module and instead use a front-facing camera to perform facial recognition based on the 2D images captured by that camera, resulting in poor security for facial recognition. Summary of the Invention

[0005] This application provides a method, device, and storage medium for processing facial data, which can be applied to electronic devices with shooting functions and can improve the accuracy and security of facial recognition.

[0006] In a first aspect, embodiments of this application propose a method for processing facial data. The method includes: at a first moment, a first electronic device controls a front-facing camera of the first electronic device to acquire multiple first polarization images containing the face of a target person, and constructs first three-dimensional facial data of the target person based on the multiple first polarization images; at a first moment, a second electronic device controls a front-facing camera of the second electronic device to acquire multiple second polarization images containing the face of the target person, and constructs second three-dimensional facial data of the target person based on the multiple second polarization images.

[0007] The top polarizer of the first liquid crystal display screen of the first electronic device has an opening, and the opening area is filled with a material that does not have a polarizing effect; the top polarizer of the second liquid crystal display screen of the second electronic device has a polarizing effect.

[0008] The matching degree between the first 3D face data and the standard 3D face data of the target person is greater than the matching degree between the second 3D face data and the standard 3D face data.

[0009] In this embodiment, at the same time, the first electronic device and the second electronic device respectively capture multiple polarized images of the same target person's face through their respective front-facing cameras, and the shooting environment and shooting angle of the two electronic devices are the same.

[0010] The first LCD screen of the first electronic device can be referenced. Figure 4 In addition to the top polarizer having an opening, and the opening area being filled with a material that does not have a polarizing effect, the first LCD screen also has openings on the bottom polarizer and the backlight layer. The axes of these openings are all coincident with the optical axes of the lens group below the backlight layer, and the size of the openings is based on the principle of not affecting the field of view of the lens group.

[0011] Materials that do not have polarization effects are those that cannot cause light waves to vibrate in a specific direction. The optical properties of such materials are isotropic, meaning that when light passes through these materials, its vibration direction is not significantly affected or changed. Examples include glass or OCA adhesive.

[0012] The second LCD screen of the second electronic device can be referenced. Figure 3 The second LCD screen has openings on the bottom polarizer and backlight layer. The axes of these openings coincide with the optical axis of the lens group, and the size of the openings is designed so as not to affect the field of view of the lens group.

[0013] It should be noted that the matching degree between the target person's 3D facial data (first 3D facial data, or second 3D facial data) and the target person's standard 3D facial data can also be described as the similarity between the target person's 3D facial data and the target person's standard 3D facial data. Here, the target person's standard 3D facial data refers to the precise 3D structural information of the target person's face, pre-collected under ideal shooting conditions.

[0014] Because the opening area of ​​the top polarizer of the first LCD screen is filled with a non-polarizing material, most light can pass through the top polarizer. Since the top polarizer of the second LCD screen has no opening, only light whose vibration direction is parallel to the polarization axis of the top polarizer can pass through. Therefore, under the same conditions of subject, shooting environment, and shooting angle, the multiple first polarized images captured by the first electronic device have richer pixel information than the multiple second polarized images captured by the second electronic device. Thus, the first 3D face data constructed by the first electronic device is more accurate, improving the accuracy and security of face recognition.

[0015] In addition, the first electronic device uses a front-facing camera to capture multiple first polarized images containing a person's face, and then combines them with a software solution to construct three-dimensional facial data. Compared with configuring a three-dimensional recognition module, this reduces the cost of the device, reduces the physical space of the device, achieves better facial recognition results, and improves the security of facial recognition.

[0016] In one optional embodiment of the first aspect, the first electronic device controls its front-facing camera to capture multiple first polarized images containing the face of the target person, including: in response to a facial data entry or facial data verification operation, the first electronic device controls the polarization angle of the liquid crystal layer of the first liquid crystal display screen and controls the front-facing camera to capture multiple first polarized images containing the face of the target person. The polarization angle includes at least three.

[0017] Reference Figure 4 The first electronic device adjusts the voltage across the liquid crystal layer of the first liquid crystal display by controlling the TFT and the common electrode, so as to change the orientation of the liquid crystal molecules in the liquid crystal layer.

[0018] In one example, the first electronic device controls the liquid crystal layer of the first liquid crystal display to have a polarization angle of 0°, 45°, 90° and 135°. At each polarization angle, the first electronic device controls the front-facing camera to capture a first polarized image containing the face of the target person, for a total of four first polarized images.

[0019] In another example, the first electronic device controls the liquid crystal layer of the first liquid crystal display to be polarized at angles of 0°, 60°, and 120°. At each polarization angle, the first electronic device controls the front-facing camera to capture a first polarized image containing the face of the target person, for a total of three first polarized images.

[0020] The first electronic device controls the polarization angle of the liquid crystal layer and controls the front-facing camera to acquire multiple first polarization images at different polarization angles. These multiple first polarization images are used to construct three-dimensional face data, which can improve the accuracy and security of face recognition.

[0021] In one optional embodiment of the first aspect, the first electronic device constructs first three-dimensional face data of a target person based on multiple first polarization images, including: the first electronic device performs pixel-by-pixel unit processing on the facial region of the target person in the multiple first polarization images to obtain the normal vector information of each micro-facet of the target person's face; the first electronic device constructs first three-dimensional face data based on the normal vector information of multiple micro-facets of the target person's face.

[0022] In this embodiment, the normal vector information of each micro-element of the target person's face includes: the zenith angle and azimuth angle of the micro-element, which can be referred to in detail. Figure 6 .

[0023] In one optional embodiment of the first aspect, the first electronic device performs pixel-by-pixel processing on the facial region of a target person in multiple first polarized images to obtain the normal vector information of each micro-facet of the target person's face, including: the first electronic device acquiring the light intensity value of a first pixel unit in the multiple first polarized images; the first pixel unit being any pixel unit in the facial region of the target person; the first electronic device determining the azimuth angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity value of the first pixel unit in the multiple first polarized images; and the first electronic device determining the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity value of the first pixel unit in the multiple first polarized images and the refractive index of the target person's face. Wherein, the normal vector information of the micro-facet includes the azimuth angle and the zenith angle of the micro-facet.

[0024] A pixel unit can be a single pixel or multiple pixels, such as 2×2, 3×3, etc. This embodiment does not limit this.

[0025] In this embodiment, the refractive index of the target person's face is a preset facial refractive index, which is a fixed value. The first electronic device, based on multiple first polarization images, processes data on each pixel unit of the target person's facial region in the first polarization images to obtain the azimuth and zenith angle of the micro-facets of the target person's face corresponding to each pixel unit, providing data support for constructing the first three-dimensional face data. For details, please refer to... Figure 5 S102 of the embodiment.

[0026] In one optional embodiment of the first aspect, the first electronic device determines the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in multiple first polarized images and the refractive index of the target person's face, including: the first electronic device determining the refractive index of the target person's face; and determining the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in multiple first polarized images and the refractive index of the target person's face.

[0027] In this embodiment, the refractive index of the target person's face includes the refractive index of multiple regions (multiple micro-facets) of the target person's face, and the refractive index of different regions is different.

[0028] In one optional embodiment of the first aspect, the first electronic device determines the refractive index of the target person's face by: the first electronic device selecting a target polarization image from a plurality of first polarization images; the first electronic device extracting the red, green, and blue (RGB) information of the target polarization image; and the first electronic device determining the refractive index of the target person's face based on the RGB information of the target polarization image and a preset dispersion equation.

[0029] The first electronic device can randomly select one first polarization image from multiple first polarization images as the target polarization image, or it can select one first polarization image with better image quality from multiple first polarization images as the target polarization image. The first electronic device can be based on... Figure 8 In S303 of the embodiment, the refractive index of the target person's face in the target polarization image is determined, which will not be elaborated here.

[0030] The first electronic device determines the refractive index of the target person's face in the current shooting environment based on multiple first polarization images and a preset dispersion equation, and uses this information to construct three-dimensional facial data. Compared to using a fixed facial refractive index, this method improves the quality of the three-dimensional facial data, thereby enhancing the accuracy and security of facial recognition.

[0031] Secondly, embodiments of this application provide a first electronic device, including: a first liquid crystal display screen, a front-facing camera, a processor, and a memory; the first liquid crystal display screen includes a top polarizer, a bottom polarizer, and a backlight layer from top to bottom, and openings are provided on the top polarizer, the bottom polarizer, and the backlight layer, and the opening area of ​​the top polarizer is filled with a material that does not have a polarizing effect, and the front-facing camera is located below the first liquid crystal display screen.

[0032] The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to perform the method of the first electronic device as described in any of the first aspects.

[0033] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the method of the first electronic device according to any one of the first aspects.

[0034] Fourthly, embodiments of this application provide a computer program product including a computer program, which, when run, causes the computer to perform the method of the first electronic device according to any one of the first aspects.

[0035] Fifthly, this application provides a chip or chip system including at least one processor and a communication interface, wherein the communication interface and at least one processor are interconnected via a circuit, and the at least one processor is used to run a computer program or instructions to perform the method of the first electronic device according to any one of the first aspects. The communication interface in the chip may be an input / output interface, pins, or circuits, etc.

[0036] In an optional embodiment of the fifth aspect, the chip or chip system further includes at least one memory storing instructions. The memory may be an internal storage unit of the chip, such as a register or cache, or it may be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0037] It should be understood that the second to fifth aspects of this application correspond to the technical solutions of the first electronic device in the first aspect of this application, and the beneficial effects achieved by each aspect and each optional embodiment are similar, and will not be described again. Attached Figure Description

[0038] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0039] Figure 2 A partial structural diagram of an electronic device with an LCD screen provided in an embodiment of this application;

[0040] Figure 3 for Figure 2 The diagram shows the hierarchical structure of the LCD screen.

[0041] Figure 4 A schematic diagram of the hierarchical structure of an improved LCD screen provided in an embodiment of this application;

[0042] Figure 5 A schematic flowchart of a facial data processing method provided in an embodiment of this application;

[0043] Figure 6 A schematic diagram of the zenith angle and azimuth angle of a micro-surface element provided in an embodiment of this application;

[0044] Figure 7 Another flowchart illustrating the facial data processing method provided in this application embodiment;

[0045] Figure 8 Another flowchart illustrating the facial data processing method provided in this application embodiment;

[0046] Figure 9 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0047] Figure 10 Another structural schematic diagram of the electronic device provided in the embodiments of this application;

[0048] Figure 11 This is a schematic diagram of the chip structure provided in an embodiment of this application. Detailed Implementation

[0049] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0050] 1. Polarized images are images acquired using the polarization properties of light. They can capture the polarization state of light reflected from the surface of a subject (such as a person's face), thereby obtaining more information about the subject.

[0051] 2. Liquid crystals are substances with unique optical properties and complex analytical structures. Their alignment can be altered by electric fields or heat, thus affecting the propagation and polarization of light. In liquid crystal displays, the liquid crystal molecules in the liquid crystal layer change their alignment under the influence of an electric field, thereby changing the polarization of light. By controlling the intensity of the electric field, the transmittance of light through the liquid crystal layer can be adjusted to achieve image display.

[0052] In this embodiment of the application, when acquiring images through the front-facing camera, the polarization angle of the liquid crystal layer can be controlled to acquire polarized images corresponding to different polarization angles.

[0053] 3. The distribution of normal vectors of a target object refers to the set of vectors describing the surface orientation at each point on the target object's surface. A normal vector is a vector perpendicular to the object's surface; it not only determines the object's surface orientation but also plays a crucial role in various aspects such as lighting calculations, collision detection, and surface rendering.

[0054] In this embodiment, the target object can be a human face. The front-facing camera of the electronic device can capture multiple polarized images containing the human face. The electronic device preprocesses the captured multiple polarized images, such as removing noise, filling in missing data, and calibrating the coordinate system. Based on the preprocessed multiple polarized images, a preset image processing algorithm is used to calculate the normal vector of each point on the human face in the image, so as to obtain the normal vector distribution of the human face and provide data support for subsequent three-dimensional reconstruction of the human face.

[0055] 4. OCA (optically clear adhesive) is a special adhesive used for bonding transparent optical components. It belongs to the pressure-sensitive adhesive category and features colorless transparency, high light transmittance (typically exceeding 90%), good bonding strength and water resistance, while also being resistant to high temperatures and ultraviolet radiation.

[0056] 5. Polarization angle is a key parameter in liquid crystal display technology. It refers to the angle at which liquid crystal molecules deflect polarized light when it passes through the liquid crystal layer. The polarization angle is usually determined by the properties of the liquid crystal material and the thickness of the liquid crystal layer, and it has a significant impact on the display effect of the liquid crystal display screen.

[0057] In this embodiment of the application, the electronic device controls the voltage at both ends of the liquid crystal layer of the liquid crystal panel to adjust the arrangement and orientation of the liquid crystal molecules in the liquid crystal layer, thereby achieving the deflection of light.

[0058] 6. A pixel unit, usually composed of multiple pixels (in some cases it may be a single pixel), is the basic unit of digital image processing.

[0059] 7. A micro-facet is a unit used to describe the optical properties of a rough surface. In digital image processing, when processing images related to surface roughness or optical properties, a pixel unit can be regarded as a projection or representation of a micro-facet on the image. Information from the pixel unit can be extracted or analyzed to indirectly understand the characteristics of the micro-facet. In the embodiments of this application, the subject is a human face, and a micro-facet is a tiny unit on the human face.

[0060] The degree of polarization of a micro-surface element is used to characterize the degree of polarization of light reflected or scattered by the micro-surface element.

[0061] 8. Other terms

[0062] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first electronic device" and "second electronic device" are used only to distinguish different electronic devices and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0063] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0064] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0065] 9. Electronic equipment

[0066] The electronic devices in this application embodiment may include handheld devices with shooting functions, vehicle-mounted devices, etc. For example, some electronic devices include: mobile phones, tablets, PDAs, laptops, mobile internet devices (MIDs), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, terminal devices in 5G networks, or future evolution of public land mobile communication networks. Terminal devices in a network (PLMN), etc., are not limited to this in the embodiments of this application.

[0067] By way of example and not limitation, in this embodiment, the electronic device can also be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses and watches. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices; they achieve powerful functions through software support, data interaction, and cloud interaction.

[0068] Furthermore, in this embodiment of the application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0069] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0070] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0071] Currently, high-end mobile phones and other electronic devices are equipped with active 3D recognition modules, such as structured light or TOF modules, near the front-facing camera.

[0072] Structured light 3D recognition modules actively project specific patterns of light (usually laser stripes or grids) onto the object being recognized (such as a human face), and then use a camera to capture the deformation images formed by these light rays on the surface of the object. By analyzing the deformation images, a 3D image of the object can be constructed.

[0073] The light source in the TOF module emits modulated pulsed light (such as infrared light). When this light encounters the object being identified, it is emitted again. The receiver in the module (such as an infrared sensor) receives the reflected light and measures the time difference between the emission and reception of the light. Using the known speed of light and the measured time difference, the depth information of each point on the object being identified can be obtained, thereby constructing a three-dimensional image of the object being identified.

[0074] The two aforementioned 3D recognition modules offer advantages such as high accuracy, high robustness, and high adaptability, performing exceptionally well in applications like facial recognition. However, using these modules inevitably increases costs and adds to the overall size and weight.

[0075] In some mobile phones and other electronic devices, two-dimensional facial recognition is performed by capturing two-dimensional images using a front-facing camera. However, two-dimensional facial recognition cannot effectively distinguish between a real face and a face in a photograph or video. Attackers could potentially gain unauthorized access or impersonate someone by printing photos or displaying them on a high-definition screen. Furthermore, two-dimensional images cannot capture depth information of the face, resulting in poor security. Therefore, improving the facial recognition performance of these electronic devices is a pressing issue that needs to be addressed.

[0076] To address this issue, this application provides an electronic device that improves facial recognition performance without adding an additional active 3D recognition module by improving the hardware of the device's display screen and combining it with software algorithms.

[0077] The hardware structure of the electronic device according to an embodiment of this application will now be described with reference to the accompanying drawings.

[0078] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 1 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor 180, a button 190, a motor 191, a camera 193, a display screen 194, etc.

[0079] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0080] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0081] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0082] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0083] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0084] The display screen 194, also known as a display screen or screen, is used to display images, videos, etc. In this embodiment, the display screen 194 includes a display panel and a display driver IC (DDIC).

[0085] Display panels can be made of liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (FLED), Mini LED, Micro LED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc.

[0086] The display driver chip is the main control element of the display panel, used to convert electrical signals into visual signals and control the brightness and color of the display panel so that image information is displayed on the screen. In some embodiments, the electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0087] To retain the front-facing camera in a full-screen design, a punch-hole solution was developed. This solution involves punching a hole in the bottom backlight layer of the LCD screen, placing the front-facing camera beneath the LCD panel, thus achieving coexistence of the full-screen display and the front-facing camera. The following diagram illustrates this punch-hole solution for a full-screen front-facing camera.

[0088] Figure 2 This is a partial structural diagram of an electronic device with an LCD screen provided in an embodiment of this application. Figure 2 As shown, this electronic device is equipped with a front-facing camera, which includes an image sensor and a lens assembly. The image sensor can be a CCD (Charge-coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. The lens assembly includes multiple lenses. Through the refraction and reflection of these lenses, light is deflected and focused, ultimately projecting the image onto the imaging surface (the plane where the image sensor is located). The image sensor captures and converts the light focused by the lens into electrical signals, thereby generating a digital image.

[0089] Continue to refer to Figure 2 The LCD panel and backlight layer cover the lens assembly, with the LCD panel on top and the backlight layer on the bottom. A glass cover plate covers the LCD panel.

[0090] Figure 3 for Figure 2 The diagram shows the hierarchical structure of the LCD screen. Figure 3 As shown, the LCD screen includes: a glass cover, a top polarizer, a color filter, a liquid crystal layer, an alignment layer, a TFT (thin film transistor) and a common electrode, a bottom polarizer, a backlight layer, etc.

[0091] The glass cover, also known as protective glass, not only protects the screen but also enables touch operation.

[0092] The top polarizer is also called the upper polarizer, and the bottom polarizer is also called the lower polarizer. The polarization functions of the two polarizers are perpendicular to each other. For example, the top polarizer allows horizontally polarized light to pass through, and the bottom polarizer allows vertically polarized light to pass through.

[0093] Color filter film, also known as color filter, is used to generate three primary colors of light: red, green, and blue. By mixing these three primary colors in different ways, various colors can be mixed to achieve full-color display on LCD screens.

[0094] The TFT and the common electrode work together to control the deflection of the liquid crystal molecules in the liquid crystal layer by adjusting the voltage across the two ends of the liquid crystal layer, thereby affecting the light transmittance and enabling different gray levels or colors to be formed on the display screen to display images.

[0095] Alignment films, also known as orientation films, enable liquid crystals to achieve uniform alignment and orientation at the microscopic level.

[0096] The backlight layer, from top to bottom, includes a prism film, a diffuser film, LED (light emitting diode) lamps, a light guide plate, and a reflector. The main function of the backlight layer is to provide a uniform and sufficient light source for the LCD screen, enabling the liquid crystal panel to display correctly. In some embodiments, the backlight layer may also be referred to as a backlight module.

[0097] Figure 3 In the LCD screen shown, there are openings on the backlight layer and the bottom polarizer. The axis of the opening coincides with the optical axis of the lens. The size of the opening is designed so as not to affect the field of view of the lens group, thus realizing a full-screen blind hole solution.

[0098] based on Figure 3 As shown in the LCD screen, when electronic devices call the facial recognition function, they usually use two-dimensional images captured by the front-facing camera for facial recognition, which has poor recognition effect and security.

[0099] In response, this application proposes an electronic device that, in addition to providing openings in the backlight layer and bottom polarizer of an LCD screen, also provides openings in the top polarizer of the LCD screen. Specifically, the axis of the opening in the top polarizer coincides with the axis of the openings in the backlight layer and the bottom polarizer; see reference for details. Figure 4 For example, the diameter of the openings on the backlight layer, bottom polarizer, and top polarizer can be set between 2-4 mm. In this embodiment, the diameter of the openings is not limited, with the principle of not affecting the field of view of the lens group.

[0100] Since the top polarizer is located on the upper layer of the LCD screen and is easily pressed by the user, a material without polarization effect can be filled in the opening area of ​​the top polarizer of the LCD screen to improve the screen's support strength without affecting the light transmittance.

[0101] In one possible implementation, a glass plate is provided in the opening area of ​​the top polarizer of the LCD screen. The thickness of the glass plate is the same as the thickness of the top polarizer, and the shape of the glass plate is consistent with the shape of the opening area.

[0102] In one possible implementation, OCA adhesive is filled into the opening area of ​​the polarizer at the top of the LCD screen.

[0103] This application does not limit the materials that do not have polarization effects, as long as they meet the requirements of high transmittance (generally requiring transmittance of more than 90%) and reasonable structural reliability.

[0104] By making the aforementioned hardware improvements to the LCD screen, it is possible to acquire multiple polarized images containing a person's face by repeatedly adjusting the voltage across the liquid crystal layer of the LCD screen without adding an active 3D recognition module. This can be achieved by changing the orientation of the liquid crystal molecules in the liquid crystal layer. Based on these multiple polarized images, 3D facial data can be constructed, providing data support for subsequent 3D facial recognition / payment.

[0105] On the one hand, since there is no need to add a 3D recognition module, the equipment cost can be reduced and the physical space required for the device can be minimized. On the other hand, because the top polarizer has an opening, multiple polarized images can be captured to construct a 3D face image, which can improve the accuracy and security of face recognition.

[0106] based on Figure 4The improved LCD screen shown in this application embodiment provides a facial data processing method that can be applied to any electronic device with a camera function. The main idea is as follows: When a user triggers the device to collect facial data or triggers the device to perform facial recognition, the electronic device controls the deflection of liquid crystal molecules in the liquid crystal layer of the LCD screen, so that the front-facing camera collects multiple images containing a person's face. The polarization information in the multiple images is extracted using a preset image processing algorithm to obtain the normal vector distribution of the person's face. Based on the normal vector distribution of the person's face, three-dimensional facial data is constructed to provide data support for subsequent facial recognition.

[0107] Because the polarizer at the top of the LCD screen has an opening, the multiple images of a person's face captured by the device's front-facing camera have virtually no image loss. Based on this, the quality of the constructed 3D facial data can be improved, thereby enhancing the device's facial recognition performance.

[0108] The equipment control scheme will be described in detail below with reference to several specific embodiments.

[0109] Figure 5 This is a schematic flowchart illustrating a facial data processing method provided in an embodiment of this application. The facial data processing method shown in this embodiment involves the input of facial data, such as... Figure 5 As shown, the method for processing this facial data includes the following steps:

[0110] S101. In response to the user's operation of triggering facial data recording, the electronic device controls the polarization angle of the liquid crystal layer of the LCD screen and controls the front-facing camera to capture multiple polarized images containing the person's face.

[0111] For example, a user's action to trigger facial data entry can be: the user clicking the "Face Enrollment" control on the interface of a third-party application or system application.

[0112] In some embodiments, in response to the above operations, the electronic device controls the tilt angle of the liquid crystal layer of the LCD screen by calling the display driver, and controls the front-facing camera to take pictures by calling the camera driver.

[0113] In some embodiments, in response to the above operations, the electronic device invokes a display driver to control the tilt angle of the liquid crystal layer of the LCD screen to a preset value. When the tilt angle of the liquid crystal layer of the LCD screen is at the preset value, the electronic device invokes a camera driver to control the front-facing camera to capture images. The preset values ​​include at least three.

[0114] For example, taking preset values ​​including 0°, 45°, 90°, and 135° as an example, when the polarization angle of the liquid crystal layer of the LCD screen is 0°, 45°, 90°, and 135° respectively, the electronic device calls the camera driver to control the front-facing camera to take four shots to capture four polarized images containing the person's face. The four polarized images are denoted as I0, I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I2 ... 45 I 90 I 135 .

[0115] For example, taking preset values ​​including 0°, 60°, and 120° as an example, when the polarization angle of the liquid crystal layer of the LCD screen is 0°, 60°, and 120° respectively, the electronic device controls the front-facing camera to take three shots through camera drive to acquire three polarized images containing the person's face. The three polarized images are denoted as I0, I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I2 ... 60 I 120 .

[0116] S102. Electronic devices construct three-dimensional face data based on multiple polarization images.

[0117] In some embodiments, the electronic device performs facial recognition on each polarized image to obtain the facial region of a person from multiple polarized images; then, it performs pixel-by-pixel processing on the facial region of the multiple polarized images to determine the normal vector information of each micro-facet of the face; and finally, based on the normal vector information of each micro-facet of the face, it constructs three-dimensional face data. The normal vector of the micro-facet of the face can be characterized by the following two parameters: the zenith angle and the azimuth angle of the micro-facet of the face.

[0118] The following examples illustrate in detail how to calculate the normal vectors of each micro-facet of a person's face.

[0119] Reference Figure 6 Taking the micro-element i of a person's face as an example, the zenith angle of this micro-element is denoted as θ. d (i), the zenith angle is the angle between the direction of the normal vector of the surface element and the positive direction of the z-axis. The azimuth angle of the micro-surface element is denoted as α(i), which is the angle between the projection direction of the normal vector of the micro-surface element onto the xoy plane and the positive direction of the x-axis.

[0120] In some embodiments, taking four polarized images as an example, the electronic device performs pixel-by-pixel processing on the facial region of each polarized image to obtain the light intensity value (usually represented by grayscale value) of each pixel unit in the facial region of each polarized image. For each pixel unit, the electronic device can determine the azimuth angle of the micro-face element of the person's face corresponding to that pixel unit based on the light intensity values ​​of that pixel unit in the four polarized images, as shown in Formula 1.

[0121]

[0122] In the formula, α(i) represents the azimuth angle of the micro-face element i corresponding to pixel unit i, I0(i) represents the light intensity value of pixel unit i in polarized image 1, and I 45 (i) represents the light intensity value of pixel unit i in polarization image 2, I 90 (i) represents the light intensity value of pixel unit i in polarization image 3, I 145 (i) represents the light intensity value of pixel unit i in polarization image 1. Polarization image 1 is a polarization image captured by the front-facing camera when the polarization angle of the liquid crystal layer of the LCD screen controlled by the electronic device is 0°; polarization image 2 is a polarization image captured by the front-facing camera when the polarization angle of the liquid crystal layer of the LCD screen controlled by the electronic device is 45°; polarization image 3 is a polarization image captured by the front-facing camera when the polarization angle of the liquid crystal layer of the LCD screen controlled by the electronic device is 90°; and polarization image 4 is a polarization image captured by the front-facing camera when the polarization angle of the liquid crystal layer of the LCD screen controlled by the electronic device is 145°.

[0123] Based on Formula 1 above, electronic devices can obtain the azimuth angles of each micro-face element of a person's face, providing data support for constructing three-dimensional face data.

[0124] In some embodiments, taking four polarized images as an example, the electronic device performs pixel-by-pixel processing on the facial region of each polarized image, and after obtaining the light intensity value of each pixel unit in the facial region of each polarized image, it further performs the following: For each pixel unit, the electronic device can determine the polarization degree of the micro-face element of the person's face corresponding to the pixel unit based on the light intensity value of the pixel unit in the four polarized images, as shown in Formula 2.

[0125]

[0126] In the formula, ρ(i) represents the polarization degree of the micro-face element i corresponding to pixel unit i.

[0127] Based on Formula 2 above, the electronic device can obtain the polarization degree of each micro-facet of the human face. Based on this, for each micro-facet, the electronic device can determine the zenith angle of the micro-facet according to the polarization degree of the micro-facet and the preset refractive index of the human face, as shown in Formula 3.

[0128]

[0129] In the formula, n represents the refractive index of the human face, and θ d (i) represents the zenith angle of surface element i.

[0130] Based on Formula 3 above, electronic devices can obtain the zenith angle of each micro-face element of a person's face, providing data support for constructing three-dimensional face data.

[0131] After acquiring the orientation angle and zenith angle of each micro-facet of the human face, the electronic device can determine the normal vector of each micro-facet based on the orientation angle and zenith angle of each micro-facet, as shown in Formula 4.

[0132]

[0133] In the formula, The normal vector represents the micro-element i of the face of the person.

[0134] Based on the aforementioned embodiments, the electronic device can obtain the normal vectors of each micro-face element of a person's face based on Formulas 1 to 4, and construct three-dimensional face data based on the normal vectors of each micro-face element.

[0135] S103. The electronic device stores the three-dimensional facial data in a preset storage space.

[0136] The facial data processing method described in the above embodiments involves the electronic device repeatedly controlling the polarization angle of the liquid crystal layer on the LCD screen when the user triggers facial data enrollment, so that the front-facing camera of the electronic device can capture multiple polarized images containing the person's face. The electronic device acquires information (e.g., the light intensity value of the pixel unit) of each pixel unit in the facial region of the multiple polarized images, constructs three-dimensional facial data through a preset image processing algorithm, and stores the three-dimensional facial data in a preset space, which can be used for facial recognition or facial payment.

[0137] Compared to capturing a single 2D facial image, 3D facial data contains more features and details of a person's face (such as facial depth information). Furthermore, because the polarizer at the top of the LCD screen of electronic devices has an opening, the multiple polarized images captured by the front-facing camera show virtually no image loss. Therefore, compared to electronic devices without an opening in the polarizer at the top of the LCD screen, the image quality of the constructed 3D facial data can be further improved, providing data support for subsequent facial recognition and contributing to improved accuracy and security.

[0138] Figure 7 This is another schematic flowchart illustrating a facial data processing method provided in an embodiment of this application. The facial data processing method shown in this embodiment involves facial data verification, such as... Figure 7 As shown, the method for processing this facial data includes the following steps:

[0139] S201. In response to a user triggering a facial data verification operation, the electronic device controls the polarization angle of the liquid crystal layer of the LCD screen and controls the front-facing camera to capture multiple polarized images containing the person's face.

[0140] For example, a user's action to trigger facial data verification could be: the user triggering facial payment or user information verification in a third-party application or system application.

[0141] S202. The electronic device constructs three-dimensional face data to be verified based on multiple polarization images.

[0142] S201-S202 in this embodiment is similar to S101-S102 described above, and can be referred to the previous text, so it will not be elaborated here.

[0143] S203. The electronic device compares the three-dimensional face data to be verified with the pre-stored three-dimensional face data and outputs the face recognition result.

[0144] In some embodiments, the electronic device registers the 3D face data to be verified with pre-stored 3D face data to establish a correspondence between face regions; then, using preset algorithms, such as the nearest neighbor rule, support vector machine (SVM) and other machine learning algorithms, it compares the face features to be verified with the pre-stored face features; if the feature similarity is greater than or equal to a preset threshold, the face recognition result indicates that the face verification has passed; if the feature similarity is less than the preset threshold, the face recognition result indicates that the face verification has failed.

[0145] The facial data processing method described in the above embodiments involves the electronic device repeatedly controlling the polarization angle of the liquid crystal layer on the LCD screen when the user triggers facial data verification. This allows the front-facing camera of the electronic device to capture multiple polarized images containing the face of the person to be verified. The electronic device then acquires information about each pixel unit (e.g., the light intensity value of the pixel unit) in the facial region of each of the multiple polarized images and constructs the three-dimensional facial data to be verified using a preset image processing algorithm.

[0146] Compared to capturing a single 2D face image, the 3D face data to be verified contains more features and details of the face. Furthermore, because the polarizer at the top of the LCD screen of electronic devices has an opening, the multiple polarized images captured by the front-facing camera show virtually no image loss. Therefore, compared to electronic devices without an opening in the polarizer at the top of the LCD screen, the image quality of the 3D face data to be verified can be further improved, which is beneficial for improving the accuracy and security of face recognition.

[0147] Based on the foregoing embodiments, in some embodiments, reference is made to... Figure 8 After acquiring multiple polarized images containing a person's face, the electronic device also performs the following steps:

[0148] S301. The electronic device selects a target polarization image from multiple polarization images.

[0149] In some embodiments, the electronic device may randomly select one polarization image from multiple polarization images as the target polarization image.

[0150] In some embodiments, the electronic device may select a polarization image with better image quality from multiple polarization images as the target polarization image. Image quality evaluation parameters include, but are not limited to, image sharpness, contrast, and resolution. The electronic device may select a polarization image with better image quality from multiple polarization images as the target polarization image based on one or more evaluation parameters.

[0151] S302. Electronic device extracts RGB information from target polarization image.

[0152] S303. The electronic device updates the refractive index of the person's face based on the RGB information of the target polarization image and the preset dispersion equation.

[0153] In some embodiments, the electronic device acquires the grayscale values ​​of the red (R), green (G), and blue (B) channels of each pixel in the target polarization image. For each pixel in the target polarization image, the electronic device updates the refractive index of the person's face based on the grayscale values ​​of the pixel's RGB three channels and a preset dispersion equation.

[0154] For example, an electronic device updates the refractive index of a person's face based on the grayscale values ​​of the RGB three channels of a pixel and the Cauchy dispersion equation, as shown in Formula 5.

[0155]

[0156] In the formula, n(μ, λ) represents the refractive index of the micro-area corresponding to each pixel μ in the image at wavelength λ (e.g., red wavelength between 620-750nm, green wavelength between 492-577nm, and blue wavelength between 435-492nm), and c k (μ) represents the k-th dispersion coefficient corresponding to each pixel μ, and M can represent the number of channels, such as M being 3.

[0157] As can be seen from Formula 5, the updated refractive index of the human face includes the refractive index of multiple micro-facets of the human face.

[0158] It should be noted that the embodiments of this application do not limit the dispersion equation. In addition to the Cauchy dispersion equation, other dispersion equations can also be used, such as the Hartmann dispersion equation, the Conrady dispersion equation, the Hetzberger dispersion equation, etc.

[0159] S304. Electronic devices construct three-dimensional face data based on multiple polarization images and updated refractive index of the human face.

[0160] In some embodiments, taking the aforementioned four polarized images as examples, the electronic device performs pixel-by-pixel (e.g., pixel-by-pixel point) processing on the facial region of each polarized image to obtain the light intensity value of each pixel unit in the facial region of each polarized image. For each pixel unit, the electronic device can obtain the normal vector of each micro-facet of the human face based on the aforementioned formulas one to four, and construct three-dimensional face data based on the normal vectors of each micro-facet. The implementation process of this embodiment can refer to S102 of the aforementioned embodiment. The difference from S102 is that the refractive index of each micro-facet of the human face is determined based on S303.

[0161] The facial data processing method illustrated in this embodiment involves an electronic device determining the refractive index of a person's face in the current shooting environment based on multiple polarized images captured by a front-facing camera and a preset dispersion equation. Then, based on the refractive index of the person's face in the current shooting environment and the multiple polarized images, it constructs three-dimensional facial data. Compared to using a fixed facial refractive index, this method improves the quality of the three-dimensional facial data, thereby enhancing the accuracy and security of facial recognition.

[0162] In this embodiment, the software system of the electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment uses the layered architecture Android system as an example to illustrate the software structure of the electronic device 100.

[0163] Figure 9 This is a software structure block diagram of an electronic device provided in an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, such as... Figure 9 As shown, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer.

[0164] The application layer can include a series of application packages. Application packages can include applications such as camera, gallery, calendar, calling, maps, navigation, WLAN (wireless local area network), Bluetooth, music, video, and SMS. Applications include third-party applications and system applications.

[0165] In this embodiment of the application, in response to the user's operation of facial data recording or facial verification triggered by a third-party application or system application, the third-party application or system application can control the front camera to capture multiple polarized images containing a person's face by calling the lower-level camera driver.

[0166] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. In this embodiment, such as... Figure 9 As shown, the application framework layer may include a camera management module. The camera management module is responsible for managing camera device information, including the number of cameras, shooting capabilities, shooting modes, etc. It is also used to transmit data between the application and the camera HAL. For example, the camera management module sends shooting commands to the camera driver through the camera HAL to drive the corresponding camera to capture images.

[0167] The hardware abstraction layer (HAL) is an abstract interface for device kernel drivers, used to provide application programming interfaces for accessing the underlying device to higher-level API frameworks. In this embodiment, as... Figure 9 As shown, the Hardware Abstraction Layer (HAL) includes a Display HAL and a Camera HAL. The Display HAL provides a unified interface for applications to manage and control the display screen, while the Camera HAL provides a unified interface for applications to manage and control the camera. Furthermore, the HAL may also include an image processing module, which can pre-configure various algorithms, such as the image processing algorithm mentioned earlier. This image processing algorithm can construct 3D facial data based on multiple polarized images of a person's face captured by the front-facing camera.

[0168] It should be noted that the image processing module described above is not limited to the hardware abstraction layer. In some embodiments, the image processing module may also reside in other layers. For example, the image processing module may be integrated into the application framework layer.

[0169] The kernel layer is the layer between hardware and software. In this embodiment, as shown... Figure 9 As shown, the kernel layer includes a display driver and a camera driver. The display driver is used to drive the display screen (e.g., an LCD screen) to work, and the camera driver is used to drive the camera (e.g., a front-facing camera) to capture images.

[0170] The hardware layer includes displays and cameras, etc. In this embodiment, for example... Figure 9 As shown, the display screen can be an LCD screen, the structure of which is described above and will not be repeated here. The camera includes a front-facing camera, which can be used to capture facial images of people in front of the display screen.

[0171] Understandable Figure 9The modules included in each layer shown are those involved in the embodiments of this application. The modules included in each layer do not constitute a limitation on the structure of the electronic device and the hierarchy of module deployment. In some embodiments, the electronic device may include more or fewer layers than shown, and each layer may include more or fewer components; this application does not impose any limitations.

[0172] It should be noted that in the above embodiments, a "module" can be a software program, a hardware circuit, or a combination of both to implement the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuits, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components that support the described functions.

[0173] Therefore, the modules of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to facial data used for analysis, stored facial data, displayed facial data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0175] Based on the foregoing embodiments, this application proposes a method for processing facial data, the method comprising:

[0176] At the first moment, the first electronic device controls the front-facing camera of the first electronic device to acquire multiple first polarization images containing the face of the target person, and constructs the first three-dimensional facial data of the target person based on the multiple first polarization images.

[0177] At the first moment, the second electronic device controls the front-facing camera of the second electronic device to acquire multiple second polarization images containing the face of the target person. Based on the multiple second polarization images, the second electronic device constructs the second three-dimensional facial data of the target person.

[0178] The top polarizer of the first liquid crystal display screen of the first electronic device has an opening, and the opening area is filled with a material that does not have a polarizing effect; the top polarizer of the second liquid crystal display screen of the second electronic device has a polarizing effect.

[0179] The matching degree between the first 3D face data and the standard 3D face data of the target person is greater than the matching degree between the second 3D face data and the standard 3D face data.

[0180] In this embodiment, at the same time, the first electronic device and the second electronic device respectively capture multiple polarized images of the same target person's face through their respective front-facing cameras, and the shooting environment and shooting angle of the two electronic devices are the same.

[0181] The first LCD screen of the first electronic device can be referenced. Figure 4 In addition to the top polarizer having an opening, and the opening area being filled with a material that does not have a polarizing effect, the first LCD screen also has openings on the bottom polarizer and the backlight layer. The axes of these openings are all coincident with the optical axes of the lens group below the backlight layer, and the size of the openings is based on the principle of not affecting the field of view of the lens group.

[0182] Materials that do not have polarization effects are those that cannot cause light waves to vibrate in a specific direction. The optical properties of such materials are isotropic, meaning that when light passes through these materials, its vibration direction is not significantly affected or changed. Examples include glass or OCA adhesive.

[0183] The second LCD screen of the second electronic device can be referenced. Figure 3 The second LCD screen has openings on the bottom polarizer and backlight layer. The axes of these openings coincide with the optical axis of the lens group, and the size of the openings is designed so as not to affect the field of view of the lens group.

[0184] It should be noted that the matching degree between the target person's 3D facial data (first 3D facial data, or second 3D facial data) and the target person's standard 3D facial data can also be described as the similarity between the target person's 3D facial data and the target person's standard 3D facial data. Here, the target person's standard 3D facial data refers to the precise 3D structural information of the target person's face, pre-collected under ideal shooting conditions.

[0185] Because the opening area of ​​the top polarizer of the first LCD screen is filled with a non-polarizing material, most light can pass through the top polarizer. Since the top polarizer of the second LCD screen has no opening, only light whose vibration direction is parallel to the polarization axis of the top polarizer can pass through. Therefore, under the same conditions of subject, shooting environment, and shooting angle, the multiple first polarized images captured by the first electronic device have richer pixel information than the multiple second polarized images captured by the second electronic device. Thus, the first 3D face data constructed by the first electronic device is more accurate, improving the accuracy and security of face recognition.

[0186] In addition, the first electronic device uses a front-facing camera to capture multiple first polarized images containing a person's face, and then combines them with a software solution to construct three-dimensional facial data. Compared with configuring a three-dimensional recognition module, this reduces the cost of the device, reduces the physical space of the device, and can achieve better facial recognition results.

[0187] In one optional embodiment, the first electronic device controls its front-facing camera to capture multiple first polarized images containing the face of the target person. This includes: in response to facial data entry or verification operations, the first electronic device controls the polarization angle of the liquid crystal layer of the first liquid crystal display screen and controls the front-facing camera to capture multiple first polarized images containing the face of the target person. The polarization angle includes at least three polarization angles.

[0188] Reference Figure 4 The first electronic device adjusts the voltage across the liquid crystal layer of the first liquid crystal display by controlling the TFT and the common electrode, so as to change the orientation of the liquid crystal molecules in the liquid crystal layer.

[0189] In one example, the first electronic device controls the liquid crystal layer of the first liquid crystal display to have a polarization angle of 0°, 45°, 90° and 135°. At each polarization angle, the first electronic device controls the front-facing camera to capture a first polarized image containing the face of the target person, for a total of four first polarized images.

[0190] In another example, the first electronic device controls the liquid crystal layer of the first liquid crystal display to be polarized at angles of 0°, 60°, and 120°. At each polarization angle, the first electronic device controls the front-facing camera to capture a first polarized image containing the face of the target person, for a total of three first polarized images.

[0191] The first electronic device controls the polarization angle of the liquid crystal layer and controls the front-facing camera to acquire multiple first polarization images at different polarization angles. These multiple first polarization images are used to construct three-dimensional face data, which can improve the accuracy and security of face recognition.

[0192] In one optional embodiment, the first electronic device constructs first three-dimensional facial data of the target person based on multiple first polarization images, including: the first electronic device performs pixel-by-pixel unit processing on the facial region of the target person in the multiple first polarization images to obtain the normal vector information of each micro-facet of the target person's face; the first electronic device constructs first three-dimensional facial data based on the normal vector information of multiple micro-facets of the target person's face.

[0193] In this embodiment, the normal vector information of each micro-element of the target person's face includes: the zenith angle and azimuth angle of the micro-element, which can be referred to in detail. Figure 6 .

[0194] In one optional embodiment, the first electronic device performs pixel-by-pixel processing on the facial region of a target person in multiple first polarized images to obtain the normal vector information of each micro-facet of the target person's face. This includes: the first electronic device acquiring the light intensity value of a first pixel unit in the multiple first polarized images; the first pixel unit being any pixel unit in the facial region of the target person; the first electronic device determining the azimuth angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity value of the first pixel unit in the multiple first polarized images; and the first electronic device determining the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity value of the first pixel unit in the multiple first polarized images and the refractive index of the target person's face. The normal vector information of the micro-facet includes the azimuth angle and the zenith angle of the micro-facet.

[0195] A pixel unit can be a single pixel or multiple pixels, such as 2×2, 3×3, etc. This embodiment does not limit this.

[0196] In this embodiment, the refractive index of the target person's face is a preset facial refractive index, which is a fixed value. The first electronic device, based on multiple first polarization images, processes data on each pixel unit of the target person's facial region in the first polarization images to obtain the azimuth and zenith angle of the micro-facets of the target person's face corresponding to each pixel unit, providing data support for constructing the first three-dimensional face data. For details, please refer to... Figure 5 S102 of the embodiment.

[0197] In one optional embodiment, the first electronic device determines the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in multiple first polarized images and the refractive index of the target person's face, including: the first electronic device determining the refractive index of the target person's face; and determining the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in multiple first polarized images and the refractive index of the target person's face.

[0198] In this embodiment, the refractive index of the target person's face includes the refractive index of multiple regions (multiple micro-facets) of the target person's face, and the refractive index of different regions is different.

[0199] In one optional embodiment, the first electronic device determines the refractive index of the target person's face by: selecting a target polarization image from multiple first polarization images; extracting the red, green, and blue (RGB) information of the target polarization image; and determining the refractive index of the target person's face based on the RGB information of the target polarization image and a preset dispersion equation.

[0200] The first electronic device can randomly select one first polarization image from multiple first polarization images as the target polarization image, or it can select one first polarization image with better image quality from multiple first polarization images as the target polarization image. The first electronic device can be based on... Figure 8 In S303 of the embodiment, the refractive index of the target person's face in the target polarization image is determined, which will not be elaborated here.

[0201] The first electronic device determines the refractive index of the target person's face in the current shooting environment based on multiple first polarization images and a preset dispersion equation, and uses this information to construct three-dimensional facial data. Compared to using a fixed facial refractive index, this method improves the quality of the three-dimensional facial data, thereby enhancing the accuracy and security of facial recognition.

[0202] Figure 10 This is another structural schematic diagram of the electronic device provided in an embodiment of this application. For example... Figure 10 As shown, the electronic device includes: a first liquid crystal display 1001, a front-facing camera 1002, a processor 1003, and a memory 1004.

[0203] The first LCD screen 1001 includes, from top to bottom, a top polarizer, a bottom polarizer, and a backlight layer. Each of the top polarizer, bottom polarizer, and backlight layer has openings, and the opening area of ​​the top polarizer is filled with a non-polarizing material. The front-facing camera 1002 is located below the first LCD screen 1001. The first LCD screen 1001 can be referenced... Figure 4 Examples are not elaborated here.

[0204] The memory 1003 stores computer execution instructions; the processor 1004 executes the computer execution instructions stored in the memory 1003 to perform the technical solution as described in the foregoing method embodiments. Its implementation principle and technical effects are similar to the aforementioned related embodiments, and will not be repeated here.

[0205] In some embodiments, the electronic device further includes: a communication line 1005 and at least one communication interface. Figure 10(The example described uses communication interface 1006 as an example.) Communication line 1005 may include circuitry for transmitting information between the aforementioned components or modules. Communication interface 1006, using any transceiver-like device, is used for communicating with other devices or communication networks, such as Ethernet, WLAN, etc.

[0206] The processor 1003 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0207] The memory 1004 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 1004 may exist independently and be connected to the processor 1003 via communication line 1005. The memory 1004 may also be integrated with the processor 1003.

[0208] The computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.

[0209] As an example, processor 1003 may include one or more CPUs.

[0210] As an example, an electronic device may include multiple processors. Each processor may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0211] This application provides a chip or chip system. The chip or chip system includes a processor, which calls a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0212] Figure 11 This is a schematic diagram of the chip structure provided in an embodiment of this application. Figure 11 As shown, chip 1100 includes one or more processors 1120 and communication interface 1130.

[0213] In some embodiments, memory 1140 stores the following elements: executable modules or data structures, or a subset of executable modules or data structures, or an extended set of executable modules or data structures.

[0214] In this embodiment, memory 1140 may include read-only memory and random access memory, and provides instructions and data to processor 1120. A portion of memory 1140 may also include non-volatile random access memory (NVRAM).

[0215] In this embodiment, the memory 1140, the communication interface 1130, and the memory 1140 are coupled together via a bus system 1110. The bus system 1110 may include a data bus, a power bus, a control bus, and a status signal bus, in addition to the data bus. For ease of description, in... Figure 11 The general designated all buses as Bus System 1110.

[0216] The methods described in the embodiments of this application can be applied to, or implemented by, processor 1120. Processor 1120 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 1120 or by instructions in software form. Processor 1120 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 1120 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0217] This application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When executed by a processor, the computer program implements the technical solutions described in the above-described method embodiments. Its implementation principle and technical effects are similar to the related embodiments described above, and will not be repeated here.

[0218] The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted on a computer-readable medium. A computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. A storage medium can be any target medium accessible by a computer.

[0219] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage or other magnetic storage devices, or any other medium targeted to carry or to store the required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, Digital Versatile Discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0220] This application provides a computer program product, which includes a computer program. When the computer program is run, it causes the computer to execute the technical solutions described in the above-described method embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0221] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0222] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for processing facial data, characterized in that, include: At the first moment, the first electronic device controls the front-facing camera of the first electronic device to acquire multiple first polarization images containing the face of the target person, and constructs the first three-dimensional face data of the target person based on the multiple first polarization images; At the first moment, the second electronic device controls the front-facing camera of the second electronic device to acquire multiple second polarization images containing the face of the target person, and the second electronic device constructs the second three-dimensional face data of the target person based on the multiple second polarization images; The top polarizer of the first liquid crystal display screen of the first electronic device has an opening, and the opening area is filled with a material that does not have a polarizing effect. The polarizer at the top of the second liquid crystal display of the second electronic device has a polarization effect; the matching degree between the first three-dimensional face data and the standard three-dimensional face data of the target person is greater than the matching degree between the second three-dimensional face data and the standard three-dimensional face data.

2. The method according to claim 1, characterized in that, The first electronic device controls its front-facing camera to capture multiple first polarization images containing the face of the target person, including: In response to the operation of facial data entry or facial data verification, the first electronic device controls the polarization angle of the liquid crystal layer of the first liquid crystal display screen and controls the front-facing camera of the first electronic device to capture multiple first polarization images containing the face of the target person. The deflection angles include at least three.

3. The method according to claim 1 or 2, characterized in that, The first electronic device constructs first three-dimensional facial data of the target person based on the multiple first polarization images, including: The first electronic device performs pixel-by-pixel unit processing on the facial region of the target person in the multiple first polarization images to obtain the normal vector information of each micro-facet of the target person's face; The first electronic device constructs the first three-dimensional face data based on the normal vector information of multiple micro-face elements of the target person's face.

4. The method according to claim 3, characterized in that, The first electronic device performs pixel-by-pixel processing on the facial region of the target person in the plurality of first polarization images to obtain the normal vector information of each micro-facet of the target person's face, including: The first electronic device acquires the light intensity value of the first pixel unit in the plurality of first polarization images; the first pixel unit is any pixel unit in the facial region of the target person; The first electronic device determines the azimuth angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in the plurality of first polarization images; and, The first electronic device determines the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit based on the light intensity value of the first pixel unit in the plurality of first polarized images and the refractive index of the target person's face. The normal vector information of the micro-surface element includes the azimuth angle and zenith angle of the micro-surface element.

5. The method according to claim 4, characterized in that, The first electronic device determines the zenith angle of the micro-area of ​​the target person's face corresponding to the first pixel unit based on the light intensity values ​​of the first pixel unit in the plurality of first polarized images and the refractive index of the target person's face, including: The first electronic device determines the refractive index of the target person's face; Based on the light intensity value of the first pixel unit in the plurality of first polarization images and the refractive index of the target person's face, the zenith angle of the micro-facet of the target person's face corresponding to the first pixel unit is determined.

6. The method according to claim 5, characterized in that, The first electronic device determines the refractive index of the target person's face, including: The first electronic device selects a target polarization image from the plurality of first polarization images; The first electronic device extracts the red, green, and blue (RGB) information of the target polarization image; The first electronic device determines the refractive index of the target person's face based on the RGB information of the target polarization image and a preset dispersion equation.

7. A first electronic device, characterized in that, include: First LCD screen, front-facing camera, processor and memory; The first liquid crystal display screen includes a top polarizer, a bottom polarizer, and a backlight layer from top to bottom. The top polarizer, the bottom polarizer, and the backlight layer are all provided with openings, and the opening area of ​​the top polarizer is filled with a material that does not have a polarizing effect. The front-facing camera is located below the first liquid crystal display screen. The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to perform the method of the first electronic device as claimed in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of the first electronic device as claimed in any one of claims 1 to 6.

9. A chip system, characterized in that, It includes at least one processor and a communication interface, the communication interface and the at least one processor being interconnected via a line, the at least one processor being configured to run a computer program or instructions to perform the method of the first electronic device as claimed in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the method of the first electronic device as claimed in any one of claims 1 to 6.

Images