Facial recognition methods, devices, equipment and media

Abstract

This application discloses a face recognition method, apparatus, device, and medium, relating to the field of image processing. The method includes: fusing a color image and a depth image of a face image to obtain a fused image of the face image, the fused image including two-dimensional information and depth information of the face image; dividing the fused image into blocks to obtain at least two image blocks; irreversibly scrambling the pixels within the at least two image blocks to obtain a pixel-obfuscated face image; and determining the user identity information corresponding to the pixel-obfuscated face image based on the pixel-obfuscated face image. This application protects the two-dimensional and depth information in the face image to prevent user privacy leakage.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a face recognition method, apparatus, device, and medium. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. Without proper privacy protection measures during facial recognition, users' biometric information may be leaked, harming their interests.

[0003] The relevant technology involves acquiring a user's video, extracting one or more frames of two-dimensional facial images from the video, performing an Arnol mapping (also known as a cat mapping) on ​​the facial image to scramble the pixels, and obtaining a pixel-scrambled facial image to protect personal information. This pixel-scrambled facial image is then input into a facial recognition model to obtain the corresponding feature vector, which is then used to determine the user's identity information from a database.

[0004] When recognizing a user's facial image, the relevant technology can only protect the two-dimensional information in the image. The amount of information protected is limited, which can easily lead to privacy leaks and losses for users. Summary of the Invention

[0005] This application provides a face recognition method, apparatus, device, and medium. The method simultaneously encrypts both depth and two-dimensional information in a face image to ensure user privacy is not compromised. The technical solution is as follows:

[0006] According to one aspect of this application, a face recognition method is provided, the method comprising:

[0007] The color image and depth image of a face image are fused to obtain a fused image of the face image, the fused image including the two-dimensional information and depth information of the face image;

[0008] The fused image is divided into blocks to obtain image blocks of the fused image;

[0009] The pixels within the at least two image blocks are irreversibly scrambled to obtain a face image with pixel confusion.

[0010] Based on the pixel-obfuscated face image, determine the user identity information corresponding to the face image.

[0011] According to another aspect of this application, a face recognition device is provided, the device comprising:

[0012] The fusion module is used to fuse the color image and depth image of a face image to obtain a fused image of the face image, wherein the fused image includes the two-dimensional information and depth information of the face image;

[0013] The segmentation module is used to segment the fused image into blocks to obtain at least two image blocks of the fused image;

[0014] The scrambling module is used to irreversibly scramble the pixels in the at least two image blocks to obtain a face image with pixel confusion.

[0015] The recognition module is used to determine the user identity information corresponding to the face image based on the face image with pixel confusion.

[0016] In an optional design of this application, the fusion module is further configured to acquire a first feature point in the color image and a second feature point in the depth image, wherein the first feature point and the second feature point have a corresponding relationship; in response to the overlap between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain the fused image of the face image.

[0017] In an optional design of this application, the fusion module is further configured to, in response to the same field of view of the color image and the same field of view of the depth image, acquire the first feature point in the color image and acquire the second feature point in the depth image.

[0018] In an optional design of this application, the fusion module is further configured to obtain first intrinsic and extrinsic parameters of the first camera corresponding to the color image, and second intrinsic and extrinsic parameters of the second camera corresponding to the depth image, wherein the first camera is used to capture the color image and the second camera is used to capture the depth image; and the field of view of the color image and the field of view of the depth image are controlled to be the same according to the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters.

[0019] In an optional design of this application, the segmentation module is further configured to uniformly segment the fused image to obtain image blocks of equal size from the fused image.

[0020] In an optional design of this application, the scrambling module is further configured to perform cat mapping on the pixels within the at least two image blocks to obtain the pixel change coordinates within the at least two image blocks; and to obtain the pixel-scrambled face image based on the pixel change coordinates within the at least two image blocks.

[0021] In an optional design of this application, the recognition module is further configured to: acquire a third feature point in the fused image; encrypt the third feature point to obtain encrypted feature point data; send the encrypted feature point data and the pixel-obfuscated face image to the server; and receive the user identity information returned by the server, wherein the user identity information is obtained by the server based on the decrypted encrypted feature point data and the pixel-obfuscated face image.

[0022] In an optional design of this application, the recognition module is further configured to input the third feature point in the pixel-confused face image and the fused image into a face recognition model to obtain a feature vector corresponding to the face image data; and determine the user identity information corresponding to the face image based on the feature vector.

[0023] In an optional design of this application, the recognition module is further configured to, in response to the failure of identity matching, obtain user identity information corresponding to the face image; use the pixel-obfuscated face image and encrypted feature point data as target face image data; and save the target face image data and the user identity information.

[0024] In an optional design of this application, the device further includes a training module;

[0025] The training module is used to acquire a training dataset, which includes sample face images and real user identity information, wherein the sample face images correspond to the real user identity information; generate pixel-obfuscated sample face images based on the sample face images; input the pixel-obfuscated sample face images into the face recognition model to obtain sample feature vectors corresponding to the sample face images; determine the sample user identity information corresponding to the sample face image data based on the sample feature vectors; and train the face recognition model based on the difference between the real user identity information and the sample user identity information.

[0026] According to another aspect of this application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the face recognition method as described above.

[0027] According to another aspect of this application, a computer storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the program code being loaded and executed by a processor to implement the face recognition method as described above.

[0028] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the face recognition method described above.

[0029] The beneficial effects of the technical solutions provided in this application include at least the following:

[0030] Aligning and fusing the color and depth maps of a face image results in a fused image that contains both two-dimensional and depth information. Dividing the fused image into blocks allows for better processing of individual pixels. By scrambling the image blocks, the two-dimensional and depth information in the fused image is preserved, making it impossible to reconstruct the two-dimensional and depth information. This protects user privacy from being leaked. At the same time, it can also determine user identity information based on the pixelated face image without affecting the use of basic face recognition functions. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application;

[0033] Figure 2 This is a flowchart illustrating a face recognition method provided in an exemplary embodiment of this application;

[0034] Figure 3 This is a schematic diagram of a face recognition method provided in an exemplary embodiment of this application;

[0035] Figure 4 This is a flowchart illustrating a face recognition method provided in an exemplary embodiment of this application;

[0036] Figure 5 This is a schematic diagram showing the same control field of view provided in an exemplary embodiment of this application;

[0037] Figure 6 This is a flowchart illustrating a face recognition model training method provided in an exemplary embodiment of this application;

[0038] Figure 7 This is a flowchart illustrating a face recognition method provided in an exemplary embodiment of this application;

[0039] Figure 8 This is a schematic diagram of a face recognition device provided in an exemplary embodiment of this application;

[0040] Figure 9 This is a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0042] First, the terms used in the embodiments of this application will be introduced:

[0043] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0044] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0045] Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0046] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0047] Color image: An image obtained by capturing natural light and performing imaging operations. For example, color images are generally used for facial comparison and recognition in facial recognition payment systems.

[0048] Depth map: In 3D computer graphics and computer vision, a depth map is an image or image channel that contains information about the distance from the surface of a scene object to the viewpoint. Each pixel in the depth map represents the vertical distance between the depth camera plane and the plane of the object being photographed, typically represented by 16-bit data in millimeters. For example, depth maps are generally used in facial recognition payment systems for liveness detection and face comparison recognition.

[0049] Infrared image: An image obtained by collecting infrared light and performing imaging operations. For example, infrared images are generally used for liveness detection in facial recognition payment systems.

[0050] RGB image (where R stands for Red, G for Green, and B for Blue): refers to an image obtained according to the RGB color mode. The RGB color mode is an industry-standard color system that creates a wide variety of colors by varying the red, green, and blue color channels and superimposing them. RGB represents the colors of the red, green, and blue channels. This standard encompasses almost all colors perceptible to human vision and is one of the most widely used color systems today.

[0051] Depth Map: In 3D computer graphics, a depth map is an image or image channel that contains information about the distances to the surfaces of scene objects from the viewpoint. It is similar to a grayscale image, except that each pixel value represents the actual distance from the sensor to the object.

[0052] RGBD image (where R represents Red, G represents Green, B represents Blue, and D represents Depth): An RGBD image is composed of an RGB image and a depth map. Because the RGB image and the depth map are registered, there is a one-to-one correspondence between pixels. Therefore, pixels from the RGB image and the depth map can be combined to generate an RGBD image.

[0053] Scrambling: In image processing, scrambling is an important data encryption technique and can serve as an effective security enhancement method.

[0054] Facial feature vector: Contains digital feature information about a person's face, and is often used in face recognition for facial similarity comparison.

[0055] Facial feature points: refer to preset points on a human face. Optionally, facial feature points include at least one of the following: left eye feature point, right eye feature point, nose tip feature point, left corner of lip feature point, and right corner of lip feature point.

[0056] Field of View (FOV): This refers to the angle formed by the two sides of the lens that allow the object being measured to pass through the lens at its apex (the camera). The FOV determines the field of view of the camera; a larger FOV results in a wider field of view. In other words, if an object is not within the FOV, it will not be captured by the camera.

[0057] Intrinsic and Extrinsic Parameters: In the embodiments of this application, these refer to the intrinsic and extrinsic parameters of the camera, including intrinsic and extrinsic parameters. Intrinsic parameters are parameters related to the camera's inherent characteristics, including 1 / dx, 1 / dy, u0, v0, r, and f. Here, dx and dy represent the length units occupied by one pixel in the x and y directions, respectively, i.e., the actual physical value represented by one pixel; u0 and v0 represent the horizontal and vertical pixel differences between the center pixel coordinates and the origin pixel coordinates of the image; r represents the aperture radius of the camera; and f represents the focal length of the camera. Extrinsic parameters are parameters related to the camera's coordinate system, including ω, δ, θ, Tx, Ty, and Tz. Here, ω, δ, and θ represent the rotation parameters of the three axes of the three-dimensional coordinate system; and Tx, Ty, and Tz represent the translation parameters of the three axes of the three-dimensional coordinate system.

[0058] Cat Mapping: Also known as the Arnold Mapping, it was proposed by the Russian mathematician Vladimir Igorevich Arnold. This is a chaotic mapping method that involves repeated folding and stretching transformations within a finite region, and is generally used in multimedia chaotic encryption. Arnold is a relatively common scrambling algorithm, and the algorithm is generated by the following transformation formula:

[0059]

[0060] Where a, b, and N are positive integers, which can be set by technical personnel; x n and y n These are the x and y coordinates of the pixel before the cat was mapped; x n+1 and y n+1 These are the x and y coordinates of the pixel after the cat is mapped; mod() represents the modulo operation. When x n and y n Let be any real number, then the cat map is a chaotic map.

[0061] Figure 1 A schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application is shown. The computer system 100 includes a terminal 120 and a server 140.

[0062] Terminal 120 is equipped with cameras 121 and 122. Camera 121 captures color images, and camera 122 captures depth images. Camera 121 connects to terminal 120 via wired or wireless means to transmit color images to terminal 120; camera 122 connects to terminal 120 via wired or wireless means to transmit depth images to terminal 120. Terminal 120 has a face recognition-related application installed. This application can be a small program within an app, a dedicated application, or a web client. For example, when a user performs face recognition-related operations on terminal 120, such as preparing to use face payment, to ensure the security and reliability of the payment process, liveness detection needs to be performed on the face image acquired by terminal 120 to prevent transactions caused by illegal attacks and protect the interests of individuals and the public. Terminal 120 is at least one of a smartphone, tablet computer, e-book reader, MP3 player, MP4 player, laptop computer, and desktop computer.

[0063] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0064] Server 140 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 140 provides background services for the face recognition application and sends the liveness detection results to terminal 120. Optionally, server 140 undertakes the main computing work, and terminal 120 undertakes secondary computing work; or, server 140 undertakes secondary computing work, and terminal 120 undertakes the main computing work; or, server 140 and terminal 120 collaborate using a distributed computing architecture. In this embodiment, each step is executed by terminal 120 alone, or by server 140 alone, or by terminal 120 and server 140 jointly.

[0065] Figure 2 A flowchart illustrating a face recognition method provided in an exemplary embodiment of this application is shown. This method can be... Figure 1 The method, executed by the terminal 120 or other computer device shown, includes the following steps:

[0066] Step 202: Fuse the color image and depth image of the face image to obtain a fused image of the face image, which includes the two-dimensional information and depth information of the face image.

[0067] Optionally, the face image may include one or more faces.

[0068] Optionally, when a face image includes multiple faces, the face with the largest area is selected as the face to be identified.

[0069] Optionally, when a face image includes multiple faces, the face within a preset area is selected as the face to be identified. For example, when a face image includes multiple faces, the face located in the central area is selected as the face to be identified.

[0070] Optionally, when a face image includes multiple faces, the face whose image quality meets a preset standard is selected as the face to be identified. Image quality includes at least one of sharpness, brightness, grayscale, contrast, and resolution.

[0071] Alternatively, the facial image can be extracted from a video.

[0072] A color image refers to an RGB image of a human face. Each pixel in an RGB image records two-dimensional information, which optionally includes at least one of color information, position information, brightness information, and grayscale information. For example, a pixel in an RGB image occupies 24 bits of data, where 8 bits represent the R component, 8 bits represent the G component, and 8 bits represent the D component.

[0073] A depth map is used to record depth information in a face image. Each pixel in the depth map records depth information, which represents the vertical distance between the camera and the face being photographed. For example, one pixel in the depth map occupies 16 bits of data.

[0074] The depth map and color map are registered, meaning there's a one-to-one correspondence between pixels in both images. Furthermore, pixels in both maps use the same data format when recording data. Therefore, corresponding pixels in the depth and color maps can be recombine to obtain the corresponding pixels in the fused image. For example, a pixel in the RGB image occupies 24 bits of data, and a pixel in the depth map occupies 16 bits. Recombining the data of these two corresponding pixels and superimposing them on the corresponding pixel data from both images yields the pixels in the fused image, which occupies 40 bits of data.

[0075] Optionally, the fused image refers to an RGBD image, which is composed of an RGB image and a depth map. Each pixel in the RGBD image records both two-dimensional information and depth information. For example, a pixel in an RGBD image occupies 40 bits, of which 8 bits represent the R component, 8 bits represent the G component, 8 bits represent the D component, and 16 bits represent the H component.

[0076] For example, such as Figure 3 As shown, the color image 301 and the depth image 302 are fused to obtain the fused image 303.

[0077] Step 204: Divide the fused image into blocks to obtain at least two image blocks of the fused image.

[0078] Optionally, the fused image can be divided into uniform blocks to obtain image blocks of equal size. For example, the fused image can be divided into 9 image blocks of the same size using a 9*9 grid structure.

[0079] Optionally, the fused image can be divided into blocks to obtain image blocks of the fused image, with the image blocks having different sizes. For example, a 36*36 fused image can be divided into blocks to obtain one 36*18 image block and two 18*18 image blocks.

[0080] For example, such as Figure 3 As shown, the fused image 303 is uniformly divided into image blocks 304. Image block 304 contains 9 image blocks, which are distinguished by numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9. For example, Figure 3 In the image, "1" represents image block 1, "2" represents image block 2, and so on.

[0081] Step 206: Irreversibly scramble the pixels in at least two image blocks to obtain a pixel-scrambled face image.

[0082] Irreversible scrambling refers to the inability to reconstruct a fused image from a face image with scrambled pixels. It's important to note that this step irreversibly scrambles image blocks within the fused image, not the fused image itself. For example, the fused image is divided into image block 1 and image block 2. Pixels in image block 1 and image block 2 are irreversibly scrambled.

[0083] Optionally, the pixels within an image block are irreversibly shuffled sequentially to obtain a face image with pixel confusion. For example, the fused image is divided into blocks to obtain image block 1, image block 2, and image block 3. First, the pixels in image block 1 are irreversibly shuffled, then the pixels in image block 2 are irreversibly shuffled, and finally the pixels in image block 3 are irreversibly shuffled.

[0084] Optionally, the pixels within the image block are irreversibly scrambled simultaneously to obtain a face image with pixel confusion.

[0085] For example, each image block in image block 304 is irreversibly scrambled to obtain a pixel-obfuscated face image 305. To illustrate the difference before and after scrambling, the pixel-obfuscated face image 305 is also divided into blocks, resulting in 9 image blocks, which are distinguished by the numbers A, B, C, D, E, F, G, H, and I, respectively. Figure 3 In this context, "A" represents image block A, "B" represents image block B, and so on. Furthermore, there is a correspondence between image block 304 and the pixel-scrambled face image 305. Image block 1 is scrambled to obtain image block A, image block 2 is scrambled to obtain image block B, image block 3 is scrambled to obtain image block C, and so on.

[0086] Step 208: Determine the user identity information corresponding to the face image based on the pixel-obfuscated face image.

[0087] User identity information includes at least one of the following: user's name, nickname, gender, age, place of origin, and user identification code.

[0088] Optionally, a pixel-scrambled face image can be input into a face recognition model, and the user's identity information corresponding to the face image can be determined based on the model's output. It should be noted that the face recognition model is not sensitive to whether the pixels of the face image are scrambled; therefore, using a pixel-scrambled face image to train the face recognition model will result in a face recognition model capable of obtaining user identity information.

[0089] For example, such as Figure 3 As shown, a pixel-obfuscated face image 305 is input into a face recognition model 306, and the face recognition model 306 outputs user identity information 307.

[0090] In summary, this embodiment aligns and fuses the color and depth maps of a face image. The resulting fused image contains both two-dimensional and depth information. Dividing the fused image into blocks allows for better processing of individual pixels. By scrambling the image blocks, the two-dimensional and depth information in the fused image is preserved, preventing them from being reconstructed. This protects user privacy from being leaked. At the same time, it can also determine user identity information based on the pixelated face image without affecting the use of basic face recognition functions.

[0091] In the following embodiments, on the one hand, the feature points in the depth map and color map are used to determine whether the depth map and color map can be fused, so that the obtained fused image can better match the actual face. On the other hand, a face recognition model is used to obtain user identity information. Since the face recognition model is not sensitive to whether the pixels of the face image are scrambled, the face recognition model can obtain relatively accurate user identity information. Furthermore, the image blocks are irreversibly scrambled through cat mapping, so that the two-dimensional information and depth information in the fused image cannot be restored, ensuring that the user's privacy will not be leaked and enhancing security.

[0092] Figure 4 A flowchart illustrating a face recognition method provided in an exemplary embodiment of this application is shown. This method can be... Figure 1 The method, executed by the terminal 120 or other computer device shown, includes the following steps:

[0093] Step 401: Obtain the first intrinsic and extrinsic parameters of the first camera corresponding to the color image, and obtain the second intrinsic and extrinsic parameters of the second camera corresponding to the depth image.

[0094] The first intrinsic and extrinsic parameters include the intrinsic and extrinsic parameters of the first camera.

[0095] The second intrinsic and extrinsic parameters include the intrinsic and extrinsic parameters of the second camera.

[0096] Alternatively, the first camera and the second camera may be the same camera, or the first camera and the second camera may be different cameras.

[0097] Optionally, the first and second internal and external parameters can be obtained by consulting the factory parameters of the first and second cameras.

[0098] Optionally, the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters can be obtained by using a checkerboard marking method.

[0099] Step 402: Based on the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters, control the field of view of the color image and the field of view of the depth image to be the same.

[0100] Optionally, based on the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters, the field of view of the color image is kept constant, while the field of view of the depth image is adjusted so that the field of view of the color image and the field of view of the depth image are the same.

[0101] Optionally, based on the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters, while keeping the field of view of the depth map unchanged, the field of view of the color map is adjusted so that the field of view of the color map is the same as that of the depth map.

[0102] For example, such as Figure 5 As shown, color image 501 and depth image 502 are face images with different field of view. Keeping the field of view of depth image 502 unchanged, the field of view of color image 501 is adjusted to obtain color image 503, wherein the field of view of color image 503 is equal to the field of view of depth image 502.

[0103] Step 403: In response to the fact that the field of view of the color image and the field of view of the depth image are the same, obtain the first feature point in the color image and the second feature point in the depth image.

[0104] When the field of view of the color image and the field of view of the depth image are the same, the color image and the depth image have the same spatial coordinate system, and the size of the color image and the depth image is the same. For example, the size of both the color image and the depth image is 100 units * 100 units.

[0105] The first feature point is a point predetermined by a technician in the color image. The first feature point can be one point or multiple points. Optionally, the first feature point includes at least one of the following in the color image: the left eye feature point, the right eye feature point, the tip of the nose feature point, the left corner of the lip feature point, and the right corner of the lip feature point. The first feature point can also be other points in the color image; this application does not specifically limit this.

[0106] The second feature point is a point pre-defined by the technician in the depth map. The second feature point can be one point or multiple points. Optionally, the second feature point includes at least one of the following in the depth map: the left eye feature point, the right eye feature point, the tip of the nose feature point, the left corner of the lip feature point, and the right corner of the lip feature point. The second feature point can also be other points in the depth map; this application does not specifically limit this.

[0107] There is a correspondence between the first feature point and the second feature point. For example, if the first feature point is the left eye feature point of a face in the color image, then the second feature point is the left eye feature point of a face in the depth image.

[0108] Optionally, the first and second feature points can be determined using a feature point extraction model.

[0109] Step 404: In response to the overlap between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain a fused image of the face image.

[0110] Since color maps and depth maps with the same field of view have the same spatial coordinate system, it is possible to determine whether pixels overlap by comparing their coordinates.

[0111] Optionally, first feature points form a first feature box, and second feature points form a second feature box. In response to the number of non-overlapping pixels in the first and second feature boxes being less than a preset value, the color image and depth image are fused to obtain a fused image of the face image. The first feature box is obtained by connecting the first feature points using a smooth curve, and the first feature box passes through multiple pixels in the color image. The second feature box is obtained by connecting the second feature points using a smooth curve, and the second feature box passes through multiple pixels in the depth image. Optionally, the non-overlapping pixels in the first and second feature boxes are determined by the coordinates of the pixels.

[0112] For example, the pre-assumption is preset to 10. The first feature point forms the first feature box, and the second feature point forms the second feature box. The first feature box has 60 pixels, and the second feature box also has 60 pixels. The first and second feature boxes have 56 overlapping pixels and 5 non-overlapping pixels. Then, the color image and the depth image are fused to obtain the fused image of the face image.

[0113] Optionally, in response to the relative distance between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain a fused image of the face. Optionally, the relative distance between the first feature point and the second feature point is determined by the coordinates of the pixels.

[0114] For example, assuming a pre-defined value of 6, the coordinates of the first feature point in the color image are (10, 10), and the coordinates of the second feature point in the depth image are (13, 14). The calculated relative distance between the first and second feature points is... Therefore, the color image and the depth image are fused to obtain a fused image of the face.

[0115] Because the depth map and color map are registered, there is a one-to-one correspondence between pixels in the two images, and the pixels in the depth map and color map use the same data format when recording data. Therefore, corresponding pixels in the depth map and color map can be recombine to obtain the corresponding pixels in the fused image. For example, a pixel in the color map occupies 24 bits of data, and a pixel in the depth map occupies 16 bits of data. The data corresponding to the two corresponding pixels are recombine and superimposed on the data of the corresponding pixels in the two images to obtain the pixels in the fused image, which will occupy 40 bits of data.

[0116] Step 405: Divide the fused image into uniform blocks to obtain image blocks of equal size for the fused image.

[0117] Image patches can be either rectangles or triangles. For example, assuming the fused image size is 10*10, dividing the fused image into uniform blocks results in four 5*5 image patches. Alternatively, assuming the fused image size is 10*10, dividing the fused image into uniform blocks results in eight isosceles right-angled triangular image patches, each with a leg length of 5.

[0118] Optionally, the fused image can be divided into blocks using a grid structure to obtain image blocks of equal size for the fused image. For example, the fused image can be divided into blocks using a 9*9 grid structure to obtain 9 image blocks of equal size for the fused image.

[0119] Optionally, the fused image is divided into blocks to obtain image blocks of the fused image, and the size of the image blocks is stored differently.

[0120] Step 406: Perform cat mapping on the pixels in at least two image blocks to obtain the changed coordinates of the pixels in at least two image blocks.

[0121] Optionally, multiple cat mappings are performed on the pixels within at least two image blocks to obtain the changed coordinates of the pixels within at least two image blocks.

[0122] For example, it is pre-assumed that the pixel coordinates of the cat before mapping are (x n y n The coordinates of the pixel after the cat is mapped are (x... n+1 y n+1 Then we have:

[0123]

[0124] Where a, b, and N are positive integers, which can be set by the technicians themselves; mod() represents the modulo operation.

[0125] The pixel coordinate change represents the coordinates of the pixel after being mapped by the cat. For example, if the original coordinates of the pixel are (1, 5), after being mapped by the cat, the pixel coordinates become (2, 8).

[0126] Step 407: Obtain a face image with pixel confusion based on the pixel change coordinates within at least two image blocks.

[0127] Optionally, the data corresponding to the pixels before cat mapping is mapped to the pixel change coordinates to obtain a face image with pixel confusion. For example, the coordinates of pixel 1 before cat mapping are (1, 5), and pixel 1 carries data A. After performing cat mapping on pixel 1, the pixel change coordinates are (2, 8), and the data A of pixel 1 is transferred to the pixel change coordinates (2, 8).

[0128] Step 408: Input the third feature point in the pixel-confusing face image and the fused image into the face recognition model to obtain the feature vector corresponding to the face image data.

[0129] Optionally, the third feature point is determined based on the first and second feature points. When the color map and depth map are aligned, the coordinates of the first, second, and third feature points are the same. For example, if the coordinates of the first and second feature points are (1, 5), then the coordinates of the third feature point are also (1, 5).

[0130] Optionally, the third feature point is determined based on the fused image. Optionally, the third feature point includes at least one of the following features in the fused image: left eye feature point, right eye feature point, nose tip feature point, left lip corner feature point, and right lip corner feature point.

[0131] The input to the face recognition model is a pixel-obfuscated face image and a third feature point, and the output is a feature vector.

[0132] Optionally, the feature vectors can be represented as arrays. For example, a 16-bit array can be used to represent the feature vectors.

[0133] Optionally, the server inputs the third feature points from the pixel-obfuscated face image and the fused image into the face recognition model to obtain the feature vector corresponding to the face image data. To ensure user privacy is not compromised, the third feature points can be encrypted. This process includes the following steps:

[0134] 1. Obtain the third feature point in the fused image.

[0135] Optionally, the third feature point includes at least one of the following features in the fused image: left eye feature point, right eye feature point, nose tip feature point, left lip corner feature point, and right lip corner feature point.

[0136] Optionally, a third feature point in the fused image can be obtained through a feature point extraction model.

[0137] 2. Encrypt the third feature point to obtain encrypted feature point data.

[0138] Optionally, the coordinates of the third feature point are encrypted using a key to obtain encrypted feature point data.

[0139] Optionally, the coordinates of the third feature point are obtained; a feature point array is generated based on the coordinates of the third feature point, and the feature point array is encrypted to obtain encrypted feature point data. For example, the coordinates of the third feature point are obtained as (1, 5), (8, 9), and (4, 7), resulting in a feature point array {1, 5, 8, 9, 4, 7}. The feature point array is then encrypted using a key to obtain encrypted feature point data {g, q, 5, i, 7, 0}.

[0140] 3. Send encrypted feature point data and pixel-obfuscated face images to the server.

[0141] The terminal sends encrypted feature point data and pixel-obfuscated face images to the server.

[0142] 4. Receive user identity information returned by the server.

[0143] User identity information is obtained by the server based on decrypted encrypted feature point data and a pixel-obfuscated facial image. Specifically, after decrypting the encrypted feature point data, a third feature point is obtained.

[0144] Optionally, the server inputs the decrypted encrypted feature point data and the fused image into the face recognition model to obtain the feature vector corresponding to the face image data; based on the feature vector, the user identity information corresponding to the face image is determined.

[0145] Step 409: Determine whether the identity matching failure condition is met.

[0146] If the identity matching failure condition is not met, proceed to step 410;

[0147] If the identity matching failure condition is met, proceed to step 411.

[0148] The identity matching failure condition is used to determine whether user identity information corresponding to the feature vector exists. For example, if user identity information matching the feature vector is found in the database, it is considered that the identity matching failure condition is not met; if no user identity information matching the feature vector is found in the database, it is considered that the identity matching failure condition is met.

[0149] Step 410: Determine the user identity information corresponding to the face image based on the feature vector.

[0150] There is a correspondence between feature vectors and user identity information. For example, the memory stores user identity information and the corresponding feature vectors, and the user identity information is retrieved through the feature vectors.

[0151] Optionally, user identity information includes feature vectors.

[0152] Step 411: Obtain the user identity information corresponding to the face image.

[0153] In response to the failure of identity matching, obtain the user identity information corresponding to the face image.

[0154] Optionally, in response to the failure of identity matching, the user identity information corresponding to the face image is obtained based on the user input data, wherein the user input data is obtained through user input, and the input method includes, but is not limited to, at least one of voice input, text input, and image input.

[0155] Step 412: Use the pixel-obfuscated face image and encrypted feature point data as the target face image data.

[0156] Encrypted feature point data refers to data obtained by encrypting the third feature point.

[0157] Step 413: Save the target face image data and user identity information.

[0158] Optionally, the target face image data and user identity information can be stored in local storage.

[0159] Optionally, the target face image data, feature vector, and user identity information are saved to the database through steps 411-413.

[0160] In summary, this embodiment aligns and fuses the color and depth maps of a face image. The resulting fused image contains both two-dimensional and depth information. Dividing the fused image into blocks allows for better processing of individual pixels. By scrambling the image blocks, the two-dimensional and depth information in the fused image is preserved, preventing them from being reconstructed. This protects user privacy from being leaked. At the same time, it can also determine user identity information based on the pixelated face image without affecting the use of basic face recognition functions.

[0161] Furthermore, the third feature point in the fused image is encrypted, further enhancing security and ensuring user privacy is not compromised. Cat mapping is also used to scramble image blocks in the fused image, guaranteeing that the 2D and depth information within these blocks is not restored, thus protecting user privacy.

[0162] In addition, a facial recognition model was used to determine user identity information, making the obtained user identity information more accurate and with a higher accuracy rate.

[0163] In the following embodiments, since the face recognition model was used in the previous embodiment to determine the feature vector corresponding to the face image, it is necessary to train the face recognition model to ensure that the output result of the face recognition model is accurate.

[0164] Figure 6 This illustration shows a flowchart of a face recognition model training method provided in an exemplary embodiment of this application. The method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0165] Step 601: Obtain the training dataset.

[0166] The training dataset includes sample face images and real user identity information, with a correspondence between the sample face images and the real user identity information.

[0167] Optionally, the sample face image may include one or more faces.

[0168] Optionally, when the sample face image includes multiple faces, the face with the largest area is selected as the face to be trained.

[0169] Optionally, when the sample face image includes multiple faces, the faces within a preset region are used as the training faces. For example, when the sample face image includes multiple faces, the face located in the central region is used as the training face.

[0170] Optionally, when the sample face image includes multiple faces, the face whose image quality meets a preset standard is selected as the face to be trained. Image quality includes at least one of sharpness, brightness, grayscale, contrast, and resolution.

[0171] Step 602: Generate a sample face image with pixel confusion based on the sample face image.

[0172] Optionally, this step includes the following sub-steps:

[0173] 1. The color image and depth image of the sample face image are fused to obtain the fused image of the sample face image.

[0174] The fused image includes both two-dimensional and depth information of the face image.

[0175] Because the depth map and color map are registered, there is a one-to-one correspondence between the pixels in the two images, and the pixels in the depth map and color map use the same data format when recording data. Therefore, corresponding pixels in the depth map and color map can be recombine to obtain the corresponding pixels in the fused image.

[0176] 2. Divide the fused image into blocks to obtain image blocks of the fused image.

[0177] Optionally, the fused image can be divided into uniform blocks to obtain image blocks of equal size.

[0178] Optionally, the fused image can be divided into blocks to obtain image blocks of the fused image, and the size of the image blocks can be different.

[0179] 3. Irreversibly shuffle the pixels in at least two image blocks to obtain a sample face image with pixel confusion.

[0180] Optionally, the pixels within the image block are irreversibly shuffled sequentially to obtain a face image with scrambled pixels.

[0181] Optionally, the pixels within the image block are irreversibly scrambled simultaneously to obtain a face image with pixel confusion.

[0182] Step 603: Input the pixel-confusing sample face image into the face recognition model to obtain the sample feature vector corresponding to the sample face image.

[0183] Optionally, the third feature point in the sample face image with pixel confusion and the fused image is input into the face recognition model to obtain the feature vector corresponding to the face image data.

[0184] Optionally, the feature vectors can be represented as arrays. For example, a 16-bit array can be used to represent the feature vectors.

[0185] Step 604: Based on the sample feature vector, determine the sample user identity information corresponding to the sample face image data.

[0186] There is a correspondence between feature vectors and user identity information. For example, the memory stores user identity information and the corresponding feature vectors, and the user identity information is retrieved through the feature vectors.

[0187] Optionally, user identity information includes feature vectors.

[0188] Step 605: Train the face recognition model based on the difference between the real user identity information and the sample user identity information.

[0189] Optionally, the face recognition model can be trained using the error backpropagation method based on the difference between the real user identity information and the sample user identity information.

[0190] In summary, this embodiment provides a method for training a face recognition model. The trained face recognition model can effectively identify the mapping relationship between sample face images with pixel confusion and feature vectors. Furthermore, the trained face recognition model is transferable and can be used on different computer devices.

[0191] Facial recognition has a wide range of applications, such as facial payment, access control systems, employee attendance, identity authentication, and security systems. However, facial information is a type of private information, and if it is leaked, user privacy will be severely violated. Therefore, it is necessary to protect facial information during the facial recognition process to prevent user privacy from being violated. This will be illustrated using facial payment as an example.

[0192] Figure 7 A flowchart illustrating a face recognition method provided in an exemplary embodiment of this application is shown. This method can be... Figure 1 The method, executed by the terminal 120 or other computer device shown, includes the following steps:

[0193] Step 701: Obtain the facial video of the first user account.

[0194] The first user account refers to the account held by the user who uses facial recognition payment.

[0195] Optionally, the facial video of the first user account can be acquired via a camera. Alternatively, the facial video of the first user account can be downloaded from the network. Alternatively, the facial video of the first user account can be received from another computer device. Alternatively, the facial video of the first user account can be read from local storage.

[0196] Step 702: Take the image of the target frame in the face video as the face image.

[0197] The target frame is any frame in the face image. For example, if the face video has 60 frames, the 24th frame out of these 60 frames is taken as the target frame.

[0198] Step 703: Determine whether the color image and depth image of the face can be fused.

[0199] If the color image and depth image of the face can be fused, then proceed to step 704;

[0200] If the color image and depth image of the face cannot be fused, return to step 702.

[0201] Optionally, a first feature point in the color image and a second feature point in the depth image are obtained; it is determined whether the overlap between the first feature point and the second feature point is less than a preset value. If the overlap between the first feature point and the second feature point is less than the preset value, step 704 is executed; if the overlap between the first feature point and the second feature point is less than the preset value, step 702 is returned.

[0202] Step 704: Fuse the color image and depth image of the face image to obtain a fused face image.

[0203] A color image refers to an RGB image of a human face. Each pixel in an RGB image records two-dimensional information, which may include at least one of color information, position information, brightness information, and grayscale information.

[0204] Depth maps are used to record depth information in facial images. Each pixel in a depth map records depth information, which represents the vertical distance between the camera and the face being photographed.

[0205] The depth map and color map are registered, meaning there is a one-to-one correspondence between pixels in the two images. Furthermore, pixels in both the depth and color maps are recorded using the same data format. Therefore, corresponding pixels in the depth and color maps can be recombine to obtain the corresponding pixels in the fused image.

[0206] Optionally, the fused image refers to an RGBD image, which is a combination of an RGB image and a depth map. Each pixel in the RGBD image records both two-dimensional information and depth information.

[0207] Step 705: Divide the fused image into blocks to obtain image blocks of the fused image.

[0208] Optionally, the fused image can be divided into uniform blocks to obtain image blocks of equal size. For example, the fused image can be divided into 9 image blocks of the same size using a 9*9 grid structure.

[0209] Optionally, the fused image can be divided into blocks to obtain image blocks of the fused image, with the image blocks having different sizes. For example, a 36*36 fused image can be divided into blocks to obtain one 36*18 image block and two 18*18 image blocks.

[0210] Step 706: Irreversibly scramble the pixels in at least two image blocks to obtain a pixel-scrambled face image.

[0211] Irreversible scrambling refers to the inability to restore a face image with disordered pixels to a fused image. It's important to note that this step irreversibly scrambles image blocks within the fused image, not the fused image itself.

[0212] Optionally, the pixels within the image block are irreversibly shuffled sequentially to obtain a face image with scrambled pixels.

[0213] Optionally, the pixels within the image block are irreversibly scrambled simultaneously to obtain a face image with pixel confusion.

[0214] Step 707: Determine the user identity information corresponding to the face image based on the pixel-obfuscated face image.

[0215] User identity information includes at least one of the following: user's name, nickname, gender, age, place of origin, and user identification code.

[0216] Optionally, a pixel-scrambled face image can be input into a face recognition model, and the user's identity information corresponding to the face image can be determined based on the model's output. It should be noted that the face recognition model is not sensitive to whether the pixels of the face image are scrambled; therefore, using a pixel-scrambled face image to train the face recognition model will result in a face recognition model capable of obtaining user identity information.

[0217] Optionally, when the user identity information corresponding to the face image is not determined, the user identity information corresponding to the face image is obtained; the face image with pixel confusion is used as the target face image data; and the target face image data and user identity information are saved.

[0218] Step 708: In response to the user's identity information matching the information of the first user account, complete the face payment.

[0219] Optionally, the information of the first user account includes at least one of the following: user's name, nickname, gender, age, place of origin, and user identification code.

[0220] Optionally, if the user's identity information does not match the information of the first user account, return to step 702.

[0221] In summary, this embodiment extracts facial images from videos and aligns and fuses the color and depth maps of the facial images. The resulting fused image contains both two-dimensional and depth information. Dividing the fused image into blocks allows for better processing of individual pixels. By scrambling the image blocks, the two-dimensional and depth information in the fused image is protected, preventing them from being reconstructed. This protects user privacy from being leaked. At the same time, it can also determine user identity information based on the pixelated facial image without affecting the use of basic facial recognition functions.

[0222] The following are device embodiments of this application. For details not described in detail in the device embodiments, please refer to the corresponding descriptions in the above method embodiments. They will not be repeated here.

[0223] Figure 8 A schematic diagram of a face recognition device provided in an exemplary embodiment of this application is shown. This device can be implemented as all or part of a computer device through software, hardware, or a combination of both. The device 800 includes:

[0224] The fusion module 801 is used to fuse the color image and the depth image of a face image to obtain a fused image of the face image, wherein the fused image includes the two-dimensional information and depth information of the face image;

[0225] The segmentation module 802 is used to segment the fused image into blocks to obtain at least two image blocks of the fused image;

[0226] The scrambling module 803 is used to irreversibly scramble the pixels in the at least two image blocks to obtain a face image with pixel confusion.

[0227] The recognition module 804 is used to determine the user identity information corresponding to the face image based on the face image with pixel confusion.

[0228] In an optional design of this application, the fusion module 801 is further configured to acquire a first feature point in the color image and a second feature point in the depth image, wherein the first feature point and the second feature point have a corresponding relationship; in response to the overlap between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain the fused image of the face image.

[0229] In an optional design of this application, the fusion module 801 is further configured to, in response to the same field of view of the color image and the same field of view of the depth image, acquire the first feature point in the color image and acquire the second feature point in the depth image.

[0230] In an optional design of this application, the fusion module 801 is further configured to obtain first intrinsic and extrinsic parameters of the first camera corresponding to the color image, and second intrinsic and extrinsic parameters of the second camera corresponding to the depth image, wherein the first camera is used to capture the color image and the second camera is used to capture the depth image; and the field of view of the color image and the field of view of the depth image are controlled to be the same according to the first intrinsic and extrinsic parameters and the second intrinsic and extrinsic parameters.

[0231] In an optional design of this application, the segmentation module 802 is further configured to uniformly segment the fused image to obtain image blocks of equal size from the fused image.

[0232] In an optional design of this application, the scrambling module 803 is further configured to perform cat mapping on the pixels in the at least two image blocks to obtain the pixel change coordinates in the at least two image blocks; and obtain the pixel-scrambled face image based on the pixel change coordinates in the at least two image blocks.

[0233] In an optional design of this application, the recognition module 804 is further configured to: acquire a third feature point in the fused image; encrypt the third feature point to obtain encrypted feature point data; send the encrypted feature point data and the pixel-obfuscated face image to the server; and receive the user identity information returned by the server, wherein the user identity information is obtained by the server based on the decrypted encrypted feature point data and the pixel-obfuscated face image.

[0234] In an optional design of this application, the recognition module 804 is further configured to input the third feature point in the pixel-confused face image and the fused image into a face recognition model to obtain a feature vector corresponding to the face image data; and determine the user identity information corresponding to the face image based on the feature vector.

[0235] In an optional design of this application, the recognition module 804 is further configured to, in response to the failure of identity matching, obtain user identity information corresponding to the face image; use the pixel-obfuscated face image and encrypted feature point data as target face image data; and save the target face image data and the user identity information.

[0236] In an optional design of this application, the device further includes a training module 805;

[0237] The training module 805 is used to acquire a training dataset, which includes sample face images and real user identity information, wherein the sample face images correspond to the real user identity information; generate pixel-obfuscated sample face images based on the sample face images; input the pixel-obfuscated sample face images into the face recognition model to obtain sample feature vectors corresponding to the sample face images; determine the sample user identity information corresponding to the sample face image data based on the sample feature vectors; and train the face recognition model based on the difference between the real user identity information and the sample user identity information.

[0238] In summary, this embodiment aligns and fuses the color and depth maps of a face image. The resulting fused image contains both two-dimensional and depth information. Dividing the fused image into blocks allows for better processing of individual pixels. By scrambling the image blocks, the two-dimensional and depth information in the fused image is preserved, preventing them from being reconstructed. This protects user privacy from being leaked. At the same time, it can also determine user identity information based on the pixelated face image without affecting the use of basic face recognition functions.

[0239] Figure 9 This is a schematic diagram of the structure of a server provided in one embodiment of this application. Specifically, the server 900 includes a Central Processing Unit (CPU) 901, a system memory 904 including Random Access Memory (RAM) 902 and Read-Only Memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. The computer device 900 also includes a basic input / output system (I / O system) 906 that facilitates the transmission of information between various devices within the computer, and a mass storage device 907 for storing the operating system 913, application programs 914, and other program modules 915.

[0240] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 for user input, such as a mouse or keyboard. Both the display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include the input / output controller 910 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, printer, or other types of output devices.

[0241] Mass storage device 907 is connected to central processing unit 901 via a mass storage controller (not shown) connected to system bus 905. Mass storage device 907 and its associated computer-readable media provide non-volatile storage for server 900. That is, mass storage device 907 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.

[0242] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 904 and mass storage device 907 described above can be collectively referred to as memory.

[0243] According to various embodiments of this application, server 900 can also be connected to a remote computer on a network, such as the Internet. That is, server 900 can be connected to network 912 via network interface unit 911 connected to system bus 905, or it can also use network interface unit 911 to connect to other types of networks or remote computer systems (not shown).

[0244] According to another aspect of this application, a computer storage medium is also provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the program code being loaded and executed by a processor to implement the face recognition method as described above.

[0245] According to another aspect of this application, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the face recognition method described above.

[0246] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0247] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0248] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A face recognition method, characterized in that, The method includes: The process involves acquiring a first intrinsic and extrinsic parameter of a first camera corresponding to a color image of a face, and acquiring a second intrinsic and extrinsic parameter of a second camera corresponding to a depth image of the face. The first camera is used to capture the color image, and the second camera is used to capture the depth image. Based on the first and second intrinsic and extrinsic parameters, the field of view of the color image and the field of view of the depth image are controlled to be the same. In response to the field of view of the color image and the field of view of the depth image of the face being the same, a first feature point in the color image and a second feature point in the depth image are acquired, wherein the first feature point and the second feature point have a corresponding relationship. In response to the relative distance between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain a fused image. The relative distance is determined by the pixel coordinates of the first feature point and the pixel coordinates of the second feature point. The fused image includes the two-dimensional information and depth information of the face image. The fused image is divided into blocks to obtain at least two image blocks of the fused image; The pixels within the at least two image blocks are irreversibly scrambled to obtain a face image with pixel confusion. A third feature point is obtained from the fused image, wherein the coordinates of the first feature point, the second feature point, and the third feature point are the same; the third feature point is encrypted to obtain encrypted feature point data. Send the encrypted feature point data and the pixel-obfuscated face image to the server; The system receives user identity information returned by the server. This user identity information is obtained by the server inputting the decrypted encrypted feature point data and the pixel-obfuscated face image into the face recognition model to obtain a feature vector, and then determining the identity information based on the feature vector.

2. The method according to claim 1, characterized in that, The step of dividing the fused image into blocks to obtain at least two image blocks of the fused image includes: The fused image is divided into uniform blocks to obtain at least two image blocks of equal size.

3. The method according to claim 1, characterized in that, The step of irreversibly scrambling the pixels within the at least two image blocks to obtain a pixel-obfuscated face image includes: Cat mapping is performed on the pixels within the at least two image blocks to obtain the changed coordinates of the pixels within the at least two image blocks; The pixel-obfuscated face image is obtained based on the pixel change coordinates within the at least two image blocks.

4. The method according to claim 1, characterized in that, The method further includes: In response to the failure of identity matching, the user identity information corresponding to the face image is obtained; The pixel-obfuscated face image and encrypted feature point data are used as the target face image data; Save the target face image data and the user identity information.

5. The method according to claim 1, characterized in that, The method further includes: Obtain a training dataset, which includes sample face images and real user identity information, wherein the sample face images correspond to the real user identity information; Based on the sample face image, generate a sample face image with pixel confusion; The pixel-confused sample face image is input into the face recognition model to obtain the sample feature vector corresponding to the sample face image; Based on the sample feature vector, determine the sample user identity information corresponding to the sample face image data; The face recognition model is trained based on the difference between the real user identity information and the sample user identity information.

6. A face recognition device, characterized in that, The device includes: A fusion module is used to acquire first intrinsic and extrinsic parameters of a first camera corresponding to a color image of a face image, and second intrinsic and extrinsic parameters of a second camera corresponding to a depth image of the face image. The first camera is used to capture the color image, and the second camera is used to capture the depth image. Based on the first and second intrinsic and extrinsic parameters, the field of view of the color image and the field of view of the depth image are controlled to be the same. In response to the field of view of the color image and the field of view of the depth image of the face image being the same, a first feature point in the color image and a second feature point in the depth image are acquired, wherein the first feature point and the second feature point have a corresponding relationship. In response to the relative distance between the first feature point and the second feature point being less than a preset value, the color image and the depth image are fused to obtain a fused image. The relative distance is determined by the pixel coordinates of the first feature point and the pixel coordinates of the second feature point. The fused image includes two-dimensional information and depth information of the face image. The segmentation module is used to segment the fused image into blocks to obtain at least two image blocks of the fused image; The scrambling module is used to irreversibly scramble the pixels in the at least two image blocks to obtain a face image with pixel confusion. The recognition module is used to acquire a third feature point in the fused image, wherein the coordinates of the first feature point, the second feature point, and the third feature point are the same; encrypt the third feature point to obtain encrypted feature point data; send the encrypted feature point data and the pixel-obfuscated face image to the server; and receive user identity information returned by the server, wherein the user identity information is determined by the server by inputting the decrypted encrypted feature point data and the pixel-obfuscated face image into a face recognition model to obtain a feature vector, and based on the feature vector.

7. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the face recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the face recognition method as described in any one of claims 1 to 5.

Images