Image processing method and related device

Abstract

The invention provides an image processing method and a related device. The method comprises the following steps: performing three-dimensional face reconstruction based on a first real image to obtain a three-dimensional face model of the first real image; projecting the three-dimensional face model to an imaging plane to obtain a first face image; performing disturbance processing on the first face image to obtain a second face image; mapping the second face image to the three-dimensional face model to obtain a three-dimensional fusion image; and determining a false face image based on the three-dimensional fused image. According to the method and the device, the relatively accurate virtual face image can be generated, so that the negative sample quality for training the face recognition model is improved, and the accuracy of model recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an image processing method and related devices. Background Art

[0002] Facial recognition algorithms are widely used in face verification modules for applications such as payment, security, and identity verification. Current facial recognition models in these modules are easily disrupted by fabricated fake facial images, resulting in erroneous recognition results. These fake facial images pose a security risk to facial recognition applications. To mitigate the security risks posed by fake facial images, fake facial images corresponding to real facial images are often pre-generated. These fake facial images can be used as negative samples to train facial recognition models, enhancing their resistance to fake facial images.

[0003] Therefore, in the field of artificial intelligence, especially face recognition, how to generate virtual facial images corresponding to real facial images has become one of the hot research issues. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method and related devices that can generate more accurate virtual facial images, thereby improving the quality of negative samples used to train facial recognition models, and further improving the accuracy of model recognition.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides an image processing method, which includes:

[0007] Performing three-dimensional facial reconstruction based on the first real image to obtain a three-dimensional facial model of the first real image;

[0008] Projecting the three-dimensional facial model onto an imaging plane to obtain a first facial image;

[0009] performing a perturbation process on the first facial image to obtain a second facial image;

[0010] Mapping the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image;

[0011] A false facial image is determined based on the three-dimensional fused image.

[0012] An embodiment of the present application provides an image processing device, comprising:

[0013] a determination module, configured to perform three-dimensional facial reconstruction based on a first real image to obtain a three-dimensional facial model of the first real image;

[0014] An acquisition module is configured to project the three-dimensional facial model onto an imaging plane to obtain a first facial image; perform perturbation processing on the first facial image to obtain a second facial image; map the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image; and determine a false facial image based on the three-dimensional fused image.

[0015] An embodiment of the present application provides an electronic device, comprising:

[0016] a memory for storing computer-executable instructions or computer programs;

[0017] The processor is used to implement the image processing method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the image processing method provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the image processing method provided in the embodiment of the present application is implemented.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] By performing three-dimensional facial reconstruction on a first real image, a three-dimensional facial model of the first real image is obtained, which accurately reflects the facial structure of the first real image in three-dimensional space. The three-dimensional facial model is projected onto the camera imaging plane to obtain a first facial image, which is then perturbed to obtain a second facial image, thereby increasing the complexity of the image features. The second facial image is mapped onto the three-dimensional facial model to obtain a three-dimensional fused image, which enhances the realism of the three-dimensional fused image. A false facial image is then determined based on the three-dimensional fused image, and the false facial image can be used as a negative sample for training a facial recognition model. The fusion between the first real image and the three-dimensional facial model can improve the fidelity of the constructed false facial image. Subsequent training of a facial recognition model based on these negative samples can improve the efficiency of facial recognition model training and the accuracy of facial recognition model recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of an application mode of the image processing method provided in an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the structure of the server provided in the embodiment of the present application;

[0024] Figure 3A This is a schematic diagram of the first flow chart of the image processing method provided in an embodiment of the present application;

[0025] Figure 3B 2 is a schematic diagram of a second flow chart of the image processing method provided in an embodiment of the present application;

[0026] Figure 3C 3 is a schematic diagram of a third flow chart of the image processing method provided in an embodiment of the present application;

[0027] Figure 3D 4 is a schematic diagram of a fourth flow chart of the image processing method provided in an embodiment of the present application;

[0028] Figure 4 5 is a schematic diagram of a fifth flow chart of the image processing method provided in an embodiment of the present application;

[0029] Figure 5 This is a real-space schematic diagram of the positions of the projector and camera provided in an embodiment of the present application;

[0030] Figure 6 is a schematic diagram of a three-dimensional facial model provided in an embodiment of the present application;

[0031] Figure 7 This is a schematic diagram of the camera imaging plane position provided in an embodiment of the present application;

[0032] Figure 8 This is a schematic diagram of the feature extraction principle of the facial recognition model provided in an embodiment of the present application.

[0033] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of distinction between the advantages and disadvantages of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0038] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0039] The collection and processing of relevant data (for example, captured facial images and target facial images) in the embodiments of this application should strictly comply with the requirements of relevant laws and regulations when applied in practice, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0040] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0041] 1) Facial recognition: It is a biometric technology that identifies individuals by analyzing and comparing facial images. It is widely used in various fields, including security verification, identity authentication, access control, traffic management, and payment systems in daily life.

[0042] 2) Three-dimensional grid: It is a regular dot matrix used to represent the three-dimensional structure of the face. It is composed of thousands of points, each of which corresponds to a position on the facial surface. These points are arranged at fixed intervals to form one or more planes, forming a three-dimensional structure that can be used to accurately describe the shape and features of the face.

[0043] 3) Texture mapping: It is an image processing technology that applies the texture information of a two-dimensional image to a three-dimensional model. It is a process of fitting the color and detail information in a two-dimensional image to the surface of a three-dimensional facial model, making the three-dimensional model look more realistic.

[0044] 4) Camera projection: This is the process of projecting an object in three-dimensional space onto a two-dimensional plane to generate a two-dimensional image. In camera projection, the object or scene is captured by the optical system of the camera lens, and the camera's imaging mechanism (such as a digital sensor or film) converts the light signal into an electrical signal, ultimately forming an image.

[0045] 5) The camera's two-dimensional plane: This refers to the plane where the camera's photosensitive element (such as film or a digital sensor) resides. It captures light signals and converts them into electrical signals, ultimately forming an image. On this plane, every point in the image corresponds to a point in three-dimensional space. This is achieved through the projection of the camera lens.

[0046] 6) 3D Morphable Model (3DMM): It is a mathematical model commonly used to represent and generate 3D faces. It represents the 3D face model as a series of parameterized shape and texture information.

[0047] 7) Adversarial samples: Samples designed to deceive AI models. In the embodiments of this application, adversarial samples are created by adding tiny, imperceptible perturbations to the original sample, with the goal of causing the machine learning model to make incorrect judgments. For facial recognition systems, adversarial samples typically add specific noise or patterns to normal facial images. These noise and patterns are carefully designed to deceive the facial recognition model, causing it to mistakenly identify the adversarial sample as a face or misjudge facial features.

[0048] 8) Attack: This refers to the use of adversarial examples to interfere with facial recognition systems. Such attacks can be motivated by various reasons, such as verifying system security, stealing sensitive information, or performing malicious operations. Attackers may try various methods to maximize the success rate of adversarial examples, such as cracking passwords, impersonating others, or disrupting the normal operation of the system.

[0049] 9) Projection Matrix: This is a mathematical matrix used in computer graphics and computer vision to project points in three-dimensional space onto a two-dimensional image plane. It is a core component of the camera model, describing the transformation from a three-dimensional world coordinate system to a two-dimensional image coordinate system. It is formed by multiplying the camera's intrinsic and extrinsic matrix. The intrinsic matrix is ​​a 3x3 matrix containing the camera's focal length and principal point coordinates. The extrinsic matrix is ​​a 3x4 matrix containing a rotation matrix and a translation vector, describing the camera's pose and position relative to a reference coordinate system. The rotation matrix is ​​a coordinate matrix describing the rotation of the coordinate system, while the translation vector describes the translation of the coordinate system's origin.

[0050] Related technologies use a projector to project adversarial texture images onto a person's face to generate adversarial samples. These fake facial images can be used as negative samples to train facial recognition models, enhancing their ability to resist fake images. However, the attack effectiveness of a fixed-position projector is limited, resulting in low accuracy of adversarial samples used to train facial recognition models, and poor training results.

[0051] The embodiments of the present application provide an image processing method and related devices that can generate more accurate virtual facial images, thereby improving the quality of negative samples used to train facial recognition models, and further improving the accuracy of model recognition.

[0052] The following describes exemplary applications of the electronic devices provided in the embodiments of the present application. The devices provided in the embodiments of the present application can be implemented as various types of terminals, such as laptops, tablet computers, desktop computers, set-top boxes, smartphones, smart speakers, smart watches, smart TVs, and in-vehicle terminals. They can also be implemented as servers. The following describes exemplary applications when the devices are implemented as terminals or servers.

[0053] See also Figure 1 , Figure 1 This is a schematic diagram of an application mode of the image processing method provided in an embodiment of the present application, for example, to support an image processing application. Figure 1 The server 200, network 300, terminal device 400 and database 500 are involved. The terminal device 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0054] In some embodiments, the user may be a technician in this field, the server 200 is a server for processing images, the terminal device 400 is a terminal operated by the user, and the database 500 stores the first real image, the disturbed image, the second real image and the false facial image obtained during the image processing process.

[0055] For example, the terminal device 400 is used to collect a first real image of a user and send the first real image to the server 200 via the network 300. The server 200 calls the image processing method of the present application, constructs a three-dimensional facial model based on the first real image, projects it onto different camera projection planes, iteratively adds a perturbed image to the image of each camera projection plane, and finally fuses multiple pixel values ​​on the projector plane to obtain a false facial image for training the facial recognition model. The false facial image is used as a negative sample to generate a training set, and the facial recognition model is trained based on the training set to obtain a trained facial recognition model. The server 200 sends the trained facial recognition model to the terminal device 400 via the network 300. The terminal device 400 places the trained facial recognition model in an application that requires facial recognition function. The user can pre-register the facial image of a legitimate user through the application. When using the relevant function in the application, the application calls the trained facial recognition model, compares the registered facial image with the user's first real image, and determines whether the user of the first real image can use the corresponding sensitive function (for example: payment, verification).

[0056] In some embodiments, the image processing method of the embodiments of the present application can also be applied in the following application scenarios:

[0057] 1. Application scenarios in the field of security and prevention: The image processing method provided in the embodiments of the present application can generate negative samples for training facial recognition models. In scenarios such as airports, stations, and public safety facilities, devices equipped with trained facial recognition models can be used to detect faces in real time, promptly discover and identify dangerous people, and effectively prevent and combat criminal activities.

[0058] 2. Application scenarios of the attendance system. The image processing method provided in the embodiment of the present application can generate negative samples for training facial recognition models. At the entrances and exits of enterprises, schools and other institutions, employees or students' attendance records can be automatically recorded by installing a device equipped with a trained facial recognition model. There is no need to use traditional attendance cards or passwords, which simplifies the attendance process and improves security.

[0059] 3. Application scenarios of financial security. The image processing method provided in the embodiments of the present application can generate negative samples for training facial recognition models. By installing a device with a trained facial recognition model, it can be used for customer identity authentication in banks and other financial institutions to ensure that only legitimate users can perform related operations, such as withdrawing money, transferring money, etc.

[0060] 4. Application scenario of personal device unlocking. The image processing method provided in the embodiment of the present application can generate negative samples for training facial recognition models. By installing smartphones and tablets with trained facial recognition models, facial recognition technology is supported as a way to unlock devices, which improves convenience of use while increasing security.

[0061] The embodiments of the present application can be implemented using database technology. A database, in short, can be considered an electronic filing cabinet that stores electronic files, allowing users to add, query, update, and delete data in these files. A "database" is a collection of data that is stored together in a specific manner, can be shared by multiple users, has minimal redundancy, and is independent of applications.

[0062] A database management system (DBMS) is a computer software system designed for managing databases, typically providing basic functions such as storage, retrieval, security, and backup. DBMSs can be categorized by the database model they support, such as relational or XML (Extensible Markup Language); by the type of computer they support, such as server clusters or mobile phones; by the query language they use, such as SQL or XQuery; by performance priorities, such as maximum scale or maximum speed; or by other classification methods. Regardless of the classification method used, some DBMSs are cross-category, for example, supporting multiple query languages ​​simultaneously.

[0063] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the server provided in the embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 410, a memory 450 and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0064] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0065] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0066] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0067] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0068] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0069] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, Wireless LAN (Wi-Fi), and Universal Serial Bus (USB).

[0070] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 2 An image processing device 455 stored in memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a determination module 4551 and an acquisition module 4552. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0071] In some embodiments, the terminal or server can implement the image processing method provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP). In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0072] The image processing method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the electronic device provided in the embodiment of the present application.

[0073] The following describes the image processing method provided by the embodiment of the present application. As mentioned above, the electronic device that implements the device management method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution entity of each step will not be repeated below.

[0074] See also Figure 3A , Figure 3A This is a first flow chart of the image processing method provided in the embodiment of the present application, which will be combined with Figure 3A The steps shown are explained, Figure 3A The executive body is Figure 1 Server 200 in.

[0075] In step 301, three-dimensional facial reconstruction is performed based on a first real image to obtain a three-dimensional facial model of the first real image.

[0076] For ease of understanding, the application scenario of the image processing method provided in the embodiment of the present application is explained. The processed images are used to train facial recognition models and serve as negative samples for training facial recognition models to improve the performance of facial recognition models. Facial recognition algorithms are widely used in fields such as payment, security, and identity authentication. For systems that rely on facial recognition, such as identity authentication for applications such as banks, payments, and attendance systems, the security of the system is of paramount importance. Current facial recognition models are easily attacked by maliciously constructed adversarial samples, resulting in incorrect recognition results. These malicious adversarial samples pose certain security risks to the application of face recognition. An attack is an act of interfering with a facial recognition system using adversarial samples. The attacker will try a variety of methods to maximize the probability that the adversarial samples will cause the facial recognition model to make incorrect judgments. Therefore, based on the above situation, the attack behavior can be simulated to obtain corresponding false images, and the obtained false images can be used as negative samples to train the facial recognition model to enhance the recognition ability and accuracy of the facial recognition model.

[0077] For example, the first real image is a 2D facial image. This can be obtained by placing a camera close to a projector and capturing a 2D frontal facial image of the attacker performing a simulated attack pose in front of the camera. 3D facial reconstruction is the process of constructing a 3D facial model corresponding to the first real image.

[0078] In some embodiments, see Figure 3B , Figure 3B 2 is a schematic diagram of a second flow chart of the image processing method provided in an embodiment of the present application; Figure 3A Step 301 can be performed by Figure 3B Steps 3011 to 3013 in are implemented as described below.

[0079] In step 3011, key point detection is performed on the first real image to obtain the positions of facial key points in the first real image.

[0080] For example, the first real image is a two-dimensional facial image. Preprocessing operations such as grayscale conversion, normalization, and noise reduction are performed on the first real image. Keypoint detection is performed on the first real image based on a facial keypoint detection algorithm to determine the locations of keypoints on the face in the first real image. Keypoints are points with specific meanings marked in a facial image that correspond to facial features. Keypoint locations typically include the locations of facial features such as the eyes, nose, and mouth.

[0081] In step 3012, facial shape parameters are generated based on the facial landmark positions.

[0082] For example, based on the detected facial key points, facial parameters are initialized using a 3D facial generative model, and the facial shape is represented as a parameterized model. The facial parameters characterize the basic shape and facial expressions of the human face. The 3D facial model can be a 3D Morphable Model, which is a mathematical model commonly used to represent and generate 3D faces. It represents the 3D facial model as a series of parameterized shape and texture information, allowing users to control the changes in facial shape and texture by adjusting a series of parameters. The 3D facial generative model usually defines a specific set of shape parameters, which can be used to describe the geometric features of the face. Based on the coordinates of the facial key points, the average facial shape and texture are calculated using statistical methods. The coordinates of each facial key point are compared with the average shape matrix to calculate the shape parameters that describe the difference between the two. The shape parameters enable the 3D facial generative model to adapt to different facial structures and generate more realistic 3D faces.

[0083] In step 3013, a three-dimensional mesh is constructed based on the facial shape parameters, and the first real image is mapped to the three-dimensional mesh to obtain a three-dimensional facial model of the first real image.

[0084] For example, a 3D mesh is a regular array of points used to represent the 3D structure of a face. Each point corresponds to a location on the facial surface, and the points are arranged at fixed intervals to form a 3D structure that can be used to accurately describe the shape and features of the face. A 3D mesh can be constructed by mapping facial shape parameters to a base mesh. That is, the base mesh is deformed according to the shape parameters corresponding to each vertex to obtain a 3D mesh representing the facial shape. In a 3D mesh, each vertex may correspond to multiple shape parameters, which determine the position and shape of the vertex in the mesh. The base mesh is a predefined average face model calculated based on a large amount of facial scan data. It has a fixed set of vertices, edges, and faces, representing a standard, average face model, and provides a basic framework for facial shape deformation.

[0085] For example, the texture of the first real image is mapped onto a three-dimensional mesh. Texture mapping is a process of fitting the color and detail information in the two-dimensional facial image to the surface of the three-dimensional facial model. The textured three-dimensional facial mesh is rendered using a rendering engine to generate a three-dimensional facial model with a realistic first real image. Figure 6 , Figure 6 3D facial model 601 is a schematic diagram of a 3D facial model provided in an embodiment of the present application. 3D facial model 601 is a 3D structure that displays the 3D shape and features corresponding to a first real image. Facial key points 602 extracted from a 2D facial image are accurately matched to 3D facial model 601 to simulate a real face. In practical applications, the number of vertices and facets on the 3D facial model can be adjusted based on actual needs. This embodiment of the present application is provided for illustrative purposes only and is not intended to be limiting.

[0086] In an embodiment of the present application, three-dimensional facial reconstruction is performed based on the first real image to construct a three-dimensional facial model of the first real image, which can provide more comprehensive facial information and more stable positioning of facial key points, so that the false image obtained by subsequent processing can better simulate the real face.

[0087] Continue to see Figure 3A In step 302, the three-dimensional facial model is projected onto an imaging plane to obtain a first facial image.

[0088] For example, the texture features of the 3D facial model are projected onto the camera's imaging plane, and the corresponding first facial image on each plane is obtained through camera projection. Camera projection is the process of projecting an object in 3D space onto a 2D plane to generate a 2D facial image. Through the projection effect of the camera lens, each point in the 2D camera image corresponds to a point in 3D space. The camera imaging plane is the plane where the different camera positions are preset.

[0089] In the embodiment of the present application, with the attacker as the center of the circle, multiple positions at the same distance as the human-machine distance at which the first real image was taken are selected as potential camera positions, and the position of each camera is distributed at an equal angle according to a preset angle. The preset offset angle is the degree of deviation from the line connecting the attacker and the projector. With the attacker as the center of the circle and connected to the position of the projector, each camera can be placed at any angle between directly facing the attacker (i.e., 0 degrees) and the preset offset angle, forming an evenly distributed arc around the attacker, and the position of each camera is distributed at an equal angle relative to the line connecting the attacker and the projector.

[0090] For ease of understanding, the position of the camera imaging plane is described below with reference to the accompanying drawings. Figure 7 , Figure 7 Schematic diagram of the camera imaging plane position provided in an embodiment of the present application; the human-machine distance is the distance between the line connecting the attacker 701 and the projector, the preset offset angle θ is the degree of deviation relative to the line connecting the attacker and the projector, and an arc range 702 is formed within the preset offset angle. The positions of cameras 7011 and 7012 are distributed at equal angles within the preset angle θ, and the distance from the attacker 701 is equal to the human-machine distance.

[0091] In some embodiments, for different camera positions, Figure 3A Step 302 can be implemented in the following manner: for each first pixel, based on a first mapping relationship between a three-dimensional coordinate value and a two-dimensional coordinate value, mapping a first pixel value of the first pixel to a second pixel value of a second pixel, where the second pixel is a pixel corresponding to the first pixel in an imaging plane, and the second pixel is located in a two-dimensional coordinate system corresponding to the imaging plane; and determining the first facial image based on the second pixel value of the second pixel corresponding to each first pixel.

[0092] Here, the three-dimensional facial model is represented by a three-dimensional grid, which includes multiple first pixels and a first pixel value of each first pixel. The first pixels are located in a three-dimensional coordinate system corresponding to the three-dimensional grid, and the imaging plane corresponds to a two-dimensional coordinate system.

[0093] For example, the position of each vertex in a 3D facial model in a texture coordinate system is calculated. Texture coordinates are position coordinates in a texture image. The projection matrix corresponding to the camera at each selected position is calculated. The projection matrix includes the camera's intrinsic and extrinsic parameters. Based on the camera's projection matrix and the 3D spatial position of the vertex, a first pixel is located in a 3D coordinate system corresponding to the 3D grid. The first pixel is any pixel in the 3D grid. Each first pixel has a color value, namely, the first pixel value of the first pixel. A first mapping relationship is established to associate each pixel in the 3D facial model with its corresponding pixel in the 2D facial image. Based on the first mapping relationship, the position of a second pixel in the 2D facial image is determined. The first pixel value of the first pixel is mapped to the second pixel value of the second pixel. The 3D facial model is converted into a 2D facial image through rendering. During the rendering process, the rendering engine uses the vertex positions of the 3D facial model and the corresponding texture coordinates to look up colors in the texture image. The color values ​​are determined based on the distance between vertices and smoothness. The color value of each pixel is sampled from the texture image. The texture coordinates are applied to each pixel, mapping the color in the texture image to the corresponding position, resulting in the first facial image.

[0094] In this embodiment, a 3D facial model is projected onto a camera imaging plane to generate a first facial image. The camera imaging plane is the plane where the selected camera positions are located. With the attacker as the center, multiple locations are selected at the same distance from the attacker as the camera used to capture the first real image. Each camera position is distributed according to a preset angle. Simulating different camera positions and angles can better understand the impact of lighting and perspective changes on facial recognition and generate diverse images.

[0095] Continue to see Figure 3A In step 303, the first facial image is disturbed to obtain a second facial image.

[0096] For example, perturbation processing is the process of iteratively adding perturbed images to a first facial image. Perturbation processing is performed on the first facial image, adding a perturbed image to each first facial image. After iteratively adding perturbed images until preconfigured conditions are met, a second facial image is obtained. Adding perturbed images is a targeted adversarial attack. An attack is the act of interfering with a facial recognition system using adversarial examples. Adversarial examples are designed to deceive the facial recognition model. The purpose of the perturbation is to cause the machine learning model to make incorrect judgments.

[0097] In some embodiments, see Figure 3C , Figure 3C 3 is a schematic diagram of a third flow chart of the image processing method provided in an embodiment of the present application; Figure 3A Step 303 can be performed by executing Figure 3CSteps 3031 to 3035 in are implemented as described below.

[0098] In step 3031, the first facial image is used as the image to be processed.

[0099] Here, the first real image is a face image of the first subject.

[0100] For example, performing perturbation processing on the first facial image is a process of adding a perturbed image, performing an independent perturbation operation on the first facial image on each camera imaging plane, and using the first facial image as an image to be processed before perturbation.

[0101] In step 3032, a disturbed image corresponding to the image to be processed is determined.

[0102] For example, a facial recognition model is used to perform perturbation on a first facial image and to identify a second real image to determine a loss function of the facial recognition model. The pixels of the perturbed image are updated using the loss function to determine a perturbed image corresponding to the image to be processed.

[0103] In some embodiments, the loss function is used to characterize the similarity between images, see Figure 3D , Figure 3D 4 is a schematic diagram of a fourth flow chart of the image processing method provided in an embodiment of the present application; Figure 3C Step 3032 can be performed by Figure 3D Steps 30321 to 30325 in the implementation are described in detail below.

[0104] In step 30321, when the image to be processed is the first facial image, an image with pixel values ​​of preset values ​​is used as a disturbance image.

[0105] For example, when the image to be processed is a first facial image, the pixels of the perturbed image are updated by the loss function. The perturbed image may initially be an all-zero image with a preset value of 0, that is, the image with a pixel value of 0 is taken as the perturbed image, and the perturbed image is recorded as δ.

[0106] In step 30322, when the image to be processed is the first superimposed image obtained in the previous iteration, the first image feature of the image to be processed and the real image feature of the second real image are determined.

[0107] Here, the first superimposed image obtained in the previous iteration is obtained by superimposing the existing disturbance image and the image to be processed in the previous iteration.

[0108] For example, when the image to be processed is the first superimposed image obtained in the previous iteration, the facial recognition model is called to extract features from the first superimposed image to obtain the first image features of the image to be processed, and feature extraction is performed on the second real image to obtain the real image features. The second real image is the image that the attacker wants the facial recognition model to mistakenly identify as the attacker's own identity. For example, the first superimposed image obtained in the previous iteration is recorded as J i +δ, call the facial recognition model (denoted as M) to extract features, and the first image feature obtained is denoted as M(J i +δ), the second real image is recorded as T, and the real image feature is recorded as M(T).

[0109] In step 30323, a disturbance variable is applied to the first image feature to obtain a second image feature.

[0110] For example, based on the loss function, the cosine similarity between the first image feature and the real image feature is determined, and the pixel value of the existing perturbed image is used as the perturbation variable. In each iteration, the pixel value of the perturbed image is solved by back propagation, and the pixel value of the perturbed image is updated, and then the perturbed image is updated, and the perturbation variable is applied to the first image feature to obtain the second image feature. For example: the perturbed image is denoted as variable δ, and the preconfigured loss function determined according to the first image feature and the real image feature is denoted as Loss(M(J i +δ), M(T)), the loss function is used to characterize the loss function (M(J i +δ), the similarity of M(T).

[0111] In step 30324, the target pixel value corresponding to the perturbation variable that maximizes the similarity between the second image feature and the true image feature is determined.

[0112] For example, based on the cosine similarity between the first image feature and the real image feature, by maximizing the cosine similarity value of the pre-configured loss function, back propagation is used to calculate the gradient of the loss function relative to the perturbed image, and the pixel value of the existing perturbed image is iteratively updated, and the pixel value of the perturbation variable that maximizes the similarity is used as the target pixel value. For example: by maximizing the loss function Loss(M(J i +δ), M(T)), the pixel values ​​of the perturbation image δ are updated based on back propagation to obtain the updated perturbation image δ', which is used as the perturbation image to be added in the next iteration.

[0113] In step 30325, the pixel value of the disturbed image determined in the previous iteration is updated based on the target pixel value to obtain the disturbed image.

[0114] For example, the pixel value of the last iteration is updated by making the target pixel value of the perturbation variable with the greatest similarity, so as to obtain a perturbed image after adding the perturbation of the target pixel value.

[0115] Continue to see Figure 3C In step 3033, the disturbance image is superimposed on the image to be processed to obtain a first superimposed image.

[0116] For example, the disturbed image is superimposed on the image to be processed, that is, each pixel value of the disturbed image is added to the corresponding pixel value in the image to be processed to obtain a first superimposed image. The image to be processed is denoted as J i , the added perturbation image is recorded as δ, and the first superimposed image is J i +δ, the first image feature is recorded as M(J i +δ), the real image feature is recorded as M(T), and the cosine similarity between the two image features is calculated as the loss function Loss(M(J i +δ), M(T)), by continuously adding perturbation images, the first superimposed image obtained is recorded as J N +δ, where N is a positive integer greater than 1.

[0117] In step 3034, if the first superimposed image does not meet the preconfigured condition, the first superimposed image is used as the image to be processed, and the step of determining the disturbance image corresponding to the image to be processed is re-executed.

[0118] Here, the preconfigured condition includes that the similarity between the first superimposed image and the second real image reaches a preconfigured similarity, the second real image is a facial image of a second object, and the first object and the second object are different.

[0119] For example, when the first superimposed image does not meet the preconfigured conditions, it indicates that there is still a large difference between the first superimposed image of the current iteration and the second real image, and it is necessary to continue to add perturbations to increase the similarity with the second real image. The first superimposed image is used as the image to be processed, and the step of determining the perturbed image is repeated, and the perturbation processing is continued on the first superimposed image. i +δ is the image J to be processed in the next iteration i+1 i+1 , continue to J i+1 Add the perturbed image δ.

[0120] In step 3035, if the first superimposed image meets the preconfigured condition, the first superimposed image is used as the second facial image.

[0121] For example, when the first superimposed image satisfies a preconfigured condition, the cosine similarity between the first superimposed image and the second real image after iteration reaches a maximum value. The preconfigured condition includes that the similarity between the first superimposed image and the second real image is equal to the preconfigured similarity, which can be used to deceive the facial recognition model and use the first superimposed image that meets the preconfigured condition as the second facial image. The first superimposed image J after N iterations that meets the preconfigured condition N +δ as the second facial image.

[0122] In an embodiment of the present application, perturbation processing is performed on the first facial image, and the pixel values ​​of the existing perturbation image are updated based on the similarity and the preconfigured loss function. When the first superimposed image meets the preconfigured conditions, the first superimposed image that meets the preconfigured conditions is used as the second facial image, thereby improving the similarity between the second real image and the second facial image, and generating a higher quality image for training the facial recognition model.

[0123] Continue to see Figure 3A In step 304, the second facial image is mapped onto the three-dimensional facial model to obtain a three-dimensional fused image.

[0124] For example, each second facial image is mapped onto a three-dimensional facial model composed of multiple vertices. During texture mapping, a corresponding texture coordinate is assigned to each vertex. The color information of each pixel in the second facial image is converted into the texture coordinates of the three-dimensional facial model. An interpolation algorithm is used to apply the texture color information to the surface of the three-dimensional model to obtain a three-dimensional fused image. Interpolation algorithms create new data points between known data points and are widely used to generate smooth curves, surfaces, or numerical sequences to meet specific application requirements. In this embodiment of the present application, the second facial image mapped onto the three-dimensional facial model can be a single image or multiple images.

[0125] In some embodiments, Figure 3A Step 304 in the embodiment can be implemented by: for each first pixel, determining a third pixel corresponding to the first pixel in the second facial image, and mapping the pixel value of the third pixel from the imaging plane to the three-dimensional grid based on a first mapping relationship between the three-dimensional coordinate value and the two-dimensional coordinate value, to obtain the third pixel value corresponding to the first pixel; performing a preset operation on the first pixel value and the third pixel value of the first pixel, and updating the first pixel value using the preset operation result to obtain a fused pixel value of the first pixel; and determining a three-dimensional fused image based on the fused pixel values ​​of the plurality of first pixels.

[0126] Here, the three-dimensional face model is represented by a three-dimensional grid, which includes a plurality of first pixels and a first pixel value of each first pixel.

[0127] For example, based on a first mapping relationship, a first three-dimensional coordinate and a first two-dimensional coordinate are determined, where a first pixel is any pixel in a three-dimensional grid, each pixel has a specific first three-dimensional coordinate, and each pixel has a color value. A pixel value is obtained by mapping the first pixel value of the first pixel to a two-dimensional plane, and a pixel with the same pixel value in the second facial image is used as the third pixel corresponding to the first pixel. A preset operation is performed on the first pixel value and the third pixel value of the first pixel, wherein the preset operation is to calculate the average value of the first pixel value and the third pixel value. The calculated average value is used to update the first pixel value to obtain a fused pixel value of the first pixel. The fused pixel value of multiple first pixels is used as the final pixel value of the first pixel in the three-dimensional facial model to form a three-dimensional fused image.

[0128] In step 305 , a false facial image is determined based on the 3D fused image.

[0129] For example, a false facial image is determined based on mapping the three-dimensional fused image to the to-be-projected plane corresponding to the projector position.

[0130] In some embodiments, Figure 3A Step 305 can be implemented in the following manner: for each first pixel, based on a second mapping relationship between three-dimensional coordinate values ​​and two-dimensional coordinate values, mapping the first pixel value of the first pixel to a fourth pixel value of a fourth pixel, where the fourth pixel is a pixel corresponding to the first pixel in the plane to be projected, and the fourth pixel is located in the two-dimensional coordinate system corresponding to the plane to be projected; and determining the false facial image according to the fourth pixel value of the fourth pixel corresponding to each first pixel.

[0131] Here, the three-dimensional facial model is represented by a three-dimensional grid, which includes multiple first pixels and a first pixel value of each first pixel; the first pixels are located in a three-dimensional coordinate system corresponding to the three-dimensional grid; and the plane to be projected corresponds to a two-dimensional coordinate system.

[0132] For example, for each first pixel in a three-dimensional grid, the first pixel value of the first pixel is mapped to the fourth pixel value of the fourth pixel through the mapping relationship between the three-dimensional coordinate value and the two-dimensional coordinate value. Each first pixel has a specific three-dimensional coordinate (the three-dimensional coordinate value of the first pixel), and each first pixel has a color value (i.e., the first pixel value). Each first pixel in the three-dimensional fused image is associated with the corresponding pixel on the plane to be projected, mapped to the fourth pixel on the projection plane, and the fourth pixel is located in the two-dimensional coordinate system corresponding to the plane to be projected. The rendering engine uses the three-dimensional coordinates of the first pixel to find the color of the fourth pixel, determine the color value, and map the color value to the corresponding two-dimensional coordinate to form a false facial image. The false facial image is an image on a two-dimensional plane, and the three-dimensional fused image is a three-dimensional stereo image presented on a three-dimensional model.

[0133] In some embodiments, after step 305, the following processing is also performed: a false facial image is projected onto the face, and the projected face is photographed to obtain a sample facial image; a facial recognition model is trained based on the sample facial image to obtain a trained facial recognition model, and the trained facial recognition model is used to identify the object type corresponding to the image to be identified.

[0134] For example, after obtaining a fake face, a projector projects the fake face image onto the face of a real attacker, simulating the fake face techniques used in real-world scenarios. The face can be a real person's face or a head model. The sampling method involves photographing the projected face to obtain sample facial images. Based on the sample facial images, a training set for a facial recognition model is constructed. The facial recognition model then undergoes supervised learning based on the constructed training set, resulting in a trained facial recognition model used to identify whether a first real image belongs to a second real image. Whether the fake face image belongs to the second real image is used as the data label, with a label of 0 indicating that the sample image does not belong to the target face, and a label of 1 indicating that the sample image belongs to the target face. The facial recognition model is then used to predict image samples from the training set. The difference between the predicted image sample labels and the actual sample labels is calculated, and the cross-entropy loss between the predicted probability distribution and the actual distribution is determined. Backpropagation is used to minimize the cross-entropy loss function to obtain a trained facial recognition model. The trained facial recognition model is then used to identify the authenticity of any face image.

[0135] In an embodiment of the present application, a three-dimensional facial reconstruction is performed based on a first real image to construct a three-dimensional facial model of the first real image. More comprehensive facial information is constructed, facial key points are obtained, and facial shape and texture features are simulated more realistically. The three-dimensional facial model is projected onto multiple camera imaging planes distributed at equal distances and angles to obtain a first facial image. Different camera positions and angles are simulated to generate diverse images. Perturbation processing is performed on the first facial image. Based on similarity and a preconfigured loss function, the pixel values ​​of the existing perturbed image are updated. If the first superimposed image meets preconfigured conditions, the first superimposed image that meets the preconfigured conditions is used as the second facial image. This helps improve the similarity between the second real image and the second facial image, generating higher-quality sample images for training more accurate and secure facial recognition models.

[0136] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0137] Facial recognition algorithms are widely used in payment, security, and identity verification. Facial authentication systems based on deep neural network technology are being implemented in mobile applications. For systems that rely on facial recognition, such as those used for authentication in banking, payment, and attendance systems, system security is paramount. Current facial recognition models are easily corrupted by maliciously constructed adversarial examples, resulting in erroneous recognition results. These malicious adversarial examples pose a security risk to facial recognition applications.

[0138] Related technologies use face adversarial attack methods based on real-world physical environments, such as wearing adversarial glasses and facial masks. These methods require the production of specialized physical materials for attacks on specific targets, which is both economically and time-consuming. Alternatively, related technologies use projectors to implement adversarial attacks on the human face. By calibrating the positional relationship between the camera and projector, iterating the color conversion relationship between the projected digital image and the captured image, and then performing an adversarial texture projection, the adversarial image is first inverted and then projected to generate adversarial samples. In the adversarial sample generation process based on calibrated positional relationships, only the transformation relationship between planar images at fixed positions is considered, resulting in limited attack effectiveness. This results in low accuracy of adversarial samples generated for training face recognition models, which in turn affects the effectiveness of training face recognition models.

[0139] In an embodiment of the present application, a three-dimensional facial generation model is called based on a first real image to perform facial reconstruction to obtain a three-dimensional facial model, and texture features on the three-dimensional facial model are projected onto multiple preset camera imaging planes to obtain corresponding multiple two-dimensional facial images (the first facial image mentioned above). An adversarial perturbation image is iteratively added to the two-dimensional facial image at each camera imaging plane position, and the cosine similarity of features between the two-dimensional facial image and the adversarial perturbation image is determined based on the facial recognition model. The pixel values ​​of the adversarial perturbation image are updated to obtain facial adversarial images corresponding to the multiple camera imaging planes (the second facial image mentioned above). The multiple facial adversarial images are mapped onto the three-dimensional facial model and fused based on the pixel values ​​to obtain false facial images, which are used as negative samples for training the facial recognition model to improve the performance of the facial recognition model.

[0140] The following is a description with reference to the accompanying drawings. Figure 4 , Figure 4 This is a fifth flow chart of the image processing method provided in the embodiment of the present application. The execution subject may be a terminal device, a server, or a combination of the two. In the embodiment of the present application, the server is used as the execution subject as an example. Figure 4 The steps shown are explained in detail.

[0141] In step 401, image parameters are extracted from a first real image, and a three-dimensional facial generation model is called based on the image parameters to perform reconstruction to obtain a three-dimensional facial model.

[0142] For example, a camera is placed close to a projector. The subject poses in front of the camera, imitating another person's posture. The camera captures a frontal shot of the subject's face, known as a first real image. This first real image is then 3D reconstructed to construct a 3D facial model corresponding to the first real image. Facial recognition is a biometric technology that identifies individuals by analyzing and comparing facial images. An attack is the use of adversarial examples to interfere with a facial recognition system. Attackers attempt to maximize the probability of the adversarial example misidentifying the system, for example by mimicking the shape of another person's face.

[0143] In some embodiments, see Figure 5 , Figure 5 This is a real-space schematic diagram of the positions of the projector and camera provided in an embodiment of the present application; the person being photographed 501 simulates an attack scene in front of the camera, and the camera 5012 photographs the person being photographed 501, that is, obtains the facial image of the attacker. In real space, the position of the camera 5012 is very close to the position of the lens of the projector 5011, and ideally, the two are in the same position.

[0144] For example, the first real image is a two-dimensional image. The first real image is received as input and preprocessed, including grayscale conversion, normalization, and noise reduction. The above processing is used to improve the accuracy of facial feature extraction. A facial key point detection algorithm is used to determine the locations of key points of the face in the image. Key point locations generally include the locations of facial features such as the eyes, nose, and mouth. Based on the detected facial key points, facial parameters are initialized using a three-dimensional facial generation model. The three-dimensional facial model can be a three-dimensional deformable facial model (3D Morphable Model). The facial parameters represent the basic shape and facial expression of the face.

[0145] A 3D mesh is a regular dot matrix used to represent the 3D structure of a face. It is composed of thousands of points, each of which corresponds to a position on the facial surface. The points are arranged at fixed intervals to form one or more planes, forming a 3D structure that is used to describe the shape and features of the face. Using the parameterization capability of the 3D facial generation model, a preliminary 3D face mesh is generated based on the initialized parameters. On the constructed 3D mesh, the first real image is mapped as a texture onto the 3D mesh. Texture mapping applies the texture information of the 2D facial image to the 3D model, ensuring that each pixel of the image can be correctly mapped to the corresponding vertex of the 3D mesh, making the 3D model look more realistic. During the texture mapping process, each vertex corresponds to a set of texture coordinates. These coordinates define the position of the vertex in the texture image. The texture coordinates are mapped to each pixel of the model to achieve the effect of applying the texture image to the surface of the 3D model. See Figure 6 , Figure 6 3D facial model 601 is a three-dimensional structure that displays the shape and features corresponding to the first real image. Based on the facial key points 602 extracted from the two-dimensional facial image, precise matching is performed on the three-dimensional facial model 601 to simulate a real face.

[0146] In step 402, the texture features on the three-dimensional facial model are projected onto different preset camera imaging planes to obtain a two-dimensional facial image on each plane.

[0147] For example, the camera imaging plane is the plane where the positions of different cameras are preset. The texture features on the three-dimensional facial model are projected onto the preset camera imaging plane, and a two-dimensional facial image (equivalent to the first facial image mentioned above) on each plane is obtained through camera projection. Camera projection is the process of projecting an object in three-dimensional space onto a two-dimensional plane to generate a two-dimensional facial image. In camera projection, the object or scene is captured by the optical system of the camera lens, and the optical signal is converted into an electrical signal through the imaging mechanism of the camera, and finally a two-dimensional image is formed. Through the projection effect of the camera lens, each point of the image on the two-dimensional plane of the camera corresponds to a point in three-dimensional space.

[0148] The preset camera imaging plane is the plane where the positions of the different selected cameras are located. In the embodiment of the present application, the attacker's face is taken as the center of the circle, the line between the attacker's face and the projector is taken as the center line, and the distance between the attacker's face and the projector is taken as the human-machine distance. An offset angle from the center line is preset, and multiple positions with equal human-machine distances and equal angle distribution within the preset offset angle are selected as potential camera positions to cover the front angle range to be attacked.

[0149] The preset offset angle is the degree of deviation from the line connecting the attacker and the projector. With the attacker as the center, the camera is connected to the projector. Each camera can be placed at any angle between directly facing the attacker (i.e., 0 degrees) and the preset offset angle, forming an evenly distributed arc around the attacker. Each camera position is evenly distributed relative to the line connecting the attacker and the projector. For example, when capturing the first real image of the attacker, the distance d between the projector and the attacker is set. A preset offset angle θ of 15 degrees is used. Each camera position is d away from the attacker. Placing five cameras within the preset offset angle will form an evenly distributed arc around the attacker. Each camera position is offset by 3.75 degrees relative to the center line connecting the attacker and the projector, covering a range from 0 degrees to the preset offset angle (15 degrees).

[0150] In some embodiments, see Figure 7 , Figure 7 This is a schematic diagram of the camera imaging plane positions provided in an embodiment of the present application; cameras 7011 and 7012 are located at preset positions, their distance from attacker 701 being equal to the human-machine distance, which is the distance between the attacker 701 and the projector. The projector and its emitted light beam 703 are projected onto the camera imaging plane where the cameras are located, projecting a two-dimensional facial image. A preset offset angle θ represents the degree of deviation from the line connecting the attacker and the projector, forming an arc-shaped range 702 within the preset offset angle. Cameras 7011 and 7012 are positioned at equal angles within the preset angle θ. This equal angular spacing reduces deviations between cameras at different positions.

[0151] The projection matrix corresponding to each selected camera position is calculated. The projection matrix describes the transformation from the 3D world coordinate system to the 2D image coordinate system and is obtained by multiplying the camera's intrinsic and extrinsic matrix. The elements of the projection matrix include the camera's intrinsic and extrinsic parameters. The intrinsic matrix contains the camera's focal length and principal point coordinates, while the extrinsic matrix includes the rotation matrix and translation vector. It maps points in 3D space onto the 2D image plane. The position of each vertex in the 3D facial model in the texture coordinate system is calculated. Texture coordinates are the position coordinates in the texture image and are determined based on the camera's projection matrix and the vertex's 3D spatial position. Texture mapping is performed based on the calculated texture coordinates. Through rendering, the 3D model is converted into a 2D image. These images consist of pixels, each with a specific coordinate position and a color value. During rendering, the rendering engine uses the vertex positions of the 3D facial model and the corresponding texture coordinates to look up colors in the texture image. The color value is determined based on the distance between vertices and the smoothness. The color value of each pixel is sampled from the texture image. Texture coordinates are applied to each pixel, and the colors in the texture image are mapped to the corresponding positions to complete the rendering of the two-dimensional facial image, thereby obtaining the corresponding two-dimensional facial image on each plane.

[0152] In step 403 , for the two-dimensional facial image of each camera imaging plane, a perturbation image is iteratively added to the two-dimensional facial image to obtain a facial adversarial image of each camera imaging plane.

[0153] For example, a targeted adversarial attack is performed on the two-dimensional facial image on each camera imaging plane. The process of the targeted adversarial attack is to add a perturbed image to the two-dimensional facial image on each camera imaging plane to obtain the corresponding facial adversarial images. The perturbed image can initially be an all-zero image (an image with all pixel values ​​​​being 0). The attack is the act of interfering with the facial recognition system using adversarial samples. The adversarial sample is a sample that is designed to deceive the facial recognition model. It is a sample generated by adding a perturbed image to the original sample. The purpose of the perturbation is to make the machine learning model make an incorrect judgment. In the embodiment of the present application, the adversarial sample is to add a perturbed image to a normal facial image. The purpose of generating the adversarial sample is to deceive the facial recognition model, causing it to mistakenly identify it as a face or mistakenly judge the features of the face.

[0154] Calculating the face adversarial image is a process of iteratively optimizing the loss function of the face recognition model. The perturbation process of each two-dimensional face image on the camera imaging plane is independent. i ) is added to the perturbed image (denoted as δ), and the facial recognition model (denoted as M) and the second real image (denoted as T) are called to extract features of the image after adding the perturbed image, and the image features after adding the perturbed image are obtained (denoted as M(J i+δ)) and the real image features (denoted as M(T)), and calculate the cosine similarity between the two image features as the loss function Loss(M(J i +δ), M(T). Each iteration solves the pixel value of the perturbed image by reverse gradient, and updates each pixel value by maximizing the cosine similarity to achieve the update of the perturbed image. By continuously adding perturbed images, the facial adversarial image corresponding to each camera imaging plane is generated, denoted as J N +δ, where N is a positive integer greater than i. The higher the cosine similarity, the easier it is for the facial recognition model to judge the facial adversarial image and the second real image as the same person.

[0155] In some embodiments, see Figure 8 , Figure 8 Schematic diagram of the feature extraction principle of the facial recognition model provided by an embodiment of the present application; facial recognition model 801 extracts features from a facial adversarial image 8011 to which a perturbation image has been added and a target image 8012, obtaining image features 8021 of the facial adversarial image and image features 8022 of the target image. The target image 8012 is a facial image of another object different from the attacker's facial image. The attacker's goal is to cause the facial recognition model to mistakenly identify the false image projected by the attacker's face as the target image. The cosine similarity between the image features 8021 of the facial adversarial image and the image features 8022 of the target image is calculated, and a loss function 802 is determined. The pixels of the perturbation image are updated using the loss function 802. The perturbation image is iteratively added to the facial adversarial image 8011 to maximize the cosine similarity between the image features of the two, thereby obtaining a final facial image close to the target image 8012.

[0156] In step 404, the facial adversarial image of each camera imaging plane is mapped onto the three-dimensional facial model for fusion to obtain a false facial image.

[0157] For example, the image texture of the facial adversarial image on each camera imaging plane is mapped onto a 3D facial model. The 3D facial model consists of multiple vertices. During texture mapping, a corresponding texture coordinate must be assigned to each vertex. The color information of each pixel in the facial adversarial image is converted into the texture coordinates of the 3D facial model. An interpolation algorithm is used to apply the texture color information to the surface of the 3D model. Interpolation algorithms are a type of mathematical algorithm used to create new data points between known data points and are widely used to generate smooth curves, surfaces, or numerical sequences to meet specific application requirements.

[0158] For example, after mapping the adversarial facial images on all camera imaging planes, texture fusion is performed on the 3D facial model. For each texture coordinate, the pixel values ​​corresponding to the texture coordinate are averaged to obtain a new texture map. This new texture map for each texture coordinate forms a fake facial image. The fake facial image is the image projected onto the attacker's face. The superposition of the attacker's face and the projected image creates an image designed to deceive the facial recognition model. The fake facial image can be used as a negative sample for the facial recognition model to train it.

[0159] The image processing method provided in the embodiment of the present application has the following beneficial effects:

[0160] Based on a first real image, a 3D facial generative model is invoked to reconstruct the face, generating a 3D facial model. Texture features from the 3D facial model are then projected onto different preset camera imaging planes, generating corresponding 2D facial images for each plane. Adversarial perturbation images are iteratively added to the 2D facial images at each camera imaging plane, enabling simulated attacks from multiple angles. This avoids the need to calibrate the relative positions of the projector and camera, increasing the flexibility of the adversarial attack process. The cosine similarity between the features of the 2D facial image and the adversarial perturbation image is determined based on the facial recognition model, and the pixel values ​​of the adversarial perturbation image are updated to further enhance its ability to deceive the facial recognition model. The resulting adversarial facial images corresponding to each of the multiple camera imaging planes are then mapped onto the 3D facial model, achieving adversarial fusion across angles to generate fake facial images, enhancing the transferability of the adversarial attack. These images are then used as negative samples for training the facial recognition model, improving its performance and assisting it in making more accurate decisions when faced with adversarial examples.

[0161] The following continues to describe the exemplary structure of the image processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the image processing device 455 of the memory 450 may include: a determination module 4551, used to perform three-dimensional facial reconstruction based on the first real image to obtain a three-dimensional facial model of the first real image; an acquisition module 4552, used to project the three-dimensional facial model onto the imaging plane to obtain a first facial image; perform perturbation processing on the first facial image to obtain a second facial image; map the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image; and determine a false facial image based on the three-dimensional fused image.

[0162] In some embodiments, the determination module 4551 is further used to perform key point detection on the first real image to obtain the positions of facial key points in the first real image; generate facial shape parameters based on the facial key point positions; construct a three-dimensional grid based on the facial shape parameters, and map the first real image to the three-dimensional grid to obtain a three-dimensional facial model of the first real image.

[0163] In some embodiments, the three-dimensional facial model is represented by a three-dimensional grid, which includes multiple first pixels and a first pixel value of each first pixel; the first pixel is located in a three-dimensional coordinate system corresponding to the three-dimensional grid; the imaging plane corresponds to a two-dimensional coordinate system; the acquisition module 4552 is also used to map the first pixel value of the first pixel to the second pixel value of the second pixel based on a first mapping relationship between the three-dimensional coordinate value and the two-dimensional coordinate value for each first pixel, where the second pixel is a pixel corresponding to the first pixel in the imaging plane, and the second pixel is located in the two-dimensional coordinate system corresponding to the imaging plane; the first facial image is determined according to the second pixel value of the second pixel corresponding to each first pixel.

[0164] In some embodiments, the first real image is a facial image of a first object; the acquisition module 4552 is further used to use the first facial image as the image to be processed; determine the disturbed image corresponding to the image to be processed; superimpose the disturbed image and the image to be processed to obtain a first superimposed image; if the first superimposed image does not meet the preconfigured conditions, use the first superimposed image as the image to be processed, and re-execute the step of determining the disturbed image corresponding to the image to be processed, wherein the preconfigured conditions include that the similarity between the first superimposed image and the second real image reaches a preconfigured similarity, the second real image is a facial image of a second object, and the first object and the second object are different; if the first superimposed image meets the preconfigured conditions, use the first superimposed image as the second facial image.

[0165] In some embodiments, the acquisition module 4552 is further used to, when the image to be processed is the first facial image, use an image whose pixel value is a preset value as a perturbed image; when the image to be processed is the first superimposed image obtained in the previous iteration, determine the first image features of the image to be processed and the real image features of the second real image; apply a perturbation variable to the first image feature to obtain a second image feature; determine a target pixel value corresponding to the perturbation variable that maximizes the similarity between the second image feature and the real image feature; and update the pixel value of the perturbation image determined in the previous iteration based on the target pixel value to obtain a perturbed image.

[0166] In some embodiments, the three-dimensional facial model is represented by a three-dimensional grid, which includes multiple first pixels and a first pixel value for each first pixel; the acquisition module 4552 is further used to determine, for each first pixel, a third pixel corresponding to the first pixel in the second facial image, and based on a first mapping relationship between the three-dimensional coordinate value and the two-dimensional coordinate value, map the pixel value of the third pixel from the imaging plane to the three-dimensional grid to obtain the third pixel value corresponding to the first pixel; perform a preset operation on the first pixel value and the third pixel value of the first pixel, and use the preset operation result to update the first pixel value to obtain a fused pixel value of the first pixel; and determine a three-dimensional fused image based on the fused pixel values ​​of the multiple first pixels.

[0167] In some embodiments, the three-dimensional facial model is represented by a three-dimensional grid, which includes multiple first pixels and a first pixel value of each first pixel; the first pixel is located in a three-dimensional coordinate system corresponding to the three-dimensional grid; the plane to be projected corresponds to a two-dimensional coordinate system; the acquisition module 4552 is also used to map the first pixel value of the first pixel to a fourth pixel value of a fourth pixel based on a second mapping relationship between the three-dimensional coordinate value and the two-dimensional coordinate value for each first pixel, where the fourth pixel is a pixel corresponding to the first pixel in the plane to be projected, and the fourth pixel is located in the two-dimensional coordinate system corresponding to the plane to be projected; and a false facial image is determined according to the fourth pixel value of the fourth pixel corresponding to each first pixel.

[0168] In some embodiments, after determining the false facial image to be projected based on the three-dimensional fused image, the acquisition module 4552 is further used to project the false facial image onto the face, and photograph the projected face to obtain a sample facial image; based on the sample facial image, the facial recognition model is trained to obtain a trained facial recognition model, and the trained facial recognition model is used to identify the object type corresponding to the image to be identified.

[0169] The present invention provides a computer program product including a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the image processing method described in the present invention.

[0170] The embodiment of the present application provides a computer-readable storage medium in which computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the image processing method provided by the embodiment of the present application, for example, Figure 3A The image processing method is shown.

[0171] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0172] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0173] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0174] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0175] In summary, through the embodiments of the present application, a three-dimensional facial reconstruction is performed on a first real image to obtain a three-dimensional facial model of the first real image, the three-dimensional facial model is projected onto an imaging plane, the two-dimensional facial image is perturbed and then mapped onto the three-dimensional facial model to obtain a three-dimensional fused image, a false facial image is determined based on the three-dimensional fused image, and the false facial image is used as a negative sample for training a facial recognition model to improve the performance of the facial recognition model.

[0176] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Performing three-dimensional facial reconstruction based on the first real image to obtain a three-dimensional facial model of the first real image; Projecting the three-dimensional facial model onto an imaging plane to obtain a first facial image; performing a perturbation process on the first facial image to obtain a second facial image; Mapping the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image; A false facial image is determined based on the three-dimensional fused image.

2. The method according to claim 1, characterized in that The performing three-dimensional facial reconstruction based on the first real image to obtain a three-dimensional facial model of the first real image includes: Performing key point detection on the first real image to obtain positions of facial key points in the first real image; generating facial shape parameters based on the facial key point positions; A three-dimensional mesh is constructed based on the facial shape parameters, and the first real image is mapped to the three-dimensional mesh to obtain a three-dimensional facial model of the first real image.

3. The method according to claim 1, characterized in that The three-dimensional facial model is represented by a three-dimensional grid, wherein the three-dimensional grid includes a plurality of first pixels and a first pixel value of each first pixel; The first pixel is located in a three-dimensional coordinate system corresponding to the three-dimensional grid; The imaging plane corresponds to a two-dimensional coordinate system; Projecting the three-dimensional facial model onto an imaging plane to obtain a first facial image includes: For each first pixel, mapping a first pixel value of the first pixel to a second pixel value of a second pixel based on a first mapping relationship between three-dimensional coordinate values ​​and two-dimensional coordinate values, where the second pixel is a pixel corresponding to the first pixel in the imaging plane and the second pixel is located in a two-dimensional coordinate system corresponding to the imaging plane; A first facial image is determined according to a second pixel value of a second pixel corresponding to each first pixel.

4. The method according to claim 1, wherein The first real image is a facial image of a first subject; and performing perturbation processing on the first facial image to obtain a second facial image includes: using the first facial image as an image to be processed; Determining a disturbed image corresponding to the image to be processed; Superimposing the disturbance image and the image to be processed to obtain a first superimposed image; If the first superimposed image does not meet a preconfigured condition, using the first superimposed image as the image to be processed and re-performing the step of determining a perturbed image corresponding to the image to be processed, wherein the preconfigured condition includes that a similarity between the first superimposed image and a second real image reaches a preconfigured similarity, the second real image is a facial image of a second subject, and the first subject and the second subject are different; In a case where the first superimposed image meets the preconfigured condition, the first superimposed image is used as the second facial image.

5. The method according to claim 4, characterized in that The determining of the disturbed image corresponding to the image to be processed includes: In a case where the image to be processed is the first facial image, using an image with a preset pixel value as a disturbance image; In a case where the image to be processed is the first superimposed image obtained by the previous superposition process, determining a first image feature of the image to be processed and a real image feature of the second real image; applying a disturbance variable to the first image feature to obtain a second image feature; Determining a target pixel value corresponding to the disturbance variable that maximizes the similarity between the second image feature and the true image feature; The pixel value of the disturbed image determined by the previous superposition process is updated based on the target pixel value to obtain a disturbed image.

6. The method according to claim 1, characterized in that The three-dimensional facial model is represented by a three-dimensional grid, wherein the three-dimensional grid includes a plurality of first pixels and a first pixel value of each first pixel; Mapping the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image includes: For each first pixel, determining a third pixel corresponding to the first pixel in the second facial image, and mapping a pixel value of the third pixel from the imaging plane to the three-dimensional grid based on a first mapping relationship between three-dimensional coordinate values ​​and two-dimensional coordinate values ​​to obtain the third pixel value corresponding to the first pixel; performing a preset operation on the first pixel value and the third pixel value of the first pixel, and updating the first pixel value using the preset operation result to obtain a fused pixel value of the first pixel; A three-dimensional fused image is determined according to the fused pixel values ​​of the plurality of first pixels.

7. The method according to claim 1, characterized in that The three-dimensional facial model is represented by a three-dimensional grid, wherein the three-dimensional grid includes a plurality of first pixels and a first pixel value of each first pixel; The first pixel is located in a three-dimensional coordinate system corresponding to the three-dimensional grid; The plane to be projected corresponds to a two-dimensional coordinate system; The determining of a false facial image based on the three-dimensional fused image comprises: For each of the first pixels, mapping a first pixel value of the first pixel to a fourth pixel value of a fourth pixel based on a second mapping relationship between three-dimensional coordinate values ​​and two-dimensional coordinate values, where the fourth pixel is a pixel corresponding to the first pixel in the plane to be projected, and the fourth pixel is located in a two-dimensional coordinate system corresponding to the plane to be projected; A false facial image is determined according to a fourth pixel value of a fourth pixel corresponding to each first pixel.

8. The method according to any one of claims 1 to 7, characterized in that After determining the false facial image based on the three-dimensional fused image, the method further includes: Projecting the false facial image onto the face, and photographing the projected face to obtain a sample facial image; The facial recognition model is trained based on the sample facial image to obtain the trained facial recognition model, and the trained facial recognition model is used to identify the object type corresponding to the image to be identified.

9. An image processing method and device, characterized in that: The device comprises: a determination module, configured to perform three-dimensional facial reconstruction based on a first real image to obtain a three-dimensional facial model of the first real image; An acquisition module is configured to project the three-dimensional facial model onto an imaging plane to obtain a first facial image; perform perturbation processing on the first facial image to obtain a second facial image; map the second facial image onto the three-dimensional facial model to obtain a three-dimensional fused image; and determine a false facial image based on the three-dimensional fused image.

10. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; The processor is configured to implement the image processing method according to any one of claims 1 to 8 when executing the computer-executable instructions or computer program stored in the memory.

Images