Counterfeit object detection method, related apparatus, and storage medium

Abstract

Embodiments of the present application disclose a counterfeit object detection method, related device and storage medium. The method comprises: obtaining a target object feature of a target object in to-be-detected data; determining a first feature distance set, a second feature distance set and a third feature distance set, the first feature distance set being a feature distance between the target object feature and each real prototype in at least one real prototype, the second feature distance set being a feature distance between the target object feature and each unique counterfeit prototype in at least two unique counterfeit prototypes, and the third feature distance set being a feature distance between the target object feature and each common counterfeit prototype in at least one common counterfeit prototype; and if the target object is determined to be a counterfeit object based on the first feature distance set, the second feature distance set and the third feature distance set, determining a counterfeit type of the target object according to the second feature distance set. The present application can improve the accuracy of counterfeit object detection and improve the processing efficiency of counterfeit object detection.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a method, related apparatus and storage medium for detecting counterfeit objects. Background Technology

[0002] With the rapid development of Artificial Intelligence Generated Content (AIGC) technology, especially the continuous iteration of Generative Adversarial Networks (GANs) and Diffusion Models, the forgery effect of deepfake face technology is becoming increasingly realistic, making it difficult to distinguish forged content. This brings many potential risks and threats, such as fake news, online fraud, and privacy violations.

[0003] Given the risks of misuse of deepfake facial recognition technology, the need for detection and defense against deepfakes is becoming increasingly urgent. For example, social media platforms need to detect and defend against the spread of deepfake content, financial institutions need to detect and defend against the application of deepfake facial recognition technology in online fraud, and governments and law enforcement agencies need deepfake facial recognition detection technology to provide false evidence.

[0004] Many fake face detection technologies have been proposed to identify whether faces in images or videos have been tampered with and to output binary labels for real and fake faces. Most of these methods use data-driven deep learning techniques, designing more advanced network structures or loss functions to improve detection performance. However, simply outputting binary labels for real and fake faces cannot meet the needs of some scenarios. For example, for malicious and illegal fake faces, administrators also need to determine how the fake content was created.

[0005] In existing technologies, most studies separate the task of classifying fake faces as real or fake and the task of tracing the source of forgery methods. The forgery method tracing model needs to be chained after the real / fake classification model. That is, the face to be detected is first sent to the deep fake classification model to determine whether it is fake, and then sent to the forgery method tracing model to determine which forgery method was used. This approach requires the deployment of multiple models, which is cumbersome, costly, and inefficient. Summary of the Invention

[0006] This application provides a method, related apparatus, and storage medium for detecting counterfeit objects, which can improve the accuracy and processing efficiency of counterfeit object detection.

[0007] In a first aspect, embodiments of this application provide a method for detecting counterfeit objects, including:

[0008] Obtain the target object features from the data to be detected;

[0009] A first feature distance set, a second feature distance set, and a third feature distance set are determined. The first feature distance set is the feature distance between the target object feature and each of the at least one real prototypes. The second feature distance set is the feature distance between the target object feature and each of the at least two unique fake prototypes. The third feature distance set is the feature distance between the target object feature and each of the at least one common fake prototypes.

[0010] If the target object is determined to be a forged object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

[0011] In some embodiments, the fourth loss value is determined according to a fourth formula, which is:

[0012]

[0013] in, For the fourth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) c |x) indicates that the target sample belongs to y. c The probability that y is the binary classification label of the target sample is a true sample. c =0, when the binary label of the target sample is a fake sample. c =y m y m A label indicating a forgery type.

[0014] In some embodiments, the fifth loss value is determined according to a fifth formula, which is:

[0015]

[0016] in, For the fifth loss value, d(z) cls ,mij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 2 represents m ij For commonalities in the prototype model, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) b |x) indicates that the target sample belongs to label y. b The probability, y b The binary label represents the target sample.

[0017] In some embodiments, the data to be detected includes at least one of audio data, video data, and image data.

[0018] In some embodiments, the data to be detected includes at least one of video data and image data, and the target object is a face.

[0019] Secondly, embodiments of this application also provide a counterfeit object detection device, comprising:

[0020] The transceiver module is used to acquire the target object features of the target object in the data to be detected;

[0021] The processing module is configured to determine a first feature distance set, a second feature distance set, and a third feature distance set. The first feature distance set consists of the feature distances between the target object features and each of the at least one real prototypes. The second feature distance set consists of the feature distances between the target object features and each of the at least two unique forgery prototypes. The third feature distance set consists of the feature distances between the target object features and each of the at least one common forgery prototypes. If the target object is determined to be a forgery object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

[0022] In some embodiments, the processing module determines whether the target object is a forged object based on the following methods:

[0023] Determine the average feature distance in the first feature distance set, and determine the minimum feature distance from the second feature distance set and the third feature set; determine whether the target object is a fake object based on the average feature distance and the minimum feature distance.

[0024] In some embodiments, when the processing module performs the step of determining whether the target object is a forgery object based on the average feature distance and the minimum feature distance, it is specifically used for:

[0025] The first probability of the target object being a real object is determined based on the average feature distance, and the second probability of the target object being a fake object is determined based on the minimum feature distance; if the first probability is greater than the second probability, the target object is determined to be a real object; if the first probability is less than or equal to the second probability, the target object is determined to be a fake object.

[0026] In some embodiments, the forgery detection device is implemented based on a forgery detection model, which includes a feature encoder, a prototype set, a detection tracing module, and a prototype update module.

[0027] The prototype set includes at least one preset real prototype, at least two preset unique fake prototypes, and at least one preset common fake prototype.

[0028] The feature encoder is used to obtain the target object features of the target object;

[0029] The detection and tracing module is used to determine whether the target object is a forged object based on the characteristics of the target object and the prototype set, and when the target object is a forged object, to determine the forgery type of the target object;

[0030] The prototype update module is used to perform prototype update processing on at least one preset real prototype, at least two preset unique fake prototypes and at least one preset common fake prototype during the training phase of the fake object detection model, so as to obtain at least one real prototype, at least two fake prototypes and at least one common fake prototype.

[0031] In some embodiments, before the transceiver module performs the step of acquiring the target object features of the target object in the data to be detected, the processing module is further configured to:

[0032] The target sample is obtained through the transceiver module, and the target sample comes from the real sample set and the fake sample set.

[0033] Obtain the first feature and the second feature of the target sample, wherein the first feature includes real features, unique forgery features and common forgery features;

[0034] The prototype update loss value is determined based on the first feature, and the prototype detection source tracing loss value is determined based on the second feature;

[0035] If it is determined that the target loss value does not meet the preset convergence condition, a new sample is obtained from the real sample set and the fake sample set, and the new sample is used as the target sample until the target loss value meets the preset convergence condition, thereby obtaining the trained fake object detection model. The target loss value is determined based on the prototype update loss value and the prototype detection source tracing loss value.

[0036] In some embodiments, when the processing module performs the step of acquiring the first feature and the second feature of the target sample, it is specifically used for:

[0037] The first feature and the second feature of the target sample are determined according to the feature encoder;

[0038] The first feature is divided into multiple real features, multiple unique fake features, and multiple common fake features based on the multi-head cross-attention submodule and the prototype set. The prototype update module includes the multi-head cross-attention submodule.

[0039] In some embodiments, when the processing module performs the steps of determining the prototype update loss value based on the first feature and determining the prototype detection and tracing loss value based on the second feature, it is specifically used for:

[0040] The prototype update module determines the prototype update loss value based on multiple real features, multiple unique forgery features, multiple common forgery features, and the prototype set, and the detection and tracing module determines the detection and tracing loss value based on the second feature.

[0041] In some embodiments, the prototype update module further includes a forgery method discriminator and a authenticity discriminator; the prototype update loss value includes a first loss value, a second loss value, and a third loss value; when performing the step of determining the prototype update loss value based on the first feature, the processing module is specifically used for:

[0042] The target real features of each real prototype are determined from multiple real features based on the sample labels of each real feature; the target unique fake features of each unique fake prototype are determined from multiple unique fake features based on the sample labels of each unique fake feature; and the target common fake features of each common fake prototype are determined from multiple common fake features based on the sample labels of each common fake feature.

[0043] The first loss value is determined based on the prototype set, the target's real features, the target's unique forgery features, and the target's common forgery features;

[0044] The second loss value is determined based on the common forgery features of the target and the forgery method discriminator.

[0045] The third loss value is determined based on the sample labels of the common forgery features, the common forgery features, and the real / fake discriminator.

[0046] In some embodiments, the first loss value is determined according to a first formula, wherein the first formula is:

[0047]

[0048] in, Let I represent the first loss value, and let I represent the indicator function used to indicate the selection of the corresponding z based on the sample label. a and m ij , z a For the target's genuine features, the target's unique forged features, or the target's common forged features, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, i = 2 indicates m ij For the common forged prototype, j represents the prototype index of the prototype in the prototype set. I(z) a ) and I(m ij The characteristic distance between ).

[0049] In some embodiments, the second loss value is determined according to a second formula, which is:

[0050]

[0051] in, The second loss value is identified, x represents the target sample, f represents the feature encoder, g represents the multi-head cross-attention submodule, and D... m Let f(x) represent the forgery method discriminator, and f(x) represent the second feature. This indicates an indicator function that selects the target common forgery feature from the common forgery features as the sample label for forgery object, y b Indicates a binary category label, y b =1 indicates that the binary classification label is a fake object, and N represents the number of target samples. Indicates x, y b and y m Expectations on y m The label indicating the type of forgery, θ f θ represents the parameters of the feature encoder. g This represents the parameters of the multi-head cross-attention submodule. The parameters represent those of the forgery method discriminator.

[0052] In some embodiments, the third loss value is determined according to a third formula, which is:

[0053]

[0054] in, The third loss value, This refers to the probability output when the common forgery features are input into the true / false discriminator.

[0055] In some embodiments, the detection and tracing loss value includes a fourth loss value and a fifth loss value; when the processing module performs the step of determining the prototype detection and tracing loss value based on the second feature, it is specifically used for:

[0056] The detection and tracing module determines a first sample feature distance set, a second sample feature distance set, and a third sample feature distance set. The first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype. The second sample feature distance set includes the feature distances between the second feature and each unique fake prototype in at least two preset unique fake prototypes. The third sample feature distance set includes the feature distances between the second feature and each common fake prototype in at least one preset common fake prototype.

[0057] The fourth loss value of the target sample is determined based on the first sample feature distance set and the second sample feature distance set;

[0058] The fifth loss value of the target sample is determined based on the first sample feature distance set and the third sample feature distance set.

[0059] In some embodiments, the fourth loss value is determined according to a fourth formula, which is:

[0060]

[0061] in, For the fourth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ijFor a unique forged prototype, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) c |x) indicates that the target sample belongs to y. c The probability that y is the binary classification label of the target sample is a true sample. c =0, when the binary label of the target sample is a fake sample. c =y m y m A label indicating a forgery type.

[0062] In some embodiments, the fifth loss value is determined according to a fifth formula, which is:

[0063]

[0064] in, For the fifth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 2 represents m ij For commonalities in the prototype model, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) b |x) indicates that the target sample belongs to label y. b The probability, y b The binary label represents the target sample.

[0065] In some embodiments, the data to be detected includes at least one of audio data, video data, and image data.

[0066] In some embodiments, the data to be detected includes at least one of video data and image data, and the target object is a face.

[0067] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0068] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the above-described method.

[0069] Fifthly, embodiments of this application provide a chip that includes a transceiver coupled to a terminal device for executing the technical solution provided in the first aspect of embodiments of this application.

[0070] Sixthly, embodiments of this application provide a chip system including a processor for supporting a terminal device in implementing the functions involved in the first aspect, such as generating or processing information involved in the counterfeit object detection method provided in the first aspect. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the terminal. The chip system may be composed of chips or may include chips and other discrete devices.

[0071] In a seventh aspect, embodiments of this application provide a computer program product containing instructions. When the computer program product is run on a computer, it causes the computer to execute the forgery detection method provided in the first aspect, thereby achieving the beneficial effects of the forgery detection method provided in the first aspect.

[0072] Compared to existing technologies, the solution provided in this application has several advantages. First, since the forgery prototypes provided in this solution include multiple unique forgery prototypes representing unique forgery features and at least one common forgery prototype representing common forgery features, the forgery prototypes provided in this solution can not only identify unique forgery features in the target object, but also common forgery features in the target object. This allows for a more comprehensive identification of the forgery features of the target object. By using the more comprehensive forgery features identified, the target object can be classified into true and false categories, thus improving the classification effect of the true and false binary classification. Second, when this solution identifies the target object as a forgery object, it further determines the forgery type of the target object using the second feature distance set calculated during the previous binary classification task, without needing to calculate the feature distance again. This solution can simultaneously obtain the data required for forgery type tracing when performing binary classification on the target object, resulting in high processing efficiency for forgery object detection. Attached Figure Description

[0073] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a schematic diagram illustrating an application scenario of the forgery detection method provided in the embodiments of this application;

[0075] Figure 2This is a schematic diagram illustrating the training process of the forgery detection model in the forgery detection method provided in this application embodiment;

[0076] Figure 3 This is a schematic diagram of a training sub-process of the forgery object detection model in the forgery object detection method provided in the embodiments of this application;

[0077] Figure 4 This is a schematic diagram of another training sub-process of the fake object detection model in the fake object detection method provided in the embodiments of this application;

[0078] Figure 5 This is a schematic diagram of another training sub-process of the fake object detection model in the fake object detection method provided in the embodiments of this application;

[0079] Figure 6 A schematic diagram of the training process for the forged object detection method provided in this application embodiment based on the specific framework of the forged object detection model;

[0080] Figure 7 A flowchart illustrating the counterfeit object detection method provided in this application embodiment;

[0081] Figure 8 A schematic diagram of the sample feature space provided in the embodiments of this application;

[0082] Figure 9 A schematic block diagram of a counterfeit object detection device provided in an embodiment of this application;

[0083] Figure 10 This is a schematic diagram of a server structure in one embodiment of this application;

[0084] Figure 11 This is a schematic diagram of the structure of a terminal device in an embodiment of this application;

[0085] Figure 12 This is a schematic diagram of a server structure in one embodiment of this application. Detailed Implementation

[0086] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0087] This application provides a method, related apparatus, and storage medium for detecting counterfeit objects. The execution subject of the method for detecting counterfeit objects can be the counterfeit object detection apparatus provided in this application, or a computer device that integrates the counterfeit object detection apparatus. The counterfeit object detection apparatus can be implemented in hardware or software, and the computer device can be a terminal or a server.

[0088] When the computer device is a server, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0089] When the computer device is a terminal, the terminal may include, but is not limited to, smart terminals with multimedia data processing functions (e.g., video data playback function, music data playback function), such as smartphones, tablets, laptops, desktop computers, smart TVs, smart speakers, personal digital assistants (PDAs), desktop computers, and smartwatches.

[0090] The solutions in this application can be implemented based on artificial intelligence technology, specifically involving computer vision technology in artificial intelligence technology and cloud computing, cloud storage and database in cloud technology, which will be described separately below.

[0091] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0092] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0093] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, model robustness testing, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as fingerprint recognition.

[0094] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0095] The solutions in this application can be implemented based on cloud technology, specifically involving cloud computing, cloud storage, and database technologies, which will be described below.

[0096] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to cloud computing business models. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing. The embodiments of this application can use cloud technology to save identification results.

[0097] Cloud storage is a new concept that extends and develops from cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to aggregate a large number of storage devices (also called storage nodes) of various types in a network through application software or application interfaces to work together and provide data storage and business access functions. In the embodiments of this application, network configuration and other information can be stored in this storage system for easy retrieval by the server.

[0098] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0099] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0100] A database, simply put, can be viewed as an electronic filing cabinet—a place to store electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, with minimal redundancy, and independent of application programs.

[0101] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile phones; or according to the query language used, such as SQL (Structured Query Language) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages. In this embodiment, the identification results can be stored in the DBMS for easy retrieval by the server.

[0102] It should be noted that the service terminal involved in the embodiments of this application can be a device that provides voice and / or data connectivity to the service terminal, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, and Personal Digital Assistants (PDAs).

[0103] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the forgery detection method provided in this application embodiment. The forgery detection method is applied to… Figure 1 The computer device 10 includes a feature encoder, a prototype set, a detection and tracing module, and a prototype update module. Figure 1(Not illustrated in the figure), wherein the prototype set includes at least one preset real prototype, at least two preset unique fake prototypes, and at least one preset common fake prototype; the feature encoder is used to acquire the target object features of the target object; the detection and tracing module is used to determine whether the target object is a fake object based on the target object features and the prototype set, and when the target object is a fake object, to determine the fake type of the target object; the prototype update module is used to perform prototype update processing on at least one preset real prototype, at least two preset unique fake prototypes, and at least one preset common fake prototype during the training phase of the fake object detection model, to obtain at least one real prototype, at least two fake prototypes, and at least one common fake prototype.

[0104] In this embodiment, the prototype update module guides the learning of the prototype set during the training phase of the fake object detection model. This module is not required to participate in the computation during the inference phase. Therefore, the prototype update module in this embodiment does not bring additional inference overhead.

[0105] Specifically, firstly, the data corresponding to the target object is detected from the data to be detected. Then, the data corresponding to the target object is input into the feature encoder. The feature encoder obtains the target object features of the target object. Then, the feature distance between the target object features and each prototype in the prototype set is calculated to obtain a first feature distance set, a second feature distance set, and a third prototype set. The first feature distance set is the feature distance between the target object features and each of the at least one real prototypes. The second feature distance set is the feature distance between the target object features and each of the at least two unique forgery prototypes. The third feature distance set is the feature distance between the target object features and each of the at least one common forgery prototypes. Then, based on the first feature distance set, the second feature set, and the third feature set, it is determined whether the target object is a forgery object. If it is a forgery object, the forgery type of the target object is further determined according to the second feature distance set.

[0106] It should be noted that the data to be detected in the embodiments of this application includes at least one of audio data, video data, and image data. The specific data type of the data to be detected is not limited here. When the data to be detected is video data or image data, the target object can be a face. In addition, the target object can also be other object types, such as a seal. The specific type of the target object is not limited in the embodiments of this application.

[0107] For ease of understanding, the following specific steps are illustrated using the data type of the data to be detected as an image, the target object as a face, and the execution subject as a fake object detection terminal.

[0108] It should be noted that the forgery object detection method provided in this application embodiment is based on a forgery object detection model, which includes a feature encoder, a prototype set, a detection tracing module, and a prototype update module; wherein, the prototype set includes at least one preset real prototype, at least two preset unique forgery prototypes, and at least one preset common forgery prototype.

[0109] Before using the forged object detection method provided in this application to detect forged objects, it is first necessary to train the forged object detection model to obtain the trained forged object detection model. The target loss value is determined based on the prototype update loss value and the prototype detection source tracing loss value.

[0110] Before training the forgery detection model, the parameters of the model must first be initialized. This involves initializing the feature encoder using ImageNet pre-trained weights, randomly initializing the prototype update module, and predefining a set of learnable prototypes P = {m}. ij}, m ij Let i represent the prototype in the prototype set, i∈{0,1,2}, and i=0 represent m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, i = 2 indicates m ij For common forged prototypes, j represents the prototype index of the prototype in the prototype set.

[0111] In this embodiment of the application, the number of real prototypes is W, the number of unique fake prototypes is K, and the number of common fake prototypes is L. Wherein, W and L are integers not less than 1, and K is an integer greater than 1. The specific values ​​of W, K, and L are obtained according to the actual number of the corresponding prototypes, and the specific values ​​are not limited here.

[0112] This application uses W = 1, K = 4, and L = 1 as an example to illustrate distance. In this case, one real prototype can use {m 01} indicates that the four unique forgery prototypes can each use {m 11}、{m 12}、{m 13} and {m 14} indicates that a common forgery prototype can be used with {m 21}express.

[0113] The training process of the forgery detection model in the embodiments of this application is described in detail below:

[0114] Please see Figure 2 , Figure 2The diagram illustrates the training process of the forgery detection model in the forgery detection method provided in this application, including steps S110-S140.

[0115] S110. Obtain the target sample, which comes from the real sample set and the fake sample set.

[0116] For example, the real sample set includes a given set of W real-type data sources, each containing the target object. The set of forged samples includes a given data source M = {M}, each containing the target object and of K forged types. j |j=1,…,K}, where W and K are not necessarily equal. Each data source represents a sample of a certain type, and each data source corresponds to multiple samples. In this embodiment, X represents the input space and Y represents the output space. For The second category label (true / false category label) y b =0, multi-category label (type label) y m =0, or a specific real type label; for x∈M, its binary label y b =1, multi-category label y m ∈{1,…,K}.

[0117] In this embodiment, each round of training requires randomly selecting at least one target sample from the real sample set and / or the fake sample set as the input to the current fake object detection model.

[0118] S120, Obtain the first feature and the second feature of the target sample.

[0119] The first feature includes genuine features, unique forgery features, and common forgery features.

[0120] In some embodiments, a feature encoder in a forgery detection model can be used to extract image features from the target sample.

[0121] Specifically, the feature encoder can be any deep learning network, such as a convolutional neural network and a visual Transformer. This application embodiment does not limit the specific type of feature encoder. This embodiment can use a basic visual Transformer as the feature encoder.

[0122] like Figure 3 As shown, step S120 includes steps S1201-S1202:

[0123] S1201. Determine the first feature and the second feature of the target sample according to the feature encoder.

[0124] In this embodiment, the target sample is image x. In some embodiments, the feature encoder specifically determines the first feature and the second feature of the target image through the following steps.

[0125] First, for a 2D image x∈R H×W×C The feature encoder unfolds it into 2D image patches Where (H,W) is the original resolution of the image, C is the number of image channels, (S,S) is the resolution of each image patch, and N = HW / S 2 It is the number of image patches, and then outputs a classification token. Used for final classification. C′=S 2 ·C, let z∈R (1+N)×C′ It is the output of the feature encoder. The feature output of a target sample mainly consists of multiple patch tokens. p ∈R N×C′ And a class token. cls ∈R 1×C′ composition.

[0126] In this embodiment, the block token is the first feature and the classification token is the second feature. A block token includes the features of an image block in the target sample, and the classification token includes the image features of the target sample.

[0127] S1202. Based on the multi-head cross-attention submodule and the prototype set, the first feature is divided into multiple real features, multiple unique forged features, and multiple common forged features.

[0128] The prototype update module includes the multi-head cross-attention submodule g(z; θ).

[0129] Specifically, each Z p (First Feature) Input the multi-head cross-attention submodule g(z; θ) as Key and Value respectively, and input the prototype set P as Query. Through cross-attention operations, Z related to each prototype can be retrieved. p This allows us to obtain the real features corresponding to each real prototype. Each unique counterfeit prototype has its own unique counterfeit features. And the common forgery characteristics of each common forgery prototype

[0130] In some embodiments, for each Z p After partitioning, feature maps can be obtained. The prototype set can then be updated based on this feature map.

[0131] S130. Determine the prototype update loss value based on the first feature, and determine the prototype detection source tracing loss value based on the second feature.

[0132] Specifically, in this embodiment, the prototype update module determines the prototype update loss value based on multiple real features, multiple unique forgery features, multiple common forgery features, and the prototype set, and the detection and tracing module determines the detection and tracing loss value based on the second feature.

[0133] In some embodiments, to enable the common forgery features learned by the prototypes in the prototype set to learn the common features of each forgery method (forgery type) and to enable the common forgery features to have a binary classification function of true and false, the prototype update module further includes a forgery method discriminator and a true / false discriminator; the prototype update loss value includes a first loss value, a second loss value, and a third loss value; please refer to Figure 4 This application determines the first loss value, the second loss value, and the third loss value through the following steps, including steps a1301-a1304:

[0134] a1301. Determine the target real feature of each real prototype from multiple real features based on the sample labels of each real feature; determine the target unique forgery feature of each unique forgery prototype from multiple unique forgery features based on the sample labels of each unique forgery feature; and determine the target common forgery feature of each common forgery prototype from multiple common forgery features based on the sample labels of each common forgery feature.

[0135] In this embodiment, although the first feature is divided into multiple real features, multiple unique fake features, and multiple common fake features through the multi-head cross-attention submodule and the prototype set, the first feature queried by each prototype may not necessarily correspond to that prototype. Therefore, it is necessary to further obtain the sample labels of each first feature, find the first features corresponding to each prototype, and determine the real features corresponding to the real prototypes as target real features, the unique fake features corresponding to the unique fake prototypes as target unique fake features, and the common fake features corresponding to the common fake prototypes as target common fake features (the corresponding target sample's sample label y). b =1).

[0136] a1302. Determine the first loss value based on the prototype set, the target's real features, the target's unique forgery features, and the target's common forgery features.

[0137] Specifically, the feature distance between each real prototype and the corresponding real feature of the target is calculated, the feature distance between each unique forged prototype and the corresponding unique forged feature is calculated, and the feature distance between each common forged prototype and the corresponding common forged feature of the target is calculated.

[0138] In some embodiments, the first loss value is the prototype center loss, which aims to minimize the distance between the prototype and its corresponding first feature. In this case, the first loss value is determined according to a first formula, which is:

[0139]

[0140] in, Let I represent the first loss value, and let I represent the indicator function used to indicate the selection of the corresponding z based on the sample label. a and m ij , z a For the target's genuine features, the target's unique forged features, or the target's common forged features, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, i = 2 indicates m ij For the common forged prototype, j represents the prototype index of the prototype in the prototype set. I(z) a ) and I(m ij The characteristic distance between ).

[0141] In this embodiment, the feature distance can be the Euclidean feature distance.

[0142] a1303. Determine the second loss value based on the common forgery features of the target and the forgery method discriminator.

[0143] In this embodiment, the common forgery features of the target need to be subjected to adversarial processing. Then, the adversarially processed common forgery features of the target are input into the forgery method discriminator. If the forgery method discriminator cannot distinguish which forgery method the common forgery features of the target come from, it means that the common forgery features of the target have learned all forgery methods (forgery types).

[0144] In some embodiments, the second loss value is determined according to a second formula, which is:

[0145]

[0146] in, The second loss value is identified, x represents the target sample, f represents the feature encoder, g represents the multi-head cross-attention submodule, and D... mLet f(x) represent the forgery method discriminator, and let I represent the second feature. [yb＝1] This indicates an indicator function that selects the target common forgery feature from the common forgery features as the sample label for forgery object, y b Indicates a binary category label, y b =1 indicates that the binary classification label is a fake object, and N represents the number of target samples. Indicates x, y b and y m Expectations on y m The label indicating the type of forgery, θ f θ represents the parameters of the feature encoder. g This represents the parameters of the multi-head cross-attention submodule. The parameters represent those of the forgery method discriminator.

[0147] a1304. Determine the third loss value based on the sample labels of the common forgery features, the common forgery features, and the true / false discriminator.

[0148] In this embodiment, specifically, the common forgery features are input into the real / fake discriminator to obtain the corresponding binary classification result. Then, the binary classification result and the sample label of the common forgery feature are used to determine the third loss value corresponding to the common forgery feature. Using the third loss value to train the model, the features learned by the common forgery prototype can have the function of real / fake binary classification.

[0149] In some embodiments, the third formula is:

[0150]

[0151] in, The third loss value, This refers to the probability output when the common forgery features are input into the true / false discriminator.

[0152] In some embodiments, the detection and source tracing loss values ​​include a fourth loss value and a fifth loss value; in this case, training the detection and source tracing module using the fourth and fifth loss values ​​allows the module to incorporate common forgery features in the true / false binary classification task. Please refer to... Figure 5 This application determines the fourth and fifth loss values ​​through the following steps, including steps b1301-b1303:

[0153] b1301. The first sample feature distance set, the second sample feature distance set, and the third sample feature distance set are determined through the detection and tracing module.

[0154] The first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype; the second sample feature distance set includes the feature distances between the second feature and each unique fake prototype in at least two preset unique fake prototypes; and the third sample feature distance set includes the feature distances between the second feature and each common fake prototype in at least one preset common fake prototype.

[0155] In some embodiments, the Euclidean distance between the second feature and each prototype in the prototype set is determined: The probability that a target sample belongs to a certain prototype is proportional to the negative of the Euclidean distance between the second feature of the target sample and the prototype, that is:

[0156]

[0157] At this point, the target sample belongs to prototype m. ij The probability can be expressed as:

[0158]

[0159] Here, γ is used to control the difficulty of the learning task.

[0160] b1302. Determine the fourth loss value of the target sample based on the first sample feature distance set and the second sample feature distance set.

[0161] In this embodiment, the first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype, and the second sample feature distance set includes the feature distances between the second feature and each unique fake prototype in at least two preset unique fake prototypes.

[0162] In order to enable the features learned by the unique forgery prototype to have the functions of true and false classification and forgery type tracing, and to enable the features learned by the common forgery prototype to have the functions of true and false classification, this embodiment determines the fourth loss function based on the real prototype and the unique forgery prototype, and determines the fifth loss function based on the real prototype and the common forgery prototype. That is, in this embodiment, the unique forgery prototype and the common forgery prototype need to calculate the loss with the real prototype respectively, so that the unique forgery prototype and the common forgery prototype have different functions.

[0163] This embodiment calculates a fourth loss value for both the real prototype and the unique forged prototype. The fourth loss value is determined according to a fourth formula, which is:

[0164]

[0165] in, For the fourth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) c |x) indicates that the target sample belongs to y. c The probability that y is the binary classification label of the target sample is a true sample. c =0, when the binary label of the target sample is a fake sample. c =y m y m A label indicating a forgery type.

[0166] b1303. Determine the fifth loss value of the target sample based on the first sample feature distance set and the third sample feature distance set.

[0167] In this embodiment, the first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype, and the third sample feature distance set includes the feature distances between the second feature and each common fake prototype in at least one preset common fake prototype.

[0168] This embodiment calculates a fifth loss value for both the real prototype and the common forged prototype. The fifth loss value is determined according to a fifth formula, which is:

[0169]

[0170] in, For the fifth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 2 represents m ij For commonalities in the prototype model, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) b|x) indicates that the target sample belongs to label y. b The probability, y b The binary label represents the target sample.

[0171] S140. Determine whether the target loss value meets the preset convergence condition; if yes, proceed to step S150; if no, proceed to step S160.

[0172] In this embodiment, the target loss value is determined based on the prototype update loss value and the prototype detection and tracing loss value, wherein the prototype update loss value includes a first loss value. Second loss value and the third loss value Prototype detection and traceability loss values ​​include a fourth loss value. And the fifth loss value

[0173] In some embodiments, the target loss value is:

[0174]

[0175] in, The target loss value is given.

[0176] Specifically, the preset convergence condition is that the target loss value is less than the preset loss function value within a preset number of training iterations.

[0177] S150. Obtain a new sample from the real sample set and the fake sample set, use the new sample as the target sample, and return to step S120.

[0178] In this embodiment, if it is determined that the target loss value does not meet the preset convergence condition, at least one new sample is randomly selected from the real sample set and the fake sample set, and the selected new sample is used as the target sample to train and update the fake object detection model.

[0179] S160. Obtain the trained fake object detection model.

[0180] In this embodiment, if the target loss value is determined to meet the preset convergence condition, the trained fake object detection model is obtained, and the trained fake object detection model is used to detect fake objects in the subsequent inference stage.

[0181] To further understand the training process of the fake object detection model in this embodiment, please refer to [link / reference]. Figure 6 , Figure 6The diagram illustrates the training process of the forgery detection model framework provided in this application. First, an image feature extraction step is performed. Then, a prototype update step and a detection and source tracing module update step are executed. Finally, gradient backpropagation is performed based on the loss values ​​obtained in the prototype update step and the detection and source tracing module update step to achieve the training and update of the forgery detection model. The specific steps are as follows:

[0182] Image feature extraction: Each target sample is input into the feature encoder, and the feature encoder outputs multiple first features (block tokens) and second features (classification tokens) of each target sample.

[0183] Prototype Update: Each primary feature is input into the multi-head cross-attention submodule as the Key and Value, respectively. Simultaneously, the prototype set P is also input into the multi-head cross-attention submodule as the Query. Through cross-attention operations, the primary features associated with each prototype are retrieved, resulting in a feature map. Then, based on the true labels of each first feature in the feature map, the first feature corresponding to each prototype is determined from the feature map. That is, the prototype center loss function value of each prototype is calculated based on the first feature corresponding to each prototype, i.e., the first loss value is calculated. First loss value Specifically, the loss value is determined according to the formula (1) above; then, the common forgery features of the target in the feature map are subjected to adversarial processing, and the adversarially processed common forgery features of the target are input into the forgery method discriminator, and the second loss value is determined based on the formula (2) above. Simultaneously, common forgery features from the feature map are input into the real / fake discriminator, and the third loss value is determined based on the above formula (3).

[0184] The detection and tracing module has been updated: a fourth loss value is calculated for both the real prototype and the unique counterfeit prototype. Among them, the fourth loss value The fifth loss value is determined according to the above formula (6); for the real prototype and the common fake prototype, the fifth loss value is calculated. The fifth loss value is determined according to the above formula (7).

[0185] Gradient backpropagation: The forgery detection model is backpropagated with gradients based on the loss values ​​obtained in the prototype update and the detection and tracing module update steps. Specifically, the target loss value is determined according to formula (8), and then the model is backpropagated with gradients based on the target loss value to complete the update of model parameters.

[0186] In summary, firstly, this embodiment only requires deploying one fake object detection model to simultaneously perform true / false binary classification and fake type tracing tasks on the target object, eliminating the need to deploy multiple models and thus reducing model training and deployment costs. Secondly, the prototype update module in the fake object detection model of this embodiment only needs to guide the learning of the prototype set during the training phase, and does not need to participate in the computation during the inference phase. Therefore, the prototype update module in the fake object detection model of this embodiment does not incur additional inference overhead. Thirdly, this embodiment trains unique fake prototypes and common fake prototypes for fake features. Through unique fake prototypes and common fake prototypes, various fake features in fake data can be learned and retained more comprehensively, enabling more comprehensive identification of fake features in the subsequent inference phase, thereby improving the performance of the model in true / false binary classification and fake type tracing.

[0187] Figures 2 to 6 The corresponding embodiments illustrate the training of the forgery detection model in the forgery detection method provided in this application. After training the forgery detection model, it is used in the inference phase to detect forgery of target objects in the detection data. The following provides a detailed description of the forgery detection method provided in this application (inference phase). Please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a flowchart illustrating the counterfeit object detection method provided in the embodiments of this application.

[0188] like Figure 7 As shown, the method includes the following steps S210-S230.

[0189] S210. Obtain the target object features of the target object in the data to be detected.

[0190] In some embodiments, after obtaining the data to be detected, it is necessary to detect the target object in the data to be detected, determine the data corresponding to the target object in the data to be detected, and then perform feature encoding on the data corresponding to the target object to obtain the target object features.

[0191] For example, a detection image containing face images is obtained, and then a preset target object detector is used to detect face images in the detection image. The detected face images are then cropped out and used as input to the feature encoder. The feature encoder then encodes the features of the input face images and uses the obtained face features as target object features.

[0192] In some other embodiments, the data to be detected only includes data of the target object. In this case, the target object features can be obtained by directly encoding the features of the data to be detected using a feature encoder.

[0193] In this embodiment, the data to be detected can be directly input by the user, obtained from another terminal, or downloaded from the cloud. This embodiment does not limit the method or path for obtaining the data to be detected.

[0194] In other embodiments, the forged object detection terminal can directly acquire the characteristics of the target object.

[0195] S220. Determine the first feature distance set, the second feature distance set, and the third feature distance set.

[0196] Wherein, the first feature distance set is the feature distance between the target object feature and each of the at least one real prototypes, the second feature distance set is the feature distance between the target object feature and each of the at least two unique fake prototypes, and the third feature distance set is the feature distance between the target object feature and each of the at least one common fake prototypes.

[0197] Specifically, the Euclidean distance between the target object feature and each prototype in the prototype set, wherein the prototype set includes at least one real prototype, at least two unique fake prototypes and at least one common fake prototype; then the Euclidean distance between the target object feature and each real prototype is classified into a first distance feature set, the Euclidean distance between the target object feature and each unique fake prototype is classified into a second distance feature set, and the Euclidean distance between the target object feature and each target object feature is classified into a third distance feature set.

[0198] S230. If the target object is determined to be a forged object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

[0199] In this embodiment, the target object is first determined based on the first feature distance set, the second feature distance set, and the third feature distance set for binary classification to determine whether the target object is a fake object. If the target object is a fake object, the fake type of the target object is further determined based on the second feature distance set. If the target object is determined to be a real object, the binary classification result of the target object being a real object is output.

[0200] In some embodiments, whether the target object is a forgery object is determined by: determining the average feature distance of the feature distances in the first feature distance set, and determining the minimum feature distance from the second feature distance set and the third feature set; and determining whether the target object is a forgery object based on the average feature distance and the minimum feature distance.

[0201] Specifically, the average feature distance in the first feature distance set is determined based on formula (9):

[0202]

[0203] Where f(x) represents the feature of the target object, m r m when i∈{0} ij N is the number of real prototypes (the number of feature distances in the first feature distance set), d(f(x),m r ) represents the average feature distance in the first feature distance set. When i∈{0}, d(f(x),m ij ) represents the feature distances in the first feature distance set.

[0204] Based on formula (10), determine the second feature distance set and the minimum feature distance in the third feature set:

[0205] d(f(x),m f =mind(f(x),m ij ), i∈{1,2}; (10)

[0206] Where d(f(x),m f f(x) is the minimum feature distance determined from the second feature distance set and the third feature set, f(x) is the target object feature, and m is the minimum feature distance. f m when i∈{1,2} ij When i∈{1,2}, d(f(x),m ij ) represents the feature distances in the second feature distance set and the third feature distance set.

[0207] In some embodiments, after determining the average feature distance and the minimum feature distance, it is necessary to further determine a first probability that the target object is a real object based on the average feature distance, and a second probability that the target object is a fake object based on the minimum feature distance; wherein, if the first probability is greater than the second probability, the target object is determined to be a real object; if the first probability is less than or equal to the second probability, the target object is determined to be a fake object.

[0208] Specifically, the first probability and the second probability are determined according to formula (11):

[0209]

[0210] Among them, y b For binary classification labels, d(f(x),m) calculated by (9) and formula (10) r) and d(f(x),m f Substituting into formula (11), we can calculate p(y). b |x), where when m takes the value m r When, the calculated p(y) b |x) is the first probability, when m takes the value m f When, the calculated p(y) b |x) represents the second probability.

[0211] In this embodiment, if the first probability is greater than the second probability, then y is determined. b =0, meaning the target object is a real object; otherwise, determine y. b =1, meaning the target object is a forged object, when y b When = 1, this embodiment needs to further predict the forgery type of the target object. Specifically, the probability of the target object corresponding to each unique forgery prototype is determined according to the feature distance in the second feature distance set, which is specifically achieved through formula (12):

[0212]

[0213] Among them, y m ∈{1,…,K}, where K is the number of unique forgery prototypes, and each unique forgery prototype represents a forgery type.

[0214] In summary, the solution provided in this application has several advantages. First, since the forgery prototypes provided include multiple unique forgery prototypes representing unique forgery features and at least one common forgery prototype representing common forgery features, the forgery prototypes provided by this solution can identify not only unique forgery features in the target object but also common forgery features in the target object. This allows for a more comprehensive identification of the forgery features of the target object. By using the more comprehensive forgery features identified, the target object can be classified into true and false categories, thus improving the classification effect of the true and false binary classification. Second, when this solution identifies the target object as a forgery object, it further determines the forgery type of the target object using the second feature distance set calculated during the previous binary classification task, without needing to calculate the feature distance again. This solution can simultaneously obtain the data required for forgery type tracing when performing binary classification on the target object, resulting in high processing efficiency for forgery object detection.

[0215] To verify the validity of this application, the inventors conducted experiments using the fake face dataset FaceForensics++. FaceForensics++ contains real face data collected from the internet and fake face data synthesized using five different forgery methods. This experiment used the ViT-Small network as the baseline model, comparing it with the ViT-Small-2way and ViT-Small-6way models and the ViT-Small-ours model of the method described in this application. For the real / fake binary classification task, the evaluation metrics selected were the real / fake classification accuracy bi_Acc, the area under the curve (AUC) (the area below the ROC curve), and TPR@FPR (recall at a fixed false positive rate). In real-world applications, the rate of fake faces is extremely low, and a large amount of data consists of real faces. Therefore, the false positive rate of identifying real samples as fake samples needs to be considered, and the cases of TPR@FPR = 0.01% and TPR@FPR = 0.1% were compared. For the forgery method tracing task, the evaluation metric selected was the classification accuracy attr_Acc of the forgery method. The experimental results are shown in Table 1:

[0216] Table 1

[0217] Model bi_Acc AUC TPR@FPR = 0.01% TPR@FPR = 0.1% attr_Acc ViT-Small-2way 0.9676 0.9911 0.7159 0.7469 - ViT-Small-6way 0.9677 0.9912 0.7687 0.8280 0.9873 ViT-Small-ours 0.972 0.9935 0.874 0.9311 0.9892

[0218] As shown in Table 1, the counterfeit object detection method provided in this application is superior to the baseline method in all indicators, which fully demonstrates the superiority of this application.

[0219] In addition, to visualize the learned features, t-SNE (t-Distributed Stochastic Neighbor Embedding) was used to visualize a subset of samples from the FaceForensics++ dataset, as shown in the results. Figure 8 As shown. In Figure 8 In the diagram, the color corresponding to label 0 represents the features corresponding to real face data, and the colors corresponding to labels 1 to 5 represent the features corresponding to fake face data of different forgery types. (a) shows the t-SNE diagram of the features before ViT_Small-6way is fed into the classifier. (b) shows the t-SNE diagram of this application. (c) shows the t-SNE diagram of the features retrieved from the real prototype and the unique forgery prototype in this application. (d) shows the t-SNE diagram of the features retrieved from the common forgery prototype of this invention. By comparing (a) and (b), it can be observed that the features extracted by the method of this invention have a higher degree of distinction between real and fake categories. By observing (c) and (d), it can be shown that the proposed prototype can learn the corresponding forgery features.

[0220] Figures 1 to 8Any technical feature mentioned in the embodiments corresponding to any one of the above also applies to the embodiments of this application. Figures 9 to 12 The corresponding implementation examples will not be repeated hereafter.

[0221] The forgery detection method in the embodiments of this application has been described above. The forgery detection device (e.g., server, user terminal) that performs the above forgery detection method is described below.

[0222] See Figure 9 ,like Figure 9 The diagram shows a counterfeit object detection device 900, which can be applied to counterfeit object authenticity detection scenarios. The counterfeit object detection device 900 in this embodiment can achieve the functions described above. Figures 1-8 The steps of the forgery detection method executed in any corresponding embodiment. The functions implemented by the forgery detection device 900 can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The forgery detection device 900 may include a transceiver module 901 and a processing module 902:

[0223] The transceiver module 901 is used to acquire the target object features of the target object in the data to be detected;

[0224] Processing module 902 is used to determine a first feature distance set, a second feature distance set, and a third feature distance set. The first feature distance set is the feature distance between the target object features and each of the at least one real prototypes. The second feature distance set is the feature distance between the target object features and each of the at least two unique forgery prototypes. The third feature distance set is the feature distance between the target object features and each of the at least one common forgery prototypes. If the target object is determined to be a forgery object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

[0225] In some embodiments, the processing module 902 determines whether the target object is a forged object based on the following methods:

[0226] Determine the average feature distance in the first feature distance set, and determine the minimum feature distance from the second feature distance set and the third feature set; determine whether the target object is a fake object based on the average feature distance and the minimum feature distance.

[0227] In some embodiments, when the processing module 902 performs the step of determining whether the target object is a forgery object based on the average feature distance and the minimum feature distance, it is specifically used for:

[0228] The first probability of the target object being a real object is determined based on the average feature distance, and the second probability of the target object being a fake object is determined based on the minimum feature distance; if the first probability is greater than the second probability, the target object is determined to be a real object; if the first probability is less than or equal to the second probability, the target object is determined to be a fake object.

[0229] In some embodiments, the forgery detection device is implemented based on a forgery detection model, which includes a feature encoder, a prototype set, a detection tracing module, and a prototype update module.

[0230] The prototype set includes at least one preset real prototype, at least two preset unique fake prototypes, and at least one preset common fake prototype.

[0231] The feature encoder is used to obtain the target object features of the target object;

[0232] The detection and tracing module is used to determine whether the target object is a forged object based on the characteristics of the target object and the prototype set, and when the target object is a forged object, to determine the forgery type of the target object;

[0233] The prototype update module is used to perform prototype update processing on at least one preset real prototype, at least two preset unique fake prototypes and at least one preset common fake prototype during the training phase of the fake object detection model, so as to obtain at least one real prototype, at least two fake prototypes and at least one common fake prototype.

[0234] In some embodiments, before the transceiver module 901 performs the step of acquiring the target object features of the target object in the data to be detected, the processing module 902 is further configured to:

[0235] The target sample is obtained through the transceiver module 901, and the target sample comes from the real sample set and the fake sample set.

[0236] Obtain the first feature and the second feature of the target sample, wherein the first feature includes real features, unique forgery features and common forgery features;

[0237] The prototype update loss value is determined based on the first feature, and the prototype detection source tracing loss value is determined based on the second feature;

[0238] If it is determined that the target loss value does not meet the preset convergence condition, a new sample is obtained from the real sample set and the fake sample set, and the new sample is used as the target sample until the target loss value meets the preset convergence condition, thereby obtaining the trained fake object detection model. The target loss value is determined based on the prototype update loss value and the prototype detection source tracing loss value.

[0239] In some embodiments, when the processing module 902 performs the step of acquiring the first feature and the second feature of the target sample, it is specifically used for:

[0240] The first feature and the second feature of the target sample are determined according to the feature encoder;

[0241] The first feature is divided into multiple real features, multiple unique fake features, and multiple common fake features based on the multi-head cross-attention submodule and the prototype set. The prototype update module includes the multi-head cross-attention submodule.

[0242] In some embodiments, when the processing module 902 performs the steps of determining the prototype update loss value based on the first feature and determining the prototype detection and tracing loss value based on the second feature, it is specifically used for:

[0243] The prototype update module determines the prototype update loss value based on multiple real features, multiple unique forgery features, multiple common forgery features, and the prototype set, and the detection and tracing module determines the detection and tracing loss value based on the second feature.

[0244] In some embodiments, the prototype update module further includes a forgery method discriminator and a genuine / fake discriminator; the prototype update loss value includes a first loss value, a second loss value, and a third loss value; when performing the step of determining the prototype update loss value based on the first feature, the processing module 902 is specifically used for:

[0245] The target real features of each real prototype are determined from multiple real features based on the sample labels of each real feature; the target unique fake features of each unique fake prototype are determined from multiple unique fake features based on the sample labels of each unique fake feature; and the target common fake features of each common fake prototype are determined from multiple common fake features based on the sample labels of each common fake feature.

[0246] The first loss value is determined based on the prototype set, the target's real features, the target's unique forgery features, and the target's common forgery features;

[0247] The second loss value is determined based on the common forgery features of the target and the forgery method discriminator.

[0248] The third loss value is determined based on the sample labels of the common forgery features, the common forgery features, and the real / fake discriminator.

[0249] In some embodiments, the first loss value is determined according to a first formula, wherein the first formula is:

[0250]

[0251] in, Let I represent the first loss value, and let I represent the indicator function used to indicate the selection of the corresponding z based on the sample label. a and m ij , z a For the target's genuine features, the target's unique forged features, or the target's common forged features, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents m ij For a unique forged prototype, i = 2 indicates m ij For the common forged prototype, j represents the prototype index of the prototype in the prototype set. I(z) a ) and I(m ij The characteristic distance between ).

[0252] In some embodiments, the second loss value is determined according to a second formula, which is:

[0253]

[0254] in, The second loss value is identified, x represents the target sample, f represents the feature encoder, g represents the multi-head cross-attention submodule, and D... m Let f(x) represent the forgery method discriminator, and f(x) represent the second feature. This indicates an indicator function that selects the target common forgery feature from the common forgery features as the sample label for forgery object, y b Indicates a binary category label, y b =1 indicates that the binary classification label is a fake object, and N represents the number of target samples. Indicates x, y b and y m Expectations on y m The label indicating the type of forgery, θ f θ represents the parameters of the feature encoder. gThis represents the parameters of the multi-head cross-attention submodule. The parameters represent those of the forgery method discriminator.

[0255] In some embodiments, the third loss value is determined according to a third formula, which is:

[0256]

[0257] in, The third loss value, This refers to the probability output when the common forgery features are input into the true / false discriminator.

[0258] In some embodiments, the detection and tracing loss value includes a fourth loss value and a fifth loss value; when the processing module 902 performs the step of determining the prototype detection and tracing loss value based on the second feature, it is specifically used for:

[0259] The detection and tracing module determines a first sample feature distance set, a second sample feature distance set, and a third sample feature distance set. The first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype. The second sample feature distance set includes the feature distances between the second feature and each unique fake prototype in at least two preset unique fake prototypes. The third sample feature distance set includes the feature distances between the second feature and each common fake prototype in at least one preset common fake prototype.

[0260] The fourth loss value of the target sample is determined based on the first sample feature distance set and the second sample feature distance set;

[0261] The fifth loss value of the target sample is determined based on the first sample feature distance set and the third sample feature distance set.

[0262] In some embodiments, the fourth loss value is determined according to a fourth formula, which is:

[0263]

[0264] in, For the fourth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 1 represents mij For a unique forged prototype, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) c |x) indicates that the target sample belongs to y. c The probability that y is the binary classification label of the target sample is a true sample. c =0, when the binary label of the target sample is a fake sample. c =y m y m A label indicating a forgery type.

[0265] In some embodiments, the fifth loss value is determined according to a fifth formula, which is:

[0266]

[0267] in, For the fifth loss value, d(z) cls ,m ij ) represents z cls With m ij The characteristic distance between them, z cls Describing the second feature, m ij This represents the prototype in the prototype set, where i = 0 indicates m. ij For the real prototype, i = 2 represents m ij For commonalities in the prototype model, j represents the prototype index of the prototype in the prototype set, x is the target sample, γ represents the coefficient controlling the difficulty of the learning task, and p(y) b |x) indicates that the target sample belongs to label y. b The probability, y b The binary label represents the target sample.

[0268] In some embodiments, the data to be detected includes at least one of audio data, video data, and image data.

[0269] In some embodiments, the data to be detected includes at least one of video data and image data, and the target object is a face.

[0270] In summary, the solution provided in this application embodiment has two advantages. First, since the forgery prototypes provided by the forgery object detection device 900 include multiple unique forgery prototypes representing unique forgery features and at least one common forgery prototype representing common forgery features, the forgery prototypes in the forgery object detection device 900 can not only identify unique forgery features in the target object, but also common forgery features in the target object. This allows for a more comprehensive identification of the forgery features of the target object. By performing binary classification of the target object based on the more comprehensive forgery features identified, the classification effect of binary classification can be improved. Second, when the forgery object detection device 900 identifies the target object as a forgery object, it can further determine the forgery type of the target object using the second feature distance set calculated during the previous binary classification task, without needing to calculate the feature distance again. The forgery object detection device 900 can simultaneously obtain the data required for forgery type tracing when performing binary classification on the target object, resulting in high processing efficiency for forgery object detection.

[0271] The forgery detection method in the embodiments of this application has been described above from the perspective of modular functional entities. The forgery detection device in the embodiments of this application will be described below from the perspective of hardware processing.

[0272] It should be noted that in the embodiments of this application (including...) Figure 9 In the embodiments shown, the physical devices corresponding to all transceiver modules can be transceivers, and the physical devices corresponding to all processing modules can be processors. When one of the devices has such Figure 9 In the structure shown, the processor, transceiver, and memory implement the same or similar functions as the transceiver module and the processing module provided in the aforementioned device embodiments corresponding to this device. Figure 10 The memory storage processor in the memory needs to call the computer program when executing the above-mentioned fake object detection method.

[0273] Figure 9 The system shown can have, for example Figure 10 The structure shown, when Figure 9 The device shown has the following characteristics: Figure 10 When the structure shown is used, Figure 10 The processor in the device can perform the same or similar functions as the processing module provided in the aforementioned device embodiments. Figure 10 The transceiver in the device can perform the same or similar functions as the transceiver module provided in the aforementioned device embodiments corresponding to this device. Figure 10 The memory in the processor stores the computer programs that need to be called when executing the above-described forgery detection method. In the embodiments of this application... Figure 9In the illustrated embodiment, the physical device corresponding to the transceiver module can be an input / output interface, and the physical device corresponding to the processing module can be a processor.

[0274] This application also provides a terminal device, such as... Figure 11 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0275] Figure 11 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 11 The mobile phone includes: a radio frequency (RF) circuit 55, a memory 520, an input unit 530, a display unit 540, a sensor 550, an audio circuit 560, a wireless fidelity (Wi-Fi) module 570, a processor 580, and a power supply 590, among other components. Those skilled in the art will understand that... Figure 11 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0276] The following is combined with Figure 11 A detailed introduction to each component of a mobile phone:

[0277] The RF circuit 55 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 580; additionally, it transmits uplink data to the base station. Typically, the RF circuit 55 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 55 can also communicate wirelessly with networks and other devices. The aforementioned wireless communications may use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0278] The memory 520 can be used to store software programs and modules. The processor 580 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 520. The memory 520 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0279] The input unit 530 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 530 may include a touch panel 531 and other input devices 532. The touch panel 531, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 531), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 531 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 580, and can also receive and execute commands sent by the processor 580. In addition, the touch panel 531 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 531, the input unit 530 may also include other input devices 532. Specifically, other input devices 532 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0280] Display unit 540 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. Display unit 540 may include display panel 541, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 541. Further, touch panel 531 may cover display panel 541. When touch panel 531 detects a touch operation on or near it, it transmits the information to processor 580 to determine the type of touch event. Subsequently, processor 580 provides corresponding visual output on display panel 541 based on the type of touch event. Although in Figure 11 In this embodiment, the touch panel 531 and the display panel 541 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 531 and the display panel 541 can be integrated to realize the input and output functions of the mobile phone.

[0281] The mobile phone may also include at least one sensor 550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 541 according to the ambient light level, and the proximity sensor can turn off the display panel 541 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0282] Audio circuit 560, speaker 561, and microphone 562 provide an audio interface between the user and the mobile phone. Audio circuit 560 converts received audio data into electrical signals and transmits them to speaker 561, where speaker 561 converts them into sound signals for output. On the other hand, microphone 562 converts collected sound signals into electrical signals, which are received by audio circuit 560, converted into audio data, and then processed by processor 580 before being transmitted via RF circuit 55 to, for example, another mobile phone, or the audio data can be output to memory 520 for further processing.

[0283] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 570, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 11 The Wi-Fi module 570 is shown, but it is understood that it is not a necessary component of the mobile phone and can be omitted as needed without changing the nature of the application.

[0284] The processor 580 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 520, and calls data stored in the memory 520 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 580 may include one or more processing units; preferably, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0285] The mobile phone also includes a power supply 590 (such as a battery) that supplies power to various components. The power supply can be logically connected to the processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0286] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0287] In this embodiment of the application, the processor 580 included in the mobile phone also has the function of controlling and executing the above-mentioned... Figure 2 and Figure 7 The flowchart shown is for a method to detect fake objects.

[0288] Figure 12 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 620 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 622 (e.g., one or more processors) and memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 642 or data 644. The memory 632 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 622 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the server 620.

[0289] Server 620 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input / output interfaces 658, and / or one or more operating systems 641, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0290] The steps performed by the server in the above embodiments can be based on this Figure 12 The structure of server 620 is shown. For example, in the above embodiment, it consists of... Figure 2 and Figure 7 The steps of the server shown can be based on this Figure 12 The server architecture is shown. For example, the processor 622 performs the following operations by calling instructions in memory 632:

[0291] Obtain the target object features from the data to be detected;

[0292] A first feature distance set, a second feature distance set, and a third feature distance set are determined. The first feature distance set is the feature distance between the target object feature and each of the at least one real prototypes. The second feature distance set is the feature distance between the target object feature and each of the at least two unique fake prototypes. The third feature distance set is the feature distance between the target object feature and each of the at least one common fake prototypes.

[0293] If the target object is determined to be a forged object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

[0294] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0295] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0296] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0297] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0298] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0299] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0300] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0301] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A method for detecting counterfeit objects, characterized in that, include: Obtain the target object features from the target object in the data to be detected, wherein the data to be detected includes at least one of video data and image data; A first feature distance set, a second feature distance set, and a third feature distance set are determined. The first feature distance set is the feature distance between the target object feature and each of the at least one real prototypes. The second feature distance set is the feature distance between the target object feature and each of the at least two unique fake prototypes. The third feature distance set is the feature distance between the target object feature and each of the at least one common fake prototypes. If the target object is determined to be a forged object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

2. The method according to claim 1, characterized in that, The following methods are used to determine whether the target object is a fake object: Determine the average feature distance in the first feature distance set, and determine the minimum feature distance from the second feature distance set and the third feature set; The target object is determined to be a fake object based on the average feature distance and the minimum feature distance.

3. The method according to claim 2, characterized in that, The step of determining whether the target object is a fake object based on the average feature distance and the minimum feature distance includes: The first probability of determining the target object as a real object is based on the average feature distance, and the second probability of determining the target object as a fake object is based on the minimum feature distance; If the first probability is greater than the second probability, then the target object is determined to be a real object; If the first probability is less than or equal to the second probability, then the target object is determined to be a fake object.

4. The method according to any one of claims 1 to 3, characterized in that, The method is based on a forged object detection model, which includes a feature encoder, a prototype set, a detection and tracing module, and a prototype update module. The prototype set includes at least one preset real prototype, at least two preset unique fake prototypes, and at least one preset common fake prototype. The feature encoder is used to obtain the target object features of the target object; The detection and tracing module is used to determine whether the target object is a forged object based on the characteristics of the target object and the prototype set, and when the target object is a forged object, to determine the forgery type of the target object; The prototype update module is used to perform prototype update processing on at least one preset real prototype, at least two preset unique fake prototypes and at least one preset common fake prototype during the training phase of the fake object detection model, so as to obtain at least one real prototype, at least two fake prototypes and at least one common fake prototype.

5. The method according to claim 4, characterized in that, Before obtaining the target object features of the target object in the data to be detected, the method further includes: Obtain target samples, which are derived from a set of real samples and a set of fake samples; Obtain the first feature and the second feature of the target sample, wherein the first feature includes real features, unique forgery features and common forgery features; The prototype update loss value is determined based on the first feature, and the prototype detection source tracing loss value is determined based on the second feature; If it is determined that the target loss value does not meet the preset convergence condition, a new sample is obtained from the real sample set and the fake sample set, and the new sample is used as the target sample until the target loss value meets the preset convergence condition, thereby obtaining the trained fake object detection model. The target loss value is determined based on the prototype update loss value and the prototype detection source tracing loss value.

6. The method according to claim 5, characterized in that, The acquisition of the first feature and the second feature of the target sample includes: The first feature and the second feature of the target sample are determined according to the feature encoder; The first feature is divided into multiple real features, multiple unique fake features, and multiple common fake features based on the multi-head cross-attention submodule and the prototype set. The prototype update module includes the multi-head cross-attention submodule.

7. The method according to claim 5, characterized in that, The steps of determining the prototype update loss value based on the first feature and determining the prototype detection and tracing loss value based on the second feature include: The prototype update module determines the prototype update loss value based on multiple real features, multiple unique forgery features, multiple common forgery features, and the prototype set, and the detection and tracing module determines the detection and tracing loss value based on the second feature.

8. The method according to claim 5, characterized in that, The prototype update module further includes a forgery method discriminator and a genuine / fake discriminator; the prototype update loss value includes a first loss value, a second loss value, and a third loss value; determining the prototype update loss value based on the first feature includes: The target real features of each real prototype are determined from multiple real features based on the sample labels of each real feature; the target unique fake features of each unique fake prototype are determined from multiple unique fake features based on the sample labels of each unique fake feature; and the target common fake features of each common fake prototype are determined from multiple common fake features based on the sample labels of each common fake feature. The first loss value is determined based on the prototype set, the target's real features, the target's unique forgery features, and the target's common forgery features; The second loss value is determined based on the common forgery features of the target and the forgery method discriminator. The third loss value is determined based on the sample labels of the common forgery features, the common forgery features, and the real / fake discriminator.

9. The method according to claim 8, characterized in that, The first loss value is determined according to a first formula, which is: ； in, This indicates an indicator function used to indicate the selection of the corresponding sample label. and , These can be the target's genuine features, the target's unique forged features, or the target's common forged features. Represents the prototype in the prototype set, i=0 indicates For the real prototype, i=1 indicates For a unique forged prototype, i=2 indicates For the common forged prototype, j represents the prototype index of the prototype in the prototype set. express The characteristic distance between them.

10. The method according to claim 8, characterized in that, The second loss value is determined according to the second formula, which is: ； in, Let f represent the target sample, f represent the feature encoder, and g represent the multi-head cross-attention submodule. This refers to the forgery method discriminator. This indicates the second feature. This indicates an indicator function that selects the target common forgery feature from the common forgery features as the sample label forgery object. Indicates a binary category label. The binary classification label indicates that the object is fake, and N represents the number of target samples. Indicates in Expectations Labels indicating counterfeit types, The parameters representing the feature encoder, This represents the parameters of the multi-head cross-attention submodule. The parameters represent those of the forgery method discriminator.

11. The method according to claim 8, characterized in that, The third loss value is determined according to a third formula, which is: ； in, The third loss value, The probability output by the true / false discriminator when the common forgery features are input into it. This indicates a binary category label.

12. The method according to claim 5, characterized in that, The detection and tracing loss value includes a fourth loss value and a fifth loss value; the step of determining the prototype detection and tracing loss value based on the second feature includes: The detection and tracing module determines a first sample feature distance set, a second sample feature distance set, and a third sample feature distance set. The first sample feature distance set includes the feature distances between the second feature and each real prototype in at least one preset real prototype. The second sample feature distance set includes the feature distances between the second feature and each unique fake prototype in at least two preset unique fake prototypes. The third sample feature distance set includes the feature distances between the second feature and each common fake prototype in at least one preset common fake prototype. The fourth loss value of the target sample is determined based on the first sample feature distance set and the second sample feature distance set; The fifth loss value of the target sample is determined based on the first sample feature distance set and the third sample feature distance set.

13. A device for detecting counterfeit objects, characterized in that, include: The transceiver module is used to acquire the target object features of the target object in the data to be detected, wherein the data to be detected includes at least one of video data and image data; The processing module is configured to determine a first feature distance set, a second feature distance set, and a third feature distance set. The first feature distance set consists of the feature distances between the target object features and each of the at least one real prototypes. The second feature distance set consists of the feature distances between the target object features and each of the at least two unique forgery prototypes. The third feature distance set consists of the feature distances between the target object features and each of the at least one common forgery prototypes. If the target object is determined to be a forgery object based on the first feature distance set, the second feature distance set, and the third feature distance set, then the forgery type of the target object is determined according to the second feature distance set.

Images