Image detection method and device, training method and device, electronic equipment, medium and program product

Abstract

The invention provides an image detection method and device, a training method and device, electronic equipment, a medium and a program product, and relates to the field of artificial intelligence, in particular to the fields of deep learning, large language models, image recognition and the like. According to the implementation scheme, first, second and third input features used for image detection are obtained, the first input feature is associated with a classification mark vector used for indicating classification of an image, and the second input feature is associated with a prompt vector used for indicating whether the image is a real image or not; the third input feature is associated with image information of the to-be-detected image; fusing the first input feature, the second input feature and the third input feature respectively, wherein the first input feature and the second input feature are contextualized respectively based on the third input feature to obtain a first output feature and a second output feature; and comparing the first output feature with the second output feature to determine an image detection result used for indicating whether the to-be-detected image is a real image or not.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to fields such as deep learning, large language models, image recognition, deepfake detection, etc. Specifically, it relates to an image detection method, an image detection training method, a device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0003] With the progress of artificial intelligence technology, especially generative artificial intelligence technology, the ability to generate images has been continuously enhanced, and the occasions where the authenticity of images needs to be detected are increasing day by day. There is an urgent need for an image detection method that can cope with technological development.

[0004] The methods described in this section are not necessarily methods that have been previously envisioned or adopted. Unless otherwise specified, no method described in this section should be considered prior art solely because it is included in this section. Similarly, unless otherwise specified, the problems mentioned in this section should not be considered to have been recognized in any prior art. Summary of the Invention

[0005] The present disclosure provides an image detection method, an image detection training method, a device, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, there is provided an image detection method, including: obtaining a first input feature, a second input feature, and a third input feature for image detection, wherein the first input feature is associated with a classification token vector for indicating classification of an image, the second input feature is associated with a hint vector for indicating whether the image is a real image, and the third input feature is associated with the image information of the image to be detected; fusing the first input feature and the third input feature to obtain a first output feature, and fusing the second input feature and the third input feature to obtain a second output feature, wherein the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature; and comparing the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

[0007] According to another aspect of the present disclosure, there is provided an image detection training method, including: obtaining a sample image for training and a hint vector to be trained, wherein the preset category of the sample image is a real image or a non-real image, and the hint vector to be trained is used to indicate whether the image is a real image; performing image detection on the sample image to obtain a sample image detection result; and training based on the sample image detection result and the preset category of the sample image through a preset loss function to optimize the hint vector to be trained.

[0008] According to another aspect of the present disclosure, there is provided an image detection apparatus, including: a feature acquisition module configured to obtain a first input feature, a second input feature, and a third input feature for image detection, wherein the first input feature is associated with a classification token vector for indicating classification of an image, the second input feature is associated with a hint vector for indicating whether the image is a real image, and the third input feature is associated with the image information of the image to be detected; a feature processing module configured to fuse the first input feature and the third input feature to obtain a first output feature, and fuse the second input feature and the third input feature to obtain a second output feature, wherein the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature; and a feature comparison module configured to compare the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

[0009] According to another aspect of the present disclosure, there is provided an image detection training device, including: a data acquisition module configured to acquire sample images for training and a prompt vector to be trained, wherein the preset category of the sample images is a real image or a non-real image, and the prompt vector to be trained is used to indicate whether the image is a real image; an image detection module configured to perform image detection on the sample images to obtain sample image detection results; and a training execution module configured to perform training based on the sample image detection results and the preset category of the sample images through a preset loss function to optimize the prompt vector to be trained.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above in the present disclosure.

[0011] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method as described above in the present disclosure.

[0012] According to another aspect of the present disclosure, there is provided a computer program product including a computer program, which implements the method as described above in the present disclosure when executed by a processor.

[0013] According to one or more embodiments of the present disclosure, the accuracy of image detection can be improved.

[0014] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings exemplarily illustrate embodiments and constitute a part of the specification, and are used together with the written description of the specification to explain the exemplary implementation manners of the embodiments. The illustrated embodiments are only for illustrative purposes and do not limit the scope of the claims. In all the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein can be implemented according to an embodiment of the present disclosure is shown;

[0017] Figure 2 A flowchart of an image detection method according to an embodiment of the present disclosure is shown;

[0018] Figure 3Shows a schematic diagram of splicing three input features into an input sequence feature according to an embodiment of the present disclosure;

[0019] Figure 4 Shows a schematic diagram of providing an input feature sequence to a Transformer to obtain an output feature sequence according to an embodiment of the present disclosure;

[0020] Figure 5 Shows a schematic diagram of the process of contextualizing a second input feature according to an embodiment of the present disclosure;

[0021] Figure 6 Shows a schematic diagram of determining a first similarity score and a second similarity score according to an embodiment of the present disclosure;

[0022] Figure 7 Shows a schematic diagram of a method for detecting whether an image is a real image according to an embodiment of the present disclosure;

[0023] Figure 8 Shows a flowchart of an image detection training method according to an embodiment of the present disclosure;

[0024] Figure 9 Shows a schematic diagram of determining a consistency loss according to an embodiment of the present disclosure;

[0025] Figure 10 Shows a structural block diagram of an image detection device according to an embodiment of the present disclosure;

[0026] Figure 11 Shows a structural block diagram of an image detection device according to another embodiment of the present disclosure;

[0027] Figure 12 Shows a structural block diagram of an image detection training device according to an embodiment of the present disclosure;

[0028] Figure 13 Shows a structural block diagram of an image detection training device according to another embodiment of the present disclosure;

[0029] Figure 14 Shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure. Detailed implementation manners

[0030] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] In the present disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, and in certain cases, based on the description of the context, they may also refer to different instances.

[0032] In the description of various examples in the present disclosure, the terms used are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in the present disclosure covers any one of the listed items and all possible combinations.

[0033] In the related art, traditional detection techniques highly rely on specific generative model features in the training data, resulting in poor performance when dealing with generative models outside the training data and being difficult to cope with the rapidly iterative artificial intelligence generation technology. In addition, traditional detection techniques usually also require structural modifications to the pre-trained model. The additional computational paths caused by these structural modifications increase the inference computational complexity. At the same time, the modification of the structure also limits the flexibility of the model in multi-task scenarios.

[0034] Therefore, embodiments of the present disclosure provide a more effective image detection method and an image detection training method.

[0035] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 FIG. shows a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein can be implemented according to an embodiment of the present disclosure. Referring Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 that couple the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more applications.

[0037] In an embodiment of the present disclosure, the server 120 can run one or more services or software applications that enable the execution of the methods described in the embodiments of the present disclosure.

[0038] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0039] In Figure 1 the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that may be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may be different from system 100. Thus, Figure 1 is an example of a system for implementing the various methods described herein and is not intended to be limiting.

[0040] Users may use client devices 101, 102, 103, 104, 105, and / or 106 to provide images to be detected, etc. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via this interface. Although Figure 1 only six client devices are depicted, those skilled in the art will be able to understand that the present disclosure may support any number of client devices.

[0041] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computing devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices, etc. These computing devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT WindowsMobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, Internet-enabled gaming devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0042] Network 110 can be any type of network known to those skilled in the art, which can support data communication using any one of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, token ring, wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (such as Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0043] Server 120 may include one or more general-purpose computers, dedicated server computers (such as PC (personal computer) servers, UNIX servers, midrange servers), blade servers, mainframes, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (such as one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the server). In various embodiments, server 120 may run one or more services or software applications that provide the functions described below.

[0044] The computing unit in server 120 can run one or more operating systems including any of the above-mentioned operating systems and any commercially available server operating systems. Server 120 can also run any one of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0045] In some embodiments, server 120 can include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and / or 106. Server 120 can also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and / or 106.

[0046] In some embodiments, server 120 can be a server of a distributed system, or a server combined with a blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability existing in traditional physical hosts and virtual private server (VPS) services.

[0047] System 100 can also include one or more databases 130. In certain embodiments, these databases can be used to store data and other information. For example, one or more of databases 130 can be used to store information such as audio files and video files. Databases 130 can reside in various locations. For example, the databases used by server 120 can be local to server 120, or can be remote from server 120 and can communicate with server 120 via a network-based or dedicated connection. Databases 130 can be of different types. In certain embodiments, the databases used by server 120 can be relational databases, for example. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0048] In certain embodiments, one or more of databases 130 can also be used by applications to store application data. The databases used by applications can be different types of databases, such as key-value repositories, object repositories, or conventional repositories supported by a file system.

[0049] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and devices described in this disclosure.

[0050] Aspects of an image detection method and an image detection training method according to embodiments of the present disclosure will be described in detail below.

[0051] Figure 2 A flowchart of an image detection method 200 according to an embodiment of the present disclosure is shown.

[0052] As Figure 2 shown, method 200 includes step S201, step S202, and step S203.

[0053] In step S201, a first input feature, a second input feature, and a third input feature for performing image detection are obtained, where the first input feature is associated with a classification label vector for indicating the classification of an image, the second input feature is associated with a hint vector for indicating whether the image is a real image, and the third input feature is associated with the image information of the image to be detected.

[0054] In an example, with the development of artificial intelligence generation technology, more and more images generated by artificial intelligence have emerged on social media platforms. In some application scenarios, it may be necessary to detect these images. In this method 200, detecting whether an image is real may refer to detecting whether the image is an image generated based on artificial intelligence.

[0055] In an example, the classification label vector is usually used in an image classification task and may have a vector form for indicating image category information. In this method 200, the first input feature being associated with the classification label vector may mean that the first input feature is directly obtained from the classification label vector or obtained after processing the classification label vector. Since this method 200 is used to detect whether an image is real, the classification label vector here can be regarded as related to a binary classification task. The classification label vector can usually be represented in the form of one-hot encoding.

[0056] In an example, a real image may refer to an image directly from the real world, such as an image obtained by a camera device without being modified or synthesized by artificial intelligence generation technology. A non-real image may refer to an image generated by artificial intelligence technology, etc. During the image detection process, the hint vector can be used to represent prior knowledge related to hint information, and it can express a hint on whether the image is a real image in the form of a vector. The second input feature being associated with the hint vector may mean that the second input feature is directly obtained from the hint vector or obtained after processing the hint vector, and can be represented by E ∈ R L×D(where L represents the number of features and D represents the feature dimension).

[0057] In the example, the third input feature can be used to represent the image information of the image to be detected itself. The obtaining process may include performing standardized preprocessing on the image to be detected, for example, dividing the image into N (N is a natural number) non-overlapping image patches, and then converting them into image patch embedding vector representations through linear projection, which can be represented by Z ∈ R N×D (where D represents the dimension of the image features). Therefore, the third input feature can retain basic visual information such as the texture, color, and shape of the image.

[0058] In step S202, the first input feature and the third input feature are fused to obtain a first output feature, and the second input feature and the third input feature are fused to obtain a second output feature, where the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature.

[0059] In the example, the input sequence may include a first input feature associated with a classification token vector, a second input feature associated with an image authenticity prompt vector, and a third input feature associated with image information. The first input feature can be fused with the third input feature during the calculation process. Therefore, the fused result, that is, the first output feature, can contain image information, which is also known as contextualization. In machine learning and deep learning, contextualization is a known process of putting data into a larger environment or context in which it is located for understanding and processing. The first output feature obtained after contextualization can be represented by h cls ∈R d (where d represents the feature dimension). Similarly, the second input feature can also be fused with the third input feature during the calculation process. Therefore, the fused result, that is, the second output feature, can not only contain its own original information associated with image authenticity but also contain information related to the image. Among them, the second output feature obtained after contextualization can be represented by (where L represents the number of features).

[0060] In step S203, the first output feature is compared with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

[0061] In an example, the first output feature may include features representing the global of the entire image to be detected, while the second input feature may include features used to express real images and non-real images. Therefore, by comparing the first output feature and the second output feature, the similarity between the features of the image to be detected and the features of real images and non-real images can be quantified, thereby realizing the judgment of image authenticity.

[0062] According to an embodiment of the present disclosure, an image detection method based on real and non-real prompts is proposed. This method introduces a prompt vector for indicating whether an image is a real image as the second input feature, and contextually processes the second input feature based on a third input feature with image information, so that the resulting second output feature incorporates the image information of the image to be detected itself. Finally, a comparative analysis is performed on the second output feature and the first output feature representing the global information of the entire image, thereby determining whether the image to be detected is closer to a real image or a non-real image. Different from traditional single-prompt methods, this method introduces structured prompt tokens for real and non-real, and through contrastive learning, a direct contrast of the feature spaces of real and non-real images is established, which can effectively capture the essential feature differences between these two types of images, improve the accuracy of image detection, and be able to cope with rapidly evolving artificial intelligence generation technologies.

[0063] The image to be detected in this embodiment may come from a public dataset, or the acquisition of the image to be detected has been authorized by the user.

[0064] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0065] In some embodiments, such as in combination with Figure 2 The steps of obtaining the first input feature, the second input feature, and the third input feature for image detection described may include: concatenating the first input feature, the second input feature, and the third input feature into an input feature sequence.

[0066] In an example, before performing image detection, the first input feature associated with the classification token vector, the second input feature associated with the image authenticity prompt vector, and the third input feature associated with the image information may be concatenated into a complete feature sequence, that is, the input feature sequence. For example, it may be concatenated in a predetermined order of the second input feature, the first input feature, and the third input feature.

[0067] This feature splicing method is adopted because information fusion in multiple dimensions may be involved in image detection tasks. If various features are input independently, it may cause the model to fail to effectively capture the connections between them. By means of sequential splicing, it is convenient for full interaction between different types of input features.

[0068] Figure 3 FIG. shows a schematic diagram of splicing three input features into an input sequence feature according to an embodiment of the present disclosure.

[0069] As Figure 3 shown, the first input feature 3011 associated with the classification token vector, the second input feature 3012 associated with the image authenticity prompt vector, and the third input feature 3013 associated with the image information can be spliced, and the splicing result is the input feature sequence 301. For example, they can be arranged and spliced in the order of the second input feature 3012, the first input feature 3011, and the third input feature 3013.

[0070] In some embodiments, as in Figure 2 the step S202 described, fusing the first input feature and the third input feature to obtain a first output feature, and fusing the second input feature and the third input feature to obtain a second output feature may include: providing the input feature sequence to a deep learning model for processing sequence data based on the self-attention mechanism to obtain an output feature sequence, and the output feature sequence includes the first output feature and the second output feature.

[0071] In the example, the above model can be, for example, a Transformer model, which can establish global connections between different parts of the sequence based on the self-attention mechanism, enabling each feature to interact with other features, thereby enhancing the expression ability of the features. In the embodiments of the present disclosure, the input feature sequence spliced from the first input feature, the second input feature, and the third input feature can be provided to the Transformer model, and through the self-attention mechanism, each input feature can pay attention to other features in the sequence. For example, for the first input feature, after self-attention calculation, it can combine the information about the image in the third input feature; after self-attention calculation for the second input feature, it not only contains the original information of the prompt vector but also combines the information about the image in the third input feature.

[0072] Therefore, by utilizing the enhanced feature expression ability of the deep learning model for processing sequence data based on the self-attention mechanism, the information of each feature can be fully considered, thereby improving the ability to judge the authenticity of the image.

[0073] Figure 4A schematic diagram showing an input feature sequence being provided to a Transformer to obtain an output feature sequence according to an embodiment of the present disclosure.

[0074] As Figure 4 shown, the input feature sequence 401 (for example, it can be the input feature sequence 301 as shown in Figure 3 ) can be provided as an input to the Transformer model, and data processing is performed based on the self-attention mechanism to obtain the output feature sequence 402. In the output feature sequence 402, the output result for the classification token vector can correspond to the first output feature 4021, and the output result for the prompt vector can correspond to the second output feature 4022.

[0075] In some embodiments, the first input feature may include a classification token vector for the above model.

[0076] In the example, still taking the Transformer model as an example, it usually carries a classification token vector when processing classification tasks, which can generally be represented by CLS (Classification). In the scenario of using the Transformer model to provide the self-attention mechanism, the required image information can be directly fused with the help of this CLS.

[0077] Therefore, the image detection method of the present disclosure can be implemented without changing the architecture of the above model, which helps to improve the universality of the method.

[0078] In some embodiments, the second input feature may include a first input prompt feature and a second input prompt feature. The first input prompt feature is used to characterize the image information of a real image, and the second input prompt feature is used to characterize the image information of a non-real image.

[0079] In the example, the second input feature can provide reference information about the authenticity of the image, enabling the model to make a more accurate judgment when determining whether the image is real by combining these prompt features. It can include a first input prompt feature and a second input prompt feature. The first input prompt feature can represent the typical features of a real image, which is used to summarize the features of a real image in different dimensions, such as the texture distribution under natural light, the noise pattern of real shooting, etc., and can be represented by E real ∈R L×D (L represents the number of features, and D represents the feature dimension). The second input prompt feature can represent the typical features of a non-real image, mainly used to describe the unique features in non-real images such as AI-generated images, such as AI-generated texture artifacts, abnormal frequency distribution, unnatural pixel arrangement, etc., and can be represented by E fake ∈R L×Drepresentation. These features play a role similar to templates in the image detection task, enabling the comparison with real and non-real feature templates during authenticity judgment, rather than relying solely on the individual feature information of the image to be detected.

[0080] Therefore, by introducing these two types of hint features for real and non-real images, a clear comparison reference can be provided for subsequent calculations, thereby more accurately judging the authenticity of the image.

[0081] In some embodiments, the second output feature includes a first output hint feature corresponding to the first input hint feature and a second output hint feature corresponding to the second input hint feature, where the first input hint feature is contextualized based on the third input feature to obtain the first output hint feature, and the second input hint feature is contextualized based on the third input feature to obtain the second output hint feature.

[0082] In the example, since the second input feature can include a first input hint feature associated with real image features and a second input hint feature associated with non-real image features, after being processed by a deep learning model such as a Transformer model, the second output feature output can include a first output hint feature associated with real image features and a second output hint feature associated with non-real image features. The first output hint feature can be obtained by providing the first input hint feature to the Transformer model for self-attention mechanism processing, which can be denoted by h real representation. During this processing, the first input hint feature can interact with the third input feature, so the resulting first output hint feature not only contains its own original features associated with real image features but also features related to the image to be detected. Similarly, the second output hint feature can be obtained by providing the second input hint feature to the Transformer model for self-attention mechanism processing, which can be denoted by h fake representation. The resulting second output hint feature not only contains its own original features associated with non-real image features but also features related to the image to be detected.

[0083] In the example, by processing the first and second hint features in this context-based manner, it can be ensured that the hint features are not limited to preset static features, but can be dynamically adjusted according to different image data to be detected. For example, the first input hint feature may represent the image features of a real image, but different real images may have different styles or shooting conditions. Therefore, relying solely on fixed hint features may lead to misjudgments. By contextualizing based on the third input feature associated with the image information, the first output hint feature can be made to incorporate the information of the current image to be detected.

[0084] Therefore, in this way, it can more flexibly adapt to different image data to be detected, improve the generalization ability of the data, and enable better stability in complex image detection tasks.

[0085] Figure 5 A schematic diagram showing the process of contextualizing the second input feature according to an embodiment of the present disclosure is shown.

[0086] As Figure 5 shown, the second input feature 501 may include a first input hint feature 5011 associated with real image features and a second input hint feature 5012 associated with non-real image features. After providing the second input feature 501 to the Transformer model and contextualizing it based on the third input feature 502 associated with the image information, a second output feature 503 can be obtained. The second output feature 503 may include a first output hint feature 5031 associated with real image features and a second output hint feature 5032 associated with non-real image features.

[0087] In some embodiments, the step of comparing the first output feature and the second output feature to determine an image detection result for indicating whether the image to be detected is a real image may include: determining a first similarity score between the first output feature and the first output hint feature, and a second similarity score between the first output feature and the second output hint feature; determining the image detection result based on the first similarity score and the second similarity score.

[0088] In an example, the first output feature may include visual features representing the global of the image to be detected, while the first output hint feature may represent the image features of a real image, and the second output hint feature may represent the image features of a non-real image. Therefore, it is possible to determine whether the image to be detected is a real image by calculating the similarity between the first output hint feature and these two hint features respectively. In an embodiment of the present disclosure, the cosine similarity between the first output feature and the first output hint feature may be calculated to obtain a first similarity score. Similarly, the cosine similarity between the first output feature and the second input hint feature may be calculated to obtain a second similarity score. If the first similarity score is greater than the second similarity score, it indicates that the image to be detected is more likely to belong to the real image category; while if the second similarity score is greater than the first similarity score, it indicates that the image is more likely to be a non-real image, thereby determining whether the image is a real image.

[0089] Therefore, by calculating the similarity scores between the image features of the image to be detected and the real and non-real hint features respectively for classification decision-making, the degree of feature approximation between the image to be detected and the real and non-real images can be comprehensively considered, thereby more accurately identifying the category of the image.

[0090] In some embodiments, the first output hint feature includes a plurality of first sub-output hint features, and the second output hint feature includes a plurality of second sub-output hint features. The steps of determining the first similarity score between the first output feature and the first output hint feature, and the second similarity score between the first output feature and the second output hint feature may include: determining the first sub-similarity scores between the first output feature and each of the plurality of first sub-output hint features, and the second sub-similarity scores between the first output feature and each of the plurality of second sub-output hint features; determining the first weights of each of the plurality of first sub-output hint features, and the second weights of each of the plurality of second sub-output hint features; determining the first similarity score based on the weighted sum of the first sub-similarity scores of each of the plurality of first sub-output hint features and the first weights; and determining the second similarity score based on the weighted sum of the second sub-similarity scores of each of the plurality of second sub-output hint features and the second weights.

[0091] In an example, the first output hint feature may represent the image features of a real image, and the image features may be further subdivided. Each subdivided part may be called a first sub-output hint feature and may be represented by (L is the number of first sub-output hint features). The first sub-output hint features can respectively represent the feature details of the real image in different local regions or different dimensions, such as the texture features, color distribution features, light and shadow detail features, etc. of the real image. Similarly, the second output hint feature can represent the image features of the non-real image, and the non-real image features can be further subdivided, and each subdivided part can be called a second sub-output hint feature, which can be represented by (L is the number of second sub-output hint features). The second sub-output hint features can respectively represent the feature details of the non-real image in different local regions or different dimensions, such as unnatural pixel distribution, singular color transition features, etc.

[0092] In the example, the first output feature h cls representing the global features of the image to be detected can be calculated respectively with the cosine similarity of each first sub-output hint feature, that is The result obtained by this expression is each first sub-similarity score; similarly, the first output feature h Cls representing the global features of the image to be detected can be calculated respectively with the cosine similarity of each second sub-output hint feature, that is The result obtained by this expression is each second sub-similarity score.

[0093] In the example, after calculating each first sub-similarity score and second sub-similarity score, it is also necessary to determine their corresponding weights. In the embodiments of the present disclosure, the weights of each hint can be calculated based on the amplitudes of the first output hint feature and the second output hint feature, that is, the first weight of each first sub-output hint feature can be calculated through the expression and the second weight of each second sub-output hint feature can be calculated through the expression calculated.

[0094] In the example, after calculating the first sub-similarity score and the corresponding first weight of each first sub-output hint feature, they can be aggregated into a first similarity score by means of weighted sum, that is S real is the first similarity score. Similarly, the second similarity score S can be calculated through the expression fake .

[0095] Therefore, by introducing multiple real and non-real sub-output hint features, calculating the similarity between the features of the image to be detected and these sub-input hint features respectively, and aggregating them into a final similarity score by means of weighted sum, the features of the image to be detected can be judged more carefully in different local regions or different dimensions. This way avoids the problem of over-reliance on a single feature in traditional methods and improves the accuracy of image detection.

[0096] Figure 6 FIG. shows a schematic diagram for determining the first similarity score and the second similarity score according to an embodiment of the present disclosure.

[0097] As Figure 6 shown, the first output hint feature 602 may include multiple first sub-output hint features, such as the first sub-output hint features 6021, 6022, and 6023. The second output hint feature 603 may include multiple second sub-output hint features, such as the second sub-output hint features 6031, 6032, and 6033. The first sub-output hint features 6021, 6022, and 6023 may be respectively calculated for similarity with the first output feature to obtain corresponding first sub-similarity scores 6041, 6042, and 6043. Based on the feature magnitude, the weights of the first sub-similarity scores 6041, 6042, and 6043 can be determined. and The first sub-similarity scores 6041, 6042, and 6043 are respectively multiplied by the corresponding weights and to obtain their respective weighted scores, and finally all the weighted scores are added together to obtain the first similarity score 605. Similarly, the second sub-output hint features 6031, 6032, and 6033 are respectively calculated for similarity with the first output feature to obtain corresponding second sub-similarity scores 6051, 6052, and 6053. Based on the feature magnitude, the weights of the second sub-similarity scores 6051, 6052, and 6053 can be determined. and The second sub-similarity scores 6051, 6052, and 6053 are respectively multiplied by the corresponding weights and to obtain their respective weighted scores, and finally all the weighted scores are added together to obtain the second similarity score 606.

[0098] It can be understood that Figure 6 only the case where the hint feature includes three sub-features is shown by way of example, but the embodiments of the present disclosure are not limited thereto.

[0099] In some embodiments, the step of determining the image detection result based on the first similarity score and the second similarity score may include: determining the one with the higher score among the first similarity score and the second similarity score; in response to determining that the first similarity score is higher, determining the image to be detected as a real image; and in response to determining that the second similarity score is higher, determining the image to be detected as a non-real image.

[0100] In an example, the first similarity score quantifies the degree of approximation between the features of the image to be detected and the features of a real image, and the second similarity score quantifies the degree of approximation between the features of the image to be detected and the features of a non-real image. After calculating these two similarity scores, the one with the higher score can be determined by comparison. If the first similarity score is higher than the second similarity score, it indicates that the features of the image to be detected are closer to the features of a real image, and thus the image can be determined as a real image; conversely, if the second similarity score is higher, it can be considered that the image more conforms to the features of a non-real image, and thus the image is determined as a non-real image.

[0101] Therefore, by adopting the method of comparison and determination, the difference between the features of the image to be detected and the features of real and non-real images can be more objectively reflected.

[0102] In some embodiments, the first similarity score and the second similarity score are normalized to be converted into probability values.

[0103] In an example, after calculating the first similarity score and the second similarity score, for the convenience of unified comparison and decision-making, normalization processing can be adopted to convert them into probability values. Normalization processing can map each score to between 0 and 1, such that the sum of the normalized probability values is 1. For example, the temperature-scaled softmax function can be used to convert the scores into probabilities, that is, the expression [p real , p fake = softmax([S real , S fake / τ), where p real is the probability that the image to be detected is a real image, p fake is the probability that the image to be detected is a non-real image, and τ is the temperature parameter, representing the sharpness of controlling the probability distribution.

[0104] Therefore, through normalization processing, the first similarity score and the second similarity score can be respectively converted into probability values, providing an intuitive basis for further decision-making.

[0105] In some embodiments, the second input feature further includes a third input hint feature, and the third input hint feature is used to characterize the general image information between a real image and a non-real image.

[0106] In the example, the third input hint feature may refer to the common visual features possessed by both real images and non-real images, such as the basic color pattern of the image, the common texture structure, etc., which can be represented by E vis ∈R L×D (where L represents the number of features and D represents the feature dimension). The purpose of adopting the third input hint feature is that during the process of processing the second input feature using the self-attention mechanism, the third input hint feature can be used as a reference, so that the second output feature does not overly focus on the general image information between real images and non-real images, but pays more attention to the information in the image used to characterize real images and non-real images, thereby improving the accuracy of image detection.

[0107] In the example, when the second input feature includes the third input hint feature E vis , the input feature sequence can be concatenated in the order of the sequence [E fake , E real , E vis , CLS, Z]. After processing the input feature sequence through the self-attention mechanism, the output feature corresponding to the third input hint feature E vis can also be obtained, and this output feature can also have multiple sub-features, for example, represented as (L is the number of sub-features), but this output feature does not participate in the subsequent similarity calculation, but is used to guide the detection process to pay more attention to the information in the image used to characterize real images and non-real images.

[0108] Therefore, by introducing the third input hint feature, the authenticity hint can be strengthened, which helps to more flexibly adapt to the detection of different images and improve the effect and efficiency of image detection.

[0109] In some embodiments, the non-real image includes a generative image generated based on artificial intelligence.

[0110] In the example, the non-real image may refer to an image generated by artificial intelligence rather than captured by a real imaging device, that is, an image generated using technologies such as generative adversarial networks and diffusion models. Such images are generated by simulating the visual features of real images, but their content is generated by algorithms and does not have a real scene source. Therefore, such images have their unique features, such as artifacts, texture abnormalities, etc.

[0111] Therefore, it is possible to determine whether the image to be detected is a real image based on the unique features of the image generated by artificial intelligence.

[0112] Figure 7 shows a schematic diagram of a method for detecting whether an image is a real image according to an embodiment of the present disclosure.

[0113] As Figure 7As shown, the generated prompt vector 701, the real prompt vector 702, the visual prompt vector 703, and the image 705 as prompt information can be input into a Vision Transformer (ViT) model 710. The CLS classification token vector 704 can be inherent in the ViT model 710. After processing based on the self-attention mechanism in the model 710, the contextualized generated prompt representation 711, the real prompt representation 712, and the CLS classification token vector representation 713 can be obtained. Finally, the similarity between the generated prompt representation 711 and the CLS classification token vector representation 713, and the similarity between the real prompt representation 712 and the CLS classification token vector representation 713 are calculated respectively. Through their respective similarity scores, it can be determined whether the image 705 is a real image or a generated image.

[0114] According to an embodiment of the present disclosure, an image detection training method is also provided.

[0115] Figure 8 The flowchart of the image detection training method according to an embodiment of the present disclosure is shown.

[0116] As Figure 8 shown, the method 800 includes step S801, step S802, and step S803.

[0117] In step S801, sample images for training and the prompt vectors to be trained are obtained, where the preset category of the sample images is a real image or a non-real image, and the prompt vectors to be trained are used to indicate whether the image is a real image.

[0118] In the example, the sample images can be image data for training, and their preset categories are real images or non-real images. Real images can be images captured by real-world camera devices, such as those obtained through news material libraries, professional photographers' portfolios, etc. Non-real images, on the other hand, can be images generated by artificial intelligence, such as those generated by common image generation models, including different styles and types. After obtaining the sample images, these images can be cropped and other operations can be performed to unify the image sizes, and then they can be labeled to clarify whether each sample image is a real image or a non-real image.

[0119] In the example, when performing image detection training, the prompt vectors to be trained also need to be obtained. The prompt vector can be regarded as a kind of prior information for prompting whether the image is a real image. Therefore, the purpose of this training method 800 is to optimize such prompt vectors.

[0120] In step S802, image detection is performed on the sample images to obtain sample image detection results.

[0121] In the example, the prompt vector to be trained can be used as the second input feature, and the sample image can be used as the third input feature. According to the above method for image detection (for example, combined with Figure 2 the image detection method 200 described above), the detection result of whether each sample image is a real image can be obtained.

[0122] In step S803, based on the sample image detection result and the preset category of the sample image, training is performed through a preset loss function to optimize the prompt vector to be trained.

[0123] In the example, after obtaining the sample image detection result, the difference between the detection result of each image and its preset label can be quantified through the loss function. This loss value reflects the deviation between the sample image detection result and the preset category, that is, the real result. If the sample image detection result is inconsistent with the preset category, the value of the loss function will be relatively large. Through the backpropagation algorithm, the gradient of the parameters of the prompt vector to be trained with respect to the loss function can be calculated. According to the gradient information, an optimizer can be used to adjust the prompt vector to be trained, so that the value of the loss function gradually decreases. Further repeating this process, the prompt vector to be trained is continuously optimized.

[0124] Therefore, in image detection training, by training with a preset loss function to optimize the prompt vector to be trained, the prompt vector can more accurately express the feature information of real images and non-real images, thereby improving the accuracy of image detection.

[0125] In some embodiments, in the initialization stage of training, the prompt vector to be trained can be obtained based on random initialization.

[0126] In the example, in the initialization stage of training, an initial value can be assigned to the prompt vector to be trained through a random algorithm (such as Gaussian random distribution) to start training.

[0127] Therefore, adopting the random initialization method can help with convergence and improve generalization.

[0128] In some embodiments, the preset loss function can include a classification loss based on cross-entropy.

[0129] In the example, for a classification task, the cross-entropy loss can measure the difference between the predicted probability distribution and the true label probability distribution to reflect the degree of classification error. The classification loss of cross-entropy can be represented by L cls denoted.

[0130] Therefore, adopting the cross-entropy loss function can effectively handle the category classification problem and is also applicable to the binary classification scenario of the present disclosure.

[0131] In some embodiments, a sample image may be segmented into multiple sample image patches. The method 800 may further include: determining a first subset of sample image patches and a second subset of sample image patches from the multiple sample image patches, wherein the first subset of sample image patches constitutes a first local sample image, and the second subset of sample image patches constitutes a second local sample image; performing image detection on the first local sample image and the second local sample image to obtain a first local sample image detection result and a second local sample image detection result respectively; and determining a consistency loss between the first local sample image detection result and the second local sample image detection result.

[0132] In an example, the first local sample image and the second local sample image may be subjected to image detection according to the image detection method 200 described in conjunction with Figure 2 the above.

[0133] In an example, a sample image Z may be segmented into multiple sample image patches, that is, the entire image is divided into several small blocks of a fixed size, and each image patch retains the local visual information of the image. Further, a part of the image patches may be extracted from these image patches according to a predetermined rule as the first subset of sample image patches and the second subset of sample image patches, which respectively constitute the first local sample image and the second local sample image. Then, the first local sample image and the second local sample image contain the local visual information of the image. For example, image sampling may be performed by random sampling, and a random sampling rate p is set, p ∼ U(p min , 1) (0 < p min < 1), and the index number of the randomly sampled image patches is S, which is a set composed of numbers. N is the total number of image patches. Then, the first local sample image obtained by sampling may be represented by , and the second local sample image may be represented by . Then, image authenticity detection may be performed on the first local sample image and the second local sample image respectively. For example, the first local sample image detection result p1 may be obtained through the expression , and the second local sample image detection result p2 may be obtained through the expression . represents the complete process of feature extraction and prediction performed according to the image detection method 200 described above. After calculating the detection results, the consistency loss between the first local sample image detection result and the second local sample image detection result may be calculated according to the following formula 1:

[0134] L cons = 1 / 2(D KL (p1||p2) + D KL (p2||p1)) (Formula 1)

[0135] Among them, D KL represents the KL divergence, which is used to measure the difference between two probability distributions. p1 can be the detection result of the first local sample image above, and p2 can be the detection result of the second local sample image above.

[0136] Therefore, by randomly sampling the local regions of the image for image detection and calculating the consistency loss between the two sampling results, and optimizing the prompt vector to be trained according to the loss, the misjudgment of image detection can be reduced, such as misjudgment caused by local noise or uneven region segmentation. If one of the two detection results obtained by image detection based on two local regions of the image shows a real image and the other shows a non-real image, the consistency loss can reduce this difference.

[0137] Figure 9 shows a schematic diagram of determining the consistency loss according to an embodiment of the present disclosure.

[0138] As Figure 9 shown, during training, the sample image 901 can be divided into multiple image blocks, such as Figure 9 the image blocks 1-16 shown in Figure 2 . Then these image blocks can be sampled twice. For example, each time 10 image blocks are sampled from 16 image blocks, and the sampling results form the first local sample image 902 and the second sample image 903. Then the above-mentioned image detection method (e.g., combined with Figure 2 the image detection method 200 described above) can be used to perform image detection on the first local sample image 902 and the second sample image 903 respectively to obtain the detection result 904 and the detection result 905. Finally, based on the detection results 904 and 905, the consistency loss 906 can be calculated through the symmetric KL divergence.

[0139] In some embodiments, the preset loss function may further include the above-mentioned consistency loss, which is used to represent the difference between the results of image detection on different local sample images derived from the sample image.

[0140] In the example, the preset loss function can include both the classification loss based on cross-entropy and the consistency loss. The consistency loss can be used to quantify the difference between the results obtained after image detection of different local sample images sampled from the same sample image. Correspondingly, the preset loss function can be calculated by Formula 2:

[0141] L = L cls + λL cons (Formula 2)

[0142] Where L clsis the classification loss, L cons is the consistency loss, λ is the consistency loss weight, and the default value can be set to 0.5.

[0143] Therefore, by using the consistency loss to measure the difference in the detection results of different local image patches of the same image and taking it as part of the preset loss, it can ensure that stable and consistent detection results can be output when processing different local regions of the same image.

[0144] In some embodiments, the image detection method of the present disclosure can be applied to a social media content review system, and can also be applied to many scenarios such as a digital media authentication and news authenticity verification system.

[0145] Figure 10 Shows a structural block diagram of an image detection device 1000 according to an embodiment of the present disclosure.

[0146] As Figure 10 shown, the device 1000 includes a feature acquisition module 1001, a feature processing module 1002, and a feature comparison module 1003. The feature acquisition module 1001 can be configured to acquire a first input feature, a second input feature, and a third input feature for image detection. Among them, the first input feature is associated with a classification marker vector for indicating the classification of the image, the second input feature is associated with a hint vector for indicating whether the image is a real image, and the third input feature is associated with the image information of the image to be detected. The feature processing module 1002 can be configured to fuse the first input feature and the third input feature to obtain a first output feature, and fuse the second input feature and the third input feature to obtain a second output feature. Among them, the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature. The feature comparison module 1003 can be configured to compare the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

[0147] The operations of the above-mentioned feature acquisition module 1001, feature processing module 1002, and feature comparison module 1003 can respectively correspond to the operations of steps S201, S202, and S203 as Figure 2 shown. Therefore, the details of each aspect will not be elaborated here.

[0148] Figure 11 Shows a structural block diagram of an image detection device 1100 according to another embodiment of the present disclosure.

[0149] As Figure 11As shown, the apparatus 1100 includes a feature acquisition module 1101, a feature processing module 1102, and a feature comparison module 1103. The operations of the above modules can be the same as those of the feature acquisition module 1001, the feature processing module 1002, and the feature comparison module 1003 as shown in Figure 10 . In addition, the above modules may further include further sub-modules.

[0150] In some embodiments, the feature acquisition module 1101 may include an input feature splicing module 1101a. The input feature splicing module 1101a may be configured to splice a first input feature, a second input feature, and a third input feature into an input feature sequence.

[0151] In some embodiments, the feature processing module 1102 may include an output feature sequence acquisition module 1102a. The output feature sequence acquisition module 1102a may be configured to provide the input feature sequence to a deep learning model for processing sequence data based on a self-attention mechanism to obtain an output feature sequence, and the output feature sequence may include a first output feature and a second output feature.

[0152] In some embodiments, the feature comparison module 1103 may include a similarity score determination module 1103a and an image detection result determination module 1103b. The similarity score determination module 1103a may be configured to determine a first similarity score between the first output feature and a first output prompt feature, and a second similarity score between the first output feature and a second output prompt feature. The image detection result determination module 1103b may be configured to determine an image detection result based on the first similarity score and the second similarity score. In addition, the above modules may further include further sub-modules.

[0153] In some embodiments, the similarity score determination module 1103a may include a sub-similarity score determination module 1103a-1, a weight determination module 1103a-2, a first similarity score determination module 1103a-3, and a second similarity score determination module 1103a-4. The sub-similarity score determination module 1103a-1 may be configured to determine a first sub-similarity score between the first output feature and each of the plurality of first sub-output hint features, and a second sub-similarity score between the first output feature and each of the plurality of second sub-output hint features. The weight determination module 1103a-2 may be configured to determine a first weight for each of the plurality of first sub-output hint features, and a second weight for each of the plurality of second sub-output hint features. The first similarity score determination module 1103a-3 may be configured to determine a first similarity score based on the weighted sum of the first sub-similarity scores of the plurality of first sub-output hint features and the first weights. The second similarity score determination module 1103a-4 may be configured to determine a second similarity score based on the weighted sum of the second sub-similarity scores of the plurality of second sub-output hint features and the second weights.

[0154] In some embodiments, the image detection result determination module 1103b may include a result determination module 1103b-1, a real image determination module 1103b-2, and a non-real image determination module 1103b-3. The result determination module 1103b-1 may be configured to determine the higher one of the first similarity score and the second similarity score. The real image determination module 1103b-2 may be configured to determine the image to be detected as a real image in response to determining that the first similarity score is higher. The non-real image determination module 1103b-3 may be configured to determine the image to be detected as a non-real image in response to determining that the second similarity score is higher.

[0155] Figure 12 The structural block diagram of an image detection training apparatus 1200 according to an embodiment of the present disclosure is shown.

[0156] As Figure 12 shown, the apparatus 1200 includes a data acquisition module 1201, an image detection module 1202, and a training execution module 1203. The data acquisition module 1201 may be configured to acquire sample images for training and training hint vectors to be trained, where the preset category of the sample images is a real image or a non-real image, and the training hint vectors to be trained are used to indicate whether the images are real images. The image detection module 1202 may be configured to perform image detection on the sample images to obtain sample image detection results. The training execution module 1203 may be configured to train based on the sample image detection results and the preset categories of the sample images through a preset loss function to optimize the training hint vectors to be trained.

[0157] In an example, the image detection module 1202 may be configured to perform image detection on a sample image according to the image detection device as described above (such as Figure 10 the device 1000 shown or such as Figure 11 the device 1100 shown).

[0158] Figure 13 FIG. shows a structural block diagram of an image detection training device 1300 according to another embodiment of the present disclosure.

[0159] As Figure 13 shown, the device 1300 may include a data acquisition module 1301, an image detection module 1302, a training execution module 1303, a subset determination module 1304, a local detection module 1305, and a loss determination module 1306. Among them, the operations of the data acquisition module 1301, the image detection module 1302, and the training execution module 1303 may be the same as the operations of the data acquisition module 1201, the image detection module 1202, and the training execution module 1203 as Figure 12 shown. The subset determination module 1304 may be configured to determine a first subset of sample image blocks and a second subset of sample image blocks from a plurality of sample image blocks, where the first subset of sample image blocks constitutes a first local sample image, and the second subset of sample image blocks constitutes a second local sample image. The local detection module 1305 may be configured to perform image detection on the first local sample image and the second local sample image to obtain a first local sample image detection result and a second local sample image detection result respectively. The loss determination module 1306 may be configured to determine a consistency loss between the first local sample image detection result and the second local sample image detection result.

[0160] In an example, the local detection module 1305 may be configured to perform image detection on a sample image according to the image detection device as described above (such as Figure 10 the device 1000 shown or such as Figure 11 the device 1100 shown).

[0161] According to an embodiment of the present disclosure, there is also provided an electronic device, a readable storage medium, and a computer program product.

[0162] According to an embodiment of the present disclosure, there is also provided an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor, where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above.

[0163] According to an embodiment of the present disclosure, there is also provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0164] According to an embodiment of the present disclosure, there is also provided a computer program product including a computer program, wherein the computer program, when executed by a processor, implements the method as described above.

[0165] Referring Figure 14 , a block diagram of an electronic device 1400 that can be a server or a client of the present disclosure will now be described. It is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0166] As Figure 14 shown, the electronic device 1400 includes a computing unit 1401, which can execute various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1402 or a computer program loaded from a storage unit 1408 into a random access memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the electronic device 1400 can also be stored. The computing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0167] Multiple components in the electronic device 1400 are connected to the I / O interface 1405, including: an input unit 1406, an output unit 1407, a storage unit 1408, and a communication unit 1409. The input unit 1406 can be any type of device capable of inputting information into the electronic device 1400. The input unit 1406 can receive input digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device, and can include but are not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and / or a remote control. The output unit 1407 can be any type of device capable of presenting information, and can include but are not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1408 can include but are not limited to a magnetic disk, an optical disk. The communication unit 1409 allows the electronic device 1400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks, and can include but are not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0168] The computing unit 1401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1401 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1401 executes the various methods and processes described above. For example, in some embodiments, the method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1400 via the ROM 1402 and / or the communication unit 1409. When the computer program is loaded into the RAM 1403 and executed by the computing unit 1401, one or more steps of the method described above can be executed. Alternatively, in other embodiments, the computing unit 1401 can be configured to execute the method in any other suitable way (e.g., by means of firmware).

[0169] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0171] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include electrical connections based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0173] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), the Internet, and blockchain networks.

[0174] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating blockchain.

[0175] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0176] Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above methods, systems, and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but is only defined by the authorized claims and their equivalent scope. Various elements in the embodiments or examples may be omitted or replaced by their equivalent elements. In addition, the steps may be executed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. Importantly, with the evolution of technology, many of the elements described herein may be replaced by equivalent elements that emerge after the present disclosure.

Claims

1. An image detection method, comprising: Obtaining a first input feature, a second input feature, and a third input feature for image detection, wherein the first input feature is associated with a classification marker vector for indicating classification of an image, the second input feature is associated with a hint vector for indicating whether the image is a real image, and the third input feature is associated with image information of the image to be detected; Fusing the first input feature and the third input feature to obtain a first output feature, and fusing the second input feature and the third input feature to obtain a second output feature, wherein the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature; and Comparing the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

2. The method according to claim 1, wherein, The obtaining the first input feature, the second input feature, and the third input feature for image detection comprises: Concatenating the first input feature, the second input feature, and the third input feature into an input feature sequence.

3. The method according to claim 2, wherein, The fusing the first input feature and the third input feature to obtain a first output feature, and fusing the second input feature and the third input feature to obtain a second output feature comprises: Providing the input feature sequence to a deep learning model for processing sequence data based on a self-attention mechanism to obtain an output feature sequence, the output feature sequence including the first output feature and the second output feature.

4. The method according to claim 3, wherein, The first input feature includes the classification marker vector for the model.

5. The method according to any one of claims 1 to 4, wherein, The second input feature includes a first input hint feature and a second input hint feature, the first input hint feature being used to characterize image information of a real image, and the second input hint feature being used to characterize image information of a non-real image.

6. The method according to claim 5, wherein The second output feature includes a first output hint feature corresponding to the first input hint feature, and a second output hint feature corresponding to the second input hint feature, wherein the first input hint feature is contextualized based on the third input feature to obtain the first output hint feature, and the second input hint feature is contextualized based on the third input feature to obtain the second output hint feature.

7. The method according to claim 6, wherein, The comparing the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image comprises: Determining a first similarity score between the first output feature and the first output hint feature, and a second similarity score between the first output feature and the second output hint feature; and Determining the image detection result based on the first similarity score and the second similarity score.

8. The method according to claim 7, wherein, The first output prompt feature includes a plurality of first sub-output prompt features, the second output prompt feature includes a plurality of second sub-output prompt features, and determining the first similarity score between the first output feature and the first output prompt feature, and the second similarity score between the first output feature and the second output prompt feature includes: Determining a first sub-similarity score between the first output feature and each of the plurality of first sub-output prompt features, and a second sub-similarity score between the first output feature and each of the plurality of second sub-output prompt features; Determining a first weight for each of the plurality of first sub-output prompt features, and a second weight for each of the plurality of second sub-output prompt features; Determining the first similarity score based on the weighted sum of the first sub-similarity scores and the first weights for each of the plurality of first sub-output prompt features; and Determining the second similarity score based on the weighted sum of the second sub-similarity scores and the second weights for each of the plurality of second sub-output prompt features.

9. The method according to claim 7 or 8, wherein Determining the image detection result based on the first similarity score and the second similarity score includes: Determining the one with the higher score among the first similarity score and the second similarity score; In response to determining that the first similarity score is higher, determining the image to be detected as a real image; and In response to determining that the second similarity score is higher, determining the image to be detected as a non-real image.

10. The method according to any one of claims 7 to 9, wherein, The first similarity score and the second similarity score are normalized to be converted into probability values.

11. The method according to claim 5, wherein, The second input feature further includes a third input prompt feature, and the third input prompt feature is used to characterize the general image information between real images and non-real images.

12. The method according to claim 11, wherein, The non-real image includes a generative image generated based on artificial intelligence.

13. An image detection training method, including: Obtaining a sample image for training and a prompt vector to be trained, where the preset category of the sample image is a real image or a non-real image, and the prompt vector to be trained is used to indicate whether the image is a real image; Performing image detection on the sample image to obtain a sample image detection result; and Based on the sample image detection result and the preset category of the sample image, training is performed through a preset loss function to optimize the prompt vector to be trained.

14. The method according to claim 13, wherein, In the initialization stage of training, the prompt vector to be trained is obtained based on random initialization.

15. The method according to claim 13 or 14, wherein, The preset loss function includes a classification loss based on cross entropy.

16. The method according to claim 15, wherein, The sample image is segmented into a plurality of sample image blocks, and the method further includes: Determining a first subset of sample image blocks and a second subset of sample image blocks from the plurality of sample image blocks, where the first subset of sample image blocks constitutes a first local sample image, and the second subset of sample image blocks constitutes a second local sample image; Performing image detection on the first local sample image and the second local sample image to respectively obtain a first local sample image detection result and a second local sample image detection result; and Determine the consistency loss between the detection result of the first local sample image and the detection result of the second local sample image.

17. The method according to claim 16, wherein The preset loss function further includes the consistency loss, which is used to represent the difference between the results of image detection on different local sample images derived from the sample image.

18. An image detection device, comprising: A feature acquisition module, configured to acquire a first input feature, a second input feature, and a third input feature for image detection, wherein the first input feature is associated with a classification marker vector for indicating the classification of an image, the second input feature is associated with a hint vector for indicating whether an image is a real image, and the third input feature is associated with the image information of the image to be detected; A feature processing module, configured to fuse the first input feature and the third input feature to obtain a first output feature, and fuse the second input feature and the third input feature to obtain a second output feature, wherein the first input feature is contextualized based on the third input feature to obtain the first output feature, and the second input feature is contextualized based on the third input feature to obtain the second output feature; and A feature comparison module, configured to compare the first output feature with the second output feature to determine an image detection result for indicating whether the image to be detected is a real image.

19. An image detection training device, comprising: A data acquisition module, configured to acquire a sample image for training and a prompt vector to be trained, wherein the preset category of the sample image is a real image or a non-real image, and the prompt vector to be trained is used to indicate whether an image is a real image; An image detection module, configured to perform image detection on the sample image to obtain a sample image detection result; and A training execution module, configured to perform training based on the sample image detection result and the preset category of the sample image through a preset loss function to optimize the prompt vector to be trained.

20. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-17.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-17.

22. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method according to any one of claims 1-17.

Images