Method, device, storage medium and electronic device for detecting an image

Abstract

Embodiments of the present specification disclose a method and device for detecting an image, a storage medium and an electronic device, obtaining a trained image detection model; obtaining corresponding global input information and local input information according to a target image; inputting the local input information into a local picture coding model to obtain output local feature information, and inputting the local feature information into a mask regression model to obtain output target mask; inputting the global input information into a global picture coding model to obtain output global feature information, and inputting the global input information into a semantic segmentation model to obtain output semantic feature information; inputting the semantic feature information, the global feature information and the local feature information into a detection score discrimination model to obtain output target image generation result, and taking the target image generation result and the target mask as the output of the image detection model for the target image.

Description

Technical Field

[0001] This invention relates to computer technology, and more particularly to a method, apparatus, storage medium, and electronic device for detecting images. Background Technology

[0002] In recent years, text-to-image (TPE) generation models have developed rapidly, capable of generating images from short text. Especially with the emergence of diffusion-based generation models, the quality of generated images has seen a qualitative leap. Various open-source communities support users in using various open-source models for diverse creative projects. On the one hand, TPE technology generates various risks, such as celebrity photos and famous paintings, involving various copyright issues; on the other hand, image-to-image (AGE) technology can stylize and edit real images, engaging in image authentication fraud, causing negative social impact. Therefore, there is an urgent need for AIGC (Artificial Intelligence Generated Content) detection technology to identify generated images, enabling traceability and accountability, and ensuring social security. Currently, AIGC-generated images include globally generated TPEs and AGEs, as well as locally edited generation methods such as image editing and image filling. Therefore, a detection method that can simultaneously address both globally generated and locally edited AIGC images is essential. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to provide a method, apparatus, storage medium, and electronic device for detecting images.

[0004] This specification provides a method for image detection. It utilizes a unified framework to study algorithms for global AIGC detection and local AIGC detection. By combining global semantic features with local perceptual features, it unifies global and local AIGC detection into a single framework. Simultaneously, a semantic segmentation model is introduced to extract semantic features, aiming to perceive locally meaningful features. This is because local editing often tends to modify regions with specific semantic meaning (e.g., foreground objects). Furthermore, edit region mask regression is introduced to guide the model to focus on the tampered regions, creating a focus on the real regions and facilitating the differentiation of tampered regions from real images. The method includes:

[0005] A trained image detection model is obtained, wherein the image detection model includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model;

[0006] Based on the target image, corresponding global input information and local input information are obtained, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image;

[0007] The local input information is input into the local image encoding model to obtain the local feature information corresponding to the output target image, and the local feature information is input into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the editing region in the target image;

[0008] The global input information is input into the global image encoding model to obtain the global feature information corresponding to the output target image, and the global feature information is input into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image;

[0009] The semantic feature information, the global feature information, and the local feature information are input into the detection score discrimination model to obtain the target image generation result corresponding to the target image. The target image generation result and the target mask are used as the output of the image detection model for the target image.

[0010] Furthermore, the target image includes any of the following:

[0011] Real images;

[0012] Generate an image globally;

[0013] Image generated through partial editing.

[0014] Further, obtaining the global input information and local input information corresponding to the target image includes:

[0015] The target image is adjusted to a preset size, and corresponding global input information and local input information are obtained based on the adjusted target image. The global input information includes the adjusted target image, and the local input information includes the target number of image units obtained by segmenting the adjusted target image.

[0016] Further, the step of inputting the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result includes:

[0017] The semantic feature information, the global feature information, and the local feature information are concatenated along the channel to obtain the final discriminative feature information;

[0018] The final discrimination feature information is input into the detection score discrimination model to obtain the output target image generation result.

[0019] Further, obtaining the trained image detection model includes:

[0020] Based on the training image samples, the corresponding mask labels and image generation labels, the image detection model is trained using the classification loss function and the mask regression loss function to obtain the trained image detection model.

[0021] Furthermore, the training image sample is a pair of training images, which includes a globally generated training image and a locally edited training image.

[0022] Furthermore, the input to the classification loss function includes the image generation label corresponding to the globally generated training image, the image generation label corresponding to the locally edited training image, the final discriminant feature information corresponding to the globally generated training image, and the final discriminant feature information corresponding to the locally edited training image.

[0023] Furthermore, the inputs to the mask regression loss function include the mask label corresponding to the globally generated training image, the mask label corresponding to the locally edited training image, the target mask output by the mask regression model for the globally generated training image, and the target mask output by the mask regression model for the locally edited training image.

[0024] This specification also provides an embodiment of a method for detecting an image, the method comprising:

[0025] An image detection model is constructed, which includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model.

[0026] Based on the training image samples, corresponding global input information and local input information are obtained, wherein the global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the training image samples;

[0027] The local input information is input into the local image encoding model to obtain the output local feature information, and the local feature information is input into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the location of the edit region in the training image sample;

[0028] The global input information is input into the global image encoding model to obtain the output global feature information, and the global feature information is input into the semantic segmentation model to obtain the output semantic feature information;

[0029] The semantic feature information, the global feature information, and the local feature information are input into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask label corresponding to the training image sample, and the image generation label, the image detection model is trained based on the classification loss function and the mask regression loss function to obtain the trained image detection model.

[0030] Furthermore, the training image sample is a pair of training images, which includes a globally generated training image and a locally edited training image.

[0031] This specification also provides an apparatus for detecting images, comprising:

[0032] The model acquisition module is used to acquire a trained image detection model, wherein the image detection model includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model.

[0033] An input acquisition module is used to obtain corresponding global input information and local input information based on a target image, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image;

[0034] The local feature output module is used to input the local input information into the local image encoding model to obtain the local feature information corresponding to the output target image, and input the local feature information into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the edit region in the target image;

[0035] The global feature output module is used to input the global input information into the global image encoding model to obtain the global feature information corresponding to the output target image, and to input the global feature information into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image;

[0036] The model output module is used to input the semantic feature information, the global feature information and the local feature information into the detection score discrimination model to obtain the target image generation result corresponding to the target image, and to use the target image generation result and the target mask as the output of the image detection model for the target image.

[0037] This specification also provides an apparatus for training an image detection model, comprising:

[0038] The model building module is used to build an image detection model, which includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model.

[0039] The input acquisition module is used to obtain corresponding global input information and local input information based on training image samples, wherein the global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the target image samples;

[0040] The local feature output module is used to input the local input information into the local image encoding model to obtain the output local feature information, and input the local feature information into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the position of the edit region in the target image;

[0041] The global feature output module is used to input the global input information into the global image encoding model to obtain the output global feature information, and input the global feature information into the semantic segmentation model to obtain the output semantic feature information;

[0042] The model training module is used to input the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask label corresponding to the training image sample, and the image generation label, the image detection model is trained based on the classification loss function and the mask regression loss function to obtain the trained image detection model.

[0043] This specification also provides a computer program product that stores at least one instruction adapted to be loaded by a processor and executed in accordance with the above-described method steps.

[0044] This specification also provides a storage medium storing a computer program adapted to be loaded by a processor and to execute the steps of the method described above.

[0045] This specification also provides an electronic device, including a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method described above.

[0046] In the embodiments of this specification, the algorithms for global AIGC detection and local AIGC detection are studied using the same framework. Global AIGC detection and local AIGC detection are unified into a single framework by combining global semantic features and local perceptual features. At the same time, a semantic segmentation model is introduced to extract semantic features, aiming to perceive local features with specific meaning, because local editing often tends to modify regions with specific semantics (such as foreground objects). In addition, edit region mask regression is introduced to guide the model to focus on the tampered region, while forming a focus on the real region, which is conducive to distinguishing the tampered region from the real image. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a method for detecting an image provided in an embodiment of this specification;

[0048] Figure 2 This is a flowchart illustrating a method for training an image detection model, as provided in an embodiment of this specification.

[0049] Figure 3 A flowchart illustrating an example of a method for detecting an image, provided as an embodiment of this specification;

[0050] Figure 4 A schematic diagram of the structure of an image detection device provided in the embodiments of this specification;

[0051] Figure 5 This is a schematic diagram of the structure of an apparatus for training an image detection model, provided in an embodiment of this specification.

[0052] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0054] Please see Figure 1 This is a schematic flowchart illustrating a method for detecting an image provided in an embodiment of this specification. In this embodiment, the method for detecting an image is applied to an image detection apparatus (hereinafter referred to as an "image detection apparatus") or an electronic device equipped with an image detection apparatus. The following will focus on... Figure 1The process shown will be described in detail. The method for detecting images may specifically include the following steps:

[0055] S102, obtain the trained image detection model, wherein the image detection model includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model.

[0056] In some embodiments, the image detection model is an image detection framework composed of multiple models, including a semantic segmentation model, a global image coding model, a local image coding model, a detection score discriminant model, and a mask regression model. The inputs to the semantic segmentation model, the global image coding model, and the local image coding model are also the inputs to the image detection model; that is, the inputs to the image detection model are respectively fed into the semantic segmentation model, the global image coding model, and the local image coding model. The output of the local image coding model is fed into the mask regression model; that is, the input to the mask regression model is the output of the local image coding model. The outputs of the semantic segmentation model, the global image coding model, and the local image coding model are fed into the detection score discriminant model; that is, the inputs to the detection score discriminant model include the outputs of the semantic segmentation model, the global image coding model, and the local image coding model. The outputs of the encoding model, the detection score discriminant model, and the mask regression model are used to indicate whether the input image is an AIGC-generated image. The mask regression model outputs the mask corresponding to the input image, which is used to indicate the location of the AIGC editing region in the input image. The outputs of the detection score discriminant model and the mask regression model are used as the outputs of the image detection model. That is, the inputs of the semantic segmentation model, the global image encoding model, and the local image encoding model are the inputs of the image detection model. The output of the local image encoding model is input to both the mask regression model and the detection score discriminant model. The output of the global image encoding model is input to the detection score discriminant model. The output of the semantic segmentation model is also input to the score discriminant model. The outputs of the detection score discriminant model and the mask regression model are the outputs of the image detection model.

[0057] In some embodiments, the semantic segmentation model is obtained after pre-training, the global image coding model is composed of a residual convolutional network res-block (res-block is a basic component of the ResNet network structure), the local image coding model is composed of a transformer structure (the transformer structure mainly includes an encoder and a decoder), the detection score discrimination model is composed of fully connected layers (FC), and the mask regression model is a statistical model used to quantitatively describe the statistical relationship between variables. It is a statistical predictive model that describes the relationship between the independent variable (also known as the explanatory variable or feature, i.e., the local feature output by the local image coding model in this scheme) and the dependent variable (also known as the response variable or target variable, i.e., the mask used in this scheme to indicate the location of the AIGC editing region in the input image).

[0058] In some embodiments, the embodiments of this specification do not limit the training method of the image detection model. Those skilled in the art should understand that any model training method can be included within the scope of protection of this specification, and this specification does not limit it.

[0059] In some embodiments, the methods for obtaining a trained image detection model include, but are not limited to, reading the trained image detection model from memory or local storage, sending an image detection model acquisition request to the corresponding server and receiving the trained image detection model returned by the server based on the image detection model acquisition request, etc. It should be noted that the above acquisition methods are merely examples and not limitations. Those skilled in the art should understand that any acquisition method can be included within the scope of protection of this specification, and this specification does not impose any limitations on it.

[0060] S104, obtain corresponding global input information and local input information based on the target image, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image.

[0061] In some embodiments, it is necessary to first obtain the global input information and local input information corresponding to the target image of the input image detection model. The global input information is the target image itself. The target image is uniformly divided into multiple image units (image patches) of the target number, and these multiple image units of the target number are used as the local input information corresponding to the target image. In some embodiments, the target image can be a real image, or it can be an image generated globally by AIGC, or it can be an image generated by AIGC through local editing of a real image.

[0062] In some embodiments, the target number can be preset, that is, different input images correspond to the same target number. Alternatively, the target number can be determined based on the attribute information of the input image, that is, different input images correspond to their own independent target number. The attribute information includes, but is not limited to, the image size information, image resolution information, image color information, image content information, etc. of the input image. This example embodiment does not make any special limitation in this regard. For example, the target number can be determined based on the size information of the input image. The larger the size of the input image, the more target number there is. Another example is that the target number can also be determined based on the number of objects in the input image. The more objects in the input image, the more target number there is. It should be noted that the above methods for determining the target number are only examples and not limitations. Those skilled in the art should understand that any method for determining the target number based on the attribute information of the input image can be included within the protection scope of this specification, and this specification does not limit it.

[0063] S106, the local input information is input into the local image encoding model to obtain the local feature information corresponding to the output target image, and the local feature information is input into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the editing region in the target image.

[0064] In some embodiments, multiple image units of a target number are input into a local image coding model to obtain local feature information (i.e., feature information corresponding to each image unit) corresponding to the target image output by the local image coding model. Then, the local feature information is input into a mask regression model to obtain a target mask corresponding to the target image output by the mask regression model. The target mask is used to indicate the location of the AIGC editing region in the target image.

[0065] In some embodiments, the mask is a matrix or image of the same size as the original image (i.e., the target image), used to indicate or identify specific regions in the original image (i.e., regions edited by AIGC). The mask is binary (i.e., contains only 0 or 1 values), where a value of 1 indicates that the region corresponding to that value in the original image has been edited by AIGC or that the region was generated by AIGC, and a value of 0 indicates that the region corresponding to that value in the original image has not been edited by AIGC or that the region was not generated by AIGC. Based on the position of a value in the mask, the corresponding position in the original image can be obtained. Based on this value, it can be determined whether the region at that corresponding position in the original image has been edited by AIGC or whether the region was generated by AIGC. For example, the mask can be a matrix containing only the values ​​0 and 1, where the position of the value 0 in the matrix corresponds to a value in the original image. For each corresponding position, the region in the original image at that position has not been edited by AIGC or the region was not generated by AIGC. Similarly, the position of the value 1 in this matrix will also correspond to a corresponding position in the original image, where the region at that position in the original image has been edited by AIGC or the region was generated by AIGC. For example, the mask can also be an image where each pixel value only includes 0 and 1. Since pixel value 1 is white and pixel value 0 is black, the mask is a black and white image that only includes black and white. The position of pixel value 0 in this image will correspond to a corresponding position in the original image, where the region at that position in the original image has not been edited by AIGC or the region was not generated by AIGC. Similarly, the position of pixel value 1 in this image will also correspond to a corresponding position in the original image, where the region at that position in the original image has been edited by AIGC or the region was generated by AIGC. In some embodiments, if the original image (i.e. the target image) is an image generated globally by AIGC, then its corresponding mask is a matrix consisting only of the value 1, or the mask is an image consisting only of the pixel value 1 (i.e., a completely white image). That is, the mask corresponding to the image generated globally by AIGC is the entire image, and the mask corresponding to the image generated by AIGC local editing is the specific editing location.

[0066] S108, the global input information is input into the global image encoding model to obtain the global feature information corresponding to the output target image, and the global feature information is input into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image.

[0067] In some embodiments, global input information is input into the global image encoding model and the semantic segmentation model respectively, which can obtain the semantic feature information corresponding to the target image output by the semantic segmentation model. This is because the local editing of real images by AIGC generally edits regions with specific semantics (such as specific objects). Semantic features can effectively capture the features of local editing, and can also obtain the global feature information (i.e., global perception features) corresponding to the target image output by the global image encoding model.

[0068] S110, the semantic feature information, the global feature information and the local feature information are input into the detection score discrimination model to obtain the target image generation result corresponding to the output target image, and the target image generation result and the target mask are used as the output of the image detection model for the target image.

[0069] In some embodiments, the semantic feature information corresponding to the target image output by the semantic segmentation model, the global feature information corresponding to the target image output by the global image coding model, and the local feature information corresponding to the target image output by the local image coding model are input into the detection score discrimination model to obtain the target image generation result corresponding to the target image output by the detection score discrimination model. The target image generation result is used to indicate whether the target image is generated by AIGC (including AIGC global generation and AIGC local editing generation). For example, if the target image is generated by AIGC, the output target image generation result is 1; if the target image is not generated by AIGC, the output target image generation result is 0.

[0070] In some embodiments, the target image generation result corresponding to the target image output by the detection score discrimination model and the target mask corresponding to the target image output by the mask regression model are used as the output of the trained image detection model for the target image input to the model. For example, the output of the image detection model includes that the target image is a real image, and the output also includes that the mask corresponding to the target image is a matrix consisting only of the value 0, or that the mask is an image consisting only of the pixel value 0 (i.e., a completely black image). Another example is that the output of the image detection model includes that the target image is an AIGC globally generated image, and the output also includes that the mask corresponding to the target image is a matrix consisting only of the value 1, or that the mask is an image consisting only of the pixel value 0. An image consisting of a prime value of 1 (i.e., a completely white image). For example, the output of an image detection model includes a target image that is a locally edited image generated by AIGC (i.e., an image generated by AIGC through local editing of a real image), and the output also includes a mask corresponding to the target image that is a matrix consisting of values ​​0 and 1, or an image consisting of pixel values ​​0 and 1 (i.e., a black and white image). Here, a value of 1 or a pixel value of 1 indicates that the region corresponding to the position of that value or pixel value in the mask in the target image has been edited by AIGC, and a value of 0 or a pixel value of 0 indicates that the region corresponding to the position of that value or pixel value in the mask in the target image has not been edited by AIGC.

[0071] In the embodiments of this specification, the algorithms for global AIGC detection and local AIGC detection are studied using the same framework. Global AIGC detection and local AIGC detection are unified into a single framework by combining global semantic features and local perceptual features. At the same time, a semantic segmentation model is introduced to extract semantic features, aiming to perceive local features with specific meaning, because local editing often tends to modify regions with specific semantics (such as foreground objects). In addition, edit region mask regression is introduced to guide the model to focus on the tampered region, while forming a focus on the real region, which is conducive to distinguishing the tampered region from the real image.

[0072] In some embodiments, the target image includes any one of the following: a real image; a globally generated image; or a locally edited generated image. In some embodiments, the target image of the input image detection model can be a real image, or it can be an image globally generated by AIGC, or it can be an image generated by AIGC through local editing of a real image.

[0073] In some embodiments, obtaining the global input information and local input information corresponding to the target image includes: adjusting the target image to a preset size, and obtaining the corresponding global input information and local input information based on the adjusted target image. The global input information includes the adjusted target image, and the local input information includes a target number of image units obtained by segmenting the adjusted target image. In some embodiments, the target image needs to be adjusted to a preset size (e.g., scaled or enlarged to 512*512). This example embodiment does not specifically limit the specific value of the preset size. In this case, the global input information is the adjusted target image. By uniformly segmenting the adjusted target image into a target number of image units, these target number of image units are used as the local input information corresponding to the target image.

[0074] In some embodiments, the step of inputting the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result includes: concatenating the semantic feature information, the global feature information, and the local feature information along the channel to obtain the final discrimination feature information; and inputting the final discrimination feature information into the detection score discrimination model to obtain the output target image generation result. In some embodiments, the semantic feature information corresponding to the target image output by the semantic segmentation model, the global feature information corresponding to the target image output by the global image coding model, and the local feature information corresponding to the target image output by the local image coding model are first concatenated along the channels to obtain the final discriminative feature information. Then, the final discriminative feature information is input into the detection score discriminative model to obtain the target image generation result corresponding to the target image output by the detection score discriminative model. For example, the data format corresponding to the semantic feature information, global feature information, and local feature information is X*Y*N, where X is the number of pixels in the x-direction, Y is the number of pixels in the y-direction, and N is the number of color channels. Concatenating these three feature information along their respective channels, the data format corresponding to the final discriminative feature information obtained after concatenation is X*Y*3N. For example, if the data format corresponding to the semantic feature information, global feature information, and local feature information is 512*512*3 (i.e., the number of color channels is 3), the data format corresponding to the final discriminative feature information obtained after concatenation is 512*512*9 (i.e., the number of color channels is 9).

[0075] In some embodiments, obtaining the trained image detection model includes: training the image detection model based on a classification loss function and a mask regression loss function using training image samples, mask labels corresponding to the training image samples, and image generation labels, to obtain the trained image detection model. In some embodiments, one training image can be input into the image detection model for training at a time, or multiple training images can be input into the image detection model for training at a time. In some embodiments, the image generation labels corresponding to the training image samples are used to indicate or identify whether the training image is generated by AIGC (including AIGC global generation and AIGC local editing generation). For example, if the training image is generated by AIGC, its corresponding image generation label is 1; if the training image is not generated by AIGC, its corresponding image generation label is 0. In some embodiments, the mask labels corresponding to the training image samples are used to indicate or identify specific regions in the training image (i.e., regions edited by AIGC). For example, if the training image is a real image, its corresponding mask label is a matrix consisting only of the value 0 or an image consisting only of the pixel value 0 (i.e., a completely black image). If the training image is an image globally generated by AIGC, its corresponding mask label is a matrix consisting only of the value 1 or an image consisting only of the pixel value 1 (i.e., a completely white image). If the training image is an image locally edited by AIGC, its corresponding mask label is a matrix consisting of the values ​​0 and 1 or an image consisting of the pixel values ​​0 and 1 (i.e., a black and white image). Here, the value 1 or pixel value 1 indicates that the region corresponding to the position of the value or pixel value in the mask label in the training image has been edited by AIGC, and the value 0 or pixel value 0 indicates that the region corresponding to the position of the value or pixel value in the mask label in the training image has not been edited by AIGC. In some embodiments, by inputting a training image into an image detection model, the target mask corresponding to the training image and the target image generation result corresponding to the training image are obtained from the output of the image detection model. Based on the target mask and mask label, the image detection model is trained using a mask regression loss function. Based on the target image generation result and image generation label, the image detection model is trained using a classification loss function, and finally, a trained image detection model is obtained.

[0076] In some embodiments, the training image samples are a pair of training images, comprising a globally generated training image and a locally edited training image. In some embodiments, each pair of training images is input into the image detection model for training. This pair includes one AIGC-generated globally and one AIGC-generated locally, thus achieving balanced training samples during training and preventing knowledge forgetting. In some embodiments, there does not need to be any correlation between the globally generated and locally edited training images in the pair. In some embodiments, during the training phase of the image detection model, a pair of training images needs to be input for training each time. However, in the prediction phase after the trained image detection model is obtained, only one target image to be predicted needs to be input, instead of a pair of images.

[0077] In some embodiments, the input to the classification loss function includes the image generation label corresponding to the globally generated training image, the image generation label corresponding to the locally edited training image, the final discriminative feature information corresponding to the globally generated training image, and the final discriminative feature information corresponding to the locally edited training image. In some embodiments, the semantic feature information corresponding to the target image output by the semantic segmentation model, the global feature information corresponding to the target image output by the global image encoding model, and the local feature information corresponding to the target image output by the local image encoding model are first concatenated along channels to obtain the final discriminative feature information. Then, the final discriminative feature information is input into the detection score discriminative model to obtain the target image generation result corresponding to the target image output by the detection score discriminative model. In some embodiments, the function formula of the classification loss function is as follows:

[0078]

[0079] in, It is classification loss. It is a pair of image generation labels corresponding to the locally edited images generated from the training images. It refers to the final discriminative feature information obtained after stitching together the locally edited image and inputting it into the image detection model. It is a pair of image generation labels corresponding to the globally generated images in the training images. It is the final discriminative feature information obtained after the globally generated image is input into the image detection model.

[0080] In some embodiments, the input to the mask regression loss function includes the mask label corresponding to the globally generated training image, the mask label corresponding to the locally edited generated training image, the target mask output by the mask regression model for the globally generated training image, and the target mask output by the mask regression model for the locally edited generated training image. In some embodiments, the function formula of the mask regression loss function is as follows:

[0081]

[0082] in, It is the mask regression loss. It is a pair of mask labels corresponding to the locally edited generated images in the training images. It is the target mask for the image output generated by the mask regression model for this local edit. It is a pair of mask labels corresponding to the globally generated images in the training images. It is the target mask for the global generated image output by the mask regression model.

[0083] Figure 2 This is a flowchart illustrating a method for training an image detection model provided in an embodiment of this specification. In this embodiment, the method for training an image detection model is applied to an apparatus for training an image detection model (hereinafter referred to as an "image detection model training apparatus") or an electronic device equipped with an image detection model training apparatus. The following will focus on... Figure 2 The process shown will be described in detail. The method for training the image detection model may specifically include the following steps:

[0084] S202, Construct an image detection model, wherein the image detection model includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model. In some embodiments, the image detection model has been described in detail above and will not be repeated here. This specification does not limit the construction method of the image detection model in any way. Those skilled in the art should understand that any model construction method can be included within the scope of protection of this specification, and this specification does not limit it. For example, the semantic segmentation model, the global image coding model, the local image coding model, the detection score discrimination model, and the mask regression model can be combined according to a preset framework to form an integrated image detection model.

[0085] S204: Obtain corresponding global input information and local input information based on the training image samples. The global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the training image samples. The implementation of step S204 is the same as or similar to the aforementioned step S104, and will not be repeated here.

[0086] S206, the local input information is input into the local image encoding model to obtain the output local feature information, and the local feature information is input into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the location of the edit region in the training image sample. The implementation of step S206 is the same as or similar to the aforementioned step S106, and will not be described again here.

[0087] S208: Input the global input information into the global image encoding model to obtain the output global feature information, and input the global feature information into the semantic segmentation model to obtain the output semantic feature information. The implementation of step S208 is the same as or similar to the aforementioned step S108, and will not be described again here.

[0088] S210, the semantic feature information, the global feature information, and the local feature information are input into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask labels corresponding to the training image samples, and the image generation labels, the image detection model is trained using a classification loss function and a mask regression loss function to obtain a trained image detection model. In some embodiments, the method for obtaining the target image generation result corresponding to the training image samples output by the detection score discrimination model has been described in detail above and will not be repeated here. In some embodiments, the method for training the image detection model based on the classification loss function and the mask regression loss function has been described in detail above and will not be repeated here.

[0089] In some embodiments, the training image samples are a pair of training images, which include a globally generated training image and a locally edited training image. In some embodiments, the case where the training image samples consist of a pair of training images including an AIGC globally generated training image and an AIGC locally edited training image has been described in detail above and will not be repeated here.

[0090] Figure 3 This is a schematic flowchart illustrating an example of a method for detecting an image, provided as an embodiment of this specification.

[0091] like Figure 3As shown, the global input information (i.e., the target image itself) corresponding to the target image to be predicted is input into the semantic segmentation model and the global image coding model, respectively. The local input information (i.e., the number of image units obtained by segmenting the target image) corresponding to the target image is input into the local image coding model. The local feature information output by the local image coding model is input into the mask regression model to obtain the target mask corresponding to the target image output by the mask regression model. The target mask is used to indicate the location of the AIGC editing region in the target image. The semantic feature information output by the semantic segmentation model, the global feature information output by the global image coding model, and the local feature information output by the local image coding model are input into the detection score discrimination model to obtain the target image generation result corresponding to the target image output by the detection score discrimination model. The target image generation result is used to indicate whether the target image is generated by AIGC (including AIGC global generation and AIGC local editing generation).

[0092] Figure 4 This is a schematic diagram of a device for detecting images provided in an embodiment of this specification. This device (hereinafter referred to as "image detection device 1") can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the image detection device 1 includes a model acquisition module 11, an input acquisition module 12, a local feature output module 13, a global feature output module 14, and a model output module 15.

[0093] The model acquisition module 11 is used to acquire a trained image detection model, wherein the image detection model includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model.

[0094] The input acquisition module 12 is used to obtain corresponding global input information and local input information based on the target image, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image;

[0095] The local feature output module 13 is used to input the local input information into the local image encoding model to obtain the local feature information corresponding to the output target image, and input the local feature information into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the editing region in the target image;

[0096] The global feature output module 14 is used to input the global input information into the global image encoding model to obtain the global feature information corresponding to the output target image, and to input the global feature information into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image.

[0097] The model output module 15 is used to input the semantic feature information, the global feature information and the local feature information into the detection score discrimination model to obtain the target image generation result corresponding to the target image, and use the target image generation result and the target mask as the output of the image detection model for the target image.

[0098] In some embodiments, the target image includes any of the following:

[0099] Real images;

[0100] Generate an image globally;

[0101] Image generated through partial editing.

[0102] In some embodiments, obtaining the global input information and local input information corresponding to the target image includes:

[0103] The target image is adjusted to a preset size, and corresponding global input information and local input information are obtained based on the adjusted target image. The global input information includes the adjusted target image, and the local input information includes the target number of image units obtained by segmenting the adjusted target image.

[0104] In some embodiments, the step of inputting the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result includes:

[0105] The semantic feature information, the global feature information, and the local feature information are concatenated along the channel to obtain the final discriminative feature information;

[0106] The final discrimination feature information is input into the detection score discrimination model to obtain the output target image generation result.

[0107] In some embodiments, obtaining the trained image detection model includes:

[0108] Based on the training image samples, the corresponding mask labels and image generation labels, the image detection model is trained using the classification loss function and the mask regression loss function to obtain the trained image detection model.

[0109] In some embodiments, the training image samples are a pair of training images, the pair of training images including a globally generated training image and a locally edited training image.

[0110] In some embodiments, the input to the classification loss function includes the image generation label corresponding to the globally generated training image, the image generation label corresponding to the locally edited training image, the final discriminant feature information corresponding to the globally generated training image, and the final discriminant feature information corresponding to the locally edited training image.

[0111] In some embodiments, the input to the mask regression loss function includes the mask label corresponding to the globally generated training image, the mask label corresponding to the locally edited training image, the target mask output by the mask regression model for the globally generated training image, and the target mask output by the mask regression model for the locally edited training image.

[0112] Figure 5 This is a schematic diagram of a device for training an image detection model, provided in an embodiment of this specification. This device (hereinafter referred to as "image detection model training device 2") can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the image detection model training device 2 includes a model building module 21, an input acquisition module 22, a local feature output module 23, a global feature output module 24, and a model training module 25.

[0113] The model building module 21 is used to build an image detection model, wherein the image detection model includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model;

[0114] The input acquisition module 22 is used to obtain corresponding global input information and local input information based on the training image samples, wherein the global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the target image samples;

[0115] The local feature output module 23 is used to input the local input information into the local image encoding model to obtain the output local feature information, and input the local feature information into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the position of the editing region in the target image;

[0116] The global feature output module 24 is used to input the global input information into the global image encoding model to obtain the output global feature information, and to input the global feature information into the semantic segmentation model to obtain the output semantic feature information;

[0117] The model training module 25 is used to input the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask label corresponding to the training image sample, and the image generation label, the image detection model is trained based on the classification loss function and the mask regression loss function to obtain the trained image detection model.

[0118] In some embodiments, the training image samples are a pair of training images, the pair of training images including a globally generated training image and a locally edited training image.

[0119] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0120] This specification also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the methods described in this specification.

[0121] This specification also provides a computer program product that stores at least one instruction, which is loaded by the processor and executes the method described in this specification embodiment.

[0122] The embodiments in this specification also provide Figure 6 The diagram shows the structure of the electronic device. Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned voice activity detection method.

[0123] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0124] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0125] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0126] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0127] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0128] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0133] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0134] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0135] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0138] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0139] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for detecting an image, comprising: A trained image detection model is obtained, wherein the image detection model includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model; Based on the target image, corresponding global input information and local input information are obtained, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image; The local input information is input into the local image encoding model to obtain the local feature information corresponding to the output target image, and the local feature information is input into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the editing region in the target image; The global input information is input into the global image encoding model to obtain the global feature information corresponding to the output target image, and the global feature information is input into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image; The semantic feature information, the global feature information, and the local feature information are input into the detection score discrimination model to obtain the target image generation result corresponding to the target image. The target image generation result and the target mask are used as the output of the image detection model for the target image.

2. The method according to claim 1, wherein the target image comprises any one of the following: Real images; Generate an image globally; Image generated through partial editing.

3. The method according to claim 1, wherein obtaining the global input information and local input information corresponding to the target image includes: The target image is adjusted to a preset size, and corresponding global input information and local input information are obtained based on the adjusted target image. The global input information includes the adjusted target image, and the local input information includes the target number of image units obtained by segmenting the adjusted target image.

4. The method according to claim 1, wherein inputting the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result includes: The semantic feature information, the global feature information, and the local feature information are concatenated along the channel to obtain the final discriminative feature information; The final discrimination feature information is input into the detection score discrimination model to obtain the output target image generation result.

5. The method according to any one of claims 1 to 4, wherein obtaining the trained image detection model comprises: Based on the training image samples, the corresponding mask labels and image generation labels, the image detection model is trained using the classification loss function and the mask regression loss function to obtain the trained image detection model.

6. The method according to claim 5, wherein the training image sample is a pair of training images, the pair of training images comprising a globally generated training image and a locally edited training image.

7. The method according to claim 6, wherein the input of the classification loss function includes the image generation label corresponding to the globally generated training image, the image generation label corresponding to the locally edited training image, the final discriminant feature information corresponding to the globally generated training image, and the final discriminant feature information corresponding to the locally edited training image.

8. The method according to claim 6, wherein the input of the mask regression loss function includes the mask label corresponding to the globally generated training image, the mask label corresponding to the locally edited generated training image, the target mask output by the mask regression model for the globally generated training image, and the target mask output by the mask regression model for the locally edited generated training image.

9. A method for training an image detection model, comprising: An image detection model is constructed, which includes a semantic segmentation model, a global image coding model, a local image coding model, a detection score discrimination model, and a mask regression model. Based on the training image samples, corresponding global input information and local input information are obtained, wherein the global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the training image samples; The local input information is input into the local image encoding model to obtain the output local feature information, and the local feature information is input into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the location of the edit region in the training image sample; The global input information is input into the global image encoding model to obtain the output global feature information, and the global feature information is input into the semantic segmentation model to obtain the output semantic feature information; The semantic feature information, the global feature information, and the local feature information are input into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask label corresponding to the training image sample, and the image generation label, the image detection model is trained based on the classification loss function and the mask regression loss function to obtain the trained image detection model.

10. The method according to claim 9, wherein the training image sample is a pair of training images, the pair of training images comprising a globally generated training image and a locally edited training image.

11. An apparatus for detecting an image, comprising: The model acquisition module is used to acquire a trained image detection model, wherein the image detection model includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model. An input acquisition module is used to obtain corresponding global input information and local input information based on a target image, wherein the global input information includes the target image, and the local input information includes the target number of image units obtained by segmenting the target image; The local feature output module is used to input the local input information into the local image encoding model to obtain the local feature information corresponding to the output target image, and input the local feature information into the mask regression model to obtain the target mask corresponding to the output target image, wherein the target mask is used to indicate the position of the edit region in the target image; The global feature output module is used to input the global input information into the global image encoding model to obtain the global feature information corresponding to the output target image, and to input the global feature information into the semantic segmentation model to obtain the semantic feature information corresponding to the output target image; The model output module is used to input the semantic feature information, the global feature information and the local feature information into the detection score discrimination model to obtain the target image generation result corresponding to the target image, and to use the target image generation result and the target mask as the output of the image detection model for the target image.

12. An apparatus for training an image detection model, comprising: The model building module is used to build an image detection model, which includes a semantic segmentation model, a global image encoding model, a local image encoding model, a detection score discrimination model, and a mask regression model. The input acquisition module is used to obtain corresponding global input information and local input information based on training image samples, wherein the global input information includes the training image samples, and the local input information includes the target number of image units obtained by segmenting the training image samples; The local feature output module is used to input the local input information into the local image encoding model to obtain the output local feature information, and input the local feature information into the mask regression model to obtain the output target mask, wherein the target mask is used to indicate the position of the edit region in the training image sample; The global feature output module is used to input the global input information into the global image encoding model to obtain the output global feature information, and input the global feature information into the semantic segmentation model to obtain the output semantic feature information; The model training module is used to input the semantic feature information, the global feature information, and the local feature information into the detection score discrimination model to obtain the output target image generation result. Based on the target mask, the target image generation result, the mask label corresponding to the training image sample, and the image generation label, the image detection model is trained based on the classification loss function and the mask regression loss function to obtain the trained image detection model.

13. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

14. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 10.

15. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 10.

Images