Synthetic image recognition method and device and electronic equipment
By employing a multi-validation mechanism that combines image detection models, image-to-text models, and hybrid classification models, along with visual and textual features, the accuracy and reliability of synthetic image recognition are improved, thus solving the problem of low accuracy in synthetic image recognition.
Patent Information
- Application Number
- CN202411298731.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-01-07
AI Technical Summary
How to improve the accuracy of identifying synthetic images and prevent them from being misused to spread misleading information or impersonate real events.
A multi-validation mechanism is adopted, which includes image detection model, image-to-text model and hybrid classification model. Synthetic images are trained and classified by convolutional neural network and machine learning model, and image recognition is performed by combining the correlation between visual features and text features.
It improves the accuracy and reliability of synthetic image recognition, enabling it to capture the essential features of images more comprehensively and reduce misidentification.
Smart Images
Figure CN119273966A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a synthetic image recognition method, device and electronic equipment. Background Art
[0002] With the development of science and technology, synthetic image technology has become increasingly mature. Modern computer technology can accurately generate synthetic images that are highly consistent with text input. These synthetic images have shown great potential and value in creative expression, visual arts and many other fields. However, if synthetic images are used improperly, such as to spread misleading information or impersonate real events, they will have a serious impact on society.
[0003] How to improve the accuracy of identifying synthetic images has become a question worth discussing. Summary of the invention
[0004] The embodiments of the present application provide a synthetic image recognition method, device and electronic device for improving the accuracy of recognizing synthetic images.
[0005] In a first aspect, an embodiment of the present application provides a synthetic image recognition method, in which:
[0006] Receive a data set to be processed, and determine whether there is a corresponding user-defined description text for the image to be processed in the data set to be processed;
[0007] If the image to be processed does not have a user-defined description text, the image to be processed is input into the image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image. The image detection model is trained on synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network;
[0008] If the probability value is less than the preset threshold, the image to be processed is input into the image-to-text model to obtain a description text used to describe the meaning of the image to be processed. The image-to-text model is a machine learning model that automatically converts the visual information in the image into a natural language text description;
[0009] The image to be processed and the descriptive text are input into the hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the meaning of the image and the image into vectors and trains them based on the correlation between the vectors.
[0010] Optionally, the above image detection model is trained in the following way:
[0011] Inputting natural training images and non-natural training images into a convolutional neural network, extracting visual features from the visual contents of the natural training images and the non-natural training images, wherein the natural training images and the non-natural training images have the same expression meaning;
[0012] Based on the visual features, the probability value of each image is calculated through the classification layer of the convolutional neural network.
[0013] Optionally, the hybrid classification model is trained in the following way:
[0014] A preset text encoder is used to convert the input training description text into a text vector, where the training description text is used to describe the expression meaning of the natural training image;
[0015] A preset image encoder is used to convert the input natural training images and non-natural training images into image vectors of the same dimension as the text vector, and the non-natural training images have the same expression meaning as the natural training images;
[0016] The text vector and the image vector are concatenated to obtain a combined feature vector, which contains the semantic information of the natural training image and the training description text;
[0017] The combined feature vector is input into the classification layer to determine whether the training image is a synthetic image.
[0018] Optionally, the non-natural training images are generated by using a text-based graph model based on descriptive text of natural training images. The text-based graph model is a machine learning model that generates images that conform to text descriptions based on semantic information in text.
[0019] Optionally, the above method further includes:
[0020] If the probability value is greater than or equal to a preset threshold, it is determined that the image to be processed is a non-synthesized image.
[0021] Optionally, the above method further includes:
[0022] If there is user-defined description text in the data set to be processed, the image to be processed and the user-defined description text are input into the hybrid classification model to determine whether the image to be processed is a synthetic image.
[0023] In a second aspect, an embodiment of the present application provides a synthetic image recognition device, the device comprising:
[0024] A transceiver module is used to receive a data set to be processed and determine whether there is a corresponding user-defined description text for the image to be processed in the data set to be processed;
[0025] A processing module, for inputting the image to be processed into an image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image if there is no user-defined description text for the image to be processed, and the image detection model is trained on synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network;
[0026] The processing module is further used to input the image to be processed into the image-to-text model to obtain a description text for describing the meaning of the image to be processed if the probability value is less than a preset threshold value. The image-to-text model is a machine learning model that automatically converts visual information in an image into a natural language text description;
[0027] The processing module is also used to input the image to be processed and the descriptive text into the hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the meaning of the image and the image into vectors and trains based on the correlation between the vectors.
[0028] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the processor implements any one of the synthetic image recognition methods described in the first aspect.
[0029] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the synthetic image recognition method of any one of the first aspects is implemented.
[0030] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement a synthetic image recognition method as described in any one of the first aspects above.
[0031] The technical effects brought about by any one of the implementation methods in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 A schematic diagram of an application scenario of synthetic image recognition provided in an embodiment of the present application;
[0033] Figure 2 A flow chart of a synthetic image recognition method provided in an embodiment of the present application;
[0034] Figure 3 An exemplary flow chart of a synthetic image recognition provided in an embodiment of the present application;
[0035] Figure 4 A schematic diagram of a device for composite image recognition provided in an embodiment of the present application;
[0036] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0038] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.
[0039] With the development of science and technology, synthetic image technology has become increasingly mature. Modern computer technology can accurately generate synthetic images that are highly consistent with text input. These synthetic images have shown great potential and value in creative expression, visual arts and many other fields. However, if synthetic images are used improperly, such as to spread misleading information or impersonate real events, they will have a serious impact on society.
[0040] How to improve the accuracy of identifying synthetic images has become a question worth discussing.
[0041] In order to solve the above problems, an embodiment of the present application provides a synthetic image recognition method. In the method, a data set to be processed is received, and it is determined whether there is a corresponding user-defined description text for the image to be processed in the data set to be processed. If there is no user-defined description text for the image to be processed, the image to be processed is input into the image detection model to obtain a probability value for determining whether the image to be processed belongs to a non-synthetic image. Among them, the image detection model trains synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network. If the probability value is less than a preset threshold, the image to be processed is input into the image-generated text model to obtain a description text for describing the expression meaning of the image to be processed. Among them, the image-generated text model is a machine learning model that automatically converts visual information in an image into a natural language text description. The image to be processed and the description text are input into a hybrid classification model to determine whether the image to be processed is a synthetic image. Among them, the hybrid classification model converts the text describing the expression meaning of the image and the image into vectors, and trains based on the correlation between the vectors.
[0042] In this way, the image is preliminarily screened through the image detection model, and it is determined whether it is a synthetic image based on the visual features of the image. If the image is preliminarily determined to be a synthetic image based on the image detection model, the multiple verification mechanisms of further verification through the image-to-text model and the hybrid classification model greatly improve the accuracy and reliability of classifying the processed image. At the same time, the hybrid classification model not only considers the visual features of the image, but also combines the text features of the descriptive text generated by the image-to-text model. By analyzing the correlation between these two features to classify the image, it can more comprehensively capture the essential features of the image, thereby improving the accuracy of identifying synthetic images.
[0043] A schematic diagram of an application scenario of an optional synthetic image recognition method in an embodiment of the present application. Figure 1 As shown, it includes a server 100 and a terminal 101. The server 100 and the terminal 101 can be communicatively connected through a network to implement the synthetic image recognition method of the present application.
[0044] The user can use the server 100 to interact with the terminal 101 through the network, such as receiving or sending messages, etc. Various client applications can be installed on the terminal 101, such as program writing applications, web browser applications, search applications, etc.
[0045] It is understandable that in the embodiment of the present application, the server 100 can be implemented as an independent server or a server cluster composed of multiple servers. The terminal 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, desktop computers, etc.
[0046] The server 100 can be used to receive a set of data to be processed, and determine whether there is a corresponding user-defined description text for the image to be processed in the set of data to be processed. If there is no user-defined description text for the image to be processed, the image to be processed is input into the image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image. The image detection model trains synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network. If the probability value is less than a preset threshold, the image to be processed is input into the image-to-text model to obtain a description text for describing the expression meaning of the image to be processed. The image-to-text model is a machine learning model that automatically converts visual information in an image into a natural language text description. The image to be processed and the description text are input into a hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the expression meaning of the image and the image into vectors, and trains based on the correlation between the vectors.
[0047] like Figure 2As shown, a flowchart of a synthetic image recognition method provided in an embodiment of the present application may specifically include the following steps.
[0048] Step S201: receiving a data set to be processed, and determining whether there is a corresponding user-defined description text for an image to be processed in the data set to be processed.
[0049] For example, if the data is stored in the form of a file system, the file containing the user-defined description text corresponding to the image to be processed can be preset to have the same name as the file of the image to be processed. Then, by checking whether there is a description text file with the same name as the image file, it can be determined whether there is a corresponding user-defined description text.
[0050] Step S202: If the image to be processed does not have a user-defined description text, the image to be processed is input into an image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image.
[0051] Among them, the image detection model uses a convolutional neural network to train synthetic images and non-synthetic images with the same meaning.
[0052] It is understood that non-synthetic images can be images that are natural or artificial but imitate natural scenes (such as landscapes, people, animals, etc.) with natural textures, color distribution and lighting conditions. Synthetic images can be artificially processed or generated and do not directly correspond to anything in nature, such as computer-generated images.
[0053] The following is an explanation of how to train the image detection model:
[0054] Input natural training images and non-natural training images into the convolutional neural network, and extract visual features from the visual content of natural training images and non-natural training images. Based on the visual features, the probability value of each image is calculated through the classification layer of the convolutional neural network. Among them, the expression meaning of synthetic images and non-synthetic images is the same.
[0055] In this way, model training based on convolutional neural networks can automatically extract useful visual features from raw image data. Compared with traditional manual feature extraction methods, automatic feature extraction is more efficient and accurate. At the same time, because convolutional neural networks can learn hierarchical feature representations from low-level to high-level through multi-layer convolution and pooling operations, they can provide richer information in the subsequent image classification process, thereby improving the accuracy of image classification. At the same time, because convolutional neural networks have the ability to continuously learn, they can learn more robust features and improve the robustness of the model.
[0056] For example, natural training images and unnatural training images are input into a convolutional neural network. The convolutional neural network extracts the visual features of the image layer by layer through a series of convolutional layers, pooling layers, etc. After extracting the visual features, the probability value of each image is calculated by the classification layer of the convolutional neural network. Among them, the classification layer is trained to predict the probability value of whether the image is a synthetic image. The probability value can be calculated based on the difference between the non-synthetic image and the synthetic image learned by the network. If the input image is more like a non-synthetic image, the corresponding probability value will be higher. If the input image is more like a synthetic image, the corresponding probability value will be lower.
[0057] In an optional embodiment, the probability value may be compared with a preset threshold. In one possible case, if the probability value is greater than or equal to the preset threshold, the image to be processed is determined to be a non-synthesized image. In another possible case, if the probability value is less than the preset threshold, step S203 is executed.
[0058] It is understandable that the preset threshold value can be preset by those skilled in the art. The preset threshold value can also be changed based on the actual application scenario. For example, the preset threshold value is 0.5. For another example, the preset threshold value is 0.6.
[0059] Step S203: If the probability value is less than a preset threshold, the image to be processed is input into the image-to-text model to obtain a description text for describing the meaning of the image to be processed.
[0060] Among them, the image-to-text model is a machine learning model that automatically converts visual information in images into natural language text descriptions.
[0061] For example, the image-to-text model can be an image description generation model based on a deep learning model (Transformer) with a self-attention mechanism.
[0062] In this way, by inputting the image to be processed into the image-to-text model and obtaining a description text for describing the expression meaning of the image to be processed, it can be ensured that the generated description text can accurately describe the expression meaning of the image to be processed in the real-world scene.
[0063] Step S204: input the image to be processed and the description text into the hybrid classification model to determine whether the image to be processed is a composite image.
[0064] Among them, the hybrid classification model converts the text describing the meaning of the image and the image into vectors, and trains them based on the correlation between the vectors.
[0065] The following is an explanation of how to train a hybrid classification model:
[0066] The input training description text is converted into a text vector using a preset text encoder. The training description text is used to describe the expression meaning of the natural training image. The input natural training image and the non-natural training image are converted into an image vector of the same dimension as the text vector using a preset image encoder. The non-natural training image has the same expression meaning as the natural training image. The text vector and the image vector are concatenated to obtain a combined feature vector, which contains the semantic information of the natural training image and the training description text. The combined feature vector is input into the classification layer to determine whether the training image is a synthetic image. The training images include natural training images and non-natural training images.
[0067] In this way, based on the corresponding non-synthetic images and synthetic images and text, the model can be trained more accurately on the subtle differences between non-synthetic images and synthetic images. By splicing the information encoded by the image and text, the correlation and complementarity between the image and text can be captured. This helps the model to more accurately determine whether the input image is a synthetic image.
[0068] It is understandable that the present application does not specifically limit the concatenation method of the text vector and the image vector. For example, the text vector and the image vector may be connected head to tail to obtain a combined feature vector. For another example, different weights may be assigned to the text vector and the image vector, respectively, and then the weighted vectors are connected head to tail to obtain a combined feature vector.
[0069] For example, the server can use the text encoding part of the Contrastive Language-Image Pre-training (CLIP) model to convert the input description text into a text vector. Using the image encoding part of the CLIP model, the input natural training images and non-natural training images are converted into image vectors of the same dimension as the text vector. The text vector and the image vector are concatenated to obtain a combined feature vector. The combined feature vector is input into a classification layer to determine whether the training image is a synthetic image.
[0070] Optionally, the non-natural training images may be generated based on the description text of the natural training images using a text-based graph model, wherein the text-based graph model is a machine learning model that generates images that conform to the text description based on semantic information in the text.
[0071] For example, the text-to-image model can be a text-to-image generation model based on a latent diffusion model (StableDiffusion), an exploration model based on a diffusion model and a large language model (Diffusion and Large LanguageModel based Explorer 2, DALL·E 2), and the like.
[0072] The following is an explanation of how to use the text graph model based on natural training images to obtain non-natural training images with the same expression meaning as the natural training images:
[0073] First, the server can collect natural training images from various sources (such as public data sets, self-photographing, etc.). In one possible case, there is a corresponding training description text for the natural training image. In another possible case, for natural training images that do not have training description text, the image-to-text model is first used to understand the visual content of the natural training image and automatically generate the corresponding training description text. Subsequently, the training description text of the natural training image is input into the text-to-image model to obtain a non-natural training image.
[0074] In this way, by determining the training description text corresponding to the natural training image and then generating the corresponding non-natural training image based on the training description text, the continuity and consistency of semantic information are ensured in the conversion process from image to text and then to image. By automatically generating training description text for natural training images and non-natural training images, the training data for subsequent models can be greatly enriched. This diverse dataset helps the model learn more features and patterns, thereby enhancing the model's generalization ability when facing unknown or complex inputs.
[0075] It is understandable that although the visual content of the non-natural training images generated based on the training description text conforms to the training description text, the details are different from the original natural training images and are not direct copies of the natural training images. Natural training images usually contain rich details that cannot be fully covered by the description training text, while non-natural training images may be limited to the visual content of the description text.
[0076] In another possible situation, if the data set to be processed contains user-defined description text, the image to be processed and the user-defined description text are input into the hybrid classification model to determine whether the image to be processed is a synthetic image.
[0077] In this way, based on the user-defined description text content, it helps the hybrid classification model to more accurately understand the content of the image to be processed, and assists the hybrid classification model to make more accurate image classification.
[0078] like Figure 3 As shown, an exemplary flow chart of a synthetic image recognition method provided in an embodiment of the present application may include the following steps:
[0079] Step S301, receiving a data set to be processed, determining whether there is a corresponding user-defined description text for the image to be processed in the data set to be processed, if yes, executing step S302, if no, executing step S303;
[0080] Step S302: input the image to be processed and the user-defined description text into the hybrid classification model to determine whether the image to be processed is a synthetic image;
[0081] Step S303: input the image to be processed into the image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image;
[0082] Step S304, determine whether the probability value is greater than or equal to a preset threshold, if so, execute step S305, if not, execute step S306;
[0083] Step S305, determining that the image to be processed is a non-synthesized image;
[0084] Step S306: input the image to be processed into the image-to-text model to obtain a description text for describing the meaning of the image to be processed;
[0085] Step S307: input the image to be processed and the description text into the hybrid classification model to determine whether the image to be processed is a composite image.
[0086] Figure 4 This is a schematic diagram of the structure of a synthetic image recognition device provided in an embodiment of the present application. Figure 4 As shown, the device includes: a transceiver module 401 and a processing module 402.
[0087] The transceiver module 401 is used to receive a data set to be processed and determine whether there is a corresponding user-defined description text for the image to be processed in the data set to be processed;
[0088] The processing module 402 is used for inputting the image to be processed into the image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image if there is no user-defined description text for the image to be processed, and the image detection model is trained on synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network;
[0089] The processing module 402 is further used to input the image to be processed into the image-to-text model to obtain a description text for describing the meaning of the image to be processed if the probability value is less than a preset threshold value. The image-to-text model is a machine learning model that automatically converts visual information in an image into a natural language text description;
[0090] The processing module 402 is also used to input the image to be processed and the descriptive text into the hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the meaning of the image and the image into vectors and trains them based on the correlation between the vectors.
[0091] Optionally, the image detection model is trained in the following way:
[0092] Inputting a natural training image and a non-natural training image into a convolutional neural network, extracting visual features from the visual contents of the natural training image and the non-natural training image, wherein the natural training image and the non-natural training image have the same expression meaning;
[0093] Based on visual features, the probability value of each image is calculated through the classification layer of the convolutional neural network. The expression meaning of non-natural training images is the same as that of natural training images.
[0094] Optionally, the hybrid classification model is trained as follows:
[0095] A preset text encoder is used to convert the input training description text into a text vector, where the training description text is used to describe the expression meaning of the natural training image;
[0096] A preset image encoder is used to convert the input natural training images and non-natural training images into image vectors of the same dimension as the text vector, and the non-natural training images have the same expression meaning as the natural training images;
[0097] The text vector and the image vector are concatenated to obtain a combined feature vector, which contains the semantic information of the natural training image and the training description text;
[0098] The combined feature vector is input into the classification layer to determine whether the image to be processed is a synthetic image.
[0099] Optionally, the non-natural training images are generated based on the description text of the natural training images using a text-based graph model, where the text-based graph model is a machine learning model that generates images that conform to the text description based on semantic information in the text.
[0100] Optionally, the processing module 402 is further configured to:
[0101] If the probability value is greater than or equal to a preset threshold, it is determined that the image to be processed is a non-synthesized image.
[0102] Optionally, the processing module 402 is further configured to:
[0103] If there is user-defined description text in the data set to be processed, the image to be processed and the user-defined description text are input into the hybrid classification model to determine whether the image to be processed is a synthetic image.
[0104] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned synthetic image recognition device.
[0105] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0106] At least one processor 501, and a memory 502 connected to the at least one processor 501. The specific connection medium between the processor 501 and the memory 502 is not limited in the embodiment of the present application. Figure 5 In the example, the processor 501 and the memory 502 are connected via a bus 500. The bus 500 is Figure 5 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.
[0107] In the embodiment of the present application, the memory 502 stores instructions that can be executed by at least one processor 501. The at least one processor 501 can execute a synthetic image recognition method discussed above by executing the instructions stored in the memory 502. The processor 501 can implement Figure 4 The functions of each module in the device shown.
[0108] Among them, the processor 501 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 502 and calling data stored in the memory 502, the various functions of the device and processing data, the device can be monitored as a whole.
[0109] In one possible design, the processor 501 may include one or more processing units, and the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a driver interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 501. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.
[0110] Processor 501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of a synthetic image recognition method disclosed in the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.
[0111] The memory 502 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 502 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0112] By programming the processor 501, the code corresponding to the synthetic image recognition method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 2 The embodiment shown is a synthetic image recognition method. How to design and program the processor 501 is a technique known to those skilled in the art and will not be described in detail here.
[0113] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0114] An embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a synthetic image recognition method in the above embodiment.
[0115] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0116] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 Flow or flows and / or boxes Figure 1 A device that performs the functions specified in a box or multiple boxes.
[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Flow or flows and / or boxes Figure 1 The functions specified in one or more boxes.
[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Flow or flows and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0119] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A synthetic image recognition method, characterized in that: The method comprises: Receive a data set to be processed, and determine whether there is a corresponding user-defined description text for an image to be processed in the data set to be processed; If the image to be processed does not have the user-defined description text, input the image to be processed into an image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image, wherein the image detection model is trained on synthetic images and non-synthetic images that express the same meaning through a convolutional neural network; If the probability value is less than a preset threshold, input the image to be processed into a picture-to-text model to obtain a description text for describing the meaning of the image to be processed, wherein the picture-to-text model is a machine learning model that automatically converts visual information in an image into a natural language text description; The image to be processed and the descriptive text are input into a hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the meaning of the image and the image into vectors and is trained based on the correlation between the vectors.
2. The method according to claim 1, characterized in that The image detection model is trained in the following way: Inputting a natural training image and a non-natural training image into the convolutional neural network, extracting visual features from the visual contents of the natural training image and the non-natural training image, wherein the natural training image and the non-natural training image have the same expression meaning; Based on the visual features, a probability value of each image is calculated through the classification layer of the convolutional neural network.
3. The method according to claim 1, characterized in that The hybrid classification model is trained in the following way: Using a preset text encoder to convert the input training description text into a text vector, wherein the training description text is used to describe the expression meaning of the natural training image; Using a preset image encoder to convert the input natural training image and the non-natural training image into an image vector of the same dimension as the text vector, the non-natural training image and the natural training image having the same expression meaning; Concatenating the text vector and the image vector to obtain a combined feature vector, wherein the combined feature vector includes semantic information of the natural training image and the training description text; The combined feature vector is input into the classification layer to determine whether the training image is a synthetic image.
4. The method according to claim 2 or 3, characterized in that: The non-natural training image is generated based on the description text of the natural training image using a text-based graph model, and the text-based graph model is a machine learning model that generates an image that conforms to the text description based on semantic information in the text.
5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the probability value is greater than or equal to the preset threshold, it is determined that the image to be processed is a non-synthesized image.
6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If the data set to be processed contains user-defined description text, the image to be processed and the user-defined description text are input into the hybrid classification model to determine whether the image is a non-synthetic image.
7. A synthetic image recognition device, characterized in that: The device comprises: A transceiver module, used for receiving a set of data to be processed, and determining whether there is a corresponding user-defined description text for an image to be processed in the set of data to be processed; A processing module, for inputting the image to be processed into an image detection model to obtain a probability value for determining whether the image to be processed is a non-synthetic image if there is no user-defined description text for the image to be processed, wherein the image detection model is trained on synthetic images and non-synthetic images with the same expression meaning through a convolutional neural network; The processing module is further configured to input the image to be processed into a picture-to-text model to obtain a description text for describing the meaning of the image to be processed if the probability value is less than a preset threshold value, wherein the picture-to-text model is a machine learning model that automatically converts visual information in an image into a natural language text description; The processing module is also used to input the image to be processed and the descriptive text into a hybrid classification model to determine whether the image to be processed is a synthetic image. The hybrid classification model converts the text describing the meaning of the image and the image into vectors and trains them based on the correlation between the vectors.
8. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to cause the computer to execute the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that When the computer program product is called by a computer, the computer executes the method according to any one of claims 1 to 6.