Image processing apparatus, control method of image processing apparatus, and storage medium

Abstract

The invention provides an image processing apparatus, a control method of the image processing apparatus, and a storage medium. The invention provides a structure capable of changing output processing according to a determination result of whether to generate image data to be output based on a learning model. The image processing apparatus determines whether to generate image data input through input processing based on a learning model. In a case where it is determined that the image data is generated based on the learning model, the image processing apparatus does not perform output processing on the image data, or adds processing to the image data, and then performs output processing on the image data to which the processing has been added.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a control method for the image processing apparatus, and a storage medium. Background Art

[0002] In recent years, image generation techniques using artificial intelligence (AI) have become widespread. For example, this enables individuals to easily generate realistic images. On the other hand, AI-based image generation techniques are also increasingly being used for malicious purposes, which is becoming a problem. For example, fabricating photos of fictional cases and accidents related to real people, facilities, etc.

[0003] To prevent such abuse of AI-based image generation techniques, various countries are making efforts to quickly formulate laws and development guidelines for AI-generated content. For example, a law under consideration would require a clear statement that the AI-generated content is generated by AI. In the future, it is likely that it will be necessary to clearly distinguish AI-generated content from other content.

[0004] Japanese Patent Application Laid-Open No. 2000-175031 discloses an image processing apparatus equipped with measures to prevent malicious use of image data by restricting the output of image data of, for example, banknotes and securities, which are usually prohibited from being copied, to paper, storage in external devices, and transmission. It is considered that such an image processing apparatus may also need to be restricted in the processing of using AI to generate images.

[0005] However, the image processing apparatus described in Japanese Patent Application Laid-Open No. 2000-175031 does not determine whether an image is generated using AI, even when such determination is required, and there is room for improvement in this regard. Summary of the Invention

[0006] The present invention provides a system capable of performing output processing of output image data and changing the output processing according to the result of determination when it is necessary or requested to determine whether an image to be output is generated based on a learning model.

[0007] According to one aspect of the present invention, an image processing apparatus includes: an input unit configured to perform input processing of input image data; a determination unit configured to determine whether the image data input through the input processing is generated based on a learning model; and an output unit configured to, when the determination unit determines that the image data is generated based on the learning model, not perform output processing of outputting the image data, and when the determination unit determines that the image data is not generated based on the learning model, perform the output processing.

[0008] According to another aspect of the present invention, an image processing apparatus includes: an input unit configured to perform input processing of input image data; and a determination unit configured to determine whether the image data input through the input processing is generated based on a learning model. The image processing apparatus further includes: a processing unit configured to perform processing on the image data when it is determined that the image data is generated based on the learning model; and an output unit configured to perform output processing of outputting the image data on which the processing has been performed.

[0009] According to another aspect of the present invention, a control method for an image processing apparatus includes: performing input processing of input image data; and determining whether the image data input through the input processing is generated based on a learning model. The method further includes: when it is determined that the image data is generated based on the learning model, not performing output processing of outputting the image data; and when it is determined that the image data is not generated based on the learning model, performing the output processing.

[0010] According to another aspect of the present invention, a control method for an image processing apparatus includes: performing input processing of input image data; and determining whether the image data input through the input processing is generated based on a learning model. The method further includes: when it is determined that the image data is generated based on the learning model, performing processing on the image data and performing output processing of outputting the image data on which the processing has been performed.

[0011] According to the present invention, when an image processing apparatus performs output processing on image data, in a case where it is necessary or requested to determine whether the image to be output is generated based on a learning model, the output processing can be changed according to the determination result.

[0012] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram showing a hardware configuration of an image processing system according to a first embodiment.

[0014] Figure 2 is a block diagram showing a hardware configuration of an AI-based image generation server.

[0015] Figure 3 is a block diagram showing a hardware configuration of a general-purpose terminal.

[0016] Figure 4A and Figure 4B is a flowchart showing processing performed by a general-purpose terminal and processing performed by an AI-based image generation server.

[0017] Figure 5 It is a block diagram showing another hardware configuration of the image processing system.

[0018] Figure 6 It is a block diagram showing the hardware configuration of the image processing device.

[0019] Figure 7 It is a block diagram showing the hardware configuration of the controller unit included in the image processing device.

[0020] Figure 8 It is a flowchart showing the processing (processing for restricted printing of AI-generated images) executed by the image processing device.

[0021] Figure 9 It is a diagram showing an example of the notification screen displayed on the image processing device.

[0022] Figure 10A and Figure 10B It is a diagram showing an example of the printed matter printed by the image processing device.

[0023] Figure 11 It is a diagram showing an example of the operation screen displayed on the image processing device when restricted printing is performed.

[0024] Figure 12 It is a flowchart showing the processing (print job processing for restricted printing of AI-generated images) executed by the general-purpose terminal.

[0025] Figure 13 It is a flowchart showing a modified example of the processing (print job processing for restricted printing of AI-generated images) executed by the general-purpose terminal.

[0026] Figure 14 It is a flowchart showing the processing (processing for restricted transmission of AI-generated images) executed by the image processing device according to the second embodiment.

[0027] Figure 15 It is a diagram showing an example of the notification screen displayed on the image processing device.

[0028] Figure 16A and Figure 16B It is a diagram showing an example of the printed matter printed by the printing device in the case where the device other than the image processing device is the printing device.

[0029] Figure 17 It is a diagram showing an example of the operation screen displayed on the image processing device when restricted transmission is performed.

[0030] Figure 18It is a flowchart showing the processing executed by a general-purpose terminal (transmission job processing for restricted transmission of AI-generated images).

[0031] Figure 19 It is a flowchart showing a modified example of the processing executed by a general-purpose terminal (transmission job processing for restricted transmission of AI-generated images). Detailed Description of the Invention

[0032] The present invention will be described in detail below with reference to the accompanying drawings showing embodiments of the present invention.

[0033] The configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in the embodiments. For example, each unit constituting the present invention can be replaced with a unit having any configuration capable of exhibiting a similar function. In addition, any component can be added to the configuration described in the embodiments. Furthermore, any two or more configurations (features) in the embodiments can be combined.

[0034] First Embodiment

[0035] Hereinafter, with reference to Figures 1 to 13 the first embodiment will be described.

[0036] Figure 1 It is a block diagram showing the hardware configuration of an image processing system according to the first embodiment. As Figure 1 shown, the image processing system 10 includes an AI-based image generation server 101 and a general-purpose terminal 102, which are communicably connected to each other via a network 100.

[0037] The AI-based image generation server 101 is an external device that generates image data based on a machine learning model or a learning model. Hereinafter, generating image data based on a learning model (including a trained learning model) may be referred to as "AI-based image generation". The generated image data is sent to the general-purpose terminal 102. The general-purpose terminal 102 is a device capable of performing various types of processing on the image data sent from the AI-based image generation server 101. The general-purpose terminal 102 is not limited to any specific computing device and may be, for example, a desktop or laptop personal computer, a tablet terminal, a smartphone, etc. It should be noted that in the present embodiment, AI-based image generation is performed by the AI-based image generation server 101, but it is not limited thereto, and for example, AI-based image generation may be performed by the general-purpose terminal 102.

[0038] Figure 2 It is a block diagram showing the hardware configuration of the AI-based image generation server 101. As Figure 2As shown, the AI-based image generation server 101 includes a CPU 201, a RAM 202, a ROM 203, a storage unit 204, a GPU 207, an AI-based identification information adding unit 208, and a network interface (I / F) 209.

[0039] The CPU 201 controls the operation of the AI-based image generation server 101 based on the program expanded in the RAM 202. The ROM 203 is a boot ROM and stores, for example, a boot program for the image processing system 10. The storage unit 204 is a non-volatile device including an HDD, an SSD, etc. The storage unit 204 stores a training model 205 to be used in AI-based image generation, an AI-based image generation program 206, etc. It should be noted that in AI-based image generation, any training model 205 and existing AI-based image generation programs 206 (such as Stable Diffusion) can be used, but it is not limited to using them. The training model 205 and the AI-based image generation program 206 are loaded into the RAM 202 and executed by the CPU 201. Since the technology related to AI-based image generation is a known technology, its description is omitted here.

[0040] In response to a request for AI-based image generation from the general-purpose terminal 102, AI-based image generation is performed. The CPU 201 instructs the GPU 207 in response to the request. The GPU 207 performs AI-based image generation according to the given instruction. Thus, image data is generated.

[0041] The image data includes identification information that enables the identification of the image data generated through AI-based image generation, that is, identification information that identifies the image data as generated based on a learning model. Such identification information is added to the image data by the AI-based identification information adding unit 208. It should be noted that the identification information may also include, for example, information about the training model 205 and the AI-based image generation program 206, information about the request for AI-based image generation, etc. In addition, the identification information can be provided as metadata similar to the header information of an image file, or can be provided as information superimposed on the image itself, such as an invisible digital watermark or text to be visualized.

[0042] The network I / F 209 is connected to the network 100 and is used for inputting and outputting various types of information. The connection between the network I / F 209 and the network 100 can be wired or wireless.

[0043] Figure 3 is a block diagram showing the hardware configuration of the general-purpose terminal 102. As Figure 3As shown, the general-purpose terminal 102 includes a CPU 301, a RAM 302, an SSD 303, a user I / F 304, and a network I / F 305.

[0044] The CPU 301 controls the operation of the general-purpose terminal 102 based on the programs expanded in the RAM 302. The SSD 303 stores various programs and the like. The programs include, for example, system programs, AI-based image generation applications, and the like. The user I / F 304 includes, for example, a display, a touch panel, a keyboard, a mouse, etc., and performs input / output processing for the user. The network I / F 305 is connected to the network 100 and is used for inputting and outputting various types of information. The connection between the network I / F 305 and the network 100 can be wired or wireless. It should be noted that the general-purpose terminal 102 can have, for example, a telephone function, a camera function, etc.

[0045] Figure 4A is a flowchart showing the processing executed by the general-purpose terminal 102. Figure 4B is a flowchart showing the processing executed by the AI-based image generation server 101. As Figure 4A shown, in step S401, the CPU 301 of the general-purpose terminal 102 determines that parameters for an image to be generated by the AI-based image generation server 101 have been received from the user via the user I / F 304. These parameters are not particularly limited and can be information or data used in known AI-based image generation techniques, such as keywords, sentences, and images related to the image.

[0046] In step S402, the CPU 301 sends an image generation request based on the parameters received in step S401 to the AI-based image generation server 101 via the network I / F 305.

[0047] As Figure 4B shown, in step S411, the CPU 201 of the AI-based image generation server 101 determines that the image generation request sent in step S402 has been received via the network I / F 209.

[0048] In step S412, the CPU 201 controls the GPU 207 to perform AI-based image generation using the training model 205 as described above.

[0049] In step S413, the CPU 201 controls the AI-based identification information adding unit 208 to add identification information to the image data generated in step S412. Hereinafter, the image data obtained by AI-based image generation can be referred to as "AI image data". In addition, the AI image data with identification information added can be referred to as "image data with identification information".

[0050] In step S414, the CPU 201 transmits the image data with the identification information added in step S413 to the general-purpose terminal 102 via the network I / F 209. Accordingly, the general-purpose terminal 102 receives the image data with the identification information.

[0051] Figure 5 FIG. is a block diagram showing the hardware configuration of the image processing system 10 in another case. As Figure 5 shown, in the image processing system 10, the general-purpose terminal 102 is connected to the image processing apparatus 500 and the AI-based image recognition server 701 via the network 100.

[0052] In the general-purpose terminal 102, an application capable of inputting a remote job to the image processing apparatus 500 is installed. This application is not particularly limited, and for example, it may be a driver application for the image processing apparatus 500, or a mobile application that supports the image processing apparatus 500 when the general-purpose terminal 102 is a tablet terminal or the like.

[0053] In the AI-based image recognition server 701, an AI-based image recognition application using known techniques (e.g., a learning model through machine learning) is installed. Using this application, the AI-based image recognition server 701 analyzes the image data input via the network 100 and determines whether the image data is created by an AI-based image generation technique. Then, the AI-based image recognition server 701 notifies the general-purpose terminal 102 of the determination result.

[0054] It should be noted that in the present embodiment, the general-purpose terminal 102 is connected to the AI-based image generation server 101 via the network 100 in one case, and is connected to the image processing apparatus 500 and the AI-based image recognition server 701 via the network 100 in another case, but is not limited thereto. For example, the network connecting the general-purpose terminal 102 and the AI-based image generation server 101 may be different from the network connecting the general-purpose terminal 102, the image processing apparatus 500, and the AI-based image recognition server 701.

[0055] Figure 6 FIG. is a block diagram showing the hardware configuration of the image processing apparatus 500. As Figure 6 shown, the image processing apparatus 500 includes an automatic document feeder (ADF) 501, an image reading unit 502, a controller unit 503, and an image forming unit 504.

[0056] The ADF 501 sends control signals to the image reading unit 502 via the data bus and receives control signals from the image reading unit 502, and conveys the original document. In addition, the ADF 501 includes various sensors (such as an original document detection sensor for detecting the original document), and notifies the values of the respective sensors when conveying the original document. The image reading unit 502 reads the original document together with the ADF 501 according to the original document reading instruction received from the controller unit 503 via the data bus.

[0057] The controller unit 503 controls the entire image processing apparatus 500 including the ADF 501, the image reading unit 502, and the image forming unit 504 via the data bus. In addition, the controller unit 503 performs input processing of input image data, analyzes the input image data, appropriately performs image processing, and generates image data to be output.

[0058] The image forming unit 504 prints the image data obtained from the controller unit 503 via the data bus as a visible image on the recording sheet while conveying the recording sheet. The printed matter is discharged from the image processing apparatus 500. Thus, in the present embodiment, the image forming unit 504 serves as an output unit (printing engine) capable of performing printing processing (which is output processing of output image data) on the printed image data.

[0059] Figure 7 is a block diagram showing the hardware configuration of the controller unit 503 included in the image processing apparatus 500. As Figure 7 shown, the controller unit 503 includes a CPU 601, an eMMC 602, a storage unit 603, a scanner I / F 604, a printer I / F 605, an image memory 606, an operation unit 607, an AI-based image recognition unit 608, an image processing unit 609, and a network I / F 610.

[0060] The CPU 601 is a computer that controls the entire image processing apparatus 500. The eMMC 602 includes a flash memory and stores a control program executed by the CPU 601. The control program includes, for example, a program for causing the computer to execute a process for realizing the functions of the units of the image processing apparatus 500 and / or a process for controlling the image processing apparatus 500.

[0061] The storage unit 603 is a non-volatile memory for holding information to be used for various types of control. The scanner I / F 604 sends data to the image reading unit 502 and receives data from the image reading unit 502. The printer I / F 605 sends data to the image forming unit 504 and receives data from the image forming unit 504. The image memory 606 stores image data and the like acquired via the scanner I / F 604. The operation unit 607 is a user interface and includes a touch panel and hard keys. For example, the touch panel displays information for the user and accepts operations by the user for job input and various settings.

[0062] The AI-based image recognition unit 608 determines whether the image data input through input processing (e.g., image data sent from the general-purpose terminal 102) is generated by AI-based image generation before output processing. This determination is made based on the recognition information (whether the image data includes recognition information). Specifically, when the image data as the determination target includes recognition information, that is, when recognition information is provided in the image data, the AI-based image recognition unit 608 determines that the image data as the determination target is generated by AI-based image generation. On the other hand, when the image data as the determination target does not include recognition information, that is, when recognition information is not provided in the image data, the AI-based image recognition unit 608 determines that the image data as the determination target is not generated by AI-based image generation.

[0063] The image processing unit 609 processes image data. The network I / F 610 is connected to the network 100 and controls the input / output of various types of information, image data, and the like.

[0064] Figure 8 is a flowchart showing the processing (processing for restricted printing of AI-generated images) executed by the image processing apparatus 500. When the network I / F 610 receives a print job sent from an external device such as the general-purpose terminal 102 via the network 100, the image processing apparatus 500 starts executing the print job. The print job includes information (such as output size and resolution) and image data that the image processing apparatus 500 uses for printing.

[0065] In step S801, the CPU 601 (controller unit 503) of the image processing apparatus 500 controls the AI-based image recognition unit 608 to determine whether the AI image data (AI-generated image) is included in the print job received by the network I / F 610 (whether the image data of the print job includes an AI-generated image). Specifically, after performing the input processing of the image data of the input print job, the AI-based image recognition unit 608 refers to the metadata of the image data that is the determination object in the print job, and performs image analysis to determine whether the image data includes identification information indicating whether the image data is AI image data. When the image data includes the identification information, the AI-based image recognition unit 608 determines that the print job includes AI image data. When the image data does not include the identification information, the AI-based image recognition unit 608 determines that the print job does not include AI image data. Then, as the determination result in step S801, when it is determined that the print job includes AI image data, the process proceeds to step S802. At this time, the identification information is stored in the storage unit 603. On the other hand, as the determination result in step S801, when it is determined that the print job does not include AI image data, the process proceeds to step S803.

[0066] It should be noted that in this embodiment, it is determined whether the print job includes AI image data based on the presence or absence of identification information, but it is not limited thereto. For example, the identification information can be digitized by a probability, a score, etc. indicating the possibility that a specific part of the image data (image) as the determination object is AI image data (AI-generated image). In this case, the AI-based image recognition unit 608 converts the identification information into a percentage value, and when the result value of the identification information is equal to or greater than the threshold (N%), it is determined that the print job includes AI image data. When the result value of the identification information is less than the threshold (N%), the AI-based image recognition unit 608 determines that the print job does not include AI image data.

[0067] The threshold can be stored in the storage unit 603 in advance, can be included in the print job, or can be set via the operation unit 607 before executing the job. In addition, the threshold is preferably changeable to adapt to the situation.

[0068] In the case where the entire image of the image data as the determination object is not an AI-generated image but a part of the image is an AI-generated image, the identification information can include information for specifying the AI-generated image part. The information for specifying the AI-generated image part can be indicated by, for example, the upper left coordinates and the lower right coordinates of the AI-generated image part, or can be information about the pixels constituting the AI-generated image part.

[0069] In this embodiment, the determination as to whether the print job includes AI image data is made by the AI-based image recognition unit 608, but is not limited thereto, and may be made by, for example, the AI-based image recognition server 701.

[0070] In step S803, the CPU 601 controls the image forming unit 504 to execute the print job, that is, to perform normal printing processing on the assumption that the input is a document that does not include an AI-generated image. After step S803 is executed, the process ends.

[0071] In step S802, the CPU 601 notifies the user who uses the image processing apparatus 500 that the document to be printed includes an AI-generated image, and notifies whether to perform restricted printing or stop printing.

[0072] Figure 9 FIG. is an example of a notification screen displayed on the image processing apparatus 500. Figure 9 The shown notification screen 900 is displayed on the touch panel of the operation unit 607. The notification screen 900 includes a preview image 901, a preview image 903, a stop printing button 905, and a perform restricted printing button 906. The preview image 901 is an image representing the document image as it is. The preview image 901 includes a partial image 902 that is an AI-generated image. The preview image 903 is an image indicating a prediction of the output result when performing restricted printing. The preview image 901 and the preview image 903 are provided for comparison.

[0073] The preview image 903 includes a partial image 904 that is an AI-generated image. The partial image 904 includes an image frame 908 and the character 909 “AI” which is information indicating that this partial image is an AI-generated image and is superimposed on this partial image. This enables the user to understand that when printing the image data, the image processing apparatus 500 provides a printed matter with the same image as the preview image 903 printed thereon. Note that the character 909 is “AI”, but is not limited thereto, and may be any character indicating that this partial image is an AI-generated image.

[0074] In step S801, the partial image 902 and the partial image 904 are determined based on the recognition information stored in the storage unit 603. In the case where the document image does not include an AI-generated image or in the case where the entire document image is indicated as an AI-generated image, the preview image 901 and the preview image 903 are exactly the same as each other.

[0075] In step S804, the CPU 601 determines whether the restricted printing button 906 has been operated on the notification screen 900 displayed in step S802. As a result of the determination in step S804, when it is determined that the restricted printing button 906 has been operated, the process proceeds to step S805. On the other hand, as a result of the determination in step S804, when it is determined that the restricted printing button 906 has not been operated (i.e., the stop printing button 905 has been operated), the printing process (output process) is not executed, and the process is terminated.

[0076] In step S805, the CPU 601 controls the image forming unit 504 to perform restricted printing. Specifically, the CPU 601 refers to the identification information stored in the storage unit 603 and the output mode setting for restricted printing. Then, the CPU 601 controls the image processing unit 609 to perform image processing for adding processing to the AI-generated image portion included in the original image according to the output mode setting. Thereafter, the CPU 601 controls the image forming unit 504 to output the image processed (in other words, modified) by the image processing unit 609. Thus, a print having the same image as the preview image 903 is obtained. After executing step S805, the process ends.

[0077] Figure 10A and Figure 10B is a diagram showing an example of a print printed by the image processing apparatus 500. In Figure 10A the shown print 1000, a partial image 1001 as an AI-generated image has a very light color over the entire image and has marks such as stars superimposed on the image. Through such image processing, the viewer of the print 1000 can understand that the partial image 1001 is an AI-generated image. In Figure 10B the shown print 1002, a partial image 1003 as an AI-generated image is an image filled with black. With such image processing, the viewer of the print 1002 can understand that the print 1002 is a print including an AI-generated image. It should be noted that the print including the AI-generated image is not limited to the print 1000 and the print 1002, and for example, it can be a print with invisible information such as an electronic watermark superimposed on the entire image or a partial image. In addition, the information indicating that the print is a print including an AI-generated image can be preset by the administrator of the image processing apparatus 500 and stored in the storage unit 603, or can be set by the user and stored in the storage unit 603.

[0078] Figure 11 is a diagram showing an example of an operation screen displayed on the image processing apparatus during restricted printing. It is displayed on the touch panel of the operation unit 607 Figure 11The operation screen 1100 shown. The operation screen 1100 includes preview images 1101 to 1104, radio buttons 1105 to 1107, and an enter button 1108.

[0079] The preview image 1101 is an image representing the original manuscript image as it is. The preview image 1101 includes a partial image 1109 which is an AI-generated image. The preview image 1102 is an image indicating the output result prediction in the case of restricted printing (first output mode). The preview image 1102 includes a partial image 1110 which is an AI-generated image. The partial image 1110 is similar to the partial image 904. The preview image 1103 is an image indicating the output result prediction in the case of restricted printing (second output mode). The preview image 1103 includes a partial image 1111 which is an AI-generated image. The partial image 1111 is similar to the partial image 1001. The preview image 1104 is an image indicating the output result prediction in the case of restricted printing (third output mode). The preview image 1104 includes a partial image 1112 which is an AI-generated image. The partial image 1112 is similar to the partial image 1003.

[0080] When the user desires to obtain a printout with the same image as the preview image 1102, the user selects the radio button 1105. When the user desires to obtain a printout with the same image as the preview image 1103, the user selects the radio button 1106. When the user desires to obtain a printout with the same image as the preview image 1104, the user selects the radio button 1107. The user is allowed to select one of the radio buttons 1105 to 1107 and then operate the enter button 1108. Thus, a printout of the preview image corresponding to the selected radio button is obtained.

[0081] As described above, in the image processing apparatus 500, input processing of input image data is performed, and then it is determined by the AI-based image recognition unit 608 whether the input image data is generated by AI-based image generation. The result of the determination by the AI-based image recognition unit 608 can be a case where it is determined that the image data to be printed (output processed) is generated by AI-based image generation, or a case where it is determined that the image data is not generated by AI-based image generation. When it is necessary or requested to determine whether the image data to be output is AI image data, the image processing apparatus 500 can change the printing process according to the determination result. Hereinafter, the case where the image data to be printed (output processed) is determined to be generated by AI-based image generation is referred to as "the first case", and the case where the image data to be printed (output processed) is determined not to be generated by AI-based image generation is referred to as "the second case".

[0082] In the first case, the CPU 601 (controller unit 503) of the image processing apparatus 500 adds processing to the image data that is the object of the printing process, and controls the image forming unit 504 to output the processed image data, thereby performing the printing process. When adding processing to the image data, the CPU 601 processes the image data so that it will be visualized that the printed matter obtained through the printing process is the printed matter of the image data including the identification information. In other words, the CPU 601 adds processing to the image data so that it will be visualized that the image data includes the identification information in the printed matter of the image data obtained through the printing process. Thus, for example, a printed matter is obtained that prints the same image as the preview image 903 (refer to Figure 9 ) or the printed matter 1000 (refer to Figure 10A ). A partial image 904 of the preview image 903 includes an image frame 908 and the character 909 "AI" superimposed on the partial image. A partial image 1001 of the printed matter 1000 has a very light color applied to the entire image and has marks such as stars superimposed on the image. Alternatively, the printing process may not be performed on the AI image part. Thus, for example, a printed matter is obtained that prints the same image as the printed matter 1002 (see Figure 10B ). A partial image 1003 of the printed matter 1002 is an image filled with black. These types of printed matters make the viewer aware that the printed matter includes an AI image. For example, this gives the viewer of the printed matter a way to suspect that the AI image may be a fake image.

[0083] It should be noted that in the first case, the image processing apparatus 500 may be configured not to perform restricted printing (output processing).

[0084] On the other hand, in the second case, the CPU 601 of the image processing apparatus 500 adds no processing to the image data that is the object of the printing process via the image processing unit 209, but performs the printing process on the image data as it is via the image forming unit 504.

[0085] In the present embodiment, the image processing system 10 has a configuration in which the AI-based image recognition unit 608 is included in the image processing apparatus 500, but may have a configuration in which, for example, an AI-based image recognition unit (hereinafter referred to as "terminal AI-based image recognition unit") having a function similar to that of the AI-based image recognition unit 608 is included in the general-purpose terminal 102. Here, a description will mainly be given of the differences from the above configuration in which the AI-based image recognition unit 608 is included in the image processing apparatus 500, and similar matters will be omitted from the description.

[0086] Figure 12It is a flowchart showing the process executed by the general-purpose terminal 102 (print job process for restricted printing of AI-generated images). As Figure 12 shown, in step S1201, the CPU 301 of the general-purpose terminal 102 controls the AI-based image recognition unit of the terminal to determine whether the AI image data (AI-generated image) is included in the print job (whether the original image data of the print job includes the AI-generated image). As a result of the determination in step S1201, when it is determined that the print job includes the AI image data, the process proceeds to step S1202. On the other hand, as a result of the determination in step S1201, when it is determined that the print job does not include the AI image data, the process proceeds to step S1203.

[0087] In step S1202, the CPU 301 sends a print job including restricted print specification and the AI-based recognition result to the image processing device 500 via the network I / F 209. As a result, restricted printing is performed by the image processing device 500 (see Figure 8 step S805 of the flowchart shown). After executing step S1202, the process ends.

[0088] In step S1203, the CPU 301 sends a print job including normal print specification and not including the AI-generated image to the image processing device 500 via the network I / F 305. As a result, normal printing is performed by the image processing device 500 (see Figure 8 step S803 of the flowchart shown). After executing step S1203, the process ends.

[0089] In the present embodiment, the image processing system 10 has a configuration in which the image processing unit 609 is included in the image processing device 500, but it may have a configuration in which, for example, an image processing unit (hereinafter referred to as "terminal image processing unit") having a function similar to that of the image processing unit 609 is included in the general-purpose terminal 102. Here, a description will be mainly given of the differences from the above configuration in which the image processing unit 609 is included in the image processing device 500, and similar matters will be omitted from the description.

[0090] Figure 13 It is a flowchart showing a modified example of the process executed by the general-purpose terminal 102 (print job process for restricted printing of AI-generated images). As Figure 13As shown, in step S1301, the CPU 301 of the general-purpose terminal 102 controls the AI-based image recognition unit of the terminal to determine whether the print job includes AI image data (whether the original image data of the print job includes AI-generated images). As a result of the determination in step S1301, if it is determined that the print job includes AI image data, the process proceeds to step S1302. At this time, the recognition information is stored in the SSD 303. On the other hand, as a result of the determination in step S1301, if it is determined that the print job does not include AI image data, the process proceeds to step S1303.

[0091] In step S1303, the CPU 301 sends a print job that includes a normal print specification and does not include an AI-generated image to the image processing device 500 via the network I / F 305. In this case, the processing of the AI-generated image of the print job by the terminal image processing unit is omitted. Therefore, normal printing is performed by the image processing device 500. After executing step S1303, the process ends.

[0092] In step S1302, the CPU 301 notifies the user using the general-purpose terminal 102 that the original to be printed includes an AI-generated image, and notifies whether to perform restricted printing or stop printing. This notification is made by displaying the Figure 9 notification screen 900 shown. Note that in step S1301, the information for displaying the Figure 9 notification screen 900 shown is obtained by referring to the recognition information stored in the SSD 303.

[0093] In step S1304, the CPU 301 determines whether the restricted printing button 906 has been operated on the notification screen 900 displayed in step S1302. As a result of the determination in step S1304, if it is determined that the restricted printing button 906 has been operated, the process proceeds to step S1305. On the other hand, as a result of the determination in step S1304, if it is determined that the restricted printing button 906 has not been operated (i.e., the stop printing button 905 has been operated), the printing process (output process) is not executed, and the process ends.

[0094] In step S1305, the CPU 301 refers to the recognition information stored in the SSD 303 and the output mode setting for restricted printing. Then, the CPU 301 controls the terminal image processing unit to perform image processing on the AI-generated image portion included in the original image by adding processing according to the output mode setting. After that, the CPU 301 sends a print job including the image data on which the image processing has been performed to the image processing apparatus 500 via the network I / F 305. Accordingly, using the image processing apparatus 500, a print having the same image as the preview image 903 printed thereon is obtained. After executing step S1305, the process ends.

[0095] Second Embodiment

[0096] Hereinafter, reference will be made to Figures 14 to 19 describe the second embodiment. A description will mainly be given of the differences from the above-described embodiment, and similar matters will be omitted from the description. This embodiment is similar to the first embodiment except for the output processing of the image processing apparatus 500. Specifically, as the output processing for outputting image data in this embodiment, this embodiment employs a transmission process of sending the image data to a device other than the image processing apparatus 500 (i.e., a device different from the image processing apparatus 500 (e.g., the general-purpose terminal 102)). Further, in this embodiment, the network I / F 610 of the image processing apparatus 500 serves as an output unit capable of performing the transmission process.

[0097] Figure 14 is a flowchart showing the processing (processing for restricted transmission of an AI-generated image) performed by the image processing apparatus 500 according to the second embodiment. When the network I / F 610 receives a transmission job sent from an external device such as the general-purpose terminal 102 via the network 100, the image processing apparatus 500 starts to execute the transmission job. The transmission job includes information (such as output size and resolution) used by the image processing apparatus 500 for printing and the image data.

[0098] In step S1401, the CPU 601 (controller unit 503) of the image processing apparatus 500 controls the AI-based image recognition unit 608 to determine whether the AI image data (AI-generated image) is included in the transmission job received by the network I / F 610 (whether the original image data of the transmission job includes an AI-generated image). Specifically, after performing the input processing of the image data of the input transmission job, the AI-based image recognition unit 608 refers to the metadata of the image data to be determined in the transmission job, and performs image analysis to determine whether the image data includes identification information indicating whether the image data is AI image data. When the image data includes the identification information, the AI-based image recognition unit 608 determines that the transmission job includes AI image data. When the image data does not include the identification information, the AI-based image recognition unit 608 determines that the transmission job does not include AI image data. Then, as the determination result in step S1401, when it is determined that the transmission job includes AI image data, the process proceeds to step S1402. At this time, the identification information is stored in the storage unit 603. On the other hand, as the determination result in step S1401, when it is determined that the transmission job does not include AI image data, the process proceeds to step S1403.

[0099] In step S1403, the CPU 601 executes the transmission job, that is, executes normal transmission processing assuming that the original document that does not include the AI-generated image is input. After performing step S1403, the process ends.

[0100] In step S1402, the CPU 601 notifies the user using the image processing apparatus 500 that the original document to be transmitted includes an AI-generated image, and notifies whether to perform restricted transmission or stop transmission.

[0101] Figure 15 FIG. is an example of a notification screen displayed on the image processing apparatus 500. Figure 15 The shown notification screen 1500 is displayed on the touch panel of the operation unit 607. The notification screen 1500 includes a preview image 1501, a preview image 1503, a stop transmission button 1505, and a perform restricted transmission button 1506. The preview image 1501 is an image representing the original document image as it is. The preview image 1501 includes a partial image 1502 that is an AI-generated image. The preview image 1503 is an image indicating the predicted print result when performing restricted transmission.

[0102] The preview image 1503 includes a partial image 1504 that is an AI-generated image. The partial image 1504 includes an image frame and the character "AI" superimposed on the partial image as information indicating that the partial image is an AI-generated image. This enables the user to understand that when printing the image data that is the restricted transmission object, a print of the same image as the preview image 1503 is provided.

[0103] In step S1404, the CPU 601 determines whether the restricted transmission button 1506 of the notification screen 1500 displayed in step S1402 has been operated. As a result of the determination in step S1404, when it is determined that the restricted transmission button 1506 has been operated, the process proceeds to step S1405. On the other hand, as a result of the determination in step S1404, when it is determined that the restricted transmission button 1506 has not been operated (i.e., the stop transmission button 1505 has been operated), the transmission process (output process) is not executed, and the process ends.

[0104] In step S1405, the CPU 601 controls the network I / F 610 to perform restricted transmission. Specifically, the CPU 601 refers to the identification information stored in the storage unit 603 and the output mode setting for restricted transmission. Then, the CPU 601 controls the image processing unit 609 to perform image processing on the AI-generated image portion included in the original image by adding processing according to the output mode setting. Thereafter, the CPU 601 transmits a transmission job including the image data on which the image processing has been performed to a device other than the image processing device 500 via the network I / F 610. After executing step S1405, the process ends.

[0105] Figure 16A and Figure 16B is a diagram showing an example of a printout printed by a printing device in the case where the device outside the image processing device is a printing device. In Figure 16A the printout 1600 shown, the partial image 1601 that is an AI-generated image has a very light color over the entire image and has, for example, marks such as stars superimposed on the image. With such image processing, the viewer of the printout 1600 can understand that the partial image 1601 is an AI-generated image. In Figure 16B the printout 1602 shown, the partial image 1603 that is an AI-generated image is an image filled with black. With such image processing, the viewer of the printout 1602 can understand that the printout 1602 is a printout including an AI-generated image.

[0106] Figure 17 is a diagram showing an example of an operation screen displayed on the image processing device during restricted transmission. It is displayed on the touch panel of the operation unit 607Figure 17 The operation screen 1700 shown. The operation screen 1700 includes preview images 1701 to 1704, radio buttons 1705 to 1707, and an input button 1708.

[0107] The preview image 1701 is an image representing the original manuscript image as it is. The preview image 1701 includes a partial image 1709 which is an AI-generated image. The preview image 1702 is an image indicating the prediction of the print result at the destination in the case of restricted transmission (first output mode). The preview image 1702 includes a partial image 1710 which is an AI-generated image. The partial image 1710 is similar to the partial image 1504. The preview image 1703 is an image indicating the prediction of the print result at the destination in the case of restricted transmission (second output mode). The preview image 1703 includes a partial image 1711 which is an AI-generated image. The partial image 1711 is similar to the partial image 1601. The preview image 1704 is an image indicating the prediction of the print result at the destination in the case of restricted transmission (third output mode). The preview image 1704 includes a partial image 1712 which is an AI-generated image. The partial image 1712 is similar to the partial image 1603.

[0108] When the user expects to obtain a printout with the same image as the preview image 1702, the user selects the radio button 1705. When the user expects to obtain a printout with the same image as the preview image 1703, the user selects the radio button 1706. When the user expects to obtain a printout with the same image as the preview image 1704, the user selects the radio button 1707. The user is allowed to select one of the radio buttons 1705 to 1707 and then operate the input button 1708. Therefore, in the case of restricted transmission, a printout of the preview image corresponding to the radio button selected in the operation screen 1700 is obtained at the destination.

[0109] As described above, in the image processing apparatus 500 according to the present embodiment, input processing of input image data is performed, and then it is determined by the AI-based image recognition unit 608 whether the input image data is generated by AI-based image generation. The result of the determination by the AI-based image recognition unit 608 may be a case where it is determined that the image data to be the transmission processing (output processing) object is generated by AI-based image generation, or a case where it is determined that the image data is not generated by AI-based image generation. When it is necessary or requested to determine whether the image data to be output is AI image data, the image processing apparatus 500 can change the transmission processing according to the determination result. Hereinafter, similar to the first embodiment, a case where the image data to be the transmission processing object is determined to be generated by AI-based image generation is referred to as "the first case", and a case where the image data to be the transmission processing object is determined not to be generated by AI-based image generation is referred to as "the second case".

[0110] In the first case, the CPU 601 (controller unit 503) of the image processing apparatus 500 adds processing to the image data to be the transmission processing object, and outputs the processed image data via the network I / F 610, thereby performing the transmission processing. When adding processing to the image data, the CPU 601 processes the image data so that the following can be visualized: the printed matter obtained by the printing process at the transmission destination is the printed matter of the image data including the identification information. In other words, the CPU 601 processes the image data so that the following can be visualized: when the image data transmitted by the transmission processing is printed, in the printed matter of the image data, the image data includes the identification information. Therefore, at the transmission destination, a printed matter is obtained that prints, for example, the same image as the preview image 1503 (see Figure 15 ) or the printed matter 1600 (see Figure 16A ). A partial image 1504 of the preview image 1503 includes an image frame and the character "AI" superimposed on the partial image. A partial image 1601 of the printed matter 1600 has a very light color applied over the entire image, and has a mark such as a star superimposed on the image. Alternatively, the transmission processing may not be performed on the AI image portion. Therefore, a printed matter is obtained that prints, for example, the same image as the printed matter 1602 (see Figure 16B ). A partial image 1603 of the printed matter 1602 is an image filled with black. These types of printed matters enable the viewer to know that the printed matter includes an AI image. For example, this gives the viewer of the printed matter a way to suspect that the AI image may be a fake image.

[0111] It should be noted that in the first case, the image processing apparatus 500 may be configured not to perform the restricted transmission processing (output processing).

[0112] On the other hand, in the second case, the CPU 601 of the image processing apparatus 500 performs a transmission process on the image data to be transmitted as it is via the network I / F 609, without adding processing to the image data that is the object of the transmission process via the image processing unit 209.

[0113] In the present embodiment, the image processing system 10 has a configuration in which the image processing apparatus 500 includes an AI-based image recognition unit 608. However, it may have a configuration in which, for example, a terminal AI-based image recognition unit having a function similar to that of the AI-based image recognition unit 608 is included in the general-purpose terminal 102. Here, a description will be mainly given of the differences from the above configuration in which the image processing apparatus 500 includes the AI-based image recognition unit 608, and similar matters will be omitted from the description.

[0114] Figure 18 It is a flowchart showing the processing (transmission job processing for restricted transmission of AI-generated images) executed by the general-purpose terminal. As Figure 18 shown, in step S1801, the CPU 301 of the general-purpose terminal 102 controls the terminal AI-based image recognition unit to determine whether the transmission job includes AI image data (AI-generated image) (whether the original image data of the transmission job includes an AI-generated image). As a result of the determination in step S1801, if it is determined that the transmission job includes AI image data, the process proceeds to step S1802. On the other hand, as a result of the determination in step S1801, if it is determined that the transmission job does not include AI image data, the process proceeds to step S1803.

[0115] In step S1802, the CPU 301 transmits a transmission job including restricted transmission specification and the AI-based recognition result to, for example, the image processing apparatus 500 via the network I / F 209. As a result, restricted printing is performed by the image processing apparatus 500 (see step S805 of the flowchart shown in Figure 8 ). After executing step S1802, the process ends.

[0116] In step S1803, the CPU 301 transmits a transmission job including normal transmission specification and not including an AI-generated image to, for example, the image processing apparatus 500 via the network I / F 305. As a result, normal printing is performed by the image processing apparatus 500 (see step S803 of the flowchart shown in Figure 8 ). After executing step S1803, the process ends.

[0117] In this embodiment, the image processing system 10 is configured to include an image processing unit 609 in the image processing apparatus 500, but may be configured to include, for example, a terminal image processing unit having a function similar to that of the image processing unit 609 in the general-purpose terminal 102. Here, a description will be mainly given of the differences from the above configuration in which the image processing unit 609 is included in the image processing apparatus 500, and similar matters will be omitted from the description.

[0118] Figure 19 It is a flowchart showing a modified example of the process (transmission job process for restricted transmission of AI-generated images) executed by the general-purpose terminal 102. As Figure 19 shown, in step S1901, the CPU 301 of the general-purpose terminal 102 controls the terminal AI-based image recognition unit to determine whether the transmission job includes AI image data (whether the original image data of the transmission job includes an AI-generated image). As a result of the determination in step S1901, if it is determined that the transmission job includes AI image data, the process proceeds to step S1902. At this time, the identification information is stored in the SSD 303. On the other hand, as a result of the determination in step S1901, if it is determined that the transmission job does not include AI image data, the process proceeds to step S1903.

[0119] In step S1903, the CPU 301 transmits a transmission job including a normal transmission designation and not including an AI-generated image to, for example, the image processing apparatus 500 via the network I / F 305. In this case, the processing of the AI-generated image of the print job by the terminal image processing unit is omitted. Therefore, normal printing is performed by the image processing apparatus 500. After executing step S1903, the process ends.

[0120] In step S1902, the CPU 301 notifies the user using the general-purpose terminal 102 that the original document to be transmitted includes an AI-generated image, and notifies whether to perform restricted transmission or stop transmission. This notification is made by displaying Figure 15 the notification screen 900 shown on the touch panel of the user I / F 304 or the like.

[0121] In step S1904, the CPU 301 determines whether the restricted transmission button 1506 has been operated on the notification screen 900 displayed in step S1902. As a result of the determination in step S1904, if it is determined that the restricted transmission button 1506 has been operated, the process proceeds to step S1905. On the other hand, as a result of the determination in step S1904, if it is determined that the restricted transmission button 1506 has not been operated, that is, if the stop transmission button 1505 has been operated, the transmission process (output process) is not executed, and the process ends.

[0122] In step S1905, the CPU 301 refers to the recognition information stored in the SSD 303 and the output mode setting for restricted transmission. Then, the CPU 301 controls the terminal image processing unit to perform image processing on the AI-generated image portion included in the original image by adding processing according to the output mode setting. Thereafter, the CPU 301 sends a transmission job including the image data on which the image processing has been performed to, for example, the image processing device 500 via the network I / F 305. Accordingly, using the image processing device 500, a printed matter on which the same image as the preview image 1503 is printed is obtained. After step S1905 is executed, the processing ends.

[0123] Other embodiments

[0124] The (multiple) embodiments of the present invention can also be implemented as follows: a computer of a system or device that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a "non-transitory computer-readable storage medium") to perform one or more functions of the above (multiple) embodiments, and / or the system or device includes one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more functions of the above (multiple) embodiments; and a method performed by a computer provided in the system or device, e.g., reading and executing computer-executable instructions from a storage medium to perform one or more functions of the above (multiple) embodiments, and / or controlling the one or more circuits to perform one or more functions of the above (multiple) embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, one or more of a hard disk, a random access memory (RAM), a read-only memory (ROM), a memory of a distributed computing system, an optical disc (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD) TM ), a flash device, and a memory card, etc.

[0125] In addition, in the above-described embodiments, the image data to be processed as output is sent from the AI-based image generation server 101 to the image processing apparatus 500 via the general-purpose terminal 102, and then undergoes input processing through the network I / F 610 and the CPU 601 of the image processing apparatus 500, but is not limited thereto. For example, the image data to be processed as output may be read by the image reading unit 502 of the image processing apparatus 500 as input processing. In this case, it is preferable to determine whether the image data is AI image by, for example, analysis or the like.

[0126] Embodiments of the present invention can also be implemented by the following method, that is, software (including a computer program product including computer programs / instructions) that executes the functions of the above-described embodiments is provided to a system or apparatus via a network or various storage media, and the computer (central processing unit (CPU), microprocessing unit (MPU)) of the system or apparatus reads and executes the computer programs / instructions.

[0127] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation so as to cover all such variations and equivalent structures and functions.

[0128] This application claims the priority of Japanese Patent Application No. 2024-006580 filed on January 19, 2024, the entire content of which is incorporated herein by reference.

Claims

1. An image processing apparatus, comprising: an input unit configured to perform input processing of input image data; a determination unit configured to determine whether the image data input through the input processing is generated based on a learning model; and an output unit configured to when the determination unit determines that the image data is generated based on the learning model, not perform output processing of outputting the image data, and when the determination unit determines that the image data is not generated based on the learning model, perform the output processing.

2. The image processing apparatus according to claim 1, wherein, The output processing is printing processing for printing the image data.

3. The image processing apparatus according to claim 1, wherein, The output processing is transmission processing for transmitting the image data to a device different from the image processing apparatus.

4. The image processing apparatus according to claim 1, Among them, wherein the image processing apparatus is communicably connected to an external device configured to transmit the image data to the image processing apparatus, and the output unit performs the output processing on the image data input from the external device through the input unit.

5. An image processing apparatus, comprising: an input unit configured to perform input processing of input image data; a determination unit configured to determine whether the image data input through the input processing is generated based on a learning model; a processing unit configured to perform processing on the image data when it is determined that the image data is generated based on the learning model; and an output unit configured to perform output processing of outputting the image data on which the processing has been performed.

6. The image processing apparatus according to claim 5, wherein, The output processing is printing processing for printing the image data.

7. The image processing apparatus according to claim 6, Among them, wherein when the image data includes identification information for identifying the image data as being generated based on the learning model, the processing unit performs processing on the image data to visualize that the image data includes the identification information in a printed matter of the image data obtained through the printing processing.

8. The image processing apparatus according to claim 5, wherein, The output processing is transmission processing for transmitting the image data to a device different from the image processing apparatus.

9. The image processing apparatus according to claim 8, Among them, wherein when the image data includes identification information for identifying the image data as being generated based on the learning model, the processing unit performs processing on the image data to visualize that when the image data transmitted through the transmission processing is printed, the image data includes the identification information in a printed matter of the image data.

10. The image processing apparatus according to claim 5, Among them, wherein when the determination unit determines that the image data is not generated based on the learning model, the output unit outputs the image data, and the processing unit does not perform processing on the image data.

11. The image processing apparatus according to claim 5, Among them, wherein the image processing apparatus is communicably connected to an external device configured to transmit the image data to the image processing apparatus, and the output unit performs the output processing on the image data input from the external device through the input unit.

12. The image processing apparatus according to claim 1 or 5, Among them, wherein the determination unit makes the determination based on whether the image data includes identification information for identifying that the image data is generated based on the learning model.

13. The image processing apparatus according to claim 12, Among them, wherein the determination unit determines that the image data is generated based on the learning model when the image data includes the identification information, and determines that the image data is not generated based on the learning model when the image data does not include the identification information.

14. The image processing apparatus according to claim 12, Among them, wherein the identification information is digitized, and the determination unit determines that the image data is generated based on the learning model when the value of the identification information is equal to or greater than a threshold, and determines that the image data is not generated based on the learning model when the value of the identification information is less than the threshold.

15. A control method for an image processing apparatus, the control method comprising the following steps: Performing an input process for inputting image data; Determining whether the image data input through the input process is generated based on a learning model; When it is determined that the image data is generated based on the learning model, not performing an output process for outputting the image data; and When it is determined that the image data is not generated based on the learning model, performing the output process.

16. A control method for an image processing apparatus, the control method comprising the following steps: Performing an input process for inputting image data; Determining whether the image data input through the input process is generated based on a learning model; and When it is determined that the image data is generated based on the learning model, performing a process on the image data and performing an output process for outputting the image data on which the process has been performed.

17. A computer-readable non-transitory storage medium storing a program for causing a computer to execute a control method for an image processing apparatus, the control method comprising the following steps: Performing an input process for inputting image data; Determining whether the image data input through the input process is generated based on a learning model; When it is determined that the image data is generated based on the learning model, not performing an output process for outputting the image data; and When it is determined that the image data is not generated based on the learning model, performing the output process.

18. A computer-readable non-transitory storage medium storing a program for causing a computer to execute a control method for an image processing apparatus, the control method comprising the following steps: Performing an input process for inputting image data; Determining whether the image data input through the input process is generated based on a learning model; and When it is determined that the image data is generated based on the learning model, performing a process on the image data and performing an output process for outputting the image data on which the process has been performed.

Images