Smartphone, program, storage medium, and method

Abstract

To provide a mechanism capable of giving a suitable instruction of image generation to a generative AI.SOLUTION: An image processing device 1 performs communication with a generative AI server 10 for generating AI image data. The image processing device 1 comprises a scanner device for reading an original. The image processing device 1 scans an original of a rough image obtained by roughly handwriting an object by a user and generates scan image data of the original. The image processing device 1 causes the generative AI server 10 to generate AI image data based on the scan image data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a control method for the image processing apparatus, and a program.

Background Art

[0002] An AIGC (AI-Generated Content) system that generates an image using a keyword input by a user is known. In the AIGC system, as a keyword, for example, a prompt consisting of a string of natural language is input (see Patent Document 1). The AIGC system generates an image including an object corresponding to the prompt input by the user. Thereby, the user can obtain an image including an object corresponding to this prompt, such as a person or a car, simply by inputting the prompt to the AIGC system.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the configuration that uses the above-described prompt as an instruction for image generation, the user may not be able to appropriately give instructions regarding the composition, such as the direction in which a person is facing, or instructions regarding the background.

[0005] An object of the present invention is to provide a mechanism capable of appropriately giving an instruction for image generation to a generative AI.

Means for Solving the Problems

[0006] In order to achieve the above object, the image processing apparatus of the present invention is characterized by comprising: reading means for reading a document; means for causing a generation AI to generate second image data based on first image data obtained by reading the document by the reading means; and means for receiving the second image data generated by the generation AI.

Effect of the Invention

[0007] According to the present invention, an instruction for image generation can be appropriately given to the generation AI.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention according to the claims, and not all combinations of the features described in this embodiment are essential for the solution means of the present invention.

[0010] FIG. 1 is a configuration diagram schematically showing the configuration of an AIGC system (AI-Generated Content: generative artificial intelligence) including an image processing apparatus 1 according to this embodiment. AI is an abbreviation for Artificial Intelligence. In FIG. 1, this AIGC system is composed of an image processing apparatus 1, a computer 9, and a generative AI server 10. The image processing apparatus 1, the computer 9, and the generative AI server 10 can communicate with each other via a LAN / Internet 8. In this AIGC system, the image processing apparatus 1 reads a rough sketch manuscript in which a user roughly hand-draws an object and generates digital image data of this manuscript (hereinafter referred to as "scanned image data"). The image processing apparatus 1 generates an image generation request based on this scanned image data and transmits it to the generative AI server 10. The generative AI server 10 includes a generative AI that generates colored realistic image data (hereinafter referred to as "AI image data") according to the received image generation request. The generative AI server 10 transmits the generated AI image data to a transmission destination specified by the image generation request, such as the image processing apparatus 1 or the computer 9.

[0011] Next, the configuration of the image processing apparatus 1 will be described. As shown in FIG. 1, the image processing apparatus 1 includes a scanner device 2, a controller 3, a printer device 4, an operation unit 5, a storage device 6, and a FAX device 7. The controller 3 is connected to the scanner device 2, the printer device 4, the operation unit 5, the storage device 6, and the FAX device 7, respectively.

[0012] The scanner device 2 includes a document feeder unit 21 capable of automatically and sequentially replacing a stack of documents, and a scanner unit 22 capable of optically scanning a document and converting it into scanned image data. The scanner device 2 optically reads an image from the document, converts it into scanned image data, and transmits this scanned image data to the controller 3.

[0013] The controller 3 executes a job by issuing instructions to the connected modules. The printer device 4 prints image data on paper. The printer device 4 includes a paper feeder unit 42 capable of sequentially feeding sheets one by one from a stack of papers, a marking unit 41 for printing image data on the fed paper, and a paper discharge unit 43 for discharging the printed paper.

[0014] The operation unit 5 receives various instructions from the user and also displays various information regarding the image processing device 1. The storage device 6 stores image data, control programs, etc. The FAX device 7 transmits scanned image data, etc. to an external device via a telephone line or the like.

[0015] The image processing device 1 performs transmission and reception of image data with the computer 9 via, for example, the LAN / Internet 8. Also, the image processing device 1 receives an instruction for issuing a job transmitted from the computer 9 via the LAN / Internet 8.

[0016] Also, the computer 9 controls the operation of the image processing device 1 via the LAN / Internet 8. For example, the computer 9 outputs a power-off instruction to the controller 3 of the image processing device 1 via the LAN / Internet 8. The controller 3 controls the power-off sequence of the image processing device 1 according to the received power-off instruction.

[0017] The image processing apparatus 1 has a plurality of functions such as a copying function, an image transmission function, an image storage function, and an image printing function. The copying function is a function of recording, by the storage device 6, the scan image data generated by optically scanning a document by the scanner device 2, and printing this scan image data by the printer device 4. The image transmission function is a function of transmitting, via the LAN / Internet 8, the scan image data generated by optically scanning a document by the scanner device 2 to an external device such as a computer 9. The image storage function is a function of recording, by the storage device 6, the scan image data generated by optically scanning a document by the scanner device 2, and transmitting and printing this scan image data as necessary. The image printing function is a function of analyzing the PDL data transmitted from the computer 9 and causing the printer device 4 to execute a printing process.

[0018] Next, the configuration of the controller 3 of the image processing apparatus 1 will be described. FIG. 2 is a block diagram schematically showing the configuration of the controller 3 in FIG. 1. In FIG. 2, the controller 3 is composed of a main system 200 and a subsystem 220.

[0019] The main system 200 is connected to a USB memory 209, an operation unit 5, a storage device 6, etc. The main system 200 is a so-called general-purpose CPU system. The main system 200 includes a main CPU 201, a boot ROM 202, a memory 203, a bus controller 204, a non-volatile memory 205, and a disk controller 206. The main system 200 further includes a flash disk 207, a USB controller 208, a network interface 210, and an RTC 211.

[0020] The main CPU 201 controls the entire main system 200. The boot ROM 202 stores the boot program. The memory 203 is used as the work memory of the main CPU 201. The bus controller 204 has a bridge function with the external bus. The non-volatile memory 205 is a storage device capable of retaining data even when the main system 200 is powered off. The disk controller 206 controls storage devices such as the flash disk 207 and the storage device 6. The flash disk 207 is a relatively small-capacity non-volatile storage device composed of semiconductor devices, for example, an SSD. The USB controller 208 controls the USB devices connected to the image processing device 1. For example, the USB controller 208 performs a process of saving image data to the USB memory 209 connected to the image processing device 1 and a process of reading out the image data stored in the USB memory 209. The network interface 210 performs data communication with external devices such as the computer 9 and the generative AI server 10 via the LAN / Internet 8. The RTC 211 has a clock function.

[0021] To the subsystem 220, a printer device 4, a scanner device 2, a FAX device 7, etc. are connected. The subsystem 220 is composed of a relatively small general-purpose sub-CPU system and image processing hardware. The subsystem 220 includes a sub-CPU 221, a memory 223, a bus controller 224, a non-volatile memory 225, an image processing processor 226, a printer controller 227, and a scanner controller 228.

[0022] The sub-CPU 221 controls the entire subsystem 220. Also, the sub-CPU 221 controls the FAX device 7. The memory 223 is used as the work memory of the sub-CPU 221. The bus controller 224 has a bridge function with an external bus. The non-volatile memory 225 is a storage device capable of retaining data even when the subsystem 220 is powered off. The image processing processor 226 performs real-time digital image processing. The printer controller 227 controls the printing process by the printer device 4. For example, the printer controller 227 sends the image data to be printed to the printer device 4. The scanner controller 228 controls the scanning process by the scanner device 2. For example, the scanner controller 228 gives an instruction to execute the scanning process to the scanner device 2, and acquires the scanned image data generated by this scanning process from the scanner device 2.

[0023] Here, the operation of the controller 3 will be described by taking the copying function as an example. When the user gives an instruction for image copying from the operation unit 5, the main CPU 201 sends an image reading command to the scanner device 2 via the sub-CPU 221. The scanner device 2 optically scans the set document and converts it into scanned image data, and sends this scanned image data to the image processing processor 226 via the printer controller 227. The image processing processor 226 performs DMA transfer to the memory 223 via the sub-CPU 221 to temporarily store this scanned image data.

[0024] When the main CPU 201 confirms that a certain amount or all of the scanned image data has been stored in the memory 223, it issues an image output instruction to the printer device 4 via the sub CPU 221. The sub CPU 221 conveys the storage area of the scanned image data in the memory 223 to the image processing processor 226. The scanned image data stored in the memory 223 is transmitted to the printer device 4 via the image processing processor 226 and the printer controller 227 according to the synchronization signal output by the printer device 4. The printer device 4 prints the received scanned image data on paper.

[0025] When printing multiple copies, the main CPU 201 saves the scanned image data stored in the memory 223 in the storage device 6. By saving the scanned image data in the storage device 6 in this way, it becomes possible to send the scanned image data for the second and subsequent copies to the printer device 4 without having to acquire it again from the scanner device 2.

[0026] Figure 3 is a block diagram schematically showing the configuration of the generation AI server 10 in Figure 1. In Figure 3, the generation AI server 10 includes an AIGC front-end server 101, an AIGC back-end server 102, and a learning database 103.

[0027] The AIGC front-end server 101 requests the AIGC back-end server 102 to generate an image based on an image generation request received from an external device such as a computer 9 or an image processing device 1. The AIGC back-end server 102 performs image generation processing using the trained model stored in the learning database 103 to generate AI image data. Further, the AIGC back-end server 102 transmits the generated AI image data to the destination (such as the image processing device 1 or the computer 9) specified in the image generation request via the front-end server 101. For example, when the image data is transmitted to the image processing device 1, the image processing device 1 prints the received image data on paper as shown in A of FIG. 4. Further, the image processing device 1 transfers the received image data to other devices such as the computer 9 or the smartphone 12 shown in FIG. 4 as shown in B of FIG. 4. The learning database 103 stores a trained model trained to output a realistic colored image from a line drawing.

[0028] Next, the configuration of the UI of the image processing device 1 will be described. FIG. 5 is a diagram showing an example of the UI screen 500 displayed on the operation unit 5 of FIG. 1. On the UI screen 500, a plurality of operation buttons corresponding to functions available on the image processing device 1, for example, a "copy" button 501, a "scan and send" button 502, an "AIGC use" button 503, and a "device setting" button 504 are displayed. Note that the configuration of the UI screen 500 is an example, and the UI screen 500 may include operation buttons other than these operation buttons.

[0029] The "Copy" button 501 is an operation button for using the copying function of the image processing apparatus 1. The "Scan and Send" button 502 is an operation button for using the image transmission function of the image processing apparatus 1. The "Device Settings" button 504 is an operation button for performing various settings of the image processing apparatus 1. The "AIGC Use" button 503 is an operation button for using the AIGC system in the present embodiment. When the user selects the "AIGC Use" button 503, an AIGC app (not shown) installed in the image processing apparatus 1 is launched, and the setting screen 600 of FIG. 6 is displayed on the operation unit 5. The setting screen 600 includes operation buttons 601 to 605.

[0030] The operation button 601 is a button for setting whether to use only an image or both an image and characters as the input of the AIGC system. For example, when it is set that "only an image is used as the input of the AIGC system", the image processing apparatus 1 generates intermediate data based on the objects identified from the generated scanned image data, and transmits this intermediate data together with an image generation request to the generation AI server 10. The intermediate data is, for example, a prompt consisting of a natural language string representing the features of a rough sketch. The prompt includes, for example, a string indicating the type of object included in the scanned image data ("person", "cat", etc.), a string indicating the position of the object included in the scanned image data ("center", etc.), and a string indicating the style of the AI image data ("comic style", etc.). Note that the intermediate data is not limited to the prompt, and may be a command interpretable by the generation AI server 10 as an instruction to generate colored and realistic AI image data from this rough sketch and including the feature information of the objects included in the scanned image data.

[0031] When it is set that "images and characters are used as inputs to the AIGC system", the image processing apparatus 1 generates intermediate data based on the objects identified from the generated scanned image data and the character information. Further, the image processing apparatus 1 transmits this intermediate data together with an image generation request to the generation AI server 10. For example, when the character string "comic style" is described beside an object in the scanned image data, the image processing apparatus 1 generates intermediate data including a natural language character string representing the characteristics of this object and the character string "comic style".

[0032] The operation button 602 is a button for setting object correction. When the user effectively sets the object correction setting, in the image processing apparatus 1, when the rough drawing is ambiguous and the characteristic information of the object cannot be narrowed down, the user is made to select the characteristic information of the object from among a plurality of assumed candidates.

[0033] Here, the case where it is set in the setting of the operation button 601 that "only images are used as inputs to the AIGC system" will be described. In this case, as shown in FIG. 7(a), when the rough drawing manuscript 701 depicting a background and a person is set in the image processing apparatus 1, the image processing apparatus 1 performs identification processing of the objects included in the scanned image data of this manuscript. The image processing apparatus 1 can identify that the object included in this scanned image data is "human", but since the rough drawing is ambiguous, it cannot identify whether this object is "male", "female", or "child". Therefore, the image processing apparatus 1 displays a selection screen 702 for allowing the user to select the characteristic information of the object from among a plurality of assumed candidates on the operation unit 5. The image processing apparatus 1 generates intermediate data based on the characteristic information selected on the selection screen 702.

[0034] Also, when it is set in the operation button 601 to "use images and text as input for the AIGC system", a selection screen is similarly displayed. For example, as shown in FIG. 7(b), when a rough draft 703 of a sketch with the character string "comic style" drawn beside a person is set in the image processing apparatus 1, the image processing apparatus 1 performs identification processing on the objects included in the scanned image data of this draft. The image processing apparatus 1 can identify that the object included in this scanned image data is a "human", but since the rough draft is ambiguous, it cannot identify which of "male", "female", or "child" this object is. Therefore, the image processing apparatus 1 displays a selection screen 704 on the operation unit 5 for allowing the user to select the feature information of the object from among a plurality of assumed candidates. The image processing apparatus 1 generates intermediate data based on the feature information selected on the selection screen 704.

[0035] The operation button 603 is a button for reading a rough draft of a sketch and giving an instruction to generate scanned image data of the draft.

[0036] The operation button 604 is a button for setting the settings regarding the output. In the present embodiment, for example, it is possible to print the AI image data generated by the generation AI server 10, transfer this AI image data to a computer 9 or the like operated by the user, display this AI image data on a computer 9 or the like, and set whether to reuse the prompt which is intermediate data.

[0037] The operation button 605 is a button for setting the print settings. When it is set in the operation button 604 to "print the AI image data", the print settings used in this print process are set. For example, it is possible to set to output in a poster format or a booklet.

[0038] FIG. 8 is a flowchart showing the procedure of the generation control process of AI image data executed by the image processing apparatus 1 of FIG. 1. The generation control process of AI image data is realized by the controller 3 executing a program stored in the memory 203, the memory 223, etc. The generation control process of AI image data is executed when the user sets a rough sketch manuscript and selects the operation button 603.

[0039] In FIG. 8, first, the controller 3 scans the set rough sketch manuscript (S801). As a result, scan image data of this manuscript is generated. Next, the controller 3 performs object identification processing on the generated scan image data. As a result, objects such as people included in the scan image data are identified. Also, when it is set in the setting of the operation button 601 to "use images and characters as input to the AIGC system", the character information included in the scan image data is also identified.

[0040] Next, the controller 3 determines whether or not the object correction setting is set to be valid (S802).

[0041] If it is determined at S802 that the setting of object correction is not set to be effective, this process proceeds to S804 described later. If it is determined at S802 that the setting of object correction is set to be effective, the controller 3 causes the operation unit 5 to display a selection screen for allowing the user to select the feature information of the object based on the result identified in the identification process (S803). Here, as an example, as shown in FIG. 9, a case where a rough draft 901 of a drawing in which a background and a person are drawn and a character string "comic style" is drawn beside the person is set will be described. In the identification process of the object included in the scanned image data of the rough draft 901, the controller 3 can identify that the background and the person are drawn, but since the rough draft is ambiguous, it cannot identify whether the person is "male", "female", or "child". Therefore, the controller 3 causes the operation unit 5 to display a selection screen 902 for allowing the user to select the feature information of the person from among a plurality of candidates. When the user selects one from among the plurality of candidates included in the selection screen 902, this process proceeds to S804.

[0042] At S804, the controller 3 generates intermediate data for causing the generation AI server 10 to generate an AI image. For example, when the setting of object correction is not set to be effective, the controller 3 generates intermediate data based on the feature information of the object identified in the identification process. Note that, as described above, the intermediate data is a prompt consisting of a natural language character string representing the features of the identified object. On the other hand, when the setting of object correction is set to be effective, the controller 3 generates intermediate data 904 based on the feature information 903 selected on the selection screen 902.

[0043] Next, the controller 3 causes the operation unit 5 to display the generated intermediate data and confirms with the user whether to perform image generation using this intermediate data (S805).

[0044] In S805, when receiving an instruction from the user to generate an image using this intermediate data, the controller 3 transmits the intermediate data generated in S804 and the image generation request to the generation AI server 10 (S806). The generation AI server 10 performs an image generation process using the received intermediate data as input. As a result, for example, AI image data 905 is generated by drawing the rough sketch depicted on the manuscript 901 realistically and in a comic style. The generation AI server 10 transmits the generated AI image data 905 to the transmission destination specified in the image generation request, for example, to the image processing device 1.

[0045] Next, the controller 3 receives the AI image data 905 from the generation AI server 10 (S807). The controller 3 prints the received AI image data 905 on paper, for example, as shown in A of FIG. 4 described above, or transmits the received AI image data 905 to an external device such as the computer 9 or the smartphone 12, as shown in B of FIG. 4 described above. Then, this process ends.

[0046] In S805, when receiving an instruction from the user not to generate an image using this intermediate data, the controller 3 corrects the intermediate data according to the correction instruction received from the user (S808). Next, in S806, the controller 3 transmits the corrected intermediate data and the image generation request to the generation AI server 10. Then, S807 described above is performed, and this process ends.

[0047] According to the above-described embodiment, the generation AI server 10 is caused to generate AI image data based on the scanned image data obtained by scanning the manuscript. As a result, information regarding the composition and background obtained from the scanned image data can be included in the instruction for image generation to the generation AI server 10, and thus, the instruction for image generation to the generation AI server 10 can be appropriately performed.

[0048] In addition, in the above-described embodiments, the manuscript includes objects drawn by the user by hand. As a result, the user can appropriately give instructions regarding composition and background to the generation AI server 10 simply by preparing a manuscript in which objects are drawn by hand.

[0049] In addition, in the above-described embodiments, intermediate data including feature information of an object identified from the scanned image data is generated, and this intermediate data is transmitted to the generation AI server 10. As a result, the feature information of the object identified from the scanned image data can be conveyed to the generation AI server 10.

[0050] In addition, in the above-described embodiments, a selection screen for allowing the user to select information to be included in the intermediate data from among a plurality of candidates that are the feature information of the identified object is displayed on the operation unit 5. As a result, the intention of the user can be reflected in the information to be included in the intermediate data.

[0051] In addition, in the above-described embodiments, a character string indicating the features of the object is drawn beside the object in the manuscript, and the intermediate data further includes the character string identified from the scanned image data. As a result, the user can combine a rough sketch and characters to give an instruction for image generation to the generation AI server 10.

[0052] In addition, in the above-described embodiments, the user is allowed to edit the intermediate data. As a result, an instruction that more reflects the intention of the user can be given to the generation AI server 10.

[0053] In addition, in the above-described embodiments, the received AI image data is printed. As a result, a printed matter of the AI image data generated by the generation AI server 10 can be obtained.

[0054] In the above-described embodiment, the generative AI is provided in the generative AI server 10, which is an external device different from the image processing apparatus 1. As a result, an instruction for image generation can be appropriately given to the generative AI server 10, which is an external device.

[0055] Note that in this embodiment, the image processing apparatus 1 is not limited to a device having a scanning function. For example, it may be a device such as a smartphone, a tablet terminal, or a PC having a photographing function. The control unit of the device having a photographing function, or an application installed in this device, performs a process of photographing a rough sketch manuscript to generate photographed image data of this manuscript, and performs the processes of S802 to S808 described above using the generated photographed image data. Even in such a device having a photographing function, an instruction for image generation can be appropriately given to the generative AI server 10.

[0056] Also, in this embodiment, the image processing apparatus 1 may be configured to include a generative AI. As a result, in the image processing apparatus 1 including the generative AI, an instruction for image generation can be appropriately given to this generative AI.

[0057] Also, in this embodiment, the process of recognizing an object from the scanned image data or the photographed image data may be performed using the AI included in the image processing apparatus 1. Many current CPUs often have a dedicated circuit for object recognition, and by using this circuit in combination, the process of recognizing an object can be accurately performed while minimizing the load of the process of recognizing an object.

[0058] Also, in this embodiment, the image processing apparatus 1 may cause the generative AI server 10 to generate AI image data corresponding to the print settings. For example, when a 2in1 setting is made in the print settings, the image processing apparatus 1 generates intermediate data for causing the generative AI server 10 to generate one AI image data to which two pieces of image data are assigned. As a result, an instruction to generate AI image data corresponding to the print settings can be given to the generative AI server 10.

[0059] Further, in this embodiment, when the generation AI server 10 is configured to be able to generate AI image data with rough drawing image data as input, instead of intermediate data, scan image data or photographed image data may be transmitted to the generation AI server 10 together with an image generation request. Thereby, the image processing apparatus 1 can appropriately instruct the generation AI server 10 to generate an image without allocating its resources to the generation of intermediate data.

[0060] In addition, as use cases of the technology according to this embodiment, for example, creation of a flowchart, creation of a poster (for example, sketching a person with a line and showing their pose in a rough drawing, and writing a string such as "comic style" beside it to indicate the style), etc. can be mentioned. Also, creation of a slide prototype, New Year's card, letter, flyer, fan-made comic for personal enjoyment, etc. can be mentioned.

[0061] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus reading and executing the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0062] Note that the disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An image processing apparatus comprising: reading means for reading a manuscript; means for causing a generation AI to generate second image data based on first image data obtained by reading the manuscript by the reading means; and means for receiving the second image data generated by the generation AI. (Configuration 2) An image processing apparatus comprising: photographing means for photographing a manuscript; means for causing a generation AI to generate second image data based on first image data obtained by photographing the manuscript by the photographing means; and means for receiving the second image data generated by the generation AI. Configuration 3: The image processing apparatus according to Configuration 1 or 2, characterized in that the original document includes an object drawn by the user by hand. Configuration 4: The image processing apparatus according to Configuration 3, characterized in that the generating means identifies the object from the first image data, generates intermediate data including feature information of the identified object, and transmits the intermediate data to the generating AI. Configuration 5: The image processing apparatus according to Configuration 4, further comprising means for allowing the user to select the feature information to be included in the intermediate data from among a plurality of candidates for the feature information of the identified object. Configuration 6: The image processing apparatus according to Configuration 4 or 5, characterized in that a character string indicating the feature of the object is drawn beside the object in the original document, and the intermediate data further includes the character string identified from the first image data. Configuration 7: The image processing apparatus according to any one of Configurations 4 to 6, further comprising means for allowing the user to edit the intermediate data. Configuration 8: The image processing apparatus according to any one of Configurations 1 to 3, characterized in that the generating means transmits the first image data to the generating AI. Configuration 9: The image processing apparatus according to any one of Configurations 1 to 8, further comprising means for printing the received second image data. Configuration 10: The image processing apparatus according to Configuration 9, characterized in that the generating means causes the generating AI to generate second image data corresponding to the print settings set by the user. Configuration 11: The image processing apparatus according to any one of Configurations 1 to 10, characterized in that the generating AI is provided in an external device different from the image processing apparatus. Configuration 12: The image processing apparatus according to any one of Configurations 1 to 10, further comprising the generating AI. (Control Method 13) A control method for an image processing apparatus, comprising: a reading step of reading a document; a step of causing a generation AI to generate second image data based on first image data obtained by reading the document in the reading step; and a step of receiving the second image data generated by the generation AI. (Control Method 14) A control method for an image processing apparatus, comprising: a photographing step of photographing a document; a step of causing a generation AI to generate second image data based on first image data obtained by photographing the document in the photographing step; and a step of receiving the second image data generated by the generation AI. (Program) A program for causing a computer to execute the control method for an image processing apparatus according to Control Method 13 or 14.

Explanation of Reference Numerals

[0063] 1 Image processing apparatus 2 Scanner device 3 Controller 4 Printer device 5 Operation unit 10 Generation AI server 210 Network interface 702 Selection screen 704 Selection screen 902 Selection screen 905 AI image data

Claims

1. A smartphone, an input means for inputting a handwritten first image and a supplemental character string; a generating means for inputting the first image and the supplemental character string into a generating AI to generate a second image having the composition shown in the first image, the second image being based on the supplemental character string; A smartphone characterized by comprising an output means for outputting the second image generated by the generation AI.

2. The input means is capable of inputting a plurality of supplementary character strings, The smartphone described in claim 1, characterized in that the generation means inputs the first image and the multiple supplementary character strings into the generation AI, causing the generation AI to generate the second image having the composition shown in the first image, the second image being based on the multiple supplementary character strings.

3. The smartphone described in Claim 1, characterized in that the input means inputs the first image by photographing the first image.

4. A smartphone as described in claim 1, characterized in that the first image and the supplementary character string are displayed side by side.

5. A smartphone as described in claim 1, characterized in that a prompt is input to the generation AI to cause the generation AI to generate the second image.

6. The smartphone described in Claim 1, further characterized in that it has the generating AI.

7. The smartphone described in Claim 1, further characterized in that it has a printing means for printing the second image.

8. A smartphone as described in claim 1, characterized in that it further has a transmitting means for transmitting the second image.

9. A program comprising: On the computer, By inputting the input hand-drawn first image and the input supplemental character string into a generation AI, the generation AI generates a second image having the composition shown in the first image, the second image being based on the supplemental character string; A program characterized by outputting the second image generated by the generation AI.

10. The computer: The program described in claim 9, characterized in that by inputting the first image and a plurality of input supplementary character strings into the generation AI, the generation AI is caused to generate the second image having the composition shown in the first image, the second image being based on the plurality of supplementary character strings.

11. The program described in Claim 9, characterized in that the first image is input by being photographed.

12. The program described in Claim 9, characterized in that the first image and the supplementary character string are displayed side by side.

13. The computer:

10. The program of claim 9, further comprising inputting a prompt to the generating AI to cause the generating AI to generate the second image.

14. The program described in Claim 9, characterized in that a smartphone having the computer has the generating AI.

15. The program described in Claim 9, characterized in that it causes the computer to execute processing for printing the second image.

16. The program described in Claim 9, characterized in that it causes the computer to transmit the second image.

17. A storage medium storing a program described in any one of claims 9 to 16.

18. A method comprising: By inputting the input hand-drawn first image and the input supplemental character string into a generation AI, the generation AI generates a second image having the composition shown in the first image, the second image being based on the supplemental character string; A method comprising outputting the second image generated by the generating AI.

Images