Print image editing system

Abstract

Provided is a print image editing system that allows print images to be edited by inputting a message or an image, thereby improving convenience for users. A print image editing system 1 includes a terminal device 300 and a server 100. The terminal device 300 executes: a step S5 for displaying a print preview of a print image; a step S15 for accepting a modification instruction by means of a message or an image from a user with respect to the print preview displayed in step S5; and a step S85 for acquiring a print image in which a modification target has been modified on the basis of the modification instruction accepted in step S15. In step S5, the terminal device 300 displays a print preview of the print image after the modification target has been modified, acquired in step S85.

Description

Printed image editing system

[0001] The present disclosure relates to a printed image editing system for displaying and editing a print preview of a printed image.

[0002] Conventionally, as disclosed in, for example, Patent Document 1, there is known a technology of a system that enables editing to modify a part of a printed image on a print preview screen.

[0003] Japanese Unexamined Patent Publication No. 2017-45322

[0004] In the system described in Patent Document 1, when editing a printed image, an editing screen is displayed, and manual setting or selection is required for each of a plurality of items in the screen. Therefore, the operation is complicated and inconvenient for users.

[0005] An object of the present disclosure is to provide a printed image editing system that enables editing of a printed image by inputting a message or an image, and can improve user convenience.

[0006] In order to achieve the above object, the printed image editing system of the present disclosure is a printed image editing system comprising: a preview display unit that displays a print preview of a printed image; and a reception unit that receives a modification instruction in the form of a user's message or image for the print preview displayed by the preview display unit, wherein the modification instruction includes specification data specifying a modification target and modification content data indicating the content of modification to the modification target; and an editing processing unit that acquires the printed image in which the modification target is modified based on the modification instruction received by the reception unit, wherein the preview display unit displays a print preview of the printed image acquired by the editing processing unit after the modification target is modified.

[0007] In the present disclosure, a user can edit a printed image only by inputting a message or an image. Therefore, compared with the conventional method that requires setting or selection on an editing screen, the operation can be simplified and convenience can be improved.

[0008] According to this disclosure, it is possible to enable editing of printed images by inputting messages or images, thereby improving user convenience.

[0009] This figure shows an example of the configuration of a print image editing system according to the embodiment. This figure shows an example of the server configuration. This figure shows an example of the terminal device configuration. This figure shows an example of the display screen when a user performs image editing on the terminal device and the modification instruction includes a designated area for specifying the object to be modified. This figure shows a first example of input by the user to the designated area. This figure shows a second example of input by the user to the designated area. This figure shows a third example of input by the user to the designated area. This figure shows an example of the display screen when a user performs image editing on the terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image. This figure shows another example of the display screen when a user performs image editing on the terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image. This figure shows an example of the display screen when a user performs image editing on the terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image, but the replacement image is not found. This figure shows another example of the display screen when a user performs image editing on the terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image, but the replacement image is not found. This figure shows further examples of screens displayed by the display unit when a user is editing an image on a terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image, but the replacement image is not found. This figure shows an example of a screen displayed by the display unit when a user is editing an image on a terminal device and the modification instruction includes text specifying the image to be replaced with the replacement image, but the modified print image is returned to its original state. This figure shows an example of a screen displayed by the display unit when a user is editing an image on a terminal device and the modification instruction includes text specifying the image to be deleted. This figure shows an example of a screen displayed by the display unit when a user is editing an image on a terminal device and the modification instruction includes text specifying the string to be replaced and text specifying the replacement string.This figure shows another example of the display screen when a user is editing an image on a terminal device and the display includes text specifying the string to be replaced and text specifying the replacement string. This is a sequence diagram showing the process executed collaboratively by the terminal device, server, first artificial intelligence system, and second artificial intelligence system.

[0010] The embodiments will be described below with reference to the drawings.

[0011] <Print Image Editing System> Figure 1 is a diagram showing an example of the configuration of a print image editing system 1 according to an embodiment. As shown in Figure 1, the print image editing system 1 includes a server 100, a printer 200, a terminal device 300, a first artificial intelligence system 400, and a second artificial intelligence system 500. The server 100, printer 200, terminal device 300, first artificial intelligence system 400, and second artificial intelligence system 500 are connected to a network NT and can communicate with each other. The network NT includes, for example, the internet or a mobile phone communication line.

[0012] <Server> Figure 2 shows an example of the configuration of server 100. As shown in Figure 2, server 100 has a processor 110, memory 115, and communication interface 190. Bus 105 connects the processor 110, memory 115, and communication interface 190 so that they can send and receive data from each other.

[0013] The memory 115 comprises a volatile memory 120 and a non-volatile memory 130. The volatile memory 120 is, for example, a DRAM and stores various programs and data to be processed. The non-volatile memory 130 is, for example, a hard disk drive or a solid-state drive and has a program storage area 131 and a data storage area 132. The program storage area 131 stores various programs. These programs include processing programs based on sequences, etc., as shown in Figure 17, which will be described later. The data storage area 132 stores various data generated or used by the various programs.

[0014] The processor 110 is a data processing device, such as a CPU, and executes various programs stored in the program storage area 131. The processor 110 performs various processes, including data communication to the printer 200, terminal device 300, first artificial intelligence system 400, and second artificial intelligence system 500. The program storage area 131 stores the AI ​​agent program. When the processor 110 executes the AI ​​agent program, it receives a request containing a message or image input from the user of the terminal device 300. The processor 110 analyzes the message or image contained in the received request and creates a prompt based on the analysis result. The processor 110 sends the created prompt to the first artificial intelligence system 400 or the second artificial intelligence system 500 and receives a response from the first artificial intelligence system 400 or the second artificial intelligence system 500. Based on the received response, the processor 110 sends a response to the terminal device 300.

[0015] The communication interface 190 is a network interface that includes a communication circuit for connecting to and communicating with the network NT.

[0016] Note that the memory 115 is not limited to the configuration of the device elements described above. For example, the memory 115 may be RAM, ROM, EEPROM, HDD, a portable recording medium such as a USB memory stick that can be attached to or detached from the server 100, or a buffer provided by the processor 110. Alternatively, it may be configured by a combination of these.

[0017] Furthermore, the memory 115 may be a storage medium that is readable by a computer. A storage medium that is readable by a computer is a non-transitor medium. Non-transitor mediums also include recording media such as CD-ROMs and DVD-ROMs. Non-transitor mediums are also tangible mediums. The memory 320 of the terminal device 300, which will be described later, is also a storage medium similar to the above.

[0018] <Terminal Device> The terminal device 300 is, for example, a mobile terminal such as a smartphone. Alternatively, the terminal device 300 may be a general-purpose personal computer such as a notebook PC or a desktop PC. Figure 3 shows an example of the configuration of the terminal device 300. As shown in Figure 3, the terminal device 300 has a CPU 310, memory 320, display unit 330, input unit 340, communication interface 360, and large-capacity memory 370.

[0019] The large-capacity memory 370 comprises a program storage area 371 and a data storage area 372. The large-capacity memory 370 is, for example, a hard disk drive or flash memory. The program storage area 371 stores various programs. These programs include processing programs based on sequences, etc., as shown in Figure 17, which will be described later. The CPU 310 executes the various programs stored in the program storage area 371. The communication interface 360 ​​includes a communication circuit for connecting to the network NT and communicating. The CPU 310 performs various processes, including data communication to the server 100, printer 200, first artificial intelligence system 400, and second artificial intelligence system 500 via the communication interface 360 ​​and the network NT.

[0020] The display unit 330 is, for example, a touch panel or a liquid crystal display, and displays various information. The input unit 340 is, for example, a touch panel, a keyboard or a mouse, and accepts various operations from the user. The user can input various instructions to the terminal device 300 by operating the input unit 340.

[0021] <First Artificial Intelligence System, Second Artificial Intelligence System> The first artificial intelligence system 400 and the second artificial intelligence system 500 are, for example, server computers equipped with artificial intelligence. The first artificial intelligence system 400 and the second artificial intelligence system 500 each include a processor, memory, and communication interface (not shown). The processor, memory, and communication interface are the same as those of the server 100. The first artificial intelligence system 400 includes a large-scale language model (LLM). Based on prompts received from the server 100, the first artificial intelligence system 400 performs various natural language processing tasks such as text classification, sentiment analysis, information extraction, text summarization, text generation, and question answering. In this embodiment, based on prompts received from the server 100, the first artificial intelligence system 400 determines which of several tools related to image analysis and image processing to use. The first artificial intelligence system 400 then sends instructions to the second artificial intelligence system 500, including a printed image and identification information of the identified tool. The second artificial intelligence system 500 has several tools related to image analysis and image processing and performs image analysis and image processing using these tools. The second artificial intelligence system 500, based on instructions received from the first artificial intelligence system 400, uses identified tools to identify and modify elements in a printed image. The first artificial intelligence system 400 and the second artificial intelligence system 500 are examples of artificial intelligence systems, and the second artificial intelligence system 500 is an example of an image analysis system. The first artificial intelligence system 400 and the second artificial intelligence system 500 may be configured as a single artificial intelligence system, or they may be divided into three or more artificial intelligence systems.

[0022] <Features of this embodiment> In this embodiment, the terminal device 300 executes an application program stored in the large-capacity memory 370. Subsequently, the terminal device 300 displays a preview of the print image based on an image file selected by the user on the display unit 330. The user can edit the image by using the chat function of the application program to input messages or images while the print image preview is displayed. Specifically, the terminal device 300 displays a print preview of the print image on the display unit 330 and receives modification instructions from the user via messages or images for the print preview. The modification instruction includes specification data that specifies the target to be modified, which is a part of the print image, and modification content data that indicates the content of the modification to the target. The terminal device 300 sends the print image and modification instructions, etc. to the server 100. The server 100 executes an AI agent program and creates a prompt to send to the first artificial intelligence system 400 based on the received modification instructions, etc. Then, the server 100 sends the print image including the target to be modified and the prompt, etc. to the first artificial intelligence system 400.

[0023] The first artificial intelligence system 400 identifies which image editing tool to use based on prompts received from the server 100. The first artificial intelligence system 400 then sends an instruction to the second artificial intelligence system 500 that includes the identification information of the identified tool and a printed image. The second artificial intelligence system 500 identifies the tool based on the identification information of the tool included in the instruction received from the first artificial intelligence system 400. The second artificial intelligence system 500 then uses the identified tool to identify the target to be modified in the printed image and to modify the target. The second artificial intelligence system 500 then sends the printed image with the target to be modified to the terminal device 300. The terminal device 300 acquires the printed image with the target to be modified and displays a print preview of the modified printed image on the display unit 330.

[0024] In this way, compared to conventional methods that required settings or selections on the editing screen, the operation can be simplified and convenience improved. The details are explained below.

[0025] <Example of a screen when performing image editing> Figure 4 shows an example of a screen displayed by the display unit 330 when a user performs image editing on the terminal device 300. In this example screen, the modification instruction includes a specification area for specifying the object to be modified. The terminal device 300 executes the application program stored in the program storage area 371. When the terminal device 300 receives a modification instruction from the user while displaying a print preview of the print image on the display unit 330, it starts displaying the screen shown in Figure 4.

[0026] In this embodiment, "print image" refers to image data copied by the terminal device 300 for editing from an image file or document file stored in the data storage area 372. The image files stored in the data storage area 372 include, for example, image files in JPEG format, and the document files stored in the data storage area 372 include, for example, document files in PDF format. "Print preview" refers to image data output from the "print image" for preview purposes. The image data output for preview purposes includes, for example, image data in JPEG format. "Print data" refers to image data converted from the "print image" for printing purposes. The image data converted for printing includes, for example, image data in BRJPC format or HBP format. "Modification of the print image" refers to modifying the "print image" as described above, but does not modify the image file or document itself stored in the data storage area 372. When the "print image" is modified, the "print preview" and "print data" are also modified in conjunction.

[0027] In Figure 4, the callout SB1 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS1, a print image PI1, and a replacement object ROB1. In the example shown in Figure 4, instruction message MS1 is, for example, the text message "Replace the image in this area with an image of a cafe latte." Print image PI1 includes multiple objects OB1 to OB4 and a designated area AR for specifying the object to be modified. The designated area AR is the area corresponding to "this area" in instruction message MS1. In the example shown in Figure 4, object OB1 is the text "croissant". Object OB2 is an image of a croissant. Object OB3 is an image of coffee. Object OB4 is text representing the price. The designated area AR is entered by the user. Object OB3 is an example of the image to be modified and replaced, and the designated area AR is an example of the specified data. The replacement object ROB1 is an object that represents the replaced appearance of object OB3 specified by the designated area AR. In the example shown in Figure 4, the replacement object ROB1 is, for example, an image of a latte. The replacement object ROB1 is an example of the modified data and the replaced image.

[0028] The speech bubble SB2 shows the object OB3 received from the second artificial intelligence system 500, the confirmation message MS2, and the "Yes" button B1 and "No" button B2. The second artificial intelligence system 500 identifies the object OB3 to be modified in the printed image PI1 based on the user's modification instruction. The second artificial intelligence system 500 then transmits the image of the identified object OB3 to the terminal device 300. The second artificial intelligence system 500 also transmits the confirmation message MS2 to the terminal device 300. In the example shown in Figure 4, the confirmation message MS2 is, for example, a text message such as "Is it okay to replace this image?" The user operates the "Yes" button B1 if the object OB3 specified by the designated area AR has been correctly identified, and the user operates the "No" button B2 if it has not been correctly identified.

[0029] In this embodiment, when the second artificial intelligence system 500 transmits information to the terminal device 300, it means that the second artificial intelligence system 500 transmits information to the terminal device 300 via the first artificial intelligence system 400 and the server 100.

[0030] Speech bubble SB3 indicates that the user pressed the "Yes" button B1 in speech bubble SB2.

[0031] The speech bubble SB4 shows the notification message MS3 and the modified print image PI2 received from the second artificial intelligence system 500. Based on the user's modification instruction, the second artificial intelligence system 500 replaces object OB3 in the print image PI1 with replacement object ROB1 and sends the modified print image PI2 to the terminal device 300. In the example shown in Figure 4, the notification message MS3 is, for example, a text message that reads, "The image has been replaced with a cafe latte image." The print image PI2 includes objects OB1, OB2, and OB4, and the replaced replacement object ROB1.

[0032] The user can then issue a print command to send print data based on the modified print image PI2 to the printer 200 and perform printing based on the modified print image PI2.

[0033] Figure 5 shows a first example of user-defined area AR input. The user inputs the area AR on a preview screen that displays a print preview of the print image PI1. In the example shown in Figure 5, when the user touches two points P1 and P2, the terminal device 300 sets a rectangular area AR with the two points P1 and P2 as diagonal vertices.

[0034] Figure 6 shows a second example of user-defined area AR input. The user inputs the area AR on the preview screen where a print preview of the print image PI1 is displayed. In the example shown in Figure 6, when the user draws an outline of the area freehand, the terminal device 300 sets a rectangular area AR that encloses the outline TR.

[0035] Figure 7 shows a third example of user-defined area AR input. The user inputs the area AR on a preview screen displaying a print preview of the print image PI1. In the example shown in Figure 7, when the user touches a point P3, the terminal device 300 sets a predetermined rectangular area AR centered on that point P3. Alternatively, the terminal device 300 may not specify the area, and instead create a prompt instructing the server 100 to identify an object based on the coordinates of point P3, and send it to the first artificial intelligence system 400.

[0036] Figure 8 shows an example of the screen displayed on the display unit 330 when a user performs image editing on the terminal device 300. In this example screen, the modification instruction includes text specifying the image to be replaced with the replacement image. The terminal device 300 starts displaying the screen shown in Figure 8 when it receives a modification instruction from the user while the display unit 330 is displaying a print preview of the print image. The same conditions apply to Figures 9 to 16, which will be described later.

[0037] In Figure 8, the callout SB5 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS4, a print image PI3, and a replacement object ROB2. In the example shown in Figure 8, instruction message MS4 is, for example, the text message "Change the lightning bolt to a star." In instruction message MS4, "lightning bolt" is an example of the specified data, as well as an example of the first and third texts. In instruction message MS4, "star" is an example of the second text. The print image PI3 includes multiple objects OB5 to OB7. In the example shown in Figure 8, object OB5 is a sun mark, object OB6 is a lightning bolt, and object OB7 is a cloud mark. The color of each mark is, for example, white. Object OB6 is an example of the image to be modified and replaced. The replacement object ROB2 is an object that represents the appearance of object OB6 after replacement. In the example shown in Figure 8, the replacement object ROB2 is, for example, a star mark colored red or blue. The replacement object ROB2 is an example of modified data and the replaced image.

[0038] The speech bubble SB6 shows the confirmation message MS5 received from the second artificial intelligence system 500, as well as the "lightning bolt" button B3 and the "star" button B4. Based on the user's modification instructions, the second artificial intelligence system 500 identifies the object OB6 to be modified in the printed image PI3 and sends the confirmation message MS5 to the terminal device 300. The confirmation message MS5 is a message to confirm whether to match the color of the replaced object to the lightning bolt image (the image to be replaced) or the star image (the image after replacement). In the example shown in Figure 8, the confirmation message MS5 is, for example, the text message "Which color do you want to match, the lightning bolt or the star?". The user operates the "lightning bolt" button B3 or the "star" button B4 depending on the color they want to match.

[0039] Furthermore, when displaying the confirmation message MS5, the second artificial intelligence system 500 performs the following processing: First, the second artificial intelligence system 500 determines whether or not the specified object exists in the printed image PI3. Next, if the specified object exists, the second artificial intelligence system 500 determines whether or not there is a difference in the color of the specified object before and after the replacement. Also, if the specified object exists, the second artificial intelligence system 500 determines whether or not there is a difference in the background color of the specified object before and after the replacement. If the specified object does not exist, the second artificial intelligence system 500 informs the user of this. If there is a difference in the color or background color of the object, the second artificial intelligence system 500 asks the user which one to choose. Asking for color preference is not mandatory; the system may first perform the replacement to match the target color and then ask the user for detailed instructions.

[0040] Speech bubble SB7 indicates that the user pressed the "lightning bolt" button B3 in speech bubble SB6.

[0041] The speech bubble SB8 shows the notification message MS6 received from the second artificial intelligence system 500 and the modified print image PI4. The second artificial intelligence system 500 replaces the object OB6 in the print image PI3 with the replacement object ROB2 based on a user's modification instruction, and transmits the modified print image PI4 to the terminal device 300. In the example shown in FIG. 8, the notification message MS6 is, for example, a text message reading "The lightning mark has been replaced with a star mark.". The print image PI4 includes objects OB5, OB7 and the replaced replacement object ROB2. Note that the replacement object ROB2 has been changed to the lightning mark color specified by the user.

[0042] The user can subsequently issue a print instruction to transmit print data based on the modified print image PI4 to the printer 200, and execute printing based on the modified print image PI4.

[0043] FIG. 9 is a diagram showing another example of a screen displayed by the display unit 330 when a user performs image editing on the terminal device 300. In this screen example, the modification instruction includes text specifying a replaced image to be replaced with a post-replacement image. In FIG. 9, the same elements as those in the aforementioned FIG. 8 are denoted by the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0044] In FIG. 9, a speech bubble SB9 indicates a modification instruction input by the user. The modification instruction includes an instruction message MS4 input by the user, the print image PI3, and a replacement object ROB3. The instruction message MS4 and the print image PI3 are the same as those in the aforementioned FIG. 8. The replacement object ROB3 is an object representing the post-replacement state of the object OB6. In the example shown in FIG. 9, the replacement object ROB3 is a star mark with a rectangular black background. The replacement object ROB3 is an example of modification content data and a post-replacement image.

[0045] The speech bubble SB10 shows a confirmation message MS7 received from the second artificial intelligence system 500, a "Yes" button B5 and a "No" button B6. The second artificial intelligence system 500 specifies the object OB6 to be modified in the print image PI3 based on a user's modification instruction, and transmits the confirmation message MS7 to the terminal device 300. The confirmation message MS7 is a message for confirming whether or not to remove the background of the replacement object ROB3. In the example shown in Fig. 9, the confirmation message MS7 is, for example, a text message reading "There is a black background around the black mark. Should we remove it and replace it?". The user operates the "Yes" button B5 when removing the black background, and operates the "No" button B6 when not removing the black background. Note that when displaying the confirmation message MS7, the second artificial intelligence system 500 performs the same process as that performed when displaying the aforementioned confirmation message MS5.

[0046] The speech bubble SB11 indicates that the user operated the "Yes" button B5 in the speech bubble SB10.

[0047] The speech bubble SB12 shows a notification message MS6 received from the second artificial intelligence system 500 and the modified print image PI5. The second artificial intelligence system 500 replaces the object OB6 in the print image PI3 with the replacement object ROB3 based on a user's modification instruction, and transmits the modified print image PI5 to the terminal device 300. The notification message MS6 is the same as that in Fig. 8 described above. The print image PI5 includes objects OB5, OB7 and the replaced replacement object ROB3. Note that the black background of the replacement object ROB3 has been removed according to a user's instruction.

[0048] Thereafter, by issuing a print instruction, the user can transmit print data based on the modified print image PI5 to the printer 200 and execute printing based on the modified print image PI5.

[0049] Figure 10 shows an example of the display screen displayed by the display unit 330 when a replacement image is not found while the user is editing an image on the terminal device 300. In this example screen, the modification instruction includes text specifying the replacement image to be replaced with the replacement image. In Figure 10, elements that are the same as those in Figures 8 and 9 are denoted by the same reference numerals, and explanations are omitted as appropriate.

[0050] In Figure 10, the callout SB13 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS8, a print image PI3, and a replacement object ROB4. In the example shown in Figure 10, the instruction message MS8 is, for example, the text message "Change the rain icon to a star icon." In the instruction message MS8, "rain icon" is an example of the specified data, as well as an example of the first and third texts. In the instruction message MS8, "star icon" is an example of the second text. The print image PI3 is the same as in Figure 8 described above. The replacement object ROB4 is an object that represents the appearance of the "rain icon" after replacement. In the example shown in Figure 10, the replacement object ROB4 is a star icon. The replacement object ROB4 is an example of the modification content data and the replaced image.

[0051] The speech bubble SB14 indicates the confirmation message MS9 received from the second artificial intelligence system 500. Based on the user's modification instructions, the second artificial intelligence system 500 identifies the "rain mark" that is to be modified in the printed image PI3 and sends a confirmation message to the terminal device 300. In the example shown in Figure 10, the printed image PI3 does not contain an object corresponding to the "rain mark". Therefore, the confirmation message MS9 is a message indicating that the "rain mark" could not be identified. In the example shown in Figure 10, the confirmation message MS9 is, for example, a text message such as "The rain mark could not be found. Please give instructions again."

[0052] The display of the callout SB14 allows the user to reconfirm and review the instruction message MS8 and the print image PI3.

[0053] Figure 11 shows another example of a screen displayed by the display unit 330 when a replacement image is not found while the user is editing an image on the terminal device 300. In this example screen, the modification instruction includes text specifying the replacement image to be replaced with the replacement image. In Figure 11, the same reference numerals are used for elements that are the same as those in Figure 10, and explanations are omitted as appropriate.

[0054] In Figure 11, the callout SB15 contains the same content as the callout SB13 in Figure 10. The callout SB16 shows the confirmation message MS10 received from the second artificial intelligence system 500, the cloud mark object OB7, and the "Yes" button B5 and "No" button B6. The confirmation message MS10 is a message to confirm whether or not to replace the "rain mark" with a similar mark because the "rain mark" could not be identified. In the example shown in Figure 11, the confirmation message MS10 is, for example, the text message "No rain mark was found. Do you want to replace it with the closest cloud mark?" The user operates the "Yes" button B5 if they want to replace the cloud mark, and the "No" button B6 if they do not want to replace it.

[0055] The appearance of the callout SB16 allows the user to automatically select a similar mark to be replaced, review the instruction message MS8, or the print image PI3, among other things.

[0056] Figure 12 shows another example of the display screen displayed by the display unit 330 when the replacement image is not found while the user is editing an image on the terminal device 300. In this example screen, the modification instruction includes text specifying the replacement image to be replaced with the replacement image. In Figure 12, the same reference numerals are used for elements that are the same as those in Figure 10 and other figures, and explanations are omitted as appropriate.

[0057] In Figure 12, the speech bubble SB17 has the same content as the speech bubble SB13 in Figure 10. The speech bubble SB18 shows the confirmation message MS11 received from the second artificial intelligence system 500. The confirmation message MS11 is a message to confirm whether to replace the "rain mark" with another mark because it could not be identified. In the example shown in Figure 12, the confirmation message MS11 is a text message such as, "The rain mark could not be found. Instead, the following marks were found in the image. If you want to replace it with one of these marks, press the target mark." The speech bubble SB17 also includes the object OB5 of the sun mark and a selection button B7 for selecting the object OB5. The speech bubble SB17 also includes the object OB7 of the cloud mark and a selection button B8 for selecting the object OB7. The speech bubble SB17 also includes the object OB6 of the lightning mark and a selection button B9 for selecting the object OB6. The user operates selection button B7 to replace the sun icon, selection button B8 to replace the cloud icon, and selection button B9 to replace the lightning icon.

[0058] The appearance of the callout SB18 allows the user to select a different mark and replace it.

[0059] Figure 13 shows an example of the display screen displayed by the display unit 330 when a user edits an image on the terminal device 300 and reverts the modified printed image back to its original state. In this example screen, the modification instruction includes text specifying the image to be replaced with the replaced image. In Figure 13, the same reference numerals are used for elements that are the same as those in Figure 10 and other figures, and explanations are omitted as appropriate.

[0060] In Figure 13, the speech bubble SB19 has the same content as the speech bubble SB13 in Figure 10. The speech bubble SB20 shows the notification message MS12 received from the second artificial intelligence system 500 and the modified print image PI6. Based on the user's modification instruction, the second artificial intelligence system 500 replaces object OB6 in the print image PI3 with replacement object ROB4 and sends the modified print image PI6 to the terminal device 300. In the example shown in Figure 13, the notification message MS12 is, for example, the text message "Replaced." The print image PI6 includes objects OB5 and OB7 and the replaced replacement object ROB4.

[0061] The callout SB21 shows the instruction message MS13 entered by the user. In the example shown in Figure 8, the instruction message MS13 is a message to revert the modified print image back to the original print image, and is, for example, a text message such as "Revert".

[0062] The callout SB22 shows the notification message MS14 and the print image PI3 before modification, both received from the second artificial intelligence system 500. Based on the user's return instruction, the second artificial intelligence system 500 reverts the replacement object ROB4 in the print image PI6 back to the original object OB6, and sends the reverted print image PI3 to the terminal device 300. In the example shown in Figure 13, the notification message MS14 is, for example, the text message "Reverted to the pre-replacement state." The print image PI3 includes the original objects OB5, OB6, and OB7.

[0063] In this way, the user can revert the modified print image back to the original print image.

[0064] Figure 14 shows an example of the screen displayed by the display unit 330 when a user performs image editing on the terminal device 300. In this example screen, the modification instruction includes text specifying the replacement image to be deleted.

[0065] In Figure 14, the callout SB23 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS15 and a print image PI7. In the example shown in Figure 14, the instruction message MS15 is, for example, the text message "Delete the part that appears to be an advertisement." The "part that appears to be an advertisement" in the instruction message MS15 is an example of specified data, as well as an example of the first text and the third text. The print image PI7 includes multiple objects OB8 to OB11. In the example shown in Figure 14, object OB8 is a document composed of text, object OB9 is an advertisement, object OB10 is the text "Free download of useful materials," and object OB11 is multiple banners. Object OB9 is an example of the image to be modified and replaced. In the example shown in Figure 8, since the modification is deletion, the replacement object can also be called a blank object. This blank object is an example of the modification content data and the replaced image. Also, deletion is an example of modification.

[0066] The speech bubble SB24 shows the notification message MS16 and the modified print image PI8 received from the second artificial intelligence system 500. Based on the user's modification instruction, the second artificial intelligence system 500 deletes object OB9 in the print image PI7 and sends the modified print image PI8 to the terminal device 300. In the example shown in Figure 14, the notification message MS16 is a text message such as, "The portion on the right that appears to be an advertisement has been deleted. If you want to revert to the original, press the 'Undo' button." The print image PI8 includes objects OB8, OB10, and OB11 as a result of object OB9 being deleted.

[0067] The speech bubble SB25 indicates a further modification instruction entered by the user. The modification instruction includes the instruction message MS17 entered by the user. In the example shown in Figure 14, the instruction message MS17 is, for example, the text message "Delete the part that says 'Free Download of Useful Materials' and the banner part at the top." The parts of the instruction message MS17 that say "Free Download of Useful Materials" and the banner part at the top are examples of specified data, as well as examples of the first and third texts. Objects OB10 and OB11 are examples of the image to be modified and the image to be replaced.

[0068] The callout SB26 shows the notification message MS18 and the modified print image PI9 received from the second artificial intelligence system 500. Based on the user's modification instruction, the second artificial intelligence system 500 deletes objects OB10 and OB11 in the print image PI8 and sends the modified print image PI9 to the terminal device 300. In the example shown in Figure 14, the notification message MS18 is, for example, a text message that reads, "The specified location has been deleted. If you want to revert, press the 'Undo' button." The print image PI9 contains only object OB8 as a result of the deletion of objects OB10 and OB11.

[0069] The callout SB27 indicates a further modification instruction entered by the user. The modification instruction includes the instruction message MS19 entered by the user. In the example shown in Figure 14, the instruction message MS19 is, for example, the text message "Delete the blank space created by the deletion." The phrase "the blank space created by the deletion" in the instruction message MS19 is an example of the specified data, as well as an example of the first and third texts.

[0070] The callout SB28 shows the notification message MS20 and the modified print image PI10 received from the second artificial intelligence system 500. Based on the user's modification instruction, the second artificial intelligence system 500 removes the margin portion of the print image PI9 and sends the modified print image PI10 to the terminal device 300. In the example shown in Figure 14, the notification message MS20 is a text message such as, "The portion that became a margin due to deletion has been removed. If you want to revert to the original, press the 'Undo' button." The print image PI10 has the margin portion around the object OB8 removed.

[0071] The user can then issue a print command to send print data based on the modified print image PI10 to the printer 200 and perform printing based on the modified print image PI10.

[0072] Figure 15 shows an example of the screen displayed by the display unit 330 when a user performs image editing on the terminal device 300. In this example screen, the modification instructions include text specifying the string to be replaced and text specifying the replacement string.

[0073] In Figure 15, the callout SB29 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS21 and a print image PI11. In the example shown in Figure 15, the instruction message MS21 is, for example, the text message "Change the price from 100 yen to 50 yen". In the instruction message MS21, "100 yen" is an example of the specified data and an example of the fifth text, and "50 yen" is an example of the fourth text. The print image PI11 includes multiple objects OB12 to OB14. In the example shown in Figure 15, object OB12 is the text "Cabbage", object OB13 is the text "1 head", and object OB14 is the text "100 yen". Object OB14 is an example of an image of the text to be modified and replaced.

[0074] The speech bubble SB30 shows the notification message MS22 and the modified print image PI12 received from the second artificial intelligence system 500. Based on the user's modification instructions, the second artificial intelligence system 500 replaces object OB14 in the print image PI11 with replacement object ROB5 and sends the modified print image PI12 to the terminal device 300. Specifically, the server 100 acquires the text information of object OB14, which is the text before replacement. The text information includes, for example, font, weight, color, size, etc. The server 100 generates replacement object ROB5, which is the text after replacement, reflecting the acquired text information. In the example shown in Figure 15, replacement object ROB5 is the text "50 yen". Note that replacement object ROB5 is an example of an image of the modified content data and the replaced string.

[0075] Server 100 creates a prompt to send to the first artificial intelligence system 400 based on the received modification instructions, and sends the printed image PI11, the generated replacement object ROB5, the prompt, etc. to the first artificial intelligence system 400. Based on the prompt, etc. received from Server 100, the first artificial intelligence system 400 determines which tool to use among the image editing tools, and sends the printed image PI11, the replacement object ROB5, the identified tool, etc. to the second artificial intelligence system 500. Based on the printed image PI11 and the replacement object ROB5 etc. received from the first artificial intelligence system 400, the second artificial intelligence system 500 uses the identified tool to replace object OB14 in the printed image PI11 with replacement object ROB5. The second artificial intelligence system 500 sends the modified printed image PI12 to the terminal device 300.

[0076] In the example shown in Figure 15, the notification message MS22 is, for example, a text message that reads "The price has been changed." The printed image PI12 includes objects OB12 and OB13, and the replaced object ROB5. The replaced object ROB5 has the same font, weight, color, size, etc. as the original object OB14.

[0077] The user can then issue a print command to send print data based on the modified print image PI12 to the printer 200 and perform printing based on the modified print image PI12.

[0078] Figure 16 shows another example of a screen displayed by the display unit 330 when a user performs image editing on the terminal device 300. In Figure 16, the same reference numerals are used for elements that are the same as those in Figure 15, and explanations are omitted as appropriate.

[0079] In Figure 16, the callout SB31 indicates a modification instruction entered by the user. The modification instruction includes a user-entered instruction message MS23 and a print image PI13. In the example shown in Figure 16, the instruction message MS23 is, for example, the text message "Change the price from 70 yen to 60 yen". In the instruction message MS23, "70 yen" is an example of the specified data and an example of the fifth text, and "60 yen" is an example of the fourth text. The print image PI13 includes multiple objects OB15 to OB17. In the example shown in Figure 16, object OB15 is the text "cucumber", object OB16 is the text "1", and object OB17 is the text "70 yen". Object OB17 is an example of an image of the text to be modified and replaced.

[0080] The speech bubble SB32 shows the notification message MS22 and the modified print image PI14 received from the second artificial intelligence system 500. Based on the user's modification instructions, the second artificial intelligence system 500 replaces object OB17 in the print image PI13 with replacement object ROB6 and sends the modified print image PI14 to the terminal device 300. Specifically, the server 100 acquires the text information of object OB17, which is the text before replacement. The text information includes, for example, font, weight, color, size, etc. The server 100 generates replacement object ROB6, which is the text after replacement, reflecting the acquired text information. In the example shown in Figure 16, the replacement object ROB6 is the text "60 yen". Note that replacement object ROB6 is an example of an image of the modified content data and the replaced string.

[0081] Server 100 creates a prompt to send to the first artificial intelligence system 400 based on the received modification instructions, and sends the printed image PI 13, the generated replacement object ROB 6, the prompt, etc. to the first artificial intelligence system 400. Based on the prompt, etc. received from Server 100, the first artificial intelligence system 400 determines which tool to use among the image editing tools, and sends the printed image PI 13, the replacement object ROB 6, the identified tool, etc. to the second artificial intelligence system 500. Based on the printed image PI 13 and the replacement object ROB 6 etc. received from the first artificial intelligence system 400, the second artificial intelligence system 500 uses the identified tool to replace object OB 17 in the printed image PI 13 with replacement object ROB 6. The second artificial intelligence system 500 sends the modified printed image PI 14 to the terminal device 300.

[0082] The notification message MS22 is the same as in Figure 15 above. The printed image PI14 includes objects OB15 and OB16, and the replaced object ROB6. The replaced object ROB6 has the same font, thickness, color, size, etc. as the original object OB17.

[0083] The user can then issue a print command to send print data based on the modified print image PI14 to the printer 200 and perform printing based on the modified print image PI14.

[0084] In the explanation of Figures 15 and 16, the server 100 is shown to perform the acquisition of text information and the generation of replacement objects, but the first artificial intelligence system 400 or the second artificial intelligence system 500 may also perform these processes. In this case, the server 100 may create a prompt instructing the first artificial intelligence system 400 or the second artificial intelligence system 500 to perform these processes and send it.

[0085] <Processing of the Print Image Editing System> Figure 17 is a sequence diagram showing the processing performed collaboratively by the terminal device 300, the server 100, the first artificial intelligence system 400, and the second artificial intelligence system 500. In Figure 17, the entities that execute each process are the CPU 310 of the terminal device 300, the processor 110 of the server 100, the processor of the first artificial intelligence system 400, and the processor of the second artificial intelligence system 500.

[0086] In step S5, the terminal device 300 executes the application program stored in the program storage area 371. After that, the terminal device 300 displays a print preview of the print image on the display unit 330. The CPU 310 of the terminal device 300 that executes step S5 is an example of the preview display unit.

[0087] In step S10, the terminal device 300 determines whether or not it has received a user instruction to modify the printed image. If the terminal device 300 has not received a modification instruction (step S10: No), it proceeds to step S90, which will be described later. On the other hand, if the terminal device 300 has received a modification instruction (step S10: Yes), it proceeds to the next step S15.

[0088] In step S15, the terminal device 300 receives a message or image from the user to indicate a modification to the print preview displayed in step S5. As described above, the modification instruction includes an instruction message entered by the user, specification data that specifies the target to be modified, and modification content data that indicates the content of the modification to the target. The CPU 310 of the terminal device 300 that executes step S15 is an example of a receiving unit.

[0089] In step S20, the terminal device 300 transmits the printed image and the modification instructions received in step S15 to the server 100.

[0090] In step S25, the server 100 creates a prompt to send to the first artificial intelligence system 400 based on the modification instructions received from the terminal device 300. This prompt relates to the identification of the object to be modified. The server 100 creates a prompt with content similar to the following, for example. Here, XXXX below indicates the text included in the modification instructions entered by the user, or the coordinates of the specified area. Based on the message entered by the user, determine which of the following tools to use and output the name of the tool and the arguments required for the tool. ・Object Identification Tool (1) ... Identifies the coordinates of an object within the specified area based on the coordinates of the specified area in the argument (Arguments: Printed image, coordinates of the specified area). Returns an error if identification is not possible. ・Object Identification Tool (2) ... Identifies the coordinates of an object with that name within the printed image based on the printed image and the object's name in the argument (Arguments: Printed image, object's name). Returns an error if identification is not possible. ・Message: XXXX

[0091] In step S30, the server 100 sends the printed image and the prompt created in step S25 to the first artificial intelligence system 400. If the message in the prompt sent in step S25 contains text indicating the name of an object as specified data, the server 100 outputs the printed image and text to the second artificial intelligence system 500 via the first artificial intelligence system 400 in step S30. Furthermore, if the message in the prompt sent in step S25 contains a specified area in the printed image that should be modified, the server 100 detects the edge of the specified area in step S30 and outputs it to the first artificial intelligence system 400 along with the prompt. The processor 110 of the server 100 that executes step S30 is an example of a prompt output unit.

[0092] In step S35, the first artificial intelligence system 400 identifies a tool based on a prompt received from the server 100. The first artificial intelligence system 400 identifies a tool based on the text indicating a modification instruction contained in the message within the prompt. For example, as shown in Figures 4 to 7 above, the first artificial intelligence system 400 selects the object identification tool (1) if the modification instruction includes a specified area. Also, for example, as shown in Figures 8 to 16 above, the first artificial intelligence system 400 selects the object identification tool (2) if the modification instruction includes text specifying the image to be replaced or text specifying the string to be replaced. The first artificial intelligence system 400 sends an instruction to the second artificial intelligence system 500 that includes the name of the identified tool and the arguments corresponding to that tool.

[0093] In step S40, the second artificial intelligence system 500 selects a tool based on the name of the tool included in the instructions received from the first artificial intelligence system 400. The second artificial intelligence system 500 then passes arguments to the selected tool, causing the tool to identify the object to be modified within the printed image. The tool identifies the coordinates, color, or name of the object to be modified within the printed image, as predetermined by the tool's functions.

[0094] In step S45, the second artificial intelligence system 500 transmits object information relating to the object to be modified, which was identified in step S40, to the first artificial intelligence system 400. In step S46, the first artificial intelligence system 400 transmits the object information to the server 100. In step S47, the server 100 transmits the object information to the terminal device 300. The object information includes, for example, the coordinates of the object to be modified. If the second artificial intelligence system 500 was unable to identify the object to be modified in step S40, it transmits error information to that effect. The object information is an example of modification information.

[0095] In step S50, the terminal device 300 receives object information or error information from the second artificial intelligence system 500. If the terminal device 300 receives object information, it displays a confirmation message on the display unit 330 to confirm the identified object to be modified. Specifically, the terminal device 300 extracts an image of the object to be modified from the printed image based on the coordinates of the object included in the object information. The server 100 then displays the extracted image of the object to be modified and a message confirming whether the image matches the content of the text. If the terminal device 300 receives error information, it displays the error information on the display unit 330. For example, if the object to be modified is unclear, the terminal device 300 displays a message to the user for confirmation. If the message in the prompt sent in step S25 includes text specifying the object to be modified, the terminal device 300 displays a message in step S50 to confirm the content of the text according to the text identification result by the second artificial intelligence system 500. Furthermore, in step S50, the terminal device 300 displays a message confirming the content of the image to be replaced, according to the identification result of the second artificial intelligence system 500 of the image to be replaced. Also, if the message in the prompt sent in step S25 contains text specifying the string to be replaced which should be replaced with the replacement string, the terminal device 300 displays a message confirming the content of the string to be replaced, according to the identification result of the text by the second artificial intelligence system 500, in step S50. The CPU 310 of the terminal device 300 that executes step S50 is an example of the first confirmation message display unit, the second confirmation message display unit, and the third confirmation message display unit.

[0096] In step S55, the terminal device 300 determines the object to be modified based on the content entered by the user in response to the confirmation message displayed in step S50, and sends the object information to the server 100.

[0097] In step S60, the server 100 creates a prompt related to object replacement or deletion to send to the first artificial intelligence system 400 based on the modification instructions and object information received from the terminal device 300. The server 100 creates a prompt with content similar to the following, for example: Here, YYYY below indicates the message included in the modification instructions entered by the user. Based on the message entered by the user, determine which of the following tools to use and output the tool and the necessary arguments. ・Object replacement tool: Replaces the object to be modified in the printed image with a replacement object (arguments: printed image, coordinates of the object to be modified, replacement object). ・Object deletion tool: Deletes the object to be modified in the printed image (arguments: printed image, coordinates of the object to be modified) ・Message: YYYY

[0098] In step S65, the server 100 transmits the printed image, the object information to be modified (confirmed in step S55), and the prompt created in step S60 to the first artificial intelligence system 400. The confirmed object information to be modified includes the confirmed object image, the object's coordinates, and the object's name. In step S65, the server 100 creates a prompt to be transmitted to the first artificial intelligence system 400 based on the object information obtained from the second artificial intelligence system 500, and outputs the printed image and the prompt to the first artificial intelligence system 400. The processor 110 of the server 100 that executes step S65 is an example of a prompt output unit.

[0099] In step S70, the first artificial intelligence system 400 identifies a tool based on the object information and prompts received from the server 100. For example, as shown in Figures 4 to 13, 15 and 16 above, the first artificial intelligence system 400 selects the object replacement tool when replacing an object. Also, for example, as shown in Figure 14 above, the first artificial intelligence system 400 selects the object deletion tool when deleting an object. The first artificial intelligence system 400 sends an instruction to the second artificial intelligence system 500 that includes the name of the identified tool and the arguments corresponding to that tool.

[0100] In step S75, the second artificial intelligence system 500 selects a tool based on the tool name and arguments included in the instructions received from the first artificial intelligence system 400. The second artificial intelligence system 500 then passes the arguments to the selected tool, causing the tool to replace or delete the object to be modified with a replacement object, thereby modifying the printed image. If the message in the prompt sent in steps S30 and S65 contains text specifying the object to be modified, the second artificial intelligence system 500 identifies the area to be modified based on the object information corresponding to the acquired text in steps S40 and S75, and performs the modification on the identified area. If the message in the prompt sent in step S65 contains text specifying a replacement image representing the modified appearance, the second artificial intelligence system 500 modifies a portion of the printed image with the replacement image in step S75. Furthermore, if the message in the prompt sent in step S65 includes text specifying the image to be replaced with the replacement image, the second artificial intelligence system 500 performs a modification in step S75, replacing the image to be replaced, which is part of the printed image, with the replacement image. Furthermore, if the message in the prompt sent in step S65 includes a replacement image representing the modified appearance after replacement, the second artificial intelligence system 500 modifies a part of the printed image with the replacement image in step S75. Furthermore, if the message in the prompt sent in step S65 includes text specifying the replacement string, the second artificial intelligence system 500 modifies a part of the printed image with an image of the replacement string in step S75. Furthermore, if the message in the prompt sent in step S65 includes text specifying the replacement string to be replaced with the replacement string, the second artificial intelligence system 500 performs a modification in step S75, replacing an image of the replacement string, which is part of the printed image, with an image of the replacement string.Furthermore, if the message in the prompt sent in step S65 contains a specified area in the printed image that should be modified, the second artificial intelligence system 500 will modify the portion identified by the edge of the specified area in step S75. Also, if the message in the prompt sent in step S65 contains text instructing deletion, the second artificial intelligence system 500 will delete a portion of the printed image in step S75.

[0101] In step S80, the second artificial intelligence system 500 transmits the modified print image from step S75 to the first artificial intelligence system 400. In step S81, the first artificial intelligence system 400 transmits the modified print image to the server 100. In step S82, the server 100 transmits the modified print image to the terminal device 300.

[0102] In step S85, the terminal device 300 receives the modified print image. After receiving the image, the terminal device 300 displays a print preview of the modified print image on the display unit 330. In other words, the terminal device 300 acquires a print image in which the object to be modified has been modified based on the modification instruction received in step S15. The terminal device 300 also acquires a print image that has been edited by the second artificial intelligence system 500 based on the prompt output in step S65. The CPU 310 of the terminal device 300 that executes step S85 is an example of a print image acquisition unit. The CPU 310 of the terminal device 300 that executes steps S30, S65, and S85 is an example of an editing processing unit.

[0103] In step S90, the terminal device 300 determines whether or not there is a print command from the user to execute printing. If there is no print command (step S90: No), the terminal device 300 returns to step S10. On the other hand, if there is a print command (step S90: Yes), the terminal device 300 proceeds to the next step S95.

[0104] In step S95, the terminal device 300 sends print data to the printer 200 based on the modified print image acquired in step S85. Alternatively, the terminal device 300 may send the print data to the printer 200 via the server 100.

[0105] In the flowchart above, the first artificial intelligence system 400 transmits information such as the object identification tool and arguments to the second artificial intelligence system 500, but it may also transmit this information to the terminal device 300. In this case, the terminal device 300 transmits the information such as the object identification tool and arguments to the second artificial intelligence system 500.

[0106] <Effects of the Embodiment> In this embodiment, users can edit images simply by inputting messages or images. Therefore, compared to conventional methods that required settings or selections on the editing screen, the operation can be simplified and convenience can be improved.

[0107] Furthermore, in this embodiment, in steps S30 and S65, the server 100 creates a prompt to send to the first artificial intelligence system 400, and sends the printed image containing the object to be modified along with the prompt to the first artificial intelligence system 400. By cooperating with the artificial intelligence system in this way, the editing of printed images can be performed even more simply and quickly.

[0108] Furthermore, in this embodiment, if the modification instruction includes text as specified data, the server 100 transmits the printed image to the second artificial intelligence system 500 via the first artificial intelligence system 400 in step S30. By coordinating with the second artificial intelligence system 500 in this way, the parts to be modified can be identified quickly and accurately.

[0109] Furthermore, in this embodiment, in steps S60 and S65, the server 100 creates a prompt based on object information acquired from the second artificial intelligence system 500 and transmits the print image and the prompt to the first artificial intelligence system 400. By coordinating with the first artificial intelligence system 400 and the second artificial intelligence system 500 in this way, the parts to be modified can be identified quickly and accurately, and the editing of the print image can be performed more simply and quickly.

[0110] Furthermore, in this embodiment, in step S50, the terminal device 300 displays a message to confirm the content of the text according to the text identification result by the second artificial intelligence system 500. In this way, if the text is unidentifiable or its content is unclear and difficult to identify, the user is made aware of this, allowing the user to re-enter the correct text content.

[0111] Furthermore, in this embodiment, if the modification instruction includes text specifying a replacement image representing the appearance after the replacement as modification content data, the second artificial intelligence system 500 modifies a portion of the printed image to the replacement image in step S75. This allows the user to specify the appearance of the image after the replacement via a message.

[0112] Furthermore, in this embodiment, if the modification instruction includes text specifying the image to be replaced as designated data, the second artificial intelligence system 500 performs a modification in step S75, replacing the image to be replaced, which is part of the printed image, with the image to be replaced. This allows the user to specify the image to be replaced by a message.

[0113] Furthermore, in this embodiment, in step S50, the terminal device 300 displays a message to confirm the content of the image to be replaced, according to the identification result of the second artificial intelligence system 500. This allows the user to recognize if the image to be replaced is unidentifiable or its content is unclear and difficult to identify, enabling the user to re-enter the correct image to be replaced.

[0114] Furthermore, in this embodiment, if the modification instruction includes a replacement image representing the replaced appearance as modification content data, the second artificial intelligence system 500 modifies a portion of the printed image with the replacement image in step S75. As a result, the user can specify the replaced appearance by inputting the replacement image itself.

[0115] Furthermore, in this embodiment, if the modification instruction includes text specifying the replacement string as modification content data, the second artificial intelligence system 500 modifies a portion of the printed image to an image of the replacement string in step S75. This allows the user to specify the string to be inserted via a message.

[0116] Furthermore, in this embodiment, if the modification instruction includes text specifying the string to be replaced as modification content data, the second artificial intelligence system 500 performs a modification in step S75 by replacing the image of the string to be replaced, which is part of the printed image, with the image of the string of the replacement string. This allows the user to specify the string to be replaced by a message.

[0117] Furthermore, in this embodiment, in step S50, the terminal device 300 displays a message confirming the content of the string to be replaced, according to the text identification result by the second artificial intelligence system 500. In this way, if the string to be replaced is unidentifiable or its content is unclear and difficult to identify, the user is made aware of this, allowing the user to re-enter the correct string to be replaced.

[0118] Furthermore, in this embodiment, if the modification instruction includes a specified area in the printed image to be modified as modification content data, the server 100 detects the edge of the specified area in step S30 and transmits it to the first artificial intelligence system 400 along with a prompt. In step S75, the second artificial intelligence system 500 performs modifications on the portion identified by the edge. As a result, the user can perform the desired image editing on the specified area simply by specifying an appropriate area in the print preview.

[0119] Furthermore, in this embodiment, if the modification instruction includes text instructing deletion, the second artificial intelligence system 500 deletes a portion of the printed image in step S75. This allows the user to delete a desired area in the printed image simply by entering text instructing deletion for the print preview.

[0120] <Modifications> This disclosure is not limited to the embodiments described above, and various modifications are possible without departing from its spirit and technical concept.

[0121] In the above embodiment, the modification of the printed image was described as being the replacement or deletion of the object to be modified, but it may also include modifications other than replacement or deletion. For example, the modification of the printed image may include changes to the size, shape, color, font, etc. of the object to be modified.

[0122] Furthermore, although the above embodiment described a case where the user gives modification instructions by inputting text or images, modification instructions may also be given by voice input.

[0123] Furthermore, the processing performed by the terminal device 300 and the processing performed by the server 100 as described in the above embodiment are not limited to the examples of being performed by the devices described in the above embodiment, but may be performed by different devices. Also, all of the processing performed by the terminal device 300 and the server 100 may be performed by either the terminal device 300 or the server 100. In addition, by incorporating a program corresponding to the functions of the server 100 and one or both of the first artificial intelligence system 400 and the second artificial intelligence system 500 into the terminal device 300, the terminal device 300 may perform processing without going through the server 100.

[0124] Furthermore, the sequence diagram shown in Figure 17 does not limit this disclosure to the procedures shown in the sequence diagram, and procedures may be added, deleted, or their order changed within the scope that does not deviate from the spirit and technical concept of this disclosure.

[0125] In addition to what has already been described above, the methods described in the embodiments and each modification may be used in appropriate combinations.

[0126] Furthermore, although not to list them individually, this disclosure will be implemented with various modifications, within the scope that does not deviate from its intent.

[0127] 1 Print Image Editing System 100 Server 110 Processor 200 Printer 300 Terminal Device 310 CPU 400 First Artificial Intelligence System (Example of an Artificial Intelligence System) 500 Second Artificial Intelligence System (Example of an Artificial Intelligence System, Image Analysis System) AR Designated Area (Example of Designated Data) MS1 Instruction Message MS2 Confirmation Message MS3 Notification Message MS4 Instruction Message MS5 Confirmation Message MS6 Notification Message MS7 Confirmation Message MS8 Instruction Message MS9 Confirmation Message MS10 Confirmation Message MS11 Confirmation Message MS12 Notification Message MS13 Instruction Message MS14 Notification Message MS15 Instruction Message MS16 Notification Message MS17 Instruction Message MS18 Notification Message MS19 Instruction Message MS20 Notification Message MS21 Instruction Message MS22 Notification Message MS23 Instruction Message OB1 Object OB2 Object OB3 Object (Example of image to be modified and replaced) OB4 Object OB5 Object OB6 Object (Example of image to be modified and replaced) OB7 Object OB8 Object OB9 Object (Example of image to be modified and replaced) OB10 Object (Example of image to be modified and replaced) OB11 Object (Example of image to be modified and replaced) OB12 Object OB13 Object OB14 Object (Example of image of string to be modified and replaced) OB15 Object OB16 Object OB17 Object (Example of image of string to be modified and replaced) PI1 Printed image PI2 Printed image PI3 Printed image PI4 Printed image PI5 Printed image PI6 Printed image PI7 Printed image PI8 Printed image PI9 Printed image PI10 Printed image PI11 Printed image PI12 Printed image PI13 Printed image PI14 Printed Image ROB1 Replacement Object (Example of modified data and replaced image) ROB2 Replacement Object (Example of modified data and replaced image) ROB3 Replacement Object (Example of modified data and replaced image) ROB4 Replacement Object (Example of modified data and replaced image)ROB5 Replacement Object (Example of modified data and image of the replaced string) ROB6 Replacement Object (Example of modified data and image of the replaced string)

Claims

1. A print image editing system comprising: a preview display unit that displays a print preview of a print image; a reception unit that receives modification instructions from a user via messages or images regarding the print preview displayed by the preview display unit, wherein the modification instruction includes designation data that specifies the target to be modified and modification content data that indicates the content of the modification to the target to be modified; and an editing processing unit that acquires the print image in which the target to be modified has been modified based on the modification instruction received by the reception unit, wherein the preview display unit displays a print preview of the print image after the modification of the target to be modified has been acquired by the editing processing unit.

2. The print image editing system according to claim 1, wherein the editing processing unit includes: a prompt output unit that creates a prompt to be sent to an artificial intelligence system to perform the modification based on the modification instruction received by the reception unit, and outputs the print image including the modification target and the prompt to the artificial intelligence system; and a print image acquisition unit that acquires the print image after it has been edited by the artificial intelligence system based on the prompt output from the prompt output unit.

3. The print image editing system according to claim 2, wherein the prompt output unit outputs the print image to the image analysis system when the modification instruction includes the first text as the specified data, and the artificial intelligence system identifies the area to be modified based on the modification information corresponding to the first text obtained from the image analysis system, and performs the modification on the identified area.

4. The print image editing system according to claim 3, wherein the prompt output unit creates the prompt to be sent to the artificial intelligence system based on the modification information obtained from the image analysis system, and outputs the print image and the prompt to the artificial intelligence system.

5. The print image editing system according to claim 3, further comprising a first confirmation message display unit that displays a message confirming the content of the first text in accordance with the identification result of the first text by the image analysis system.

6. The print image editing system according to claim 2, wherein the artificial intelligence system modifies a portion of the print image to the replaced image when the modification instruction includes a second text specifying a replaced image representing the modified appearance as the data of the modified content.

7. The print image editing system according to claim 6, wherein the artificial intelligence system performs the modification by replacing the image to be replaced, which is part of the print image, with the image to be replaced, when the modification instruction includes a third text specifying the image to be replaced as the designated data.

8. The print image editing system according to claim 7, further comprising a second confirmation message display unit that displays a message confirming the content of the image to be replaced in accordance with the identification result of the image to be replaced by the artificial intelligence system.

9. The print image editing system according to claim 2, wherein the artificial intelligence system modifies a portion of the print image to the replaced image if the modification instruction includes a replaced image representing the modified appearance as the modification content data.

10. The print image editing system according to claim 2, wherein the artificial intelligence system modifies a portion of the print image to an image of the replacement string when the modification instruction includes a fourth text specifying the replacement string as the modification content data.

11. The print image editing system according to claim 10, wherein the artificial intelligence system performs the modification by replacing the image of the string to be replaced, which is part of the print image, with the image of the string of the string to be replaced, when the modification instruction includes a fifth text specifying the string to be replaced to be replaced as the designated data.

12. The print image editing system according to claim 11, further comprising a third confirmation message display unit that displays a message confirming the content of the replacement string in accordance with the identification result of the fifth text by the artificial intelligence system.

13. The print image editing system according to claim 2, wherein the prompt output unit detects the edge of the specified region in the print image to be modified when the modification instruction includes the specified region as specified data and outputs it to the artificial intelligence system along with the prompt, and the artificial intelligence system performs the modification on the portion identified by the edge.

14. The print image editing system according to claim 2, wherein the artificial intelligence system deletes a portion of the print image if the modification instruction includes a sixth text that instructs deletion.

Images