Computer program for causing printing execution unit of printing device to print image, device, and method

Abstract

The present invention provides a novel technique related to image correction. The present description discloses a computer program for causing the printing execution unit of a printing device to print an image. The computer program may cause a computer to function as: a first reception unit that receives, from a user, designation of target data representing the target image to be printed; a supply unit that, when designation of the target data is received, supplies a learning model with correspondence data corresponding to the target data and correction information indicating that correction corresponding to at least one correction item among one or more predetermined correction items should be executed on the target image; a second reception unit that receives, from the learning model, correction data generated according to the correspondence data and the correction information; and a print control unit that causes a print execution unit to execute printing of a corrected image represented by the correction data.

Description

Computer program, device, and method for causing a print execution unit of a printing device to print an image

[0001] The present specification relates to a technique for causing a print execution unit of a printing device to print an image.

[0002] Japanese Patent Application Laid-Open No. 2006-129999 discloses a technique for causing a printer to execute printing using an application installed on a terminal.

[0003] Japanese Patent Application Laid-Open No. 2022-085588

[0004] Provided herein are novel techniques for image correction.

[0005] This specification discloses a computer program for causing a print execution unit of a printing device to print an image. The computer program may cause a computer to function as the following units: a first reception unit that receives, from a user, designation of target data representing a target image to be printed; a supply unit that, when the designation of the target data is received, supplies, to a learning model, correspondence data corresponding to the target data and correction information indicating that a correction corresponding to at least one of one or more predetermined correction items should be performed on the target image; a second reception unit that receives, from the learning model, correction data generated according to the correspondence data and the correction information; and a print control unit that causes the print execution unit to print the corrected image represented by the correction data.

[0006] According to the above configuration, when the designation of target data is accepted, correspondence data corresponding to the target data and correction information indicating that correction corresponding to at least one of one or more predetermined correction items should be performed on the target image are supplied to the learning model. Then, the correction data is accepted from the learning model, and the corrected image represented by the correction data is printed. In this way, a novel technology for image correction is realized.

[0007] A computer-readable recording medium storing the above computer program is also novel and useful. Also, a device and method for causing a print execution unit of a printing device to print an image are novel and useful. The recording medium may be a single medium or multiple media.

[0008] FIG. 10 is a diagram illustrating the configuration of a communication system. FIG. 11 is a diagram illustrating screen transitions of screens displayed on a terminal. FIG. 12 is a diagram illustrating images represented by each file. FIG. 13 is a flowchart of processing executed by an app. FIG. 14 is a diagram illustrating screen transitions of case A. FIG. 15 is a diagram illustrating screen transitions of case B. FIG. 16 is a diagram illustrating screen transitions of case C. FIG. 17 is a diagram illustrating screen transitions of case D. FIG. 18 is a diagram illustrating screen transitions continuing from FIG. 10. FIG. 19 is a flowchart of processing executed by an app.

[0009] (First embodiment) (Configuration of communication system; FIG. 1) As shown in FIG. 1, a communication system 2 includes a terminal 10, a printer 100, and an artificial intelligence (AI) server 200. The terminal 10 and the printer 100 are connected to a local area network (LAN) 4. The terminal 10 and the printer 100 can communicate with each other via the LAN 4. The LAN 4 may be a wired LAN or a wireless LAN. The AI ​​server 200 is installed on the Internet 6. The devices 10, 100 and the AI ​​server 200 can communicate with each other via the LAN 4 and the Internet 6.

[0010] (Configuration of terminal 10) The terminal 10 is a portable terminal device such as a mobile phone, smartphone, PDA, or tablet PC. In a modified example, the terminal 10 may be a stationary PC, laptop PC, or the like. The terminal 10 includes an operation unit 12, a display unit 14, a communication interface 16, and a control unit 30. Each unit 12 to 30 is connected to a bus line. Hereinafter, the interface will be referred to as "I / F."

[0011] The operation unit 12 is a user interface that allows a user to input various information into the terminal 10. The operation unit 12 includes, for example, a touch panel for displaying software keys (operation objects), hardware keys, or both. The hardware keys include, for example, buttons or switches. The display unit 14 is a display or panel for displaying various information. The panel may or may not be a touch panel. The panel is, for example, a liquid crystal panel or an organic EL panel. The communication I / F 16 is connected to the LAN 4.

[0012] The control unit 30 includes a CPU 32 and a memory 34. The memory 34 includes a main storage device and an auxiliary storage device. For example, the main storage device includes RAM and cache memory. For example, the auxiliary storage device may be a ROM, flash memory, solid-state drive (SSD), hard disk drive (HDD), or a combination thereof. The auxiliary storage device of the memory 34 stores an OS program 36, application programs 38, and a modification flag 40. Hereinafter, the OS program and application programs will be referred to as "OS" and "apps," respectively. The OS 36 controls the basic operations of the terminal 10. For example, the app 38 causes a printer to perform a printing function. The app 38 is downloaded, for example, from a server on the Internet 6 and installed on the terminal 10. In other words, the app 38 is a so-called native app. The CPU 32 performs various processes in accordance with programs loaded from the auxiliary storage device to the main storage device. In the following description, various processes that are implemented by the CPU 32 in accordance with a program may be described as being performed primarily by the OS 36 or the application 38 .

[0013] The application 38 can cause the AI ​​server 200 to correct the target image to be printed. As will be described in detail later, "correcting the target image" in this specification includes (1) proofreading and / or editing text included in the target image, (2) converting a table included in the target image into a graph, (3) drawing an image included in the target image that is drawn across multiple pages on a single page, (4) transforming a graphic included in the target image, (5) converting the size of text included in the target image to a size that matches the print settings, and (6) changing the image included in the target image to an image that matches the printer's capabilities. Here, "proofreading" means correcting errors in the text content. "Proofreading" means correcting typographical errors. The above examples are merely examples of "correction," and "correction" is not limited to these examples. "Transforming a graphic" does not simply enlarge or reduce the graphic, but also includes, for example, changing the relative positional relationship of each element that makes up the graphic, changing the relative angles of various lines that make up the graphic, etc.

[0014] The memory 34 further stores a correction flag 40 and a plurality of correction items. When a user specifies a file representing a target image to be printed, the correction flag 40 indicates either "on," which indicates that the AI ​​server 200 is to execute correction of the target image, or "off," which indicates that the AI ​​server 200 is not to execute correction of the target image. The user can change the correction flag 40. Each of the plurality of correction items is, for example, an item that is predetermined by the vendor of the printer 100. Each of the plurality of correction items in this embodiment corresponds to the above corrections (1) to (6).

[0015] (Configuration of Printer 100) The printer 100 is a peripheral device capable of executing a printing function, such as a peripheral device for the terminal 10. In a modified example, the printer 100 may be a multi-function device capable of executing a scanning function, a facsimile function, and the like in addition to a printing function. The printer 100 is assigned a device ID "dv1" that identifies the printer 100. The device ID is, for example, a model name, an IP address, or a combination thereof. In this specification, the device ID is the model name. The model name is model identification information for the device. The model name may be a higher-level model name that encompasses multiple model names. For example, the model name may be "ABC Series," which encompasses the model names ABC1 and ABC2. The printer 100 includes an operation unit 112, a display unit 114, a communication I / F 116, a print execution unit 118, and a control unit 130. Each of the units 112 to 130 is connected to a bus line.

[0016] The operation unit 112 is a user interface that allows a user to input various information to the printer 100. The operation unit 112 includes, for example, a touch panel for displaying software keys, hardware keys, or both. Hardware keys include, for example, buttons or switches. The display unit 114 is a display or panel for displaying various information. The panel may or may not be a touch panel. The panel may also be, for example, a liquid crystal panel or an organic EL panel. The communication I / F 116 is connected to the LAN 4. The print execution unit 118 includes an inkjet print engine. The inkjet print engine includes a print head that ejects ink droplets. In a modified example, the print execution unit 118 may include an electrophotographic or thermal print engine. The electrophotographic print engine includes a photosensitive element and an exposure device that emits light to expose the photosensitive element. The thermal print engine includes a print head that generates heat using a heater.

[0017] The control unit 130 includes a CPU 132 and a memory 134. The memory 134 includes a main storage device and an auxiliary storage device. As an example, the main storage device includes a RAM and a cache memory. As an example, the auxiliary storage device may be a ROM, a flash memory, an SSD, an HDD, or a combination thereof. A program 136 is stored in the auxiliary storage device of the memory 134. The CPU 132 performs various processes in accordance with the program loaded from the auxiliary storage device to the main storage device.

[0018] (Configuration of AI Server 200) The AI ​​server 200 performs image correction. In this embodiment, as an example, a configuration in which the AI ​​server 200 is a physical server will be described. The AI ​​server 200 includes a communication I / F 216 and a control unit 230. The communication I / F 216 and the control unit 230 are connected to a bus line. The communication I / F 216 is connected to the Internet 6.

[0019] The control unit 230 includes a processor 232 and a memory 234. The processor 232 includes a CPU, a graphics processing unit (GPU), a neural processing unit (NPU), or a combination thereof. The memory 234 includes a main storage device and an auxiliary storage device. For example, the main storage device includes a RAM and a cache memory. For example, the auxiliary storage device may be a ROM, a flash memory, an SSD, an HDD, or a combination thereof. A program 236 is stored in the auxiliary storage device of the memory 234. The processor 232 performs various processes in accordance with the program loaded from the auxiliary storage device to the main storage device.

[0020] A trained model is incorporated into the program 236. The AI ​​server 200 performs image correction based on the trained model. The trained model includes a program and various parameters. The trained model includes, for example, a language model such as a large language model (LLM) or a small language model (SLM).

[0021] (Specific Case; Figure 2) Next, a specific case will be described with reference to Figure 2. Note that, in the following, for ease of understanding, when describing the processing performed by the CPU (e.g., 32, 132) or processor (e.g., 232) of each device (e.g., terminal 10, printer 100, AI server 200), each device will be described as the subject of the processing, rather than each CPU or processor as the subject of the processing. Furthermore, communication between each device is performed via a communication I / F (e.g., 16, 116, 216). Therefore, in the following description, when describing communication, the phrase "via a communication I / F" will be omitted.

[0022] In the initial state of FIG. 2, information about one or more printers including the printer 100 (for example, the printer's device ID, IP address, etc.) has already been registered in the memory 34 (i.e., the application 38) of the terminal 10.

[0023] When the terminal 10 receives an application launch operation from the user to launch the application 38, the terminal 10 launches the application 38 and displays a home screen SC1 on the display unit 14. As shown in Fig. 2, the home screen SC1 includes one or more device objects OB11 and OB12 and a plurality of function objects OB13 to OB15.

[0024] Device object OB11 is an object corresponding to a registered printer 100 and includes the device ID "dv1" of the printer 100. As described above, device ID "dv1" is the model name of the printer 100. Note that device ID "dv1" may also include the printer 100's IP address in addition to the printer 100's model name. Although partially hidden in FIG. 2 , device object OB12 is an object corresponding to another printer registered on terminal 10 and includes the device ID of that other device. Note that in a situation where only one device (i.e., printer 100) is registered on terminal 10, device object OB12 may be an object for registering a new printer. Function objects OB13 to OB15 are objects corresponding to a document printing function, a photo printing function, and an audio printing function, respectively. The document printing function is a function for printing images represented by a document file (e.g., a Word file, a PDF file, a Text file, etc.). The photo printing function is a function for printing an image represented by a photo file (e.g., a JPEG file, etc.). The audio printing function is a function for printing an image represented by an audio file (e.g., an MP3 file, etc.). Here, an image represented by an audio file is an image represented by a document file obtained by transcribing the audio represented by the audio file.

[0025] When the terminal 10 receives from the user at T10 the selection of a function object OB13 corresponding to the document printing function, it displays a file selection screen SC2 on the display unit 14. As shown in Fig. 2, the file selection screen SC2 includes a plurality of file objects OB21 to OB23 corresponding to a plurality of document files stored in the memory 34, and a cancel object OB24. Each of the file objects OB21 to OB23 includes a file name (e.g., F1 to F3).

[0026] Now, with reference to FIG. 3 , the files with file names F1 and F2 will be described. The file with file name F1 contains one page of image data representing one page of image data. The one page of image data includes the string "XXX" and the string "A is is X, B is Y, and C is Z." In particular, the latter string contains a typographical error. That is, "is" is written twice, and the sentence ends with a comma instead of a period. The one page of image data also includes one table and one illustration. The file with file name F2 contains three pages of image data representing three pages of image data. Each of the three pages of image data is a divided image obtained by dividing a single image having a tree structure. That is, in the file with file name F2, one image is drawn across three pages.

[0027] (Processing Realized by the App; Figure 4) Next, processing realized by the app 38 of the terminal 10 will be described with reference to Figure 4. The processing in Figure 4 is triggered by the selection of one file object on the file selection screen SC2 (see T20 in Figure 2). Hereinafter, the file corresponding to the file object selected here will be referred to as the "target file."

[0028] In S2, the CPU 32 converts the image data (e.g., vector image data) included in the target file to generate print preview image data. The print preview image data is data corresponding to the print preview image to be displayed on the print preview screen, and has an RGB bitmap format. Because this print preview image corresponds to the original image before modification, it will be referred to as the "original preview image" below.

[0029] In S10, the CPU 32 determines whether the correction flag 40 in the memory 34 indicates ON or OFF. If the correction flag 40 indicates ON (YES in S10), the CPU 32 proceeds to S12, and if the correction flag 40 indicates OFF (NO in S10), the CPU 32 proceeds to S50.

[0030] In S12, the CPU 32 displays the print preview screen on the display unit 14, and also displays a correction item dialog box superimposed on the print preview screen on the display unit 14. For example, Fig. 5 shows how the correction item dialog box D1 is displayed superimposed on the print preview screen SC3.

[0031] The correction item dialog box D1 includes six objects OB31 to OB36 corresponding to the six correction items in the memory 34, an OK button OB37, and a cancel button OB38. Each of the objects OB31 to OB36 includes a check box. When the correction item dialog box D1 is first displayed, the check boxes included in two of the six objects OB31 to OB36, objects OB31 and OB32, are checked, and the check boxes included in the other objects OB33 to OB36 are unchecked. In this way, correction items that are checked by default are pre-stored in the memory 34. In the correction item dialog box D1, the user can select the content of the correction to be performed on the target image.

[0032] Although hidden by the correction item dialog box D1, the print preview screen SC3 also includes various other objects. For example, when a cancel button OB38 included in the correction item dialog box D1 is selected, the correction item dialog box D1 is cleared and those objects are displayed. The print preview screen SC3 further includes an original preview image, a device object OB41, a settings object OB42, an AI correction object OB43, and a print object OB44 (see the print preview screen SC4 in FIG. 5 ).

[0033] The device object OB41 includes the device ID and paper size of the printer 100 (i.e., the selected printer 100) corresponding to the device object OB11 whose entirety is displayed when the function object OB13 is selected on the home screen SC1. That is, the device object OB41 is an object indicating the device that prints the image corresponding to the print preview image and the size of the print medium. Note that, in a modified example, the device object OB41 may not include the paper size, or may include information other than the paper size (e.g., paper type (plain paper, glossy paper, etc.)). Furthermore, the device object OB41 may be configured to allow settings such as paper size to be changed when the user selects the device object OB41. The setting object OB42 is an object for displaying a print setting screen. The AI ​​correction object OB43 is an object for correcting the target image. The print object OB44 is an object for printing the image.

[0034] In S14, the CPU 32 accepts the selection of one or more correction items (i.e., selection of check boxes) from the user in the correction item dialog D1, and then accepts the selection of the OK button OB37.

[0035] In S16, the CPU 32 transmits the print preview image data, one or more selected correction items, device information, and print setting values ​​to the AI ​​server 200 via the communication I / F 16. The device information includes information indicating the capabilities of the printer that will print the target image (i.e., the selected printer 100). In particular, in this embodiment, the device information includes information indicating that the print execution unit 118 of the printer 100 is equipped with an inkjet print engine. The print setting values ​​include various setting values ​​(e.g., paper size, color / monochrome, layout settings, etc.) that should be used when the printer prints the target image. The print setting values ​​are default values ​​that are predetermined for the selected printer 100.

[0036] When the AI ​​server 200 receives the above various information from the terminal 10, it corrects the target image represented by the print preview image data based on the various information to generate corrected data. Details of the correction of the target image will be described later. The corrected data has the same data format as the print preview image data (i.e., RGB bitmap format in this embodiment). The AI ​​server 200 then transmits the corrected data to the terminal 10. The AI ​​server 200 also transmits information identifying the corrections to the terminal 10 together with the correction data. This information may be included in the correction data.

[0037] In S18, the CPU 32 receives the correction data and information identifying the correction portion from the AI ​​server 200 via the communication I / F 16.

[0038] In S20, the CPU 32 displays a print preview screen on the display unit 14. FIG. 5 shows a print preview screen SC4 as an example of the print preview screen displayed in S20. The print preview screen SC4 includes two pages of print preview images. The print preview image for the first page is a corrected preview image P11 corresponding to the correction data (i.e., a print preview image representing the target image after correction). The print preview image for the second page is an original preview image corresponding to the print preview image data (i.e., a print preview image representing the target image before correction). In the corrected preview image P11 for the first page, the corrected parts are highlighted based on information identifying the corrected parts. This allows the user to easily recognize the parts corrected by the AI ​​server 200.

[0039] The character string "1 / 2" in the print preview screen SC4 indicates that the currently displayed modified preview image P11 is the print preview image of the first page of two pages of print preview images. Although not shown in the figure, when the CPU 32 receives an operation from the user to slide the area displaying the modified preview image P11 of the first page left or right, it displays the original preview image of the second page in place of the modified preview image P11 of the first page. In this way, by viewing the print preview screen displayed in S20, the user can easily compare the target image after modification with the target image before modification.

[0040] In S22, the CPU 32 displays a dialog D2 (see FIG. 5) on the display unit 14, superimposed on the print preview screen displayed in S20. The dialog D2 includes a message indicating that the target image has been corrected by the AI ​​server 200, an OK button OB45, and an Undo button OB46.

[0041] In S24, the CPU 32 determines whether the modification made to the target image is acceptable to the user. Specifically, when the CPU 32 receives the user's selection of the OK button OB45 in the dialog D2, it determines that the modification made to the target image is acceptable to the user (YES in S24) and proceeds to S30. On the other hand, when the CPU 32 receives the user's selection of the Undo button OB46 in the dialog displayed in S22, it determines that the user does not accept the modification made to the target image (NO in S24) and proceeds to S40.

[0042] The CPU 32 updates the print preview screen in S30. FIG. 5 shows a print preview screen SC5 as an example of the print preview screen updated in S30. Specifically, the CPU 32 deletes the original preview image from the revised preview image and original preview image included in the print preview screen displayed in S20. Furthermore, the CPU 32 displays a revised preview image P12 in which the revised portions are not highlighted, instead of the revised preview image P11 in which the revised portions are highlighted. The print preview screen SC5 includes only one page of print preview images. The character string "1 / 1" in the print preview screen SC5 indicates that the currently displayed print preview image is the revised preview image P12 for the first page of the one-page print preview image.

[0043] In S40, the CPU 32 updates the print preview screen. Specifically, the CPU 32 deletes the modified preview image from the modified preview image and original preview image included in the print preview screen displayed in S20. In this way, a print preview screen including the print preview image desired by the user is displayed in accordance with the user's selection in the dialog D2.

[0044] In addition, in S50, the CPU 32 displays a print preview screen on the display unit 14. The print preview screen includes an original preview image. Fig. 9 shows a print preview screen SC43 as an example of the print preview screen displayed in S50. As shown in Fig. 9, the print preview screen SC43 includes an AI-modified object OB43 that has an unselectable display mode (grayed out in this embodiment).

[0045] In S52, the CPU 32 monitors whether a selection of an object included in the print preview screen is accepted from the user. If the CPU 32 accepts the selection of the AI ​​correction object OB43 from the user (YES in S52), the CPU 32 returns to the processing of S12. If the CPU 32 accepts the selection of the print object OB44 from the user (NO in S52 (print object selected)), the CPU 32 proceeds to S60. If the CPU 32 accepts the selection of the setting object OB42 (NO in S52 (setting object selected)), the CPU 32 proceeds to S70.

[0046] In S60, the CPU 32 converts the print preview image data (or modified data) corresponding to the currently displayed print preview image into print data and sends the converted print data to the printer 100. The print data has a data format that can be interpreted by the printer 100, for example, a CMYK bitmap format. As a result, the target image is printed by the printer 100. In particular, if YES is returned in S24, the printer 100 prints the modified target image. When the processing of S60 ends, the processing of FIG. 4 ends.

[0047] Furthermore, the CPU 32 executes a setting change process in S70. The setting change process includes a process of displaying various setting screens such as a print setting screen and an AI correction setting screen, and a process of storing setting values ​​changed by the user in the memory 34. When the setting change process is completed, the CPU 32 returns to displaying the print preview screen and returns to the monitoring process in S52.

[0048] (Case A; Figure 5) Next, a specific case realized by the process of Figure 4 will be described. First, case A will be described with reference to Figure 5. Figure 5 is a continuation of Figure 2. In the initial state of case A, the modification flag 40 in the memory 34 indicates ON.

[0049] At T20 in FIG. 2, the terminal 10 accepts the selection of file name F1 (trigger in FIG. 4). In this case, the terminal 10 converts one page of image data (see FIG. 3) contained in the file having file name F1 into one page of RGB bitmap image data to generate print preview image data (S2). Because the correction flag 40 indicates "on" (YES in S10), the terminal 10 displays the print preview screen SC3 on the display unit 14 and also displays the correction item dialog D1 on the display unit 14, superimposed on the print preview screen SC3 (S12). As described above, when the correction item dialog D1 is first displayed, each check box contained in the two objects OB31 and OB32 is checked.

[0050] In T30, the terminal 10 accepts the selection of the OK button OB37 from the user without changing the check boxes (i.e., selecting the two correction items "Review and Proofread Text" and "Visualize into Graph" corresponding to objects OB31 and OB32) (S14). In this case, the terminal 10 transmits the print preview image data, the two selected correction items "Review and Proofread Text" and "Visualize into Graph," device information, and print setting values ​​to the AI ​​server 200.

[0051] When the AI ​​server 200 receives the above various information from the terminal 10, it corrects the target image represented by the print preview image data based on the various information to generate corrected data. In this case, the AI ​​server 200 receives two correction items, "review and proofread text" and "visualize into graph," from the terminal 10, and therefore performs the following processing. First, the AI ​​server 200 performs correction for the correction item, "review and proofread text." Specifically, the AI ​​server 200 first performs OCR processing on the print preview image data to identify the text included in the print preview image. Then, the AI ​​server 200 uses the trained model to review, proofread, etc. the identified text. In this case, the trained model of the AI ​​server 200 determines that the string "A is is B" is a typographical error and corrects it to "A is B," and also determines that the string "C is Z," is a typographical error and corrects it to "C is Z."

[0052] Next, the AI ​​server 200 performs correction for the correction item "Visualization into a graph." Specifically, the AI ​​server 200 uses the trained model to identify one table included in the print preview image data. Then, the AI ​​server 200 uses the trained model to generate a bar graph in which each item in the first column of the table (i.e., "A," "B," and "C") is arranged on the horizontal axis and the height corresponds to each numerical value in the second column of the table (i.e., "70," "100," and "50"). Note that each item and each numerical value included in the table is identified by the above-mentioned OCR processing. In this way, the AI ​​server 200 generates corrected data in which the character string is corrected and which includes a bar graph instead of a table. The AI ​​server 200 transmits the corrected data and information identifying the corrections to the terminal 10.

[0053] When the terminal 10 receives the correction data and information identifying the correction portions from the AI ​​server 200, it displays a print preview screen SC4 on the display unit 14 (S20) and also displays a dialog D2 on the display unit 14 superimposed on the print preview screen SC4 (S22). The correction portions are highlighted in the correction preview image P11 included in the print preview screen SC4. Specifically, the correction portions in the text are underlined and the bar graph is surrounded by a frame. The user can easily recognize the correction portions by looking at the correction preview image P11 in which the correction portions are highlighted.

[0054] The user checks the edited preview image P11 and determines whether or not to accept the edits made to the target image. If the user accepts the edits, the user selects the OK button OB45 in the dialog D2. On the other hand, if the user rejects the edits, the user selects the Undo button OB46 in the dialog D2.

[0055] In this case, the user selects the OK button OB45 in the dialog D2 in T40 (YES in S24). In this case, the terminal 10 displays a print preview screen SC5 on the display unit 14 (S30). The edited portion is not highlighted in the edited preview image P12 included in the print preview screen SC5. This allows the user to recognize that the image corresponding to the edited preview image P12 will be printed.

[0056] In T50, the user selects print object OB44 in print preview screen SC5 (NO (print object selected) in S52). In this case, terminal 10 converts the modified data representing modified preview image P12 into print data in accordance with the print setting values ​​and sends the converted print data to printer 100 (S60). As a result, printer 100 prints the modified target image.

[0057] (Case B; Fig. 6) Next, case B will be described with reference to Fig. 6. In case B, the file selected by the user and the item to be modified are different from those in case A. Fig. 6 is a continuation of Fig. 2. In addition, in the initial state of case B, the modification flag 40 in the memory 34 is on.

[0058] The terminal 10 accepts the selection of file name F2 at T120 in Fig. 2 (trigger in Fig. 4). In this case, the terminal 10 converts three pages of image data (see Fig. 3) contained in the file having file name F2 into three pages of RGB bitmap image data to generate print preview image data (S2). Because the correction flag 40 indicates ON (YES in S10), the terminal 10 displays a print preview screen SC13 on the display unit 14 and also displays a correction item dialog box D1 on the display unit 14, superimposed on the print preview screen SC13 (S12; see correction item dialog box D1 in Fig. 5).

[0059] In T122, the terminal 10 accepts, from the user, an operation to uncheck the check boxes of the objects OB31 and OB32 and an operation to check the check boxes of the objects OB33 and OB34 (S14). In this case, the terminal 10 displays a correction item dialog D11 on the display unit 14, superimposed on the print preview screen SC13. Then, in T130, the terminal 10 accepts selection of the OK button OB37 in the correction item dialog D11 (S14). In this case, the terminal 10 transmits the print preview image data, the two selected correction items "Combine images from multiple pages onto one page" and "Adjust layout by transforming graphics," device information, and print setting values ​​to the AI ​​server 200.

[0060] Upon receiving the various pieces of information from the terminal 10, the AI ​​server 200 modifies the target image represented by the print preview image data based on the various pieces of information to generate modified data. In this case, the AI ​​server 200 receives two modification items from the terminal 10: "Combine multiple pages of images onto one page" and "Transform graphics to adjust layout." The AI ​​server 200 then executes the following process. Specifically, the AI ​​server 200 uses the trained model to identify three pages of images represented by the print preview image data (i.e., images with a tree structure). Then, based on the modification item "Transform graphics to adjust layout," the AI ​​server 200 transforms the image while maintaining the tree structure using the trained model. Specifically, the AI ​​server 200 places lower-level character strings such as "AAA" and "A1" below and to the right of the top-level character string "YYY" in the tree structure. In particular, the AI ​​server 200 combines the tree-structured image spanning three pages onto one page while maintaining the tree structure based on the modification item "Combine multiple pages of images onto one page." In this way, the AI ​​server 200 generates correction data representing an image of the transformed tree structure arranged together on one page. The AI ​​server 200 transmits the correction data and information identifying the correction portion to the terminal 10.

[0061] When the terminal 10 receives the correction data and information identifying the correction portions from the AI ​​server 200, it displays the print preview screen SC14 on the display unit 14 (S20) and also displays a dialog D2 on the display unit 14 superimposed on the print preview screen SC14 (S22). The print preview screen SC14 includes four pages of print preview images, including one page of corrected preview image P21 and three pages of original preview images. In particular, the corrected portions are highlighted in the corrected preview image P21. In this case, the entire image having a tree structure has been transformed, so the entire image is surrounded by a frame.

[0062] In this case, the user selects the OK button OB45 in the dialog D2 in T140 (YES in S24). In this case, the terminal 10 displays the print preview screen SC15 on the display unit 14 (S30). In the edited preview image P22 included in the print preview screen SC15, the edited portion is not highlighted.

[0063] In T150, the user selects print object OB44 in print preview screen SC15 (NO (print object selected) in S52). In this case, terminal 10 converts the modified data representing modified preview image P22 into print data in accordance with the print setting values ​​and sends the converted print data to printer 100 (S60). As a result, printer 100 prints the modified target image.

[0064] (Case C; Fig. 7) Next, case C will be described with reference to Fig. 7. In case B, the item to be modified selected by the user is different from case A. Fig. 7 is a continuation of Fig. 2. In addition, in the initial state of case C, the modification flag 40 in the memory 34 indicates ON.

[0065] At T20 in Fig. 2, the terminal 10 accepts the selection of file name F1 (trigger in Fig. 4). In this case, the terminal 10 converts one page of image data (see Fig. 3) contained in the file having file name F1 into one page of RGB bitmap image data to generate print preview image data (S2). Because the correction flag 40 indicates ON (YES in S10), the terminal 10 displays a print preview screen SC3 on the display unit 14 and also displays a correction item dialog box D1 on the display unit 14, superimposed on the print preview screen SC3 (S12; see correction item dialog box D1 in Fig. 5).

[0066] In T222, the terminal 10 accepts from the user an operation to uncheck the check boxes of the objects OB31 and OB32 and an operation to check the check box of the object OB35 (S14). In this case, the terminal 10 displays a correction item dialog D21 on the display unit 14, superimposed on the print preview screen SC3. Then, in T230, the terminal 10 accepts selection of the OK button OB37 in the correction item dialog D21 (S14). In this case, the terminal 10 transmits the print preview image data, the selected correction item "Optimize Character Size," device information, and print setting values ​​to the AI ​​server 200.

[0067] Upon receiving the above various information from the terminal 10, the AI ​​server 200 corrects the target image represented by the print preview image data based on the various information to generate corrected data. In this case, the AI ​​server 200 receives one correction item, "character size optimization," from the terminal 10 and therefore executes the following process. Specifically, the AI ​​server 200 first performs OCR processing on the print preview image data to identify the text included in the print preview image. Next, the AI ​​server 200 identifies the paper size included in the print setting values. The AI ​​server 200 then uses the trained model to determine whether the character size of the identified character string is appropriate for the identified paper size. Here, "character size inappropriate for the paper size" includes cases where the character size is too small for the paper size, making it difficult for the user to recognize characters when printed, or where the character size is too large for the paper size, resulting in unnecessary line breaks, page breaks, etc. In this case, the AI ​​server 200 determines that the character size of the string "A is is X, B is Y, and C is Z," is too small for the paper size, and performs a correction to increase the character size of the string, thereby generating corrected data. The AI ​​server 200 transmits the corrected data and information identifying the corrections to the terminal 10.

[0068] When the terminal 10 receives the correction data and information identifying the correction portion from the AI ​​server 200, it displays the print preview screen SC24 on the display unit 14 (S20) and also displays the dialog D2 on the display unit 14 superimposed on the print preview screen SC24 (S22). The correction portion is highlighted in the correction preview image P31 included in the print preview screen SC24. In this case, the character size of the character string "A is is X, B is Y, and C is Z," has been changed, so the character string is surrounded by a frame.

[0069] In this case, the user selects the OK button OB45 in the dialog D2 in T240 (YES in S24). In this case, the terminal 10 displays the print preview screen SC25 on the display unit 14 (S30). In the edited preview image P32 included in the print preview screen SC25, the edited portion is not highlighted.

[0070] In T250, the user selects print object OB44 in print preview screen SC25 (NO (print object selected) in S52). In this case, terminal 10 converts the modified data representing modified preview image P32 into print data in accordance with the print setting values ​​and sends the converted print data to printer 100 (S60). As a result, printer 100 prints the modified target image.

[0071] (Case D; Fig. 8) Next, case D will be described with reference to Fig. 8. In case D, the item to be modified selected by the user is different from case A. Fig. 8 is a continuation of Fig. 2. In addition, in the initial state of case D, the modification flag 40 in the memory 34 indicates ON.

[0072] At T20 in Fig. 2, the terminal 10 accepts the selection of file name F1 (trigger in Fig. 4). In this case, the terminal 10 converts one page of image data (see Fig. 3) contained in the file having file name F1 into one page of RGB bitmap image data to generate print preview image data (S2). Because the correction flag 40 indicates ON (YES in S10), the terminal 10 displays a print preview screen SC3 on the display unit 14 and also displays a correction item dialog box D1 on the display unit 14, superimposed on the print preview screen SC3 (S12; see correction item dialog box D1 in Fig. 5).

[0073] In T322, the terminal 10 accepts from the user an operation to uncheck the check boxes of the objects OB31 and OB32 and an operation to check the check box of the object OB36 (S14). In this case, the terminal 10 displays a correction item dialog D31 on the display unit 14, superimposed on the print preview screen SC3. Then, in T330, the terminal 10 accepts selection of the OK button OB37 in the correction item dialog D31 (S14). In this case, the terminal 10 transmits the print preview image data, the selected correction item "Consider device information," the device information, and the print setting values ​​to the AI ​​server 200.

[0074] Upon receiving the various pieces of information from the terminal 10, the AI ​​server 200 modifies the target image represented by the print preview image data based on the various pieces of information to generate modified data. In this case, the AI ​​server 200 receives one modification item, "Consider Device Information," from the terminal 10, and executes the following process. Specifically, the AI ​​server 200 first determines that the received device information indicates that the print execution unit 118 of the printer 100 is equipped with an inkjet print engine. In general, inkjet print engines are better at rendering image gradations (i.e., gradients) richer than, for example, electrophotographic print engines. Therefore, the AI ​​server 200 uses the trained model to adjust the gradation of the print preview image data and generate modified data representing an image with higher gradations than the original image. Specifically, in this case, the AI ​​server 200 adds shading to a portion of an illustration of an apple to create a multi-toned image. The AI ​​server 200 transmits the correction data and information identifying the correction portion to the terminal 10.

[0075] When the terminal 10 receives the correction data and information identifying the correction portion from the AI ​​server 200, it displays the print preview screen SC34 on the display unit 14 (S20) and also displays a dialog D2 on the display unit 14 superimposed on the print preview screen SC34 (S22). In the correction preview image P41 included in the print preview screen SC34, the portion corrected by the AI ​​server 200 is highlighted. In this case, the gradation of the illustration has been changed, so the illustration is surrounded by a frame.

[0076] In this case, the user selects the OK button OB45 in the dialog D2 in T340 (YES in S24). In this case, the terminal 10 displays the print preview screen SC35 on the display unit 14 (S30). The edited portion is not highlighted in the edited preview image P42 included in the print preview screen SC35.

[0077] In T350, the user selects print object OB44 in print preview screen SC35 (NO (print object selected) in S52). In this case, terminal 10 converts the modified data representing modified preview image P42 into print data in accordance with the print setting values ​​and sends the converted print data to printer 100 (S60). As a result, printer 100 prints the modified target image.

[0078] (Case E; Figures 9 and 10) Next, case E will be described with reference to Figures 9 and 10. Case E differs from case A in that the modification flag 40 in the memory 34 is initially off.

[0079] At T20 in Fig. 2, the terminal 10 accepts the selection of file name F1 (trigger in Fig. 4). In this case, the terminal 10 converts one page of image data (see Fig. 3) contained in the file having file name F1 into one page of RGB bitmap image data to generate print preview image data (S2). Because the correction flag 40 indicates OFF (NO in S10), the terminal 10 displays a print preview screen SC43 on the display unit 14 (S50). In the print preview screen SC43, the AI ​​correction object OB43 is displayed in an unselectable manner.

[0080] When the terminal 10 receives selection of a setting object OB42 in the print preview screen SC43 in T430 (NO (setting object selected) in S52), it displays a print setting screen SC44 on the display unit 14 (S70). The print setting screen SC44 includes a plurality of objects OB51 to OB56 and a back button OB57. Each of the objects OB51 to OB56 includes the current setting value of the corresponding item.

[0081] Object OB51 is an object for specifying the number of copies to print. Object OB52 is an object for specifying whether or not to perform double-sided printing (for example, "single-sided," "long-edge binding," or "short-edge binding"). Object OB53 is an object for specifying color / monochrome settings (i.e., either "color" or "monochrome"). Object OB54 is an object for specifying layout settings such as 1-in-1 or 2-in-1. Object OB55 is an object for selecting the page to be printed. Object OB56 is an object for selecting the correction flag 40 and the correction items. Back button OB57 is a button for displaying a print preview screen (for example, SC43).

[0082] When the terminal 10 accepts the selection of the object OB56 at T440, it displays the AI ​​correction setting screen SC45 of FIG. 10 on the display unit 14. The AI ​​correction setting screen SC45 includes an AI correction setting object OB61, six objects OB62 to OB67 corresponding to six correction items, an OK button OB68, and a cancel button OB69. The AI ​​correction setting object OB61 includes a radio button for specifying whether the correction flag 40 is on or off. Each of the objects OB62 to OB67 includes a check box. An item object that is checked on the AI ​​correction setting screen SC45 is a default correction item, and corresponds to a correction item that is checked by default in the correction item dialog D1 (see FIG. 5).

[0083] In T450, the terminal 10 accepts the selection of the radio button corresponding to ON of the AI ​​correction setting object OB61, and then accepts the selection of the OK button OB68, changing the value of the correction flag 40 from "OFF" to "ON" (S70). In this way, the user can change the correction flag 40 after the print preview screen SC43 is displayed. The user can also change the default correction items by operating the check boxes of each object OB62 to OB67. The terminal 10 then displays the print preview screen SC46 on the display unit 14. The print preview screen SC46 is similar to the print preview screen SC43 (see FIG. 9 ) except that the AI ​​correction object OB43 has a selectable display mode.

[0084] When the terminal 10 receives the selection of the AI ​​correction object OB43 in T460 (YES in S52), it displays the print preview screen SC3 on the display unit 14 and also displays the correction item dialog D1 on the display unit 14 so as to be superimposed on the print preview screen SC3 (S12, see FIG. 5). The subsequent processing is the same as in Case A.

[0085] (Effects of the First Embodiment) With the above configuration, when the selection of a target file is accepted, print preview image data obtained by converting the image data included in the target file and at least one correction item selected by the user from six predetermined correction items are supplied to the AI ​​server 200 (S16 in FIG. 4). Then, correction data is accepted from the AI ​​server 200 (S18), and the corrected target image represented by the correction data is printed (S60). In this way, a new technology for image correction is realized.

[0086] (Correspondence) The printer 100 and the print execution unit 118 are examples of a "printing device" and a "print execution unit," respectively. The AI ​​server 200 is an example of a "learning model." The CPU 32 and the app 38 of the terminal 10 are examples of a "computer" and a "computer program," respectively. The display unit 14 of the terminal 10 is an example of a "display device." A file having a file name F1, F2, etc. (i.e., a target file) is an example of "target data." Print preview image data generated by converting image data included in the target file is an example of "corresponding data (and image data)." The six correction items stored in the memory 34 are an example of "one or more correction items (and multiple correction items)." The one or more correction items transmitted from the terminal 10 to the AI ​​server 200 in S16 of FIG. 4 are an example of "correction information." The corrected target image is an example of a "corrected image." The correction flag 40 is an example of "flag information." In particular, the values ​​"on" and "off" of the correction flag 40 are examples of a "first value" and a "second value," respectively.

[0087] The correction item dialog D1 and dialog D2 are examples of a "selection screen" and an "inquiry screen", respectively. The print preview screen SC43 is an example of a "print preview screen". The setting object OB42 is an example of a "predetermined object". The AI ​​correction setting screen SC45 is an example of a "setting screen". The corrected preview image P11 is an example of a "print preview image corresponding to the correction data, in which the correction portion is highlighted". The corrected preview image P12 is an example of a "print preview image corresponding to the correction data". The original preview image P1 is an example of a "print preview image corresponding to the target data".

[0088] The process of T20 in FIG. 2 is an example of a process executed by a "first reception unit." The processes of S2, S12, S16, and S18 in FIG. 4 are examples of a process executed by a "conversion unit," a "first display control unit," a "supply unit," and a "second reception unit," respectively. The processes of S20, S22, S30, and S40 in FIG. 4 are examples of a process executed by a "second display control unit," a "third display control unit," a "fourth display control unit," and a "fifth display control unit," respectively. The process of S60 in FIG. 4 is an example of a process executed by a "print control unit." The process of S70 in FIG. 4 is an example of a process executed by a "sixth display control unit" and a "storage control unit."

[0089] Second Example Next, a second example will be described with reference to Fig. 4. The second example differs from the first example in that the modification item dialog box D1 is not displayed when a target file is selected in a situation where the modification flag 40 indicates ON.

[0090] As shown in FIG. 4 , when the correction flag 40 indicates ON (YES in S10 in FIG. 4 ), the CPU 32 skips S12 and S14 in FIG. 4 and executes S16. That is, when the CPU 32 accepts the selection of file name F1 at T20 in FIG. 2 , it generates print preview image data (S2) and transmits to the AI ​​server 200, without displaying the print preview screen SC3 and the correction item dialog D1 on the display unit 14, the correction item pre-stored in the memory 34 as a default (i.e., the correction item selected on the AI ​​correction setting screen SC45 in FIG. 10 ). As a result, the terminal 10 displays the print preview screen SC4 on the display unit 14 without displaying the print preview screen SC3 and the correction item dialog D1. Thus, according to the configuration of the second embodiment, the user does not need to select a correction item.

[0091] Third Example Next, a third example will be described with reference to FIG. 12 . The third example differs from the first example in that an app 338 (i.e., a web app) implemented on the print server 300 is used instead of the app 38 (i.e., a native app) on the terminal 10. In a modified example, a cloud app may be used instead of the native app or the web app, or an app realized by a combination of at least two of the native app, the web app, and the cloud app may be used. Here, "a combination of at least two" refers to, for example, a combination of a native app and a web app (a so-called hybrid app), a combination of a native app and a cloud app, etc.

[0092] As shown in Fig. 1, the communication system 2 of the third embodiment further includes a print server 300. The print server 300 mediates communication between the terminal 10 and a printer (e.g., 100) and causes the printer to execute printing. In this embodiment, as an example, a configuration will be described in which the print server 300 is a physical server. The print server 300 includes a communication I / F 316 and a control unit 330. The communication I / F 316 is connected to the Internet 6.

[0093] The control unit 330 includes a CPU 332 and a memory 334. The memory 334 includes a main storage device and an auxiliary storage device. For example, the main storage device includes RAM and cache memory. For example, the auxiliary storage device may be ROM, flash memory, SSD, HDD, or a combination thereof. A program 336 and an app 338 are stored in the auxiliary storage device of the memory 334. The program 336 controls the basic operation of the print server 300. The app 338 is similar to the app 38 of the terminal 10 in the first embodiment. The CPU 332 performs various processes in accordance with the programs loaded from the auxiliary storage device to the main storage device. The memory 334 further stores a correction flag 340 and multiple correction items. The correction flag 340 and multiple correction items are similar to the correction flag 40 and multiple correction items stored in the memory 34 of the terminal 10 in the first embodiment.

[0094] First, a process for displaying the home screen SC1, etc., of FIG. 2 will be described. A terminal ID for identifying the terminal and information about one or more printers, including the printer 100 (e.g., the printer's device ID, IP address, etc.), are stored in advance in the memory 334 (i.e., the app 338) of the print server 300 in association with each other. The terminal 10 accepts an app launch operation from a user to launch the app 338 implemented on the print server 300. The app launch operation is, for example, an operation to access the print server 300 via a web browser. Upon accepting the app launch operation to launch the app 338, the terminal 10 transmits the terminal ID and an app launch operation ID to the print server 300. Upon receiving the terminal ID and the app launch operation ID from the terminal 10, the print server 300 identifies information about one or more printers stored in association with the terminal ID. Then, the print server 300 transmits home screen data (e.g., web page data) representing the home screen SC1 to the terminal 10, thereby displaying the home screen SC1 on the display unit 14 of the terminal 10 via a web browser or WebView.

[0095] When the terminal 10 accepts a selection of a functional object OB13 corresponding to the document printing function from the user, the terminal 10 transmits an object ID indicating that the functional object OB13 has been selected to the print server 300. Upon receiving the object ID from the terminal 10, the print server 300 transmits file selection screen data representing a file selection screen SC2 to the terminal 10, thereby displaying the file selection screen SC2 on the display unit 14 via a web browser or WebView. Each of the file objects OB21 to OB23 on the file selection screen SC2 corresponds to a document file stored in the memory 34 of the terminal 10. In a modified example, the document file may be stored in the memory 334 of the print server 300. In another modified example, when the terminal 10 accepts a selection of a functional object OB13 corresponding to the document printing function from the user, the file selection screen SC2 may be displayed on the display unit 14 by, for example, the OS 36, without transmitting the object ID to the print server 300.

[0096] (Processing Realized by the App; FIG. 11 ) Next, referring to FIG. 11 , processing realized by the app 338 installed on the print server 300 will be described. In a third embodiment in which the app 338 installed on the print server 300 is used instead of the app 38 of the terminal 10, processing similar to that shown in FIG. 4 is realized by the print server 300. The processing shown in FIG. 11 is triggered by the reception of file data representing the target file corresponding to a selected file object from the terminal 10 in response to the selection of a file object on the file selection screen SC2 (see FIG. 2 ) displayed on the display unit 14 of the terminal 10. The print server 300 also receives the device ID "dv1" of the printer 100 that will execute printing along with the file data. The processing of S102 and S110 is similar to the processing of S2 and S10 of FIG. 4 , respectively, except that the processing is performed by the CPU 332 of the print server 300.

[0097] In S112, the CPU 332 transmits print preview screen data representing a print preview screen (e.g., SC3) including the generated print preview image and correction item dialog data representing the correction item dialog D1 to the terminal 10 via the communication I / F 316. As a result, the print preview screen SC3 and the correction item dialog D1 are displayed on the terminal 10. The print server 300 transmits the print preview screen data and the correction item dialog data to the terminal 10, thereby displaying the print preview screen SC3, the correction item dialog D1, etc. on the display unit 14 of the terminal 10 via a web browser or WebView. When the terminal 10 receives the selection of an item to be corrected in the correction item dialog D1 and then receives the selection of the OK button OB37, it transmits to the print server 300 a correction item selection ID indicating the selected item to be corrected and an OK selection ID indicating that the OK button OB37 was selected.

[0098] In S114, the CPU 332 receives the correction item selection ID and the OK selection ID from the terminal 10 via the communication I / F 316. The processes of S116 and S118 are similar to the processes of S16 and S18 in FIG. 4, respectively, except that the main subject of the processes is the CPU 332 of the print server 300.

[0099] In S120, CPU 332 transmits print preview screen data representing a print preview screen (e.g., SC4) including the modified preview image represented by the modification data and the original preview image to terminal 10 via communication I / F 316. As a result, print preview screen SC4 is displayed on terminal 10.

[0100] In S122, the CPU 332 transmits dialog data representing the dialog D2 to the terminal 10 via the communication I / F 316. As a result, the dialog D2 is displayed on the terminal 10, superimposed on the print preview screen SC4.

[0101] In S124, the CPU 332 determines whether the user accepts the modifications made to the target image. Specifically, when the terminal 10 accepts the user's selection of the OK button OB45 in the dialog D2 displayed on the terminal 10, the terminal 10 transmits an OK selection ID to the print server 300. On the other hand, when the terminal 10 accepts the user's selection of the Undo button OB46 in the dialog D2 displayed on the terminal 10, the terminal 10 transmits an Undo selection ID to the print server 300. When the CPU 332 receives the OK selection ID from the terminal 10, the CPU 332 determines that the user accepts the modifications made to the target image (YES in S124) and proceeds to S130. On the other hand, when the CPU 332 receives the Undo selection ID from the terminal 10, the CPU 332 determines that the user does not accept the modifications made to the target image (NO in S124) and proceeds to S140.

[0102] In S130, the CPU 332 transmits print preview screen data representing a print preview screen that includes the modified preview image but does not include the original preview image to the terminal 10 via the communication I / F 316. As a result, the print preview screen (e.g., SC4) is displayed on the terminal 10.

[0103] In S140, the CPU 332 transmits print preview screen data representing a print preview screen that includes the original preview image and does not include the modified preview image to the terminal 10 via the communication I / F 316. As a result, the print preview screen is displayed on the terminal 10.

[0104] In S150, the CPU 332 transmits print preview screen data representing a print preview screen including the original preview image to the terminal 10 via the communication I / F 316. The print preview screen data transmitted here also includes information indicating that the AI-modified object OB43 is to be displayed in an unselectable display mode.

[0105] In S152, the CPU 332 monitors whether a selected object ID is received from the terminal 10. When the CPU 332 receives an AI-corrected object ID from the terminal 10 in response to receiving the selection of the AI-corrected object OB43 at the terminal 10 (YES in S152), the process returns to S112. When the CPU 332 receives a print object ID from the terminal 10 in response to receiving the selection of the print object OB44 at the terminal 10 (NO in S152 (print object ID received)), the process proceeds to S160. When the CPU 332 receives a setting object ID from the terminal 10 in response to receiving the selection of the setting object OB42 at the terminal 10 (NO in S152 (setting object ID received)), the process proceeds to S170. S160 is similar to the process of S60 in FIG. 4 except that the subject of the process is the CPU 332 of the print server 300. 4 except that the processing of S170 is performed by the CPU 332 of the print server 300 and that instead of processing such as displaying the print setting screen, processing such as sending print setting screen data representing the print setting screen to the terminal 10 is executed. Note that in the third embodiment, the device ID "dv1" may include, in addition to the model name, a Token for the print server 300 to communicate with the printer 100, authentication information, the global IP address to which the printer 100 belongs, and the like.

[0106] Furthermore, if the print server 300 cannot acquire local information such as a device ID from a web browser or the like due to security settings, a hybrid application may be created by combining the application 338 (i.e., a web application) implemented on the print server 300 with the application 38 (i.e., a native application) of the terminal 10. In this case, the native application can acquire local information such as a device ID stored in the memory 34 of the terminal 10 and transmit the acquired information to the print server 300. In this case, the print server 300 performs the same processing as described above via the native application. The same applies when a cloud application is used instead of a web application.

[0107] As with the first embodiment, the configuration of the third embodiment also realizes a new technique for image correction. In the third embodiment, the CPU 332 of the print server 300 is an example of a "computer," and the web application is an example of a "computer program." Furthermore, the display unit 14 of the terminal 10 is an example of a "display device."

[0108] Although specific examples of the technology disclosed in this specification have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and variations of the specific examples exemplified above. Modifications of the above examples are listed below.

[0109] (Variation 1) In the above embodiment, the application 38 is described as a native application installed on the terminal 10. In a variation, the application 38 may be an application installed on the printer 100, for example. In this case, the printer 100 may display the various screens of the above embodiment on the display unit 114 of the printer 100. In this variation, the CPU 132 of the printer 100 is an example of a "computer," and the display unit 114 of the printer 100 is an example of a "display device."

[0110] (Variant 2) The communication system 2 may not include the AI ​​server 200. In this variant, for example, an AI engine that executes various processes may be provided in the terminal 10 instead of the AI ​​server 200. The AI ​​engine includes a trained model. In this variant, the terminal 10 includes the "trained model." In another variant, an AI engine that executes various processes instead of the AI ​​server 200 may be provided in the printer 100. The AI ​​engine includes a trained model. In this variant, the printer 100 includes the "trained model."

[0111] (Variation 3) In the first embodiment described above, in S60 of FIG. 4 , the CPU 32 converts the correction data received from the AI ​​server 200 into print data and transmits the converted print data to the printer 100. In a variation, the CPU 32 may transmit the correction data itself received from the AI ​​server 200 and the print setting values ​​to the printer 100. In this case, the printer 100 may convert the correction data into print data in accordance with the correction data and print setting values ​​received from the terminal 10. In another variation, the terminal 10 may receive, from the AI ​​server 200, correction data having the same data format as the image data included in the target file. In this case, the terminal 10 may convert the correction data into image data having an RGB bitmap format, convert the image data into print data, and transmit the print data to the printer 100.

[0112] (Variation 4) The process of S2 in FIG. 4 can be omitted. That is, when accepting the selection of a target file, the CPU 32 does not need to convert the image data included in the target file into print preview image data. In this case, the CPU 32 may send the image data included in the target file itself to the AI ​​server 200 in place of the print preview image data in S16. In this variation, the image data included in the target file is an example of "corresponding data." Also, in this variation, the "conversion unit" can be omitted.

[0113] (Variation 5) In the print preview screen SC4, the revised preview image P12 may be displayed instead of the revised preview image P11. That is, the revised portion may not be highlighted in the print preview screen SC4. In this variation, the "second display control unit" may be omitted.

[0114] (Variation 6) In the above embodiment, the corrections in the text are highlighted by underlining them, surrounding them with a frame, or the like. The highlighting method is not limited to the method in the above embodiment. For example, in a variation, the corrections may be displayed in a different color from the uncorrected parts. In another variation, an object pointing to the corrections (for example, an arrow indicating the corrections) may be displayed.

[0115] (Variation 7) The CPU 32 can omit the processes of S22 and S24. That is, the CPU 32 does not need to display the dialog D2. In this case, the CPU 32 can execute the process of S30 to display a print preview screen that includes a modified preview image but does not include an original preview image, or can display a print preview screen that includes both the modified preview image and the original preview image. In this variation, the "third display control unit," "fourth display control unit," and "fifth display control unit" can be omitted.

[0116] (Variation 8) In the first embodiment described above, the print setting screen SC44 is displayed in response to the selection of setting object OB42 on the print preview screen SC43. Then, the AI ​​correction setting screen SC45 is displayed in response to the selection of object OB56 on the print setting screen SC44. In this variation, the print preview screen SC43 may include an AI correction setting object for displaying the AI ​​correction setting screen SC45. In this variation, the AI ​​correction setting object included in the print preview screen SC43 is an example of a "predetermined object."

[0117] (Modification 9) The AI ​​correction setting screen SC45 does not have to include the objects OB62 to OB67. That is, the AI ​​correction setting screen SC45 may only allow the correction flag 40 to be changed.

[0118] (Variation 10) In the above first embodiment, cases A to D, specific examples of correction of a target image performed by the AI ​​server 200 have been described. The correction performed by the AI ​​server 200 is not limited to the combination of correction items selected by the user in each of cases A to D. For example, in each of cases A and B above, a case in which two correction items out of six correction items are selected has been described, but one correction item may be selected, or three or more correction items may be selected. Similarly, in each of cases C and D above, a case in which one correction item out of six correction items is selected has been described, but two or more correction items may be selected.

[0119] (Variation 11) In the above embodiment, an example has been described in which the AI ​​server 200 proofreads the text (i.e., corrects spelling errors) when it receives the correction item "text review / proofreading" from the terminal 10. In addition to this, the AI ​​server 200 may proofread the text (i.e., correct content errors) based on a trained model.

[0120] (Variation 12) In the above embodiment, an example was described in which the AI ​​server 200 converts a table into a bar graph when it receives the correction item "Visualize into graph" from the terminal 10. The graph to be generated does not have to be a bar graph, and may be, for example, a line graph or a pie chart. The type of graph to be generated is determined by the trained model.

[0121] (Variation 13) In the above embodiment, an example was described in which the AI ​​server 200 receives the correction item "Combine multiple pages of images into one page" from the terminal 10 in a situation in which one image is included in three pages of print preview image data. As a variation, consider a situation in which two images are included in four pages of print preview image data. In such a situation, the AI ​​server 200 may perform a correction to combine the first of the two images into one page and a correction to combine the second of the two images into one page, thereby generating two pages of correction data. Generally speaking, the "image included in the target image" may be the entire target image or a portion of the target image.

[0122] (Variant 14) When the AI ​​server 200 receives the correction item "Adjust layout by transforming shapes" from the terminal 10, it may make a correction to adjust the layout, for example, by changing the position of line breaks in the text.

[0123] (Variant 15) When the AI ​​server 200 receives the correction item "optimize character size" from the terminal 10, it may, for example, convert the size of a table and the text contained in the table.

[0124] (Variation 16) In the above embodiment, when the AI ​​server 200 receives from the terminal 10 the correction item "Consider device information" and device information indicating that the printer has an inkjet print engine, the AI ​​server 200 applies multiple gradations to the illustration included in the original preview image. As described above, printers equipped with inkjet print engines excel at rendering images with rich gradations. That is, printers equipped with inkjet print engines excel at rendering images such as illustrations and graphs rather than character strings such as text. Therefore, when the AI ​​server 200 receives from the terminal 10 the correction item "Consider device information" and device information indicating that the printer has an inkjet print engine, the AI ​​server 200 may perform corrections such as reducing the amount of text and increasing the proportion of images, converting tables to graphs, etc. Conversely, printers equipped with electrophotographic print engines excel at rendering character strings such as text rather than rendering images such as illustrations and graphs, compared to printers equipped with inkjet print engines. Therefore, when the AI ​​server 200 receives from the terminal 10 the correction item "Consider device information" and device information indicating that the device is equipped with an electrophotographic printing engine, the AI ​​server 200 may make corrections such as reducing the amount of image and increasing the proportion of text, reducing the image quality, or converting a color image to a monochrome image.

[0125] (Variation 17) In the above embodiment, the server (e.g., the AI ​​server 200, the print server 300, etc.) was a physical server installed on the Internet 6 by the vendor of the printer 100 (hereinafter simply referred to as the "vendor"). In a variation, the server does not have to be a physical server. For example, the AI ​​server 200 may be a virtual server on the cloud, a virtual machine, etc. The AI ​​server 200 may be a single server or a collection of multiple servers. The AI ​​server 200 may be a server on an intranet or a server on the Internet 6. The AI ​​server 200 may also be installed on the Internet 6 by a business operator other than the vendor. The vendor may also use an environment provided by a cloud computing service rather than preparing the hardware for the AI ​​server 200 itself. In this case, the vendor may prepare a program (i.e., software) for the AI ​​server 200 and implement the AI ​​server 200 by installing it in the above-mentioned environment. The same applies to the print server 300.

[0126] (Variant 18) In the above embodiment, the processing of each step in Figure 4, etc. is realized by software (e.g., OS 36, app 38, program 136, etc.), but at least one of these processes may be realized by hardware such as a logic circuit.

[0127] The technical elements described in this specification or drawings may exhibit technical utility either alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technologies illustrated in this specification or drawings may achieve multiple objectives simultaneously, and achieving one of those objectives alone is technically useful.

[0128] Even if a claim is dependent on only some of the claims at the time of filing, this does not mean that the claim can depend on only those some of the claims. Each claim can also depend on other claims that were not dependent on the claim at the time of filing, to the extent that there is no technical contradiction. That is, the technology of each claim can be combined in various ways as follows. (Item 1) A computer program for causing a print execution unit of a printing device to print an image, the computer program causing a computer to function as the following units: a first reception unit that receives, from a user, designation of target data representing a target image to be printed; a supply unit that, when the designation of the target data is received, supplies, to a learning model, correspondence data corresponding to the target data and correction information indicating that a correction corresponding to at least one of one or more predetermined correction items should be performed on the target image; a second reception unit that receives, from the learning model, correction data generated according to the correspondence data and the correction information; and a print control unit that causes the print execution unit to print the corrected image represented by the correction data. (Item 2) The computer program according to item 1, further causing the computer to function as a first display control unit that, when the designation of the target data is accepted, causes a display device to display a selection screen for selecting the at least one correction item from a plurality of correction items. (Item 3) The computer program according to item 1, further causing the supply unit to supply the corresponding data and the correction information indicating that the correction corresponding to the at least one predetermined correction item should be performed on the target image to the learning model without a user selecting a correction item. (Item 4) The computer program according to any one of items 1 to 3, further causing the computer to function as a conversion unit that converts the target data to generate image data for displaying a print preview image, the corresponding data being the image data.(Item 5) The computer program according to any one of items 1 to 4, further causing the computer to function as a second display control unit that, when the correction data is accepted from the learning model, causes a display device to display a print preview image corresponding to the correction data, where the correction is highlighted. (Item 6) The computer program according to any one of items 1 to 5, further causing the computer to function as: a third display control unit that, when the correction data is accepted from the learning model, causes a display device to display a query screen to query a user whether or not to accept the correction; a fourth display control unit that, when the user instructs on the query screen to accept the correction, causes the display device to display a print preview image corresponding to the correction data; and a fifth display control unit that, when the user instructs on the query screen to not accept the correction, causes the display device to display a print preview image corresponding to the target data. (Item 7) The computer program further causes the computer to function as: a sixth display control unit that causes a display device to display a setting screen for setting whether or not to execute the correction; and a storage control unit that stores flag information in a memory, the flag information indicating a first value when a user sets on the setting screen to execute the correction, and the flag information indicating a second value when a user sets on the setting screen to not execute the correction; wherein the supply unit supplies the corresponding data and the correction information to the learning model when designation of the target data is accepted in a situation where the flag information in the memory indicates the first value; and does not supply the corresponding data and the correction information to the learning model when designation of the target data is accepted in a situation where the flag information in the memory indicates the second value.(Item 8) The computer program according to item 7, wherein the sixth display control unit causes the display device to display the setting screen when a predetermined object is selected on a print preview screen. (Item 9) The computer program according to item 7 or 8, wherein the setting screen is configured to allow the user to set at least one correction item from the one or more correction items. (Item 10) The computer program according to any one of items 1 to 9, wherein the one or more correction items include an item corresponding to reviewing and / or proofreading text included in the target image. (Item 11) The computer program according to any one of items 1 to 10, wherein the one or more correction items include an item corresponding to converting a table included in the target image into a graph. (Item 12) The computer program according to any one of items 1 to 11, wherein the one or more correction items include an item corresponding to drawing, within one page, an image included in the target image that is drawn across multiple pages in the target data. (Item 13) The computer program according to any one of items 1 to 12, wherein the one or more correction items include an item corresponding to transforming a shape included in the target image. (Item 14) The computer program of any one of items 1 to 13, wherein the one or more correction items include an item corresponding to converting a size of text included in the target image to a size that conforms to print setting values. (Item 15) The computer program of any one of items 1 to 14, wherein the one or more correction items include an item corresponding to changing an image included in the target image to an image that is suitable for the capabilities of the printing device.

[0129] 2: Communication system, 4: LAN, 6: Internet, 10: Terminal, 12, 112: Operation unit, 14, 114: Display unit, 16, 116, 216, 316: Communication I / F, 30, 130, 230, 330: Control unit, 32, 132, 332: CPU, 34, 134, 234, 334: Memory, 36: OS program, 38: Application program, 40, 340: Correction flag, 118: Print execution unit, 136, 236, 336: Program, 200: AI server, 232: Processor, 300: Print server

Claims

1. A computer program for causing a print execution unit of a printing device to print an image, the computer program causing a computer to function as the following units: a first reception unit that receives, from a user, designation of target data representing a target image to be printed; a supply unit that, when the designation of the target data is received, supplies, to a learning model, correspondence data corresponding to the target data and correction information indicating that a correction corresponding to at least one of one or more predetermined correction items should be performed on the target image; a second reception unit that receives, from the learning model, correction data generated in accordance with the correspondence data and the correction information; and a print control unit that causes the print execution unit to print the corrected image represented by the correction data.

2. The computer program according to claim 1, further causing the computer to function as a first display control unit that, when the designation of the target data is accepted, displays on a display device a selection screen for selecting the at least one item to be modified from a plurality of items to be modified.

3. The computer program of claim 1, wherein when the designation of the target data is accepted, the supply unit supplies the corresponding data and the correction information indicating that the correction corresponding to the at least one predetermined correction item should be performed on the target image to the learning model without the user selecting any correction items.

4. The computer program according to claim 1, further causing the computer to function as a conversion unit that converts the target data and generates image data for displaying a print preview image, and the corresponding data is the image data.

5. The computer program according to claim 1, further causing the computer to function as a second display control unit that, when the corrected data is accepted from the learning model, displays on a display device a print preview image corresponding to the corrected data, in which the corrected portion is highlighted.

6. The computer program according to claim 1, further comprising: a third display control unit that, when the corrected data is accepted from the learning model, causes a display device to display a query screen to query the user as to whether or not to allow the correction; a fourth display control unit that, when the user instructs on the query screen to allow the correction, causes the display device to display a print preview image corresponding to the corrected data; and a fifth display control unit that, when the user instructs on the query screen to not allow the correction, causes the display device to display a print preview image corresponding to the target data.

7. The computer program further causes the computer to function as: a sixth display control unit that causes a display device to display a setting screen for setting whether or not to execute the correction; and a storage control unit that stores flag information in memory, the flag information indicating a first value when a user sets on the setting screen that the correction is to be executed, and the flag information indicating a second value when a user sets on the setting screen that the correction is not to be executed; wherein the supply unit supplies the corresponding data and the correction information to the learning model when designation of the target data is accepted in a situation where the flag information in the memory indicates the first value; and does not supply the corresponding data and the correction information to the learning model when designation of the target data is accepted in a situation where the flag information in the memory indicates the second value.

8. The computer program according to claim 7, wherein the sixth display control unit causes the display device to display the setting screen when a predetermined object is selected on the print preview screen.

9. The computer program according to claim 7 or 8, wherein the setting screen is configured to allow setting of the at least one correction item from among the one or more correction items.

10. The computer program of claim 1, wherein the one or more correction items include an item corresponding to reviewing and / or proofreading text included in the target image.

11. The computer program of claim 1, wherein the one or more correction items include an item corresponding to converting a table included in the target image into a graph.

12. A computer program as described in claim 1, wherein the one or more correction items include an item corresponding to drawing an image included in the target image, the image being drawn across multiple pages in the target data, within one page.

13. The computer program according to claim 1, wherein the one or more correction items include an item corresponding to modifying a graphic included in the target image.

14. The computer program according to claim 1, wherein the one or more correction items include an item corresponding to converting the size of text included in the target image to a size that conforms to print setting values.

15. The computer program according to claim 1, wherein the one or more correction items include an item corresponding to changing an image included in the target image to an image suited to the capabilities of the printing device.

16. An apparatus for causing a print execution unit of a printing device to print an image, comprising: a first reception unit that receives, from a user, designation of target data representing a target image to be printed; a supply unit that, when the designation of the target data is received, supplies to a learning model correspondence data corresponding to the target data and correction information indicating that a correction corresponding to at least one correction item out of one or more predetermined correction items should be performed on the target image; a second reception unit that receives, from the learning model, correction data generated in accordance with the correspondence data and the correction information; and a print control unit that causes the print execution unit to print the corrected image represented by the correction data.

17. A method for causing a print execution unit of a printing device to print an image, comprising: a first receiving step for receiving, from a user, designation of target data representing a target image to be printed; a supply step for, when the designation of the target data is received, supplying to a learning model correspondence data corresponding to the target data and correction information indicating that a correction corresponding to at least one of one or more predetermined correction items should be performed on the target image; a second receiving step for receiving, from the learning model, correction data generated in accordance with the correspondence data and the correction information; and a print control step for causing the print execution unit to print the corrected image represented by the correction data.

Images