Printing system, printing control device, and printing method

Abstract

A printing system, a printing control device, and a printing method capable of effectively reducing a deviation in a remaining amount of a recording material are provided. The printing system includes an acquisition unit (81), an instruction unit (82), a reception unit (83), and a printing unit (50). The acquisition unit (81) acquires recording material information (RI) related to a plurality of recording materials different from each other in color. The instruction unit (82) causes a generation AI module (31) to generate an image based on the recording material information (RI). The reception unit (83) receives a generated image (GI) generated by the generation AI module (31). The printing unit (50) prints the received generated image (GI) using the recording material.

Description

Technical Field

[0001] This invention relates to a printing system, a printing control device, and a printing method. Background Technology

[0002] For example, Patent Document 1 discloses a color page printer using an electrophotographic printing method that employs various YMCK toners for printing. The CPU, constituting a microcomputer, calculates the remaining amount of each YMCK toner. Based on the remaining amount of each color toner obtained as a result of the calculation, the CPU controls the printer by reducing the unit consumption of the two colors of toner corresponding to the minimum and second minimum remaining amounts.

[0003] Furthermore, Patent Document 2 discloses an illustration background image generation system in which a server and multiple user terminals are connected via a network. The server includes: a prompt creation unit that creates prompts containing elements corresponding to the labels when it receives one or more labels selected by a user from a user terminal; an AI image generation unit that generates an image based on the prompts; and an image output unit that outputs the image. In recent years, there has been an increase in the likelihood of users printing AI images generated by AI with their preferred colors, which sometimes leads to discrepancies in the amount of remaining material recorded.

[0004] However, while the technology in Patent Document 1 can suppress deviations in the recording material, the amount of recording material that can be adjusted in a single print run is relatively small because the adjustment remains within a range that does not disrupt the tonal balance based on the printed image. Therefore, there is still room for further improvement in suppressing deviations in the recording material.

[0005] Patent document 1: Japanese Patent Application Publication No. 2004-93817.

[0006] Patent Document 2: Japanese Patent Application Publication No. 2024-120131. Summary of the Invention

[0007] A printing system for solving the above problems includes: an acquisition unit that acquires recording material information related to multiple recording materials with different colors; an instruction unit that instructs a generation AI module to generate an image based on the recording material information; a receiving unit that receives the generated image generated by the generation AI module; and a printing unit that prints the received generated image using the recording materials.

[0008] The printing control device that solves the above-mentioned problem is a printing control device installed in a terminal device for giving printing instructions to a printing apparatus, and includes: an acquisition unit that acquires recording material information related to multiple recording materials with different colors; an instruction unit that causes a generation AI module to generate an image based on the recording material information; a receiving unit that receives the generated image generated by the generation AI module; and an image transmission unit that transmits the received generated image to a printing apparatus that performs printing using the recording materials.

[0009] The printing method for solving the above problem includes: obtaining recording material information related to multiple recording materials with different colors; generating an image based on the recording material information and through a generation AI module; and printing the generated image generated by the generation AI module using the recording materials. Attached Figure Description

[0010] Figure 1 This is a schematic diagram illustrating the printing system in the implementation method.

[0011] Figure 2 A block diagram illustrating the electrical structure of a printing system.

[0012] Figure 3 This is a three-dimensional view of the printing apparatus.

[0013] Figure 4 This is a schematic diagram representing the input image.

[0014] Figure 5 This is a schematic diagram representing a screen containing recorded material information.

[0015] Figure 6 This is a schematic diagram illustrating the comparison processing performed by the comparison unit.

[0016] Figure 7 This is a schematic diagram illustrating the process of reducing the deviation in the recorded remaining amount of material.

[0017] Figure 8 A schematic diagram to reduce the deviation in recording the remaining amount of material.

[0018] Figure 9 This is a schematic diagram illustrating the image generation confirmation screen.

[0019] Figure 10 A schematic diagram illustrating the selection of a screen to represent an image generation method.

[0020] Figure 11 This is a schematic diagram showing the image generation condition confirmation screen.

[0021] Figure 12 A schematic diagram representing a frame specified for a composite image.

[0022] Figure 13 This is a schematic diagram illustrating the generated image.

[0023] Figure 14 This is a diagram illustrating the printing confirmation screen.

[0024] Figure 15 This is a diagram illustrating the save confirmation screen.

[0025] Figure 16 A sequence diagram illustrating a series of processes in a printing system.

[0026] Figure 17 This is a schematic diagram illustrating the input image in the second embodiment.

[0027] Figure 18 This is a schematic diagram illustrating a print selection screen that allows users to choose printable objects from generated images.

[0028] Figure 19 This is a sequence diagram illustrating a series of processes in the printing system of the third embodiment.

[0029] Figure 20 This is a schematic diagram representing the input image.

[0030] Figure 21 This is a schematic diagram illustrating a print selection screen that allows users to choose a printable object from an input image and a generated image.

[0031] Figure 22 This is a schematic diagram illustrating the comparison process performed by the comparison unit in the fourth embodiment.

[0032] Figure 23 A schematic diagram to reduce the deviation in recording the remaining amount of material.

[0033] Figure 24 This is a schematic bottom view illustrating the structure involved in the ejection of recording material from the printhead in the fifth embodiment. Detailed Implementation

[0034] Hereinafter, one embodiment of the printing system will be described with reference to the accompanying drawings.

[0035] Structure of printing system 10 First, refer to Figure 1 The structure of the printing system 10 will be explained below. For example... Figure 1 As shown, the printing system 10 includes a printing device 11, a terminal device 20, and servers 30 and 40. The printing device 11 performs printing on a medium M such as paper.

[0036] The printing apparatus 11 and the terminal device 20 are connected to each other via a LAN (Local Area Network). The terminal device 20 and the servers 30 and 40 are connected to each other via a network NW. The printing apparatus 11 can also be connected to the servers 30 and 40 via a network NW.

[0037] Terminal device 20 can be, for example, either Figure 1 The personal computer shown can also be a smartphone. The terminal device 20 includes an input unit 21 and a display unit 22. The terminal device 20 can also be, for example, a mobile phone, a mobile information terminal (PDA (Personal Digital Assistant)), or a tablet PC.

[0038] The first server 30 includes an AI generation module 31 (see reference). Figure 2 The second server 40 is a server that provides AI generation services. The second server 40 has an AI generation service providing module 41 (see reference). Figure 2 The second server 40 may also be managed by the manufacturer or sales company of the printing apparatus 11. The second server 40 provides services, for example, that are easy for users to utilize by mediating the AI ​​generation module 31 of the terminal device 20 and the first server 30.

[0039] Structure of printing apparatus 11 Next, refer to Figure 3 The structure of the printing apparatus 11 will now be explained. For example... Figure 3 As shown, the printing apparatus 11 is, for example, an inkjet printer. Furthermore, while the printing apparatus 11 is a so-called consumer printer used as a home printer in the home, it can also be a commercial printer used in offices, etc. The printing apparatus 11 is not limited to an inkjet printer; it can also be an electrophotographic device such as a laser printer that prints onto a medium M using toner as ink. The printing apparatus 11 can also be a dyeing printer that prints onto a fabric medium M, or a printer that prints onto a raw material such as a T-shirt as the medium M. The printing apparatus 11 can also be a printer that prints onto a transfer film capable of transferring images using the DTF (Direct to Film) method as the medium M.

[0040] The printing apparatus 11, as an inkjet printer, prints text or images by ejecting ink, an example of a recording material, toward a medium M. The printing apparatus 11 is also a liquid ejection device that ejects liquids such as ink toward a medium such as paper.

[0041] like Figure 3As shown, the printing apparatus 11 has a generally square box-shaped main body 12. An image reading unit 13 is provided on the upper side of the main body 12. An openable and closable cover 14 is also provided on the upper side of the main body 12. The image reading unit 13 and the cover 14 are opened and closed together. The cover 14... Figure 3 The closed position shown is open and closed between the closed position and the open position that exposes the interior of the main body 12.

[0042] The printing apparatus 11 includes an input unit 15 and a display unit 16. The input unit 15 is operated by the user to provide instructions to the printing apparatus 11. The display unit 16 displays various images. For example, the display unit 16 displays menu screens or various message information.

[0043] The printing apparatus 11 includes a supply box 17. The supply box 17 is located at the lower part of the main body 12. The supply box 17 is installed on the main body 12 in a pluggable state. The supply box 17 is configured to hold multiple media M such as paper. The supply box 17 supplies the held media M to the transport path within the main body 12. The printing apparatus 11 includes a transport section 55 (see reference 55) that transports the media M supplied from the supply box 17 along the transport path. Figure 2 ).

[0044] The printing apparatus 11 includes a printing unit 50. The printing unit 50 uses recording materials such as ink to print on a medium M that is conveyed along a transport path. The printing unit 50 includes a carriage 51 that can move in a width direction X1 intersecting the transport direction Y1 of the medium M, and a printing head 52 disposed at the lower part of the carriage 51. The printing head 52 prints on the medium M as the carriage 51 reciprocates in the width direction X1. The printing head 52 may also be a line head fixedly positioned opposite the transport path of the medium M.

[0045] The printing apparatus 11 has a discharge port 12A that opens on the front surface of the main body 12. The discharge port 12A discharges the printed medium M. The printing apparatus 11 has a discharge tray 18 that is slidably disposed at the front of the main body 12. The printed medium M discharged from the discharge port 12A is accumulated on the discharge tray 18.

[0046] Furthermore, the printing apparatus 11 has multiple recording material storage units 53 for storing recording materials such as ink. The printing apparatus 11 has the same number of recording material storage units 53 as the number of colors that the printing unit 50 can print on the medium M. Each of the multiple recording material storage units 53 individually stores recording materials such as ink of various colors. The printing apparatus 11 is configured such that the recording material storage units 53 are replaceable, for example, cartridges.

[0047] Figure 3The printing apparatus 11 shown in the example is a carriage-loaded type, in which a recording material reservoir 53 is mounted on a mounting portion (not shown) provided on the upper part of a carriage 51. When the recording material, such as ink, in the recording material reservoir 53 in the printing apparatus 11 is nearly exhausted or exhausted, the user will replace the recording material reservoir 53 for the printing apparatus 11. The user replaces the recording material reservoir 53 by opening the cover 14 and exposing the mounting portion on the upper part of the carriage 51.

[0048] Alternatively, the printing apparatus 11 can also be a non-carriage-mounted type, in which the recording material storage unit 53 is disposed at a predetermined position within the main body 12, which is different from the carriage 51. In the case of the non-carriage-mounted type, the recording material storage unit 53 mounted on the mounting section can be replaced by opening the cover provided at the front of the main body 12. When the liquid supply source is a liquid cartridge or other liquid container, the user needs to prepare a new recording material storage unit 53 in advance, for example, before the printing apparatus 11 is nearly depleted or exhausted.

[0049] Next, refer to Figure 2 The electrical and functional structures of the printing system 10 will be described below. Figure 2 As shown, the printing device 11 and the terminal device 20 are connected together in a communicable manner via a home network (HNW) such as a LAN. The communication via the home network (HNW) can be wireless communication such as Wi-Fi (registered trademark) or wired communication. Alternatively, it can be a short-range wireless communication device such as Bluetooth (registered trademark) or infrared communication. The home network (HNW) is connected to a network (NW) such as the Internet.

[0050] Terminal device 20 is connected to servers 30 and 40 via networks HNW and NW. Although printing device 11 can communicate within the range of home network HNW, it can also be connected to servers 30 and 40 via networks HNW and NW.

[0051] The printing apparatus 11 includes a communication unit 19, a control unit 60, an input unit 15, a display unit 16, an image reading unit 13, and a printing unit 50. The printing unit 50 includes a print head 52, a transport unit 55, and a maintenance unit 56.

[0052] The control unit 60 includes a computer 61. The computer 61 includes a CPU and a storage unit 62. The control unit 60 is not limited to components that perform software processing for all the processes it executes. For example, the control unit 60 may also include dedicated hardware circuitry (e.g., an application-specific integrated circuit: ASIC) that performs hardware processing for at least a portion of the processes it executes. That is, the control unit 60 can be configured to include one or more processors that operate according to a computer program (software), one or more dedicated hardware circuitry that executes at least a portion of various processes, or a combination thereof. The processor includes a CPU and storage units 62 (memory) such as RAM and ROM. The storage unit 62 stores program code or instructions configured to cause the CPU to execute processes. Memory, or computer-readable medium, includes all usable media that can be accessed by a general-purpose or special-purpose computer.

[0053] The computer 61 executes a program stored in the storage unit 62, thereby having a printing control unit 63 and a recording material consumption measurement unit 64 as its functional units.

[0054] The printing control unit 63 controls the printing unit 50. The printing apparatus 11 performs printing based on printing data received from the terminal device 20. The printing control unit 63 controls the print head 52 and the transport unit 55 based on the printing data, thereby printing text or images on the medium M. The printing control unit 63 maintains the print head 52 by controlling the maintenance unit 56. In the inkjet printing apparatus 11, as maintenance, cleaning is performed to remove thickened ink or foreign matter such as air bubbles from the nozzles of the print head 52 by forcibly discharging ink from the nozzles. The print head 52 consumes recording material through printing and maintenance.

[0055] The control unit 60 is capable of communicating with storage elements (not shown) disposed on the recording material storage units 53 mounted on the carriage 51. When the power to the printing apparatus 11 is turned on, the control unit 60 reads recording material information RI from each storage element of the plurality of recording material storage units 53 and stores it in the storage unit 62. The recording material information RI is information related to the recording material in each recording material storage unit 53. The recording material information RI may also include quantity information related to the amount of recording material in each recording material storage unit 53. The quantity information is only required to determine the amount of recording material in the recording material storage unit 53, and may be either the remaining amount or the consumed amount. In this embodiment, as an example of quantity information, information on the remaining amount, i.e., remaining amount information, is used. That is, the recording material information RI may also include information on the remaining amount of recording material. The recording material information RI may also include color information, consumption period, product number information, etc. of the recording material in each recording material storage unit 53.

[0056] The recording material consumption measurement unit 64 measures the amount of recording material consumed by the print head 52. The recording material consumption measurement unit 64 updates the remaining recording material data by subtracting the consumed recording material from the remaining recording material information RI. When the power to the printing apparatus 11 is turned off, the control unit 60 writes the recording material information RI, including the latest remaining recording material amount, stored in the storage unit 62, into the storage element of the recording material storage unit 53. When the power to the printing apparatus 11 is turned on again, the control unit 60 can read the recording material information RI, including the latest remaining recording material amount at the time of the previous power outage, from each storage element of the plurality of recording material storage units 53.

[0057] Terminal device 20 functions as printing control device 70 by installing a printing driver and a printing-related application. Terminal device 20 includes a printing control device 70 that gives printing instructions to printing device 11. Terminal device 20 includes a computer 71 and a storage unit. Computer 71 functions as printing driver 72 by executing the printing driver stored in the storage unit. Computer 71 also functions as image generation unit 73 by executing the printing-related application. The printing control device 70 included in terminal device 20 includes a software-based printing driver 72 and an image generation unit 73.

[0058] Although the terminal device 20 includes an image generation unit 73 in this embodiment, the printing device 11 may also include a device with the same function as the image generation unit 73. Figure 2 The image generation unit 65 is shown in the middle with a double-dotted line. Figure 2In order to distinguish between the two, the printing control device 70 side is designated as the first image generation unit 73, and the printing device 11 side is designated as the second image generation unit 65. In this embodiment, as an example, the image generation unit 73 provided by the terminal device 20 is used to generate an image by the AI ​​generation module 31. Furthermore, in a direct connection to the network NW... Figure 1 In the printing device 11 located at the bottom, the second image generation unit 65 on the side of the printing device 11 can also cause the AI ​​generation module 31 to generate an image.

[0059] The print driver 72 controls the printing apparatus 11. The terminal device 20 stores applications such as document creation applications, spreadsheet calculation applications, and presentation creation applications in its storage unit as applications not belonging to the print control device 70. These applications are used not only when the user creates documents, spreadsheets, charts, presentation materials (hereinafter also referred to as "presentation materials"), but also during the subsequent printing process. Each of these applications invokes the print driver 72 when the user provides printing instructions through the input unit 21. The invoked print driver 72 displays either the print condition setting screen 88, which allows selection of print condition information, or the recording material information RI obtained through communication with the printing apparatus 11 (both referred to as...). Figure 5 The information is displayed on the display unit 22. The user, through the operation of the input unit 21 of the terminal device 20, selects the required printing conditions and then instructs the execution of printing.

[0060] The printing driver 72 generates printing data containing printing instructions based on any one of the data from a document, form, chart, or presentation material that instructs the execution of printing. The printing data includes printed image data composed of dot data for each color of the recording material. The dot data can also be grayscale data with a specified grayscale range of 2 to 4 grayscale levels. The printing driver 72 sends the generated printing data to the printing apparatus 11.

[0061] Furthermore, the recording material consumption measurement unit 64 can also measure the amount of recording material consumed by the print head 52 during printing by counting the number of dots in the printed image data. When the printed image data contains multiple dots of different sizes, the recording material consumption measurement unit 64 can also measure the amount of recording material consumed for each color by accumulating the value obtained by multiplying the count value corresponding to each dot size by a coefficient corresponding to the dot size.

[0062] The printing control device 70 of the terminal device 20 includes the aforementioned first image generation unit 73. The first image generation unit 73 includes an acquisition unit 81, an indication unit 82, and a receiving unit 83. The first image generation unit 73 may further include a prompt generation unit 84. The prompt generation unit 84 may also include a comparison unit 85 for comparing the remaining amount of multiple recording materials.

[0063] The acquisition unit 81 acquires recording material information RI related to multiple recording materials of different colors. Recording material information RI refers to information related to multiple recording materials, including a first recording material whose color is a first color, and a second recording material whose color is a second color different from the first color. As an example, multiple colors can be four colors, six colors, eight colors, ten colors, etc. Multiple colors can be two or more, and can also include two, three, five, seven, nine, eleven, or more colors. Furthermore, multiple recording materials can also include different types of recording materials with the same color. For example, multiple recording materials can include both dye ink and pigment ink of the same color. For example, it can also include both dye ink and pigment ink, both types of black recording materials (e.g., black ink).

[0064] In the case of recording materials in multiple colors, such as four colors, including black (K), turquoise (C), magenta (M), and yellow (Y). Figure 2 In the example shown, the recorded material uses six colors. These six colors could also include, for example, black (K), turquoise (C), light turquoise (LC), magenta (M), light magenta (LM), and yellow (Y).

[0065] The first image generation unit 73 can also use a printed image PI, which the user has specified for printing, as an input image II (see reference). Figure 4 The input is also possible. The acquisition unit 81 can also acquire the recording material information RI after the printing of the printed image PI. The recording material information RI acquired after the printing of the printed image PI is the most up-to-date information compared to the recording material information RI acquired before printing.

[0066] The instruction unit 82 causes the AI ​​generation module 31 to generate an image based on the recorded material information RI. The instruction unit 82 may also, in addition to basing the image on the recorded material information RI, also base it on an input image II (e.g., reference image II). Figure 4The AI ​​generation module 31 generates an image based on the input image information IS. Furthermore, the instruction unit 82 can also generate an image based on information input by the user that includes prompts (e.g., text information), in addition to the recorded material information RI. The input image information IS may include prompts composed of text information in addition to the input image II. The instruction unit 82 can also display an input screen (see reference RI) prompting the user to select or input the information required for prompt generation. Figures 10-12 The image is displayed on the display unit 22. Furthermore, the indicator unit 82 can also cause the AI ​​generation module 31 to generate an image based not only on the recording material information RI, but also on the color information of the medium M to be printed. The color information may include, for example, the color of the substrate of the medium M before printing. The substrate of the medium M may also include a pattern. The color of the substrate containing the pattern may also be two or more colors. In addition, the medium M may include paper, thick paper, envelopes, memo pads, letter paper, OHP film (synthetic resin film), etc. Furthermore, details regarding various screens, including the input screen displayed on the display unit 22 of the terminal device 20 by the first image generation unit 73, will be described later.

[0067] The instruction unit 82 may also use the prompt generation unit 84 to generate prompts. The prompt generation unit 84 generates a prompt as an image generation command based on the recorded material information RI. The generated prompt may also include the input image information IS used by the instruction unit 82 in the instruction. The prompt may be only the recorded material information RI or a part thereof, or it may replace or include information obtained from the recorded material information RI or a part thereof.

[0068] The prompt generation unit 84 may also include a comparison unit 85. The comparison unit 85 compares quantities related to multiple recording materials based on the recording material information RI. The comparison unit 85 only needs to be able to compare quantities related to multiple recording materials that can be used in printing by the printing unit 50. For example, the comparison unit 85 may also compare the quantities of recording materials within multiple recording material receptacles 53 that supply multiple recording materials to the printing unit 50. Regarding the quantity comparison performed by the comparison unit 85, it can compare the quantities themselves, such as the amount of recording material consumed or the amount remaining, or it can indirectly compare the quantities by comparing quantity determination information that can indirectly determine the amount consumed or the amount remaining. As an example, the comparison unit 85 may also compare the remaining quantities of multiple recording materials. The comparison of remaining quantities is not limited to comparing the remaining quantities themselves; it can also indirectly compare the remaining quantities by comparing remaining quantity determination information.

[0069] The comparison unit 85 selects the color of the primary recording material consumed during the printing of the image GI by comparing quantities related to multiple recording materials. The comparison unit 85 can also select the consumed color by comparing the remaining quantities of multiple recording materials. That is, the comparison unit 85 may also include a consumed color selection unit that selects the primary consumed color among the colors of multiple recording materials based on the comparison result of the remaining quantities. When the comparison unit 85 selects the consumed color, the quantity of recording material used as the comparison object can be all of the multiple recording materials or a portion of two or more of the multiple recording materials. The comparison unit 85 can also select the color with the largest remaining quantity of recording material as the consumed color. The comparison unit 85 can also use a threshold to compare quantities related to multiple recording materials. Details regarding the comparison process of the comparison unit 85 using a threshold to select the consumed color will be described later.

[0070] The receiving unit 83 receives the generated image GI generated by the AI ​​generation module 31. The receiving unit 83 may also display the received generated image GI on the display unit 22.

[0071] Furthermore, the printing control device 70 may also include an image transmitting unit 86. The image transmitting unit 86 transmits the generated image GI received by the receiving unit 83 to the printing apparatus 11 that performs printing using recording material. The image transmitting unit 86 may be concurrently functioned by a transmitting function unit that transmits the printing data to the printing apparatus 11 after the printing driver 72 converts the printing image PI, which is the object of printing, into printing data that can be printed, or it may be provided separately from the printing driver 72. In the latter case, the generated image GI transmitted by the image transmitting unit 86 can also be converted into printing data that can be printed by the control unit 60 of the printing apparatus 11.

[0072] In this manner, the first image generation unit 73 causes the generation AI module 31 to generate image GI by means of an image generation command (prompt) based on the acquired recording material information RI, and causes the printing unit 50 to print the generated image GI.

[0073] The printing unit 50 uses recording material to print the received generated image GI. The printing apparatus 11 receives printing data of the generated image GI from the image transmission unit 86 (printing driver 72). The control unit 60 of the printing apparatus 11 controls the printing unit 50 based on the received printing data, thereby printing the generated image GI on the medium M. The control unit 60 controls the print head 52 based on the printing image data of the generated image GI contained in the printing data. The printing unit 50 ejects recording material (ink) of the color required for printing the colored generated image GI through the print head 52, thereby prioritizing the consumption of the recording material that consumes the color.

[0074] In addition, such as Figure 2 As shown, the second image generating unit 65 on the printing apparatus 11 side may also include an acquisition unit 81, an indication unit 82, and a receiving unit 83, similar to the first image generating unit 73. The second image generating unit 65 may also include a prompt generating unit 84. The prompt generating unit 84 may also include a comparison unit 85. The second image generating unit 65 can either enable the display unit 16 to display the desired content, or enable the display unit 22 of the terminal device 20 to display it via the network HNW.

[0075] Regarding the generation of AI module 31 Next, the AI ​​generation module 31 will be described. The AI ​​generation module 31 provided by server 30 can include the following modules: OpenAI's "ChatGPT", OpenAI's image generation AI tool "DALL-E" (e.g., DALL•E3), Google's "Gemini", Midjourney's "Midjourney", StabilityAI's "StableDiffusion", etc. Additionally, it can include "DeepArt", "PICSOROBAN (picso)", "Adobe Firefly", "MyEdit", "Bing ImageCreator", "Canva", "NovelAI", "AICpico", "StableStudio", "GAN (Generative Adversarial Network)", "VAE (Variational Auto-Encoder)", etc. Furthermore, the AI ​​generation module 31 can also be "Gamma" or "NapkinAI" as data generation AI. The AI ​​generation module 31, for example, incorporates a large-scale language model (LLM). Large-scale language models include learned models such as GPT (Generative Pre-trained Transformer). Large-scale language models can also include converter mechanisms.

[0076] The second server 40 is a generative AI service providing server that includes a generative AI service providing module 41. The second server 40 may also provide generative AI services to users using APIs (Application Programming Interfaces) provided by the company providing the generative AI module 31. Examples of such APIs include ChatGPT's API or Gemini's API. The second server 40 provides at least one image generation application (hereinafter also simply referred to as "the application") to the user of the printing device 11. The user downloads the application from the second server 40 to their own terminal device 20. By launching the application on the terminal device 20, the user can utilize the AI ​​image generation service provided by the second server 40. The application may also include a structure programmed into or cooperating with the printing driver 72. Thus, a software-based first image generation unit 73 is formed within the terminal device 20 where the application has been downloaded. The generative AI service providing module 41 provides the user's request received from the terminal device 20 via the application to the generative AI module 31 of the first server 30 through the API. In addition, the image generation unit 73 in the terminal device 20 can also directly access the generation AI module 31 of the first server 30 to generate the image GI without going through the second server 40.

[0077] Regarding the input image II as the printed image PI Figure 4 The image shown is the image of the object to be printed, i.e., the printing image PI. In this embodiment, the image input by the printing control device 70 of the terminal device 20 is called the input image II. The input image II contains the printing image PI input by the printing driver 72 for the purpose of printing, and the input image for generating AI input by the image generation unit 73 for the purpose of generating AI images.

[0078] Figure 4 The input image II shown is displayed on the display unit 22 of the terminal device 20. When the user operates the input unit 21 to print the input image II as a print image PI and selects printing, for example, in an image display application, the printing driver 72 is invoked. Figure 4 The input image II shown is, for example, a family photograph. As a result of the printed image PI containing this family photograph, the remaining amount of multiple recording materials based on the latest recording material information RI becomes... Figure 5 The state shown.

[0079] Regarding the recorded material information RI and its screen display When the printing driver 72 is activated, it obtains recording material information RI from the printing device 11 and determines whether there is recording material that is nearly exhausted or exhausted. When there is recording material that is nearly exhausted or exhausted, the printing driver 72... Figure 5 The remaining amount notification screen 90 displays information indicating that the recorded material is nearing or exhausted, thereby informing the user of this situation.

[0080] In addition, through the Figure 4 The printed image PI shown is displayed by the print driver 72, which is invoked by the application when printing is selected. Figure 5 The printing condition setting screen 88 shown is displayed on the display unit 22. Figure 5 The remaining ink notification screen 90 shown can be included as part of the printing condition setting screen 88, or it can be displayed as a pop-up or other screen when the remaining ink button (illustration omitted) in the printing condition setting screen 88 is selected. In addition, in the printing condition setting screen 88, printing condition information such as media size, media type, printing color (color / monochrome), and printing quality (printing resolution) can be set through selection operations.

[0081] The printing driver 72 is generated based on the recorded material information RI. Figure 5 The remaining quantity notification screen 90 is displayed on the display unit 22. The remaining quantity notification screen 90 has multiple remaining quantity scales 91 to 96, which mimic columnar containers that store the remaining quantities of multiple recording materials according to color. The user can visually confirm the remaining quantities of multiple recording materials using the multiple remaining quantity scales 91 to 96.

[0082] When at least one of multiple recorded materials is nearing its depletion threshold, the user can visually confirm the near-depletion of recorded materials from the remaining quantity notification screen 90. Because Figure 5The multiple remaining quantity scales 91-96 shown reflect remaining quantity information indicating the remaining quantity of each recording material in the recording material information RI, and are therefore one of the recording material information RI. Alternatively, multiple remaining quantity scales 91-96 or information indicating that the recording material is nearly exhausted may also be displayed on the display unit 16 of the printing apparatus 11. The recording material information RI in this embodiment is not limited to the data of the recording material information RI stored in the storage unit 62. The recording material information RI may also include image information related to the multiple remaining quantity scales 91-96 generated based on the recording material information RI, or notification information related to near exhaustion and exhaustion. Furthermore, the recording material information RI does not need to be all information related to multiple recording materials; it only needs to be information related to two or more colors of recording materials among the multiple recording materials. Further, the recording material information RI may also include quantity determination information that can directly or indirectly determine the remaining quantity, etc., of two or more of the multiple recording materials.

[0083] exist Figure 5 The remaining quantities are displayed on scales 91-96 for various colors, including black (B), light blue-green (LC), magenta (M), blue-green (C), yellow (Y), and light magenta (LM). Figure 5 In the example shown, turquoise (C) has the largest remaining amount, followed by light turquoise (LC). Furthermore, light magenta (LM) has the smallest remaining amount, followed by magenta (M). Black (K) and yellow (Y) have intermediate remaining amounts.

[0084] exist Figure 5 In the example shown, it is evident that there is an excess of recording material in turquoise (C) and light turquoise (LC), and a deficiency in magenta (M) or light magenta (LM). If the user repeatedly prints without considering this, the replacement time for the magenta-only recording material holder 53 may arrive prematurely. Therefore, in this embodiment, based on this remaining quantity state, and based on the recording material information RI, the AI ​​generation module 31 generates a color image GI (refer to...) that reduces the deviation in the remaining quantity. Figure 13 When a user prefers to print the generated image (GI), the print head 52, which prints the generated image GI, consumes more recording material of the specific consumable color. This reduces the deviation in the amount of remaining recording material. Furthermore, details regarding the process of generating the image using the AI ​​generation module 31 will be described later.

[0085] Regarding the comparison processing performed by the Comparison Department 85 Next, refer to Figures 6-8The comparison processing of the comparison unit 85, which compares the remaining amounts of multiple recorded materials, will be explained. The comparison unit 85 is based on... Figure 6 The remaining amounts of multiple recording materials are compared using the recorded material information RI shown. Based on the comparison result, the comparison unit 85 selects the color of the recording material that should be consumed to reduce the deviation of the remaining amount, i.e., the consumption color. This comparison process can also be performed using a threshold. That is, the consumption color selection condition, which selects the consumption color from multiple colors (e.g., six colors) that are colors of multiple recording materials, can also include a threshold condition. In this embodiment, the comparison unit 85 can also select the consumption color from multiple colors other than black (K) (e.g., five colors). Specifically, the comparison unit 85 can also select the consumption color that satisfies the consumption color selection condition from blue-green, magenta, yellow, light blue-green, and light magenta, which belong to the color system of multiple recording materials.

[0086] Alternatively, the color of the recording material with the largest remaining quantity can be selected as the consumption color. The comparison unit 85 can also select the color of the recording material with a remaining quantity exceeding a threshold as the consumption color. Furthermore, the threshold can also be varied based on the minimum remaining quantity among multiple recording materials.

[0087] The threshold can also vary based on the difference ΔS between the remaining amount and the minimum remaining amount. Alternatively, the color of the recording material that satisfies the condition that the difference ΔRn between the remaining amount and the minimum remaining amount among multiple recording materials is greater than or equal to a predetermined difference ΔS can be selected as the consumption color. In this case, the value of the remaining amount obtained by adding the difference ΔS to the minimum remaining amount is the threshold. In this way, the color of the recording material that exceeds the threshold specified based on the consumption color selection criteria can also be selected as the consumption color.

[0088] exist Figure 6 In the example shown, blue-green (C), representing the largest remaining amount difference ΔR1, and light blue-green (LC), representing the second largest remaining amount difference ΔR2, are selected as consumption colors. Alternatively, all colors of recording material meeting the above conditions can be set as consumption colors. Furthermore, as a condition for determining consumption colors, they can be selected in descending order of remaining amount difference ΔRn; either only the first-ranked color can be chosen, or the selection can proceed up to the first two colors. The consumption color selection criteria can also include a condition that colors of recording material with remaining amounts below the lower threshold are not selected as consumption colors.

[0089] Thus, after the printing of the printed image PI, which serves as the input image II, the printing of the generated image GI, which mainly consumes the selected color recording material, is performed according to the user's selection.

[0090] The instruction unit 82 can also generate a color image GI of the color of the recording material with the most consumed color, selected based on the remaining amount information in the recording material information RI.

[0091] The recording material of one or more selected colors is primarily consumed during the printing of the generated image GI. When multiple colors are selected, the indicator unit 82 can also instruct the generation of the image GI to be based on a color scheme that determines the consumption ratio of the recording material for each of the multiple colors, according to the ratio of the remaining amount difference ΔRn between the minimum remaining amount of the recording material for each of the multiple colors, or the ratio of the remaining amount Rn of the recording material for each of the multiple colors. Furthermore, when N colors (where N is a natural number) are selected based on the recording material information RI, the indicator unit 82 can also generate the color generated image GI with a color scheme in which the consumption ratio of the N colors of recording material during printing is 1 / (N+1) or higher. When there is only one color, for example, the indicator unit 82 generates the generated image GI with a color scheme in which the consumption ratio of the recording material for that single color is 1 / 2 or higher. Furthermore, when there are two colors, for example, the indicator unit 82 generates the generated image GI with a color scheme in which the consumption ratio of the recording material for each of the two colors is 1 / 3 or higher.

[0092] Figure 7 , Figure 8 Indicates according to Figure 6 The information shown is the remaining amount of the recorded material RI after being printed through multiple generation images GI. Figure 7 The remaining quantity is represented by a double-dotted line. Figure 6 The remaining amount in Figure 8 The remaining quantity is represented by a double-dotted line. Figure 7 The remaining amount in. By from Figure 6 The remaining amount shown is used to print the user-preferred generated image GI in an append-only manner, thus... Figure 7 As shown, the recording materials of the two colors cyan (C) and light cyan (LC), which are set as consumption colors, are consumed more than the other colors. Therefore, the deviation in the remaining amount of multiple recording materials and Figure 6 Compared to that, it is smaller. Furthermore, by means of... Figure 7 The remaining amount shown indicates that the user has printed the generated image GI more than once, thus... Figure 8As shown, the recording materials of the two colors, cyan (C) and light cyan (LC), which are set as consumed colors, are consumed more than the other colors. In this way, whenever a user prints a generated image GI based on the recording material information RI, the deviation in the remaining amount of the various colored recording materials gradually decreases. As a result, for example, the frequency with which the five colored recording materials approach or run out simultaneously is increased. Figure 8 In the example shown, five different colored recording material holders 53 can be replaced at once. Alternatively, the number of recording material holders 53 that can be replaced at once can be two, three, or four.

[0093] Furthermore, when printing begins from an unused state of multiple recording material receptacles 53, as during the initial operation of the printing apparatus 11, there is a possibility that a particular color of recording material among the multiple recording materials is excessively consumed during normal printing. In this case, if normal printing is performed just before the material is exhausted, there is a possibility that the replacement timing of the multiple recording material receptacles 53 will not overlap because only one color of recording material is nearly depleted. To address this, when printing begins from an unused state of multiple recording material receptacles 53, as during the initial operation, by performing multiple printings of the generated image GI based on the recording material information RI, the multiple recording materials can be consumed while suppressing any deviation in the remaining amount. Therefore, the number of printings until the material is initially nearly exhausted can be increased. In other words, after a user purchases the printing apparatus 11 and begins printing, the number of printings until the initial replacement of the recording material receptacles 53 can be increased. As a result, more printing experiences can be performed without being interrupted by the replacement of the recording material receptacles 53.

[0094] Although in this embodiment, the consumed color is selected from a variety of colors other than black, the reason is as follows: Black recording material is used more frequently in the printing of text such as documents. When the recording material storage unit 53 of at least one of the various colors is exhausted, the printing apparatus 11 becomes unable to perform color printing. Even if the printing apparatus 11 cannot perform color printing due to the exhaustion of one of the colors, it can still perform monochrome printing using black recording material. Therefore, in order to perform color printing, it is necessary to replace the recording material storage unit 53 that has been exhausted. At this time, if other recording material storage units 53 are also close to being exhausted or exhausted, multiple recording material storage units 53 can be replaced at once. On the other hand, if the black recording material is exhausted, for example, black and white printing can be performed using black, which is a mixture of blue-green, magenta, and yellow. Therefore, even if the black recording material is exhausted, it is not necessarily necessary to replace the black recording material storage unit 53. In addition, monochrome printing of text such as documents is sometimes performed between photo printing, as in this embodiment. In this situation, since only black recording material is consumed, even if there is a significant difference in remaining quantity with other recording materials before monochrome printing, this difference may disappear. For this reason, in this embodiment, the black recording material is consumed sparingly and actively, while for the multi-color recording materials, the deviation in remaining quantity is reduced by printing the generated image GI. This increases the frequency at which the color recording material storage unit 53 is used up or nearly used up at once.

[0095] Regarding the various screens displayed on display unit 22 Next, refer to Figures 9-12 The various screens displayed on the display unit 22 during the generation of the image GI will be explained.

[0096] Figure 9 Image generation confirmation screen 100 is displayed. When the user prints the image PI, the image generation unit 73 causes the display unit 22 to display the image generation confirmation screen 100. The display unit 22 displays the image generation confirmation screen 100, which confirms permission to generate an image based on the recorded material information RI. The image generation confirmation screen 100 is displayed based on the user's printing-related operations.

[0097] like Figure 9As shown, the image generation confirmation screen 100 has a message MS1 indicating whether to generate an AI-generated image based on the recording material information RI, and options 101 and 102 for selecting whether the ink remaining deviation reduction mode (recording material remaining deviation reduction mode) is active or inactive. For example, the active option 101 can be selected by default. Furthermore, the comparison unit 85 can perform the above-mentioned comparison processing before the image generation confirmation screen 100 is displayed. And, if the result of the comparison processing is that the implementation conditions of the ink remaining deviation reduction mode are met by selecting more than one consumable color, the image generation unit 73 can also display the image generation confirmation screen 100 with the active option 101 selected. Users who want to disable the ink remaining deviation reduction mode can select the inactive option 102 through the operation of the input unit 21. Users who want to perform AI image generation can select the "Yes" button 103, and users who do not want to perform AI image generation can select the "No" button 104. The image generation unit 73 that receives the selection of the "Yes" button 103 will switch to the AI ​​image generation mode. The image generation unit 73 stores in its memory whether the ink remaining amount deviation reduction mode is valid or invalid.

[0098] Figure 10 Image generation method selection screen 110. The user can select an image generation method from two options. Image generation method selection screen 110 includes a message MS2 indicating a prompt for the user to select an image generation method, and selection buttons 111 and 112 for selecting an image generation method. Selection button 111 is for selecting "AI Image Generation," which generates an AI image by using a printed image PI as input image II. Selection button 112 is for selecting "AI Image Composition," which generates an AI image from a desired image such as a photograph uploaded by the user. When selection button 111 is received, the image generation unit 73 causes the display unit 22 to display... Figure 11 The image generation condition confirmation screen 120 is shown. On the other hand, when the selection button 112 is selected, the display unit 22 displays... Figure 12 The composite image shown is specified as screen 130.

[0099] Figure 11The image generation condition confirmation screen 120 shown includes a message MS3 indicating that the user is prompted to input conditions to fulfill the desired image if so, an option 121 for selecting a style, an option 122 for selecting an art style, and an input field 123 for entering keywords. In addition to keywords, the input field 123 can also accept text indicating expectations for the image to be generated within a specified character limit. Furthermore, the image generation condition confirmation screen 120 includes a generate button 124 indicating the generation of an AI image, and a cancel button 125 selected when canceling image generation.

[0100] Figure 12 The image composition specification screen 130 shown includes a message MS4 prompting the user to specify (upload) the image to be composed, and an option 131 allowing selection of whether to use a printed image. Furthermore, the image composition specification screen 130 includes a selection button 132 allowing selection of the image to be composed via upload, a selection button 133 allowing selection of the composition method, and an input field 134 for entering keywords. In addition to keywords, the input field 134 can also accept text within a specified character limit.

[0101] For example, in Figure 11 In the image generation condition confirmation screen 120 shown, after selecting "Natural" as the style, "Photograph" as the art style, and "Motion" as the keyword, the generation button 124 is selected.

[0102] Regarding prompts and the generation of the image GI Figure 2 The prompt generation unit 84, as shown, includes input image information, including the input image II as the printed image PI, in addition to recording material information RI. Specifically, the prompt generation unit 84 includes the two consumable colors "blue-green" and "light blue-green," selected by the comparison unit 85 based on the comparison processing of the recorded material information RI, as recorded material information RI in the prompt. Furthermore, the prompt generation unit 84 includes user input information such as the style "natural," the art style "photograph," and the keyword "movement" in the prompt. The prompt generation unit 84 generates the prompt based on the printed image PI, the consumable colors, and the user's input information. The prompt generation unit 84 can also include an instruction statement indicating the extraction of a person or face in the prompt when detecting a person or face in the printed image PI. The instruction unit 82 instructs the generation AI module 31 to generate an image GI based on the prompt generated by the prompt generation unit 84. That is, the instruction unit 82 instructs the generation AI module 31 to generate the image GI by sending the prompt generated by the prompt generation unit 84 and the printed image PI to the server 30.

[0103] Figure 13 The received image GI is received by the receiving unit 83 from the AI ​​generation module 31. For example... Figure 13 As shown, the generated image GI is a photographic image of the person BI (the "father") in the printed image PI, enjoying "surfing" in the "sea" with the keyword "sports" and the style being "natural". The generated image GI is a color image of the main colors consumed by the recording material, namely "blue-green (C)" and "light blue-green (LC)," which are included in the recording material information RI in the prompt.

[0104] Thus, in display unit 22, the photo of the father surfing in the sea within the printed image PI, which is the input image II, is displayed on display unit 22. Image generation unit 73 confirms with the user whether to print the generated image GI displayed on display unit 22 of terminal device 20 in an additional manner. Then, if the user sees the photo of the generated image GI and finds it interesting, printing will proceed, thereby consuming more blue-green and light blue-green recording material. As a result, the deviation in the remaining amount of multiple recording materials can be reduced, thereby reducing the frequency of replacing the recording material storage unit 53. In addition, it can provide the user with a printing experience that has not been provided before. Furthermore, it is also possible to generate generated image GIs that are more easily selected by the user for printing by learning whether generated image GIs have been printed. In addition, generated image GIs can be realistic images, or illustrations or comics that have been modified based on feature points (hair color or the presence or absence of a beard). Furthermore, generated image GIs can also be dynamic images such as animations. In this case, it is also possible for the user to select and print one or more generated image GIs from the dynamic images generated by AI.

[0105] Confirmation screen after AI image generation Figure 14 The display unit 22 displays a print confirmation screen 140 to confirm the printing of the generated image GI that is being displayed. The image generation unit 73 uses this print confirmation screen 140 to confirm with the user whether they agree to the print confirmation process displayed on the display unit 22. Figure 13The generated image GI shown is printed. The print confirmation screen 140 includes a message MS5 indicating whether the user wants to confirm printing the generated image GI, a "Yes" button 141 for selecting if the user wishes to print, and a "No" button 142 for selecting if the user does not wish to print. When the "Yes" button 141 is selected, the image generation unit 73 notifies the image transmission unit 86 that print acceptance has been received. The image transmission unit 86 sends the generated image GI to the printing apparatus 11. The image transmission unit 86, for example, converts the generated image GI into print data and then sends it to the printing apparatus 11.

[0106] Figure 15 The save confirmation screen is displayed on screen 150. The image generation unit 73 of the terminal device 20 can also allow the user to choose whether to save the generated image GI received by the receiving unit 83 as another file. The image generation unit 73... Figure 14 After selecting either the "Yes" button 141 or the "No" button 142 on the print confirmation screen 140, the display unit 22 displays the save confirmation screen 150. The save confirmation screen 150 is displayed on the display unit 22 to confirm whether the user wants to save the file. Figure 13 The screen shown is for generating the image GI. The save confirmation screen 150 includes a message MS6 indicating whether the user wants to save the generated image GI to another file, a "Yes" button 141, and a "No" button 142. Users who wish to save the generated image GI will select the "Yes" button 141. Conversely, users who do not wish to save the generated image GI will select the "No" button 142. When the "Yes" button 141 is selected, the image generation unit 73 saves the data for the generated image GI in the memory of the terminal device 20. When the "No" button 142 is selected, the image generation unit 73 closes the save confirmation screen 150.

[0107] The role of the first embodiment Next, refer to Figures 1 to 16 The function of the printing system 10 will be explained, etc. Additionally, in... Figure 16 In this context, the printed image PI, which will form the basis for generating the image GI, is also called the "original image". Furthermore, the instructions given by the AI ​​generation module 31 to generate the image GI are also called "image generation commands".

[0108] The following is for reference Figure 16 An example of the process for reducing the deviation of the remaining amount of recording material implemented by the printing system 10 is illustrated.

[0109] First, when a user instructs the printing apparatus 11 to print from the terminal device 20, the printing driver 72 and the application within the terminal device 20 are activated. When the user prints an image displayed on the display unit 22 of the terminal device 20, they select printing in the application through operation of the input unit 21. The invoked printing driver 72... Figure 5 The printing condition setting screen 88 is displayed. The user, who has set the printing condition information in the printing condition setting screen 88, performs printing.

[0110] First of all, Figure 16 In step S11, the terminal device 20 receives a printing instruction for the original image. The printing driver 72 of the terminal device 20 converts the printing image PI, which is the original image, into printing data and sends it to the printing device 11.

[0111] In step S12, the terminal device 20 sends the original image to the printing device 11.

[0112] In step S13, the printing device 11 receives the original image.

[0113] In step S14, the printing apparatus 11 prints the original image. The control unit 60 of the printing apparatus 11 controls the printing unit 50 based on printing data, thereby printing on the medium M, for example... Figure 4 The printed image PI shown is printed.

[0114] In step S15, the terminal device 20 receives an instruction to generate an image. The image generation unit 73 generates the image based on the user's printing-related operations. Figure 9 The image generation confirmation screen 100 is displayed on the terminal device 20. The user who wishes to generate an AI image selects the "Yes" button 103. For example, the ink remaining deviation mode is set to enabled.

[0115] In step S16, the terminal device 20 uses the recording material information RI to determine whether to generate an image. If the ink remaining deviation mode is active and the "Yes" button 103 is selected, it is determined that an image will be generated using the recording material information RI, and the process proceeds to step S18. On the other hand, if the ink remaining deviation mode is inactive and the "Yes" button 103 is selected, or if the "No" button 104 is selected, it is determined that image generation using the recording material information RI will not be performed, and the process proceeds to step S19.

[0116] In step S17, the printing apparatus 11 sends recording material information RI. That is, when the terminal device 20 determines that image generation is to be performed, it requests recording material information RI from the printing apparatus 11. Upon receiving the request, the printing apparatus 11 sends the recording material information RI to the terminal device 20. This recording material information RI is the latest recording material information RI after the printing of the original image has been completed.

[0117] In step S18, the terminal device 20 receives the recorded material information RI.

[0118] In step S19, the terminal device 20 generates an image generation command. The image generation unit 73 of the terminal device 20... Figures 10-12 Screens 110, 120, and 130 are displayed as shown. Users select or input necessary information on screens 110, 120, and 130. Figure 11 In the image generation condition confirmation screen 120 shown, select a style or art style, or enter keywords. Additionally, in... Figure 12 In the composite image designated screen 130 shown, an image can be uploaded or the use of a printed image PI can be selected as needed. The prompt generation unit 84 generates a prompt based on the recording material information RI, the printed image PI, and input information entered by the user via screens 120 and 130. The comparison unit 85 selects the consumed color by performing a comparison process that compares the remaining amounts of multiple recording materials based on the recording material information RI. When an image is generated using the recording material information RI, the prompt may also include information related to the consumed color and the remaining amount of recording material for that consumed color, either the recording material information RI itself or obtained as a result of the comparison process based on the recording material information RI.

[0119] In step S20, the terminal device 20 sends an image generation command to the generation AI module 31. Specifically, the instruction unit 82 sends an image generation command based on a prompt including the printed image PI to the generation AI module 31 of the server 30. Alternatively, the terminal device 20 can also send the image generation command to the generation AI module 31 of the server 30 via the generation AI service provision module 41 of the server 40.

[0120] In step S21, the AI ​​generation module 31 receives the image generation command.

[0121] In step S22, the AI ​​generation module 31 generates the generated image GI. Specifically, the AI ​​generation module 31 generates the generated image GI based on an image generation command that includes the input image II and prompts. When image generation is performed using recorded material information RI (a positive determination is made in step S16), since the image generation command includes information related to the consumed color, a generated image GI with the consumed color as the primary color is generated. For example, as... Figure 13 As shown, a generated image GI is produced, primarily using blue-green (C) and light blue-green (LC) as the main colors. The AI ​​generation module 31 then sends the generated image GI to the terminal device 20.

[0122] In step S23, terminal device 20 receives the generated image GI. Terminal device 20... Figure 13 The generated image GI is shown and displayed, and... Figure 14 The print confirmation screen 140 is displayed. Users who wish to print should click the "Yes" button 141 on the print confirmation screen 140.

[0123] In step S24, the terminal device 20 sends the generated image GI to the printing device 11. That is, when the terminal device 20 receives a printing instruction for the generated image GI based on the operation of the "Yes" button 141, it converts the generated image GI into printing data and then sends it to the printing device 11. The transmission of the generated image GI is performed by the image transmission unit 86 within the terminal device 20.

[0124] In step S25, the printing apparatus 11 receives the generated image GI.

[0125] In step S26, the printing apparatus 11 prints the generated image GI. When image generation is performed using recorded material information RI, Figure 13 The generated image GI shown is printed. Through the printing of this generated image GI, the recording materials, namely cyan (C) and light cyan (LC), which are consumed colors, are primarily consumed. In this way, by generating the image GI based on the user's preferred recording material information RI and printing it onto the printed image PI in an append-on manner, the deviation in the remaining amount of multiple recording materials is reduced. By repeatedly printing such generated images GI, thereby… Figure 6 The larger deviation of the remaining amount shown will be like Figure 7 , Figure 8 The reduction is carried out in stages as shown. As a result, since multiple recording material receptacles 53 become nearly exhausted or exhausted at the same time, multiple recording material receptacles 53 can be replaced at once.

[0126] Printing methods The printing method of this embodiment includes the following (A) to (C).

[0127] (A) Obtain the Recording Material Information (RI) associated with multiple recording materials of different colors.

[0128] (B) An image is generated based on the recorded material information RI and by generating the AI ​​module 31.

[0129] (C) Use recording material to print the generated image GI generated by the AI ​​module 31.

[0130] in addition, Figure 16 The processes in steps S17 and S18 are equivalent to an example of the process described in (A) above. Furthermore, the process in step S22 is equivalent to an example of the process described in (B) above. Further, the process in step S26 is equivalent to an example of the process described in (C) above.

[0131] According to this embodiment, the following effects can be obtained.

[0132] (1-1) The printing system 10 includes an acquisition unit 81, an instruction unit 82, a receiving unit 83, and a printing unit 50. The acquisition unit 81 acquires recording material information RI related to multiple recording materials of different colors. The instruction unit 82 causes the generation AI module 31 to generate an image based on the recording material information RI. The receiving unit 83 receives the generated image generated by the generation AI module 31. The printing unit 50 uses recording material to print the received generated image GI. According to this structure, since the colors based on the recording material information RI related to multiple recording materials of different colors are reflected in the generated image GI, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image GI.

[0133] (1-2) The instruction unit 82 causes the AI ​​generation module 31 to generate an image based not only on the recording material information RI, but also on the input image information IS, including the input image II. According to this structure, since an image GI that reflects the color based on the recording material information RI and the input image II is generated, the printing frequency of the generated image GI can be easily increased by using users who are interested in the generated image GI.

[0134] (1-3) The instruction unit 82, in addition to generating the recorded material information RI, also generates an image based on information including prompts input by the user, causing the AI ​​generation module 31 to generate an image. According to this structure, the AI ​​generation module 31 can generate the image desired by the user. Therefore, by increasing the frequency with which the user prints the generated image GI, deviations in the remaining amount of recorded material can be effectively reduced.

[0135] (1-4) The printing system 10 includes a display unit 22 that displays an image generation confirmation screen 100, which confirms permission to generate an image based on the recorded material information RI.

[0136] According to this structure, since the user can only permit the generation of images when desired, the delay in printing input images II, etc., caused by the useless generation of images can be suppressed.

[0137] (1-5) The printing system 10 includes a display unit 22 that displays a print confirmation screen 140, which displays a generated image GI and confirms the printing execution of the generated image GI. With this configuration, if a user confirms the displayed generated image GI and wishes to print, printing can be executed quickly.

[0138] (1-6) The printing system 10 also includes a prompt generation unit 84 that generates a prompt as an image generation command based on the recording material information RI. According to this structure, a generated image GI can be generated according to the image generation command generated based on the prompt generated based on the recording material information RI.

[0139] (1-7) The prompt generation unit 84 has a comparison unit 85 that compares the remaining amounts of multiple recording materials. According to this structure, by comparing the remaining amounts of multiple recording materials, the color of the recording material that should be reflected in the prompt can be determined. Therefore, since the consumption of the determined color of the recording material is promoted by printing the generated image GI, deviations in the remaining amount of recording material can be effectively reduced.

[0140] (1-8) Input image II is a printed image with printing instructions, and acquisition unit 81 acquires recording material information RI after the printed image is printed. According to this structure, since the image is generated based on the latest recording material information RI after the printed image is printed, the deviation of the remaining amount of recording material can be effectively reduced.

[0141] (1-9) The image generation confirmation screen 100 is displayed based on the user's printing-related operations. According to this structure, since the user can display the generation confirmation screen at an appropriate timing based on the printing-related operations, it is less likely that the user will miss the confirmation.

[0142] (1-10) Recorded material information RI includes information on the remaining amount of recorded material. According to this structure, since an image reflecting the color corresponding to the remaining amount of recorded material can be generated, deviations in the remaining amount of recorded material can be reduced by printing the generated image GI.

[0143] (1-11) Select whether to save the generated image GI received by the receiving unit 83 as another file. According to this structure, the user can easily save their favorite generated image GI as another file.

[0144] (1-12) The instruction unit 82 generates a generated image GI of the color that is most consumed during printing, based on the remaining amount information in the recording material information RI, which is selected by the color consumption information. According to this structure, since the color corresponding to the remaining amount information of the recording material is most consumed during printing of the generated image, the deviation of the remaining amount of the recording material can be effectively reduced.

[0145] (1-13) When the number of colors (where N is a natural number) whose remaining amount corresponds to a threshold or higher based on the remaining amount information of the recording material information RI, the instruction unit 82 generates a color-generated image GI in which the consumption ratio of the N colors of recording material consumed during printing is 1 / (N+1) or higher. According to this structure, since the colors corresponding to the remaining amount information of the recording material are consumed most by printing the generated image, the deviation of the remaining amount of the recording material can be effectively reduced.

[0146] (1-14) The threshold varies based on the minimum remaining amount among multiple recording materials. According to this structure, by generating the printing of the image GI, deviations in the remaining amount of recording material can be effectively reduced. For example, if the threshold is fixed, it is possible for the recording material to be over-consumed despite a small difference from the minimum remaining amount, or for the recording material to be under-consumed despite a large difference from the minimum remaining amount. Such problems can also be solved when the threshold is fixed.

[0147] (1-15) The threshold varies based on the difference ΔS between the remaining amount and the minimum remaining amount among multiple recording materials. According to this structure, by generating the printing of the image GI, the deviation of the remaining amount of the recording material can be effectively reduced. Furthermore, it also solves the problem when the threshold is a fixed value.

[0148] (1-16) Since the consumption of a specific color of recording material corresponding to the amount of recording material based on the recording material information RI can be significantly changed, the adjustment amount that reduces the deviation of the remaining amount of the average number of prints can be increased. The replacement frequency of the recording material holder 53 of the average number of prints can be reduced. The frequency at which multiple recording material holders 53 can be replaced at one time is increased. For example, in the case of an inkjet printing apparatus 11, an initial filling is performed whenever the recording material holder 53 is replaced. In the initial filling, new ink is filled into the flow path from the supply port of the recording material holder 53 to the nozzle of the print head 52, thereby removing air that has entered during replacement from the ink in the flow path. In this initial filling, an equal amount of ink is also consumed from non-replaced ink cartridges other than the replaced ink cartridge. Therefore, from the viewpoint of suppressing ink consumption in uses other than printing, it is preferable that the replacement frequency of the recording material holder 53 is less. According to this embodiment, since the deviation of the remaining amount of recording material can be effectively reduced, the frequency of synchronizing the replacement time of two or more recording material holders 53 is increased. In other words, the replacement frequency of the recording material reservoir 53, which has an average number of print runs, can be reduced. Therefore, the consumption of recording material caused by the initial filling that accompanies the replacement of the recording material reservoir 53 can be suppressed. Furthermore, after the initial operation of the printing apparatus 11, which starts printing from a state where multiple recording material reservoirs 53 are unused, the number of print runs can be increased until the material is nearly exhausted. Therefore, a more comprehensive printing experience, including printing of generated images (GI), can be achieved without being interrupted by the replacement of the recording material reservoir 53.

[0149] (1-17) It can generate images with specifications suitable for the number of colors (four colors or six colors, etc.) that can be matched with the printing apparatus 11. As a result, compared with the printing apparatus described in Patent Document 1, it is possible to reduce the difference in sharpness between the image displayed on the display unit 22 of the terminal device 20 such as a smartphone and the actual printed image.

[0150] (1-18) The indicator unit 82 can also cause the generation AI module 31 to generate an image based not only on the recording material information RI, but also on the color information of the printing medium M. This color information indicates the color of the medium M before printing. The color information may also include information related to the color of the substrate of the medium M. According to this structure, it is possible to reduce the undesirable situation where the outline of the printed image GI easily becomes indistinct due to printing the image GI on the medium M with a color scheme that is the same as or similar to the color of the medium M. In other words, since the color scheme of the image GI, which accompanies the consumption of recording material based on the recording material information RI, is determined by contrast with the color of the medium M, the generation AI module 31 can generate an image GI with a sharp outline when printing on the medium M. Therefore, it is possible to reduce the deviation in the remaining amount of consumed material and to print an image GI with a sharp outline on the medium M.

[0151] (1-19) To instruct the printing apparatus 11 to print, the printing control device 70 of the terminal device 20 includes an acquisition unit 81, an instruction unit 82, a receiving unit 83, and an image transmission unit 86. The acquisition unit 81 acquires recording material information RI related to various recording materials of different colors. The instruction unit 82 causes the generation AI module 31 to generate an image based on the recording material information RI. The receiving unit 83 receives the generated image GI generated by the generation AI module 31. The image transmission unit 86 sends the received generated image GI to the printing apparatus 11 that performs printing using recording materials. According to this structure, since the colors based on the recording material information RI related to various recording materials of different colors are reflected in the generated image GI, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image GI.

[0152] (1-20) Printing methods include the following (A) to (C).

[0153] (A) Obtain the Recording Material Information (RI) associated with multiple recording materials of different colors.

[0154] (B) An image is generated by generating AI module 31 based on the recorded material information RI.

[0155] (C) Use recording material to print the generated image GI generated by the AI ​​module 31.

[0156] According to this printing method, since the color of the recording material information RI, which is related to multiple recording materials with different colors, is reflected in the generated image GI, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image GI.

[0157] Second Implementation Method Next, refer to Figures 16-18 The second embodiment will now be described. In the second embodiment, an image file including image data consisting of presentation slides is generated from a document file or spreadsheet file containing text or numbers, such as a Word or Excel file. The document file or spreadsheet file also serves as the input image II for the printed image PI. Based on the user's printing-related operations, the terminal device 20 displays an image generation confirmation screen 100.

[0158] When printing document files or spreadsheet files, upon receiving an instruction to generate image data consisting of presentation slides using AI (AI) generation, the terminal device 20 confirms whether it wishes to perform AI image generation using Recorded Material Information (RI). The terminal device 20 can either select the printing apparatus 11 from the automatically connected printing apparatus 11 that provides the Recorded Material Information (RI) source, or allow the user to make the selection. Here, when the "Yes" button 103 is selected on the image generation confirmation screen 100, image data consisting of presentation slides is generated based on the Recorded Material Information (RI), similar to the first embodiment. The image data includes charts within the slides. The image generation unit 73 sets the color of the charts or text within the slides where AI image generation is performed to the consumed color selected based on the Recorded Material Information (RI). This reduces the deviation in ink remaining after printing the image data consisting of slides.

[0159] Figure 17 Document data 160 serves as input image II. Document data 160 is also set as the printed image PI of the printed object. Document data 160 is, for example, a report. Within the report, as information capable of generating charts, it includes items A, B, ..., F, and corresponding numerical values. For example, document data 160 includes text 161 such as "October sales figures," multiple items A to F detailing these items, and text 162 to 167 containing numerical values ​​associated with each item A to F. Items may be, for example, branches or products, and the numerical values ​​in the text may be, for example, sales figures.

[0160] Figure 18 The print selection screen 170 shown has a generated image GI containing a chart 171. The chart includes bars 172 that form a bar chart. The bars 172 are colors that can be printed using a recording material that consumes ink. The print selection screen 170 has a next button 173 that is operated when entering the next generated image GI, a previous button 174 that is operated when returning to the previous generated image GI, and an adopt button 175 that uses the generated image GI as the printing object.

[0161] A display unit 22 is equipped with a printing selection screen 170, which allows a user to select an image for printing from multiple images, including a generated image GI. The printing selection screen 170 includes an image GI generated by a generation AI module 31 based on recorded material information RI, and an image generated by the generation AI module 31 without being based on recorded material information RI. The printing selection screen 170, which allows the user to select slides including charts generated based on recorded material information RI, and slides including charts generated without being based on recorded material information RI, can also be displayed on the display unit 22. Furthermore, the slides can be generated not based on documents or spreadsheets such as Word or Excel, but based on text entered by the user.

[0162] Printing system 10 is based on and Figure 16 The processing is performed in the same order as shown. When the terminal device 20 receives a printing instruction for document data 160 as the original image, it converts the document data 160 as the original image into printing data and sends it to the printing device 11 (steps S11, S12). The printing device 11, having received the printing data, prints the document based on the document data (steps S13, S14). When the terminal device 20 receives an instruction to generate an image, it determines whether to use the recording material information RI to generate the image (steps S15, S16). If the determination is affirmative (YES), the recording material information RI is obtained from the printing device 11 (steps S17, S18). The terminal device 20 generates an image generation command including a prompt generated using the recording material information RI and the document data 160 and sends it to the generation AI module 31 (steps S19, S20).

[0163] Upon receiving the image generation command, the generation AI module 31 parses the document data 160 according to the prompts, thereby obtaining multiple items A to F and text 162 to 167 containing the corresponding values ​​for each item. Further, the generation AI module 31 generates a generated image GI including a chart 171 based on the obtained multiple sets of items and values ​​(steps S21, S22). The terminal device 20 displays the generated image GI received from the generation AI module 31, and when printing permission is received from the user on the print confirmation screen 140, it converts the generated image GI into print data and sends it to the printing apparatus 11 (steps S23, S24). The image transmission unit 86 transmits the print data to the printing apparatus 11. The printing apparatus 11 prints the generated image GI based on the received print data (steps S25, S26).

[0164] In the graph 171 of the generated image GI, the multiple bars 172 are color-matched with the colors of recording materials, such as blue-green (C) and light blue-green (LC), which are the main colors consumed. Therefore, by generating the image GI through printing, it is possible to reduce the deviation in the amount of remaining recording material among the multiple recording material receptacles 53. As a result, the frequency at which multiple recording material receptacles 53 can be replaced at one time is increased.

[0165] Effects of the second implementation method According to the second embodiment, in addition to obtaining the effects of (1-1) to (1-20) described in the first embodiment in the same way, the following effects can also be obtained.

[0166] (2-1) The printing system 10 includes a display unit 22 that displays a printing selection screen 170, which allows a user to select an image to be printed from a plurality of images, including a generated image GI. According to this configuration, the user can select the desired image to be printed from a plurality of images, including the generated image GI.

[0167] (2-2) The print selection screen 170 includes an image GI generated by the generation AI module 31 based on the recording material information RI, and an image generated by the generation AI module 31 without being based on the recording material information RI. According to this structure, the user can select a desired image from the image GI generated based on the recording material information RI and the image generated without being based on the recording material information RI.

[0168] (2-3) The AI ​​generation module 31 generates image GI based on the document. According to this structure, since the content of the document is easily understood visually through image GI corresponding to the document's content, the printing frequency of image GI can be easily increased. Therefore, the effect of suppressing residual material deviation is improved.

[0169] Third Implementation Method Next, refer to Figures 19-21 The third embodiment will now be described. While in the various embodiments described above, the input image II, serving as the raw data, is a printed image PI, in the third embodiment, the input image II is a file of presentation materials before printing. The application used to create the presentation materials is, for example, presentation software manufactured by Microsoft Corporation, namely "Microsoft PowerPoint (registered trademark)".

[0170] The printing system 10 of this embodiment is based on Figure 19 The printing process, which includes recording the reduction of remaining material, is performed in the order shown.

[0171] Figure 20 Input image II is input by the user into terminal device 20. Input image II is set as the raw data (raw image) for AI image generation. Before printing, the printing control device 70 of terminal device 20, which receives input image II, uses input image II as raw data to cause the AI ​​generation module 31 to generate image GI.

[0172] Figure 20 The input image II shown may, for example, contain image data 180 composed of presentation slides. Input image II may also be, for example, a file containing multiple image data composed of slides. Figure 20 Image data 180 represents a single slide image from a file that serves as input image II. Input image II may also be a file containing multiple image data 180. Image data 180 includes objects 181 and 182 as editable elements and a template 183 as a non-editable element.

[0173] Here, objects 181 and 182 include text or graphics such as arrow marks containing text. Template 183 includes logos such as company logos that cannot be edited. Template 183 is set as a slide master, for example, as a common part of multiple slides. For example, the background, logo, item name, etc., that are already set in the slide master are template 183 as non-editable elements. In addition, the parts that have editing restrictions set are also template 183 as non-editable elements. On the other hand, the background or item name, etc., that are not set as slide masters and whose editing restrictions are not set are editable and are therefore objects that are editable elements. Users can also freely set the parts included in template 183 as non-editable elements.

[0174] Figure 21 A print selection screen 190 is generated by the image generation unit 73 of the terminal device 20 using the input image II and the generated image GI. The terminal device 20 includes a display unit 22 that displays the print selection screen 190. The print selection screen 190 is a screen for the user to select an image for printing from multiple images, including the generated image GI. The print selection screen 190 includes the generated image GI generated by the generation AI module 31 based on the recording material information RI, and an image generated by the generation AI module 31 without being based on the recording material information RI. In the generated image GI generated by the generation AI module 31 using the input image II as raw data and based on the recording material information RI, the colors of objects 181 and 182 in the image data 180 are changed based on the recording material information RI, while the color of the template 183 is not changed.

[0175] exist Figure 21In the print selection screen 190 shown, the input image II, serving as the original image, and the generated image GI are positioned in a way that allows them to be compared with each other. Figure 21 The image displayed shows a page containing one of the multiple image data 180 constituting the input image II. The image data 180 constituting the input image II and the image data 191 constituting the generated image GI are arranged in positions that allow for comparison, page by page. Figure 21 The example of the print selection screen 190 shown is configured to compare multiple image data 191 constituting the generated image GI while switching between image data 180 constituting one of the input image II.

[0176] The print selection screen 190 includes a selection button 192 for using an input image II and a selection button 193 for using a generated image GI, per page. Furthermore, the print selection screen 190 includes a next button 194 operated when switching the generated image GI to the next image, and a previous button 195 operated when switching to the previous image. The multiple image data 191 constituting the generated image GI are composed of various generated images GI of different colors used in the objects 181 and 182 being printed using a recording material that consumes color.

[0177] exist Figure 21 In the example shown, the color of the object 181 marked by the arrow in the input image II is, for example, a magenta color. The color of the object 181 marked by the arrow in the generated image GI is, for example, a blue-green color. When the generated image GI is switched, the color of the object 181 marked by the arrow is switched to a different blue-green color or a blue-green color mixed with other colors in a small proportion. Here, the instruction unit 82 gives the instruction that the color scheme of the image GI generated by the AI ​​generation module 31 is set to a color scheme that increases the consumption of recording material for colors that should be consumed more based on the recording material information RI, and decreases the consumption of recording material for colors that should be suppressed based on the recording material information RI. Therefore, in Figure 21 In the example shown, the color scheme of the generated image GI is determined to be a color scheme that increases the consumption of recording material based on the consumption color (blue-green, C) of the recording material information RI, and decreases the consumption of recording material based on the consumption suppression color (magenta, M) of the recording material information RI. For example, in Figure 6In this context, the threshold for determining the consumption suppression color is defined as the value of the remaining amount that is smaller than the difference ΔSL between the remaining amount and the threshold for determining the consumption color relative to the minimum remaining amount. Furthermore, the indicator unit 82 can also select the color of the recording material whose remaining amount differs from the minimum remaining amount by ΔSL or less as the consumption suppression color based on the recording material information RI.

[0178] The print selection screen 190 has a selection button 196 for proceeding to the next slide. While selecting the selection button to switch to the next slide, the user compares the input image II and the generated image GI on each page, and selects either the preferred generated image GI or the input image II as the print object by switching between the generated image GI and the output image GI on each page.

[0179] When the image data 180 consisting of slides has been selected for all pages on the print selection screen 190, the display unit 22 sequentially displays... Figure 14 The print confirmation screen shown is 140, and Figure 15 The save confirmation screen shown is 150. The user completes the process via... Figure 14 On the print confirmation screen 140 shown, selecting the "Yes" button will print, for example, all slides that constitute the presentation file. Furthermore, regardless of whether printing occurs, the presentation file, composed of image data 180, 191 selected by the user per page, can be saved to another file.

[0180] The role of the third implementation method The following is for reference Figure 19 The function of the printing system 10 will now be explained.

[0181] First, in step S31, the terminal device 20 receives a printing instruction for the original image. The input image II is set as the original image. The terminal device 20 displays an image generation confirmation screen 100. The user selects the "Yes" button 103 while the ink remaining deviation reduction mode is active.

[0182] In step S32, the terminal device 20 uses the recording material information RI to determine whether to correct the image. If the ink remaining amount deviation reduction mode is effective, the terminal device 20 determines that the image should be corrected using the recording material information RI. If the image is corrected, the process proceeds to step S34; otherwise, it proceeds to step S40.

[0183] In step S33, the printing apparatus 11 sends the recording material information RI. The recording material information RI after the printing of the original image is completed is sent to the terminal device 20.

[0184] The processing of steps S33 to S42 is the same as that of steps S17 to S26 in the first embodiment.

[0185] When the terminal device 20 receives the recording material information RI sent from the printing device 11, it generates an image generation command (steps S33-S35). The terminal device 20 sends the image generation command to the generation AI module 31 (step S36). The generation AI module 31, having received the image generation command, generates a generated image GI based on the image generation command (steps S37, S38). Based on the consumed color selected from the recording material information RI, the input image II, and the image generation command implemented by the prompt, the generation AI module 31 generates multiple generated images GI for each page, where the colors of objects 181 and 182 are different colors using the consumed color.

[0186] Terminal device 20 pairs of printing selection screens 190 ( Figure 21 The printing selection screen 190 compares the generated image GI received from the AI ​​generation module 31 with the input image II in step S39. The user selects the image to be printed from either the input image II or the generated image GI in the printing selection screen 190. The terminal device 20 sends the image of the printing object to the printing device 11 (step S40). At this time, in step S32, without correcting the image using the recording material information RI, the input image II is sent to the printing device 11 as the image of the printing object. The printing device 11 prints the image received from the terminal device 20 (steps S41, S42).

[0187] Effects of the third implementation method According to the third embodiment, in addition to obtaining the effects of (1-1) to (1-7) and (1-9) to (1-20) as described in the first embodiment, the following effects can also be obtained.

[0188] (3-1) The printing system 10 includes a display unit 22 for displaying a printing selection screen 190, which allows selection of an image to be printed from a plurality of images, including a generated image GI. According to this configuration, a user can select the desired image to be printed from a plurality of images, including the generated image GI.

[0189] (3-2) The print selection screen 190 includes an image GI generated by the generation AI module 31 based on the recording material information RI, and an image generated by the generation AI module 31 without being based on the recording material information RI. According to this structure, the user can select a desired image from the image GI generated based on the recording material information RI and the image generated without being based on the recording material information RI.

[0190] (3-3) Input image II is a file containing multiple image data including objects 181 and 182 as editable elements and a template 183 as a non-editable element. In the generated image GI generated by the AI ​​module 31 using input image II as raw data and based on the recording material information RI, the colors of objects 181 and 182 in the image data are changed based on the recording material information RI, while the color of template 183 is not changed. According to this structure, since the generated image GI is generated and the colors of objects 181 and 182 are reflected in the colors of multiple recording materials with different colors based on the recording material information RI, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image GI.

[0191] (3-4) The instruction unit 82 sets the color scheme of the image GI generated by the AI ​​generation module 31 to a color scheme that increases the consumption of recording material for colors consumed based on the recording material information RI, and decreases the consumption of recording material for colors whose consumption is suppressed based on the recording material information RI. Therefore, it is possible to more effectively reduce the deviation of the remaining amount of multiple recording materials.

[0192] Fourth Implementation Method Next, refer to Figure 22 , Figure 23 The fourth embodiment will now be described. In this embodiment, the comparison unit 85, which constitutes the prompt generation unit 84, compares the recorded material information RI, thereby making the selection criteria for the selected consumable color different from those in the previous embodiments.

[0193] The comparison unit 85 determines the color of recording material to be consumed (i.e., the consumed color) based on a comparison result of the remaining amounts of various colors according to the recording material information RI. The threshold for the consumed color selection condition based on this comparison process differs from that in the various embodiments described above.

[0194] like Figure 22 As shown, the minimum remaining amount relative to multiple colors (in) Figure 22When the value of the light magenta (in the middle) is only greater than the first difference ΔS1, and the value of the second difference ΔS2 relative to the minimum remaining amount is set as the upper limit, the comparison unit 85 selects the color within the range RS where the remaining amount is above the lower limit and below the upper limit as the consumed color. Figure 22 In the example shown, the comparison unit 85 selects the remaining amounts R1 and R2 within the range RS, namely light blue-green (LC) and yellow (Y), as the consumed colors. Since blue-green (C), which represents the largest remaining amount R3, is skewed beyond the range RS defined by the consumed color selection criteria, it is not selected as a consumed color. Furthermore, magenta (M), representing the remaining amount R4, and light magenta (LM), representing the smallest remaining amount, are skewed below the range RS, and therefore are not selected as consumed colors.

[0195] The prompt generation unit 84 generates a prompt containing the consumable color selected by the comparison unit 85 based on the comparison processing. The instruction unit 82 sends the prompt as an image generation command to the generation AI module 31 to generate an image. The generation AI module 31 generates an image GI with the consumable color included in the prompt as the main color and sends it to the terminal device 20. The receiving unit 83 of the terminal device 20 receives the generated image GI from the generation AI module 31. When the user selects to print the generated image GI through the operation of the input unit 21, the printing driver 72 of the printing control device 70 instructs the printing unit 50 to print the generated image GI. The printing unit 50 prints the generated image GI. In this way, by printing the generated image GI by the printing unit 50, the recording material of the consumable color is consumed more than the other recording material.

[0196] Therefore, as Figure 23 As shown in the recording material information RI, the consumption of light blue-green (LC) and yellow (Y), which are consumed colors, is greater than the consumption of other colors of recording material. Therefore, compared with the pre-printing material used to generate the image GI... Figure 22 In comparison, the printed version of the generated image GI Figure 23 The deviation in the remaining amount of recording material on one side is reduced. As a result, it is possible to replace the recording material holders 53 for four of the six colors of recording material (LC, M, Y, LM) at once. Thus, when the recording material holders 53 for the four colors are replaced, the deviation between the remaining amount of these four colors of recording material and the remaining amount of the unreplaced blue-green (C) recording material is reduced.

[0197] Effects of the fourth implementation method According to the fourth embodiment, in addition to obtaining the effects of (1-1) to (1-20) in the first embodiment, as well as the effects of the second and third embodiments, the following effects can also be obtained.

[0198] (4-1) The threshold for determining the consumed color based on the remaining amount information in the recording material information RI includes a lower threshold and an upper threshold. Specifically, the color of the recording material within the range RS where the remaining amount is only slightly larger than the minimum remaining amount by a first difference ΔS1 is set as the lower threshold, and the color is slightly larger than the minimum remaining amount by a second difference ΔS2 (greater than ΔS1) is set as the upper threshold. Therefore, since the consumption of recording material with a remaining amount significantly larger than the minimum remaining amount is suppressed, the replacement of the recording material holder 53 corresponding to the color that promotes the consumption of recording material through the printing of the generated image GI results in a reduction of the deviation in the remaining amount of multiple recording materials. Furthermore, multiple recording material holders 53 can be replaced at once. Therefore, the replacement frequency of the recording material holder 53 per average number of prints can be reduced.

[0199] Fifth Implementation Method Next, refer to Figure 24 The fifth embodiment will now be described. In this embodiment, the recording material information RI includes information on the unused period of printing using the recording material. The indicator unit 82 generates an image in the AI ​​generation module 31 based on the recording material information RI, which includes information on the unused period of the printing unit 50. The recording material information RI may also include, in addition to information on the remaining amount of recording material, information on the unejected period of the print head 52 constituting the printing unit 50. First, before explaining the unused period, the structure involved in the ejection of the print head 52 will be explained.

[0200] like Figure 24 As shown, the print head 52 has a nozzle array N1 to N6, which is the same number (e.g., six) of ink colors formed by n (e.g., 100 to 500) nozzles 54, numbered #1 to #n, opening on their nozzle opening faces 52A (bottom surface). The print head 52 in... Figure 24 For each nozzle row, n ejection drive elements 58 are built-in, corresponding to the positions of the n nozzles 54 constituting the nozzle row N1. A plurality of ejection drive elements 58, equal to the number of nozzle rows, constitute an ejection drive element group 57. The nozzles 54 and the ejection drive elements 58 constitute the ejection section 59. The ejection drive elements 58 are, for example, piezoelectric or electrostatic drive elements capable of generating vibrations that cause liquids such as ink to be ejected from the nozzles 54. The ejection drive elements 58 may also be heater elements that utilize the pressure (expansion pressure) of bubbles generated by heating the ink and causing it to boil through film to eject ink droplets from the nozzles 54.

[0201] When a specific color nozzle 54 is used frequently during printing, the ink in other nozzles 54 is not replaced because they are used less frequently. As a result, the ink in the less frequently used nozzles 54 tends to thicken. This thickening of the ink in the nozzles becomes a cause of poor ejection. Therefore, by ejecting ink unrelated to printing onto an object other than the medium M (e.g., a cover) at fixed intervals, the ink in the nozzles 54 is refreshed. This suppresses ejection problems caused by the thickening of the ink in the nozzles 54.

[0202] The control unit 60 can also measure the non-ejection period of the nozzle 54 during printing and store the measured non-ejection period information in the storage unit 62. In the case of an inkjet printing apparatus 11, the non-ejection period information is equivalent to an example of the unused period information of the recording material during printing. The non-ejection period information is defined as the non-ejection period of the nozzle 54 with the longest non-ejection period among the multiple (n) nozzles 54 belonging to each color's nozzle column. In this way, the control unit 60 stores the non-ejection period information as unused period information in the storage unit 62 by color (e.g., by nozzle column). If the unused period based on the unused period information is long, there is a possibility that the nozzle column contains nozzles 54 whose ink has thickened. The control unit 60 performs a nozzle inspection to check for the presence or absence of nozzle 54 ejection defects. When a nozzle ejection defect is detected based on the nozzle inspection results, the maintenance unit 56 performs forced cleaning to expel ink from the nozzle 54. When the nozzle is severely clogged, the cleaning intensity increases. Since cleaning involves forcibly expelling ink from nozzle 54, ink is consumed unrelated to printing. Therefore, it is preferable to perform printing with ink ejected from the nozzle at appropriate intervals to prevent the period of inactivity from becoming too long.

[0203] Therefore, in this embodiment, the instruction unit 82 instructs the generation AI module 31 to generate an image based on the recording material information RI, including unused period information. For example, the instruction unit 82 determines the nozzle row to which the nozzle 54 with a longer unused period belongs based on the unused period information. The instruction unit 82 selects the color corresponding to the determined nozzle row as a candidate color for consumption, in a manner that the color is used for printing. On the other hand, the instruction unit 82 selects the color that satisfies the consumption color selection conditions shown in the first embodiment as a consumption color based on the remaining amount information indicating the remaining amount of each color in the recording material information RI. If the consumption color is the same as the candidate color for consumption, a generated image using that color is generated. If the consumption color is different from the candidate color for consumption, the control unit 60 instructs the control unit 60 to include the candidate color for consumption in the color of the generated image, as long as the remaining amount of the candidate color for consumption is not below a threshold.

[0204] Effects of the fifth implementation method According to the fifth embodiment, in addition to obtaining the effects of (1-1) to (1-20) in the first embodiment and the effects in the second to fourth embodiments, the following effects can also be obtained.

[0205] (5-1) The recording material information RI includes information on the unused period of printing using the recording material. In addition to the recording material information RI, an image GI reflecting the color of the unused period of printing using the recording material can also be generated. Therefore, printing with generated image GI can reduce deviations in the remaining amount of recording material and decrease the frequency of printing errors caused by long unused periods of printing using the recording material due to the printing unit 50. For example, in the case of an inkjet printing apparatus 11, printing errors such as leaks caused by poor ejection from the nozzle 54 can be reduced. Furthermore, it can suppress the increase in cleaning frequency or the implementation of high-intensity cleaning.

[0206] Furthermore, the above-described embodiments can also be modified in ways such as the following variations. Further, variations obtained by appropriately combining the above-described embodiments and the variations shown below can be designated as further variations, as can variations obtained by appropriately combining the variations shown below with each other can be designated as further variations.

[0207] • The printing control device 70 can also be configured such that, upon receiving an instruction to print multiple input images II, it determines whether there is enough recording material to print a specified number of copies, and if there is a color with insufficient recording material, the instruction unit 82 instructs the AI ​​generation module 31 to change the color of the input image II to enable printing the specified number of copies. With this configuration, deviations in the remaining amount of multiple recording materials can be reduced, and the user can print a specified number of images. For example, it can prevent situations where there are insufficient copies of presentation materials.

[0208] • In the first and second embodiments Figure 16 as well as Figure 19 The sequence shown can also be structured as follows: cancel printing based on the original image's printing instruction acceptance, and begin processing from the point where the image-generated instruction acceptance is received. In other words, in Figure 16 In this process, steps S11 to S14 are cancelled, and processing begins from step S15. Furthermore, in... Figure 19In this process, step S31 is set as an instruction acceptance for image generation, replacing the instruction acceptance for printing the original image. The user initiates the application of the first image generation unit 73 by operating the input unit 21 of the terminal device 20, thereby enabling the user to instruct the image generation performed by the generation AI module 31. When the first image generation unit 73 is activated, for example... Figure 9 The image generation confirmation screen 100 shown is displayed on the display unit 22. The message MS1 in the image generation confirmation screen 100 can also be set to include content on image generation not based on the recording material information RI, for example, it can be set to simply state "Generate AI image?" In this way, for the user, the application structure can also be used for the purpose of AI image generation unrelated to printing. In this case, the application structure can also be used for the purpose of AI image generation using input image II. The instruction unit 82 can also cause the AI ​​generation module 31 to generate image GI based on the recording material information RI and the input image information IS including input image II. The instruction unit 82 can also cause the AI ​​generation module 31 to generate image GI based not only on the recording material information RI, but also on information including prompts (e.g., text information) input by the user. The first image generation unit 73 can also cause the display unit 22 to display a print confirmation screen 140 that confirms the printing execution of the generated image GI. The first image generation unit 73 can also cause the display unit 22 to display print selection screens 170 and 190 for the user to select an image for printing from multiple images including the generated image GI. The printing selection screens 170 and 190 may also include an image GI generated by the AI ​​generation module 31 based on the recorded material information RI, and an image generated by the AI ​​generation module 31 without being based on the recorded material information RI. In the case where the instruction unit 82 causes the AI ​​generation module 31 to generate the image GI based on the recorded material information RI, the printing system 10 may also include a prompt generation unit 84 that generates a prompt as an image generation command based on the recorded material information RI. It is also possible to select whether to save the generated image as another file after image generation.

[0209] • In the first embodiment and the above-described modifications, it is also possible to base on Figure 13 The generated image GI generated by the AI ​​generation module 31 based on the recorded material information RI, and the generated image generated by the AI ​​generation module 31 without being based on the recorded material information RI, are displayed on the display unit 22 in a selectable manner.

[0210] • In the second embodiment and the above-described modifications, it is also possible to only use the method based on Figure 18 A generated image GI of the recorded material information RI is displayed on the display unit 22.

[0211] • In the second embodiment and the above-described modifications, based on Figure 18 The generated image GI of the recorded material information RI can also be selected from multiple generated images GI of different types of charts.

[0212] • In the second embodiment and the above-described modifications, instructions for image generation can also be received (e.g., Figure 16 The processing begins from step S15, and the instruction received includes audio data or motion image data. Thus, for example, meeting minutes containing diagrams such as mind maps or fishbone diagrams can be generated based on the meeting's audio data or motion image data. As an example of this generative AI module 31, examples include "Mapify," which can generate mind maps containing keywords based on audio or images (including motion images).

[0213] • The instruction unit 82 may also instruct the AI ​​module 31 to perform extraction processing to find frame images that satisfy the color scheme based on the recording material information RI from the dynamic image data that serves as the input image II. In this case, the instruction unit 82 may also extract a specific region, including the part that implements the color scheme, from the frame image it finds and use it as a generated image GI.

[0214] • The instruction unit 82 can also generate an image GI by extracting frame images including charts, tables, mind maps, etc. from dynamic image data and implementing color matching instructions for the charts, etc. based on the recorded material information RI.

[0215] • The instruction unit 82 can also generate an image GI based on the color matching of the recorded material information RI by cooperating with the processing of the respective generation AI modules 31 of the multiple servers 30 to give their respective instructions.

[0216] • In the second embodiment and the above-described modifications, the generated image GI can also be an image in which a chart 171 or a table is incorporated into the document data 160, which serves as the original data. The incorporated chart 171 or table can also be generated by the AI ​​generation module 31 based on the text or values ​​in the document data 160.

[0217] • Although in the various embodiments described above, the multiple recording materials in the recording material information RI related to multiple recording materials of different colors are set to recording materials of colors other than black, the recording material information RI may also include information related to black recording materials. Furthermore, the recording material information RI only needs to include information related to two or more of the multiple recording materials. That is, it is sufficient for the recording material information RI to include information related to the recording material of the first color and information related to the recording material of the second color, which is a color different from the first color. The recording material information RI is not limited to including all colors; it is sufficient to include at least two colors from the all colors.

[0218] • Although the acquisition unit 81 includes the image generation unit 73 of the terminal device 20 in the described embodiment, the acquisition unit may also include a first server 30 or a second server 40. In this case, the instruction unit may also be an instruction that, instead of including the recording material information RI itself in the instruction to generate an image by the AI ​​generation module 31 based on the recording material information, the AI ​​generation module 31 generates an image after the recording material information has been acquired. In this case, the structure may also be such that the AI ​​generation module 31 that receives the instruction from the instruction unit has an acquisition unit, and the acquisition unit of the AI ​​generation module 31 accesses the terminal device 20 or the printing device 11 to acquire the recording material information RI.

[0219] • Although the terminal device 20 (printing control device 70) includes an image generation unit 73 in the described embodiment, the second server 40 may also include an image generation unit 73. That is, the second server 40 may also include an acquisition unit 81, an instruction unit 82, and a receiving unit 83. In this case, the second server 40 may also instruct the display of a confirmation screen to the display unit 22, etc., by communicating with the terminal device 20.

[0220] • The acquisition unit 81, instruction unit 82, and receiving unit 83 do not need to be centrally located in either the terminal device 20 or the printing device 11; they can be separately located in either one. Furthermore, at least one of the acquisition unit 81, instruction unit 82, and receiving unit 83 can also be located in the second server 40.

[0221] • Although the terminal device 20 generates an image generation command (prompt), the printing device 11 can also generate an image generation command. The instruction unit 82 of the second image generation unit 65 can also generate an image generation command through the prompt generation unit 84.

[0222] • The AI ​​generation module 31 can also be provided by the second server 40. For example, the second server 40 may also provide the AI ​​generation module 31, which has a generated AI model consisting of a learned LLM designed by the manufacturer of the printing apparatus 11 or the like. In this case, it can be a structure that does not use the AI ​​generation module 31 of the first server 30, or it can be a structure that uses the respective AI generation modules 31 of each server 30 and 40 simultaneously. When instructing the dedicated AI generation module 31, it is also possible not to provide prompt-based instructions, but to send only ink balance data as recorded material information RI.

[0223] • The terminal device 20 may also include an AI generation module 31. In this case, the printing control device 70 may also include an AI generation module 31. The printing control device 70 may also include all of the following: an acquisition unit 81, an instruction unit 82, a receiving unit 83, and an AI generation module 31. The printing control device 70 may also include a prompt generation unit 84.

[0224] • The printing apparatus 11 may also include an AI generation module 31. The printing apparatus 11 may also include all of the following: an acquisition unit 81, an instruction unit 82, a receiving unit 83, a printing unit 50, and an AI generation module 31. The printing apparatus 11 may also include a prompt generation unit 84.

[0225] • The printing apparatus 11 can also be a can, i.e., a can type, which can hold liquids such as ink, which is an example of recording material, instead of a replaceable box type for the recording material storage body 53. When the recording material storage body 53 is a can, the user replenishes the liquids such as ink through the bottle. The can type printing apparatus 11 can also have multiple visual confirmation windows on the main body 12, which allow the user to visually confirm the remaining amount of recording material (remaining amount of ink) in the recording material storage body 53. The user can also use the remaining amount image obtained by taking a picture of the remaining amount in the multiple visual confirmation windows with the camera of a terminal device 20 such as a smartphone as the recording material information RI. The acquisition unit 81 can also acquire the remaining amount image as the recording material information RI. For example, it can also be configured such that an upload unit is provided in the screens 100, 110, 120, 130, etc., displayed on the display unit 22 to upload images related to the recording material information RI, and a structure that can upload the remaining amount image is provided. The instruction unit 82 can either directly attach the remaining quantity image as is, or it can instruct the AI ​​generation module 31 to generate an image by providing a prompt containing recording material information RI composed of text or tables obtained from parsing the remaining quantity image. The prompt generation unit 84 can also generate a prompt as an image generation command based on the remaining quantity image. In this case, the prompt can include the remaining quantity image, the remaining quantity image alone, or text containing the parsing results of the remaining quantity image. In the can-type printing apparatus 11, by reducing the deviation of the remaining amount of recording material, the frequency at which the user can replenish recording material to multiple recording material containers 53 (cans) at one time is increased. Therefore, the frequency of replenishing recording material to the recording material containers 53 can be reduced by the average number of printing cycles.

[0226] • It is also possible to use the camera of a terminal device 20, such as a smartphone, to... Figure 5 The remaining quantity image obtained by capturing images of the multiple remaining quantity scales 91-96 in the remaining quantity notification screen 90 is set as the recording material information RI. In this case, the acquisition unit 81 can also acquire the remaining quantity image as the recording material information RI. The indication unit 82 can also include the remaining quantity image in part of the prompt. The prompt can also be just the remaining quantity image. Even if the prompt only includes the remaining quantity image, the generation AI module 31 can obtain the remaining quantity of each color by analyzing the captured image, thereby generating a generated image GI that uses the color of the recording material with the larger remaining quantity as the consumed color.

[0227] • While in the various embodiments described above, it is confirmed whether the image can be generated using the recording material information in the image generation confirmation screen 100 before sending the prompt, it is also possible to configure the system such that the feasibility of AI image generation is confirmed and set in the initial settings of the printing driver 72. In this case, if AI image generation is permitted, the conditions for granting permission can be set during the initial settings. For example, permission can be granted when printing a photograph, or when the deviation of the remaining amount of recording material exceeds a threshold. Furthermore, it is not limited to the initial settings of the printing driver, but can also be set from the setting screen displayed on the display units 16, 22 of the printing apparatus 11 or the terminal device 20, including the feasibility of AI image generation and the permission conditions.

[0228] • Users can also set whether AI image generation is possible in the settings screen displayed on the terminal device 20 or the printing device 11.

[0229] • Alternatively, the display of the image generation confirmation screen 100 or the generation of an AI image may be performed only when it is determined, based on the comparison results of the remaining amounts of multiple recording materials, that the implementation of the recording material remaining amount deviation reduction mode is required.

[0230] • The consumed color can also include the color of the recording material with the lowest remaining quantity among multiple recording materials, following the recording material with the lowest remaining quantity. According to this structure, by bringing the remaining quantities of two or more recording materials on the side with the lowest remaining quantity closer together, the frequency at which multiple recording material receptacles 53 can be replaced at one time is increased.

[0231] • Alternatively, the color of the recording material with the second smallest remaining amount, where the difference ΔS between the second smallest remaining amount and the minimum remaining amount is below a specified value, can be selected as the consumption color. According to this structure, by preventing the color from being selected as the consumption color for recording material with a large difference between the second smallest remaining amount and the minimum remaining amount, it is prevented from being forcibly consumed during the printing of the generated image GI. Furthermore, after multiple recording material receptacles 53 containing the minimum remaining amount and the second smallest remaining amount of recording material can be replaced at once with remaining amounts close to or below depletion, the deviation in the remaining amount of the multiple recording material receptacles 53 after replacement can be reduced.

[0232] • Multiple colors can also include gray. In this case, the generated image can also be a grayscale image using gray recording material. Examples of grayscale images include ink paintings, pencil sketches, pencil illustrations, and monochrome photographs. Furthermore, when gray ink is the primary consumable recording material, a small amount of black recording material can be used in the printing of grayscale images if the remaining amount of black recording material is above a threshold.

[0233] • The recorded material information RI may not be ink remaining information, but rather ink usage information or ink non-ejection period information, etc. The amount of recorded material in the recorded material container 53 can be inferred based on the non-ejection period information for each color. If the image GI is generated by using the non-ejection period information for each color and using the ink of the color with the longer non-ejection period as the primary consumed color, it is possible to reduce the frequency of cleaning when ejection defects occur due to thickened ink in the nozzle of the print head 52, since the thickened ink in the nozzle is ejected through printing.

[0234] •Although in the first and second embodiments, Figure 16 The configuration is such that the recording material information RI is sent from the printing apparatus 11 after the printing of the original image (the printed image). However, this timing can be arbitrary as long as it precedes the original image. If the transmission is performed after the printing of the original image, the remaining quantity information that better reflects the current situation can be obtained. On the other hand, if the recording material information is sent before the printing of the original image begins, the time lag from the end of the printing of the original image to the start of the printing of the generated image GI can be reduced.

[0235] • While the prompts in the first to third embodiments include data of printed images, they are not limited to this. The images included in the prompts can be images printed in the past (e.g., images in the printing history) or images stored in the memory within the terminal device 20. Furthermore, the prompts can be text that is added to the user's input without being related to the printed image, or they can be created simply by recording material information RI.

[0236] • Although in the various embodiments described above, the instruction unit 82 causes the AI ​​generation module 31 to generate an image based on the color information of the medium M, which is the object of printing, the material of the medium M is not limited to paper or synthetic resin film. For example, the printing apparatus 11 may be a garment printer configured to print images on fabric products such as T-shirts or tote bags, or a dyeing and printing apparatus configured to print images on fabrics such as textiles. In this case, the medium M is, for example, cloth, and the color information includes, for example, the color of the cloth base. The color of the cloth base can be one color, or, if there is a pattern, two or more colors. In the various embodiments and this modified example, the instruction (e.g., a prompt) given by the instruction unit 82 to the AI ​​generation module 31 may also include information indicating that the color scheme of the generated image GI is determined based on whether the combination of the color information of the medium M and the color scheme matched in the generated image GI is appropriate, based on the recording material information RI and the color information of the medium M. The instruction may, for example, include information indicating that, for regions in the generated image GI adjacent to the base of the medium M, a color scheme that is the same as or similar to the background color of the medium M has been avoided. In the presence of a consumed color (first consumed color) that is the same as or similar to the background color of the medium M, the color scheme may also include a mixture of the first consumed color and the second consumed color, which are distinct colors. Furthermore, the instruction may also include at least one of a prompt and an image (e.g., a photograph) of the medium M before printing.

[0237] • The instruction unit 82 may, in addition to recording material information RI, also cause the AI ​​generation module 31 to generate an image based on positional information indicating the position in the medium M. For example, the instruction unit 82 may also cause the AI ​​generation module 31 to generate an image GI based on positional information of design elements pre-implemented on the medium M, which is the printing object. As an example, design elements include illustrations, patterns, logos, grid lines, and text. The instruction (e.g., prompt) given by the instruction unit 82 to the AI ​​generation module 31 may include information indicating the meaning of image generation that allows printing in areas avoiding design elements based on the positional information of design elements in the medium M. The instruction may also include at least one of a prompt and an image (e.g., a photograph) of the medium M. The AI ​​generation module 31 generates an image at a position avoiding design elements in the medium M based on the positional information in the instruction (image generation command) received from the instruction unit 82. Therefore, it is possible to avoid situations where design elements such as logos are obscured by the generated image GI when printed on the medium M. Furthermore, the positional information is not limited to information indicating the position of the image being avoided, but may also be information indicating the position of the image being arranged in the medium M.

[0238] • Since the generated image GI may not comply with the policies of a company or organization, the prompt may also include restrictions on image generation, such as restrictions based on the protection of personal information.

[0239] • The image read by the image reading unit 13 of the printing device 11 can also be used as the input image II (original image).

[0240] • While in the various embodiments described above, the user is confirmed to confirm whether image generation is possible by displaying the image generation confirmation screen 100 throughout all printing operations, this is not a limitation. Confirmation of image generation may also be made on the confirmation screen only if the information related to "media type (media type)" as one of the printing condition information is "photo paper," thus determining that it is photo printing. Alternatively, the structure may be such that when the "Yes" button is selected on the confirmation screen, the AI ​​generation module 31 generates the image.

[0241] • Although the method of comparing the remaining amount of recorded material with a threshold is used in the various embodiments described, the consumed color can also be determined solely by the relative difference in the remaining amount of various colors, and the consumption ratio in the generated image GI of the determined consumed color can also be determined.

[0242] • It also allows users to freely choose the mode of generating images by the AI ​​module 31, which increases the proportion of color consumption in the generated image GI, and other modes.

[0243] • If the remaining amount is below a predetermined threshold, an image GI can be generated without using the recording material of that color.

[0244] • The AI ​​generation module 31 can also acquire the recorded material information RI. That is, the AI ​​generation module 31 can also have an acquisition unit 81. In this case, the instruction unit 82 can, for example, include a statement (text) in the prompt indicating "acquire the remaining quantity information from the printing device 11 at the specified address within the same network". For example, the terminal device 20 with the instruction unit 82 can be any PC or the like that can access the AI ​​generation module 31 via the network NW.

[0245] • Although in the various embodiments described above, the information related to black recording material in the recording material information RI is not used for image generation instructions, it is also possible to use the information related to black recording material for image generation instructions. The comparison unit 85 can select the consumed color by comparing the remaining amount of multiple recording materials, including black recording material.

[0246] • Compared to color recording material, black recording material can be used less frequently in image generation. For example, when generating an image structure in a way that consumes less than a predetermined threshold of the amount of recording material remaining, the black threshold can be set higher than the color threshold.

[0247] • Although the prompt generation unit 84 has a comparison unit 85, the comparison unit 85 can be omitted, and instead, the comparison processing can be instructed by prompts, so that the AI ​​generation module 31 can perform the comparison processing.

[0248] • Instead of the recording material consumption measuring unit 64, the structure that measures the amount of recording material consumed can also be a structure that provides a sensor in the recording material storage body 53 to detect the amount of recording material and detects the amount of recording material (e.g., the remaining amount) based on the sensor's detection result.

[0249] • The recording material information RI may also include information about the replacement period of the recording material storage container 53. Since the remaining amount of the recording material storage container 53 can be inferred to be less as the replacement period progresses, the consumption color can also be selected by comparing the replacement periods. In this way, it is not limited to comparing the amounts of multiple recording materials, but at least one other piece of information in the recording material information RI that is inferred to yield a comparison result that is the same as or has the same tendency as the comparison result of the amounts can also be used.

[0250] The information in the recording material information RI used for selecting the consumption color can be, for example, consumption period information related to the consumption period of the recording material. The consumption color can also be selected based on the consumption period information to prioritize the consumption of recording materials with shorter consumption periods. In this case, a threshold can be set for the remaining period until the consumption period, and the color of the recording material with a remaining period below the threshold can be selected as the consumption color. Furthermore, if the recording materials that meet the consumption period conditions include recording materials with a minimum remaining quantity, the colors of other recording materials besides the recording material with the minimum remaining quantity can be set as consumption colors. Here, it can be inferred that the shorter the consumption period of the recording material storage unit 53, the less remaining quantity there is. Therefore, the deviation in the remaining quantity of multiple recording materials can be reduced. This increases the frequency of responding to one replacement or one liquid replenishment for multiple (two or more) recording material storage units 53, and prioritizes the consumption of recording materials with shorter consumption periods. Additionally, the consumption period information only needs to determine the consumption period; it can also be a manufacturing batch number that can determine the manufacturing date or manufacturing period.

[0251] • In cases where there are two or more groups of recording materials with similar quantities, the deviation in the remaining quantity of the two or more recording materials belonging to one group can also be reduced. In this way, the deviation in the remaining quantity of the two or more recording materials belonging to that group can also be reduced to one group of multiple (e.g., multiple colors) recording materials that are selected as the consumed colors. As a result, since the deviation in the remaining quantity of a portion of the multiple recording materials that are selected as the consumed colors, i.e., two or more recording materials, is easier to reduce, the frequency at which it can be handled by replacing or replenishing the liquid in one or more recording material containers 53 can be increased.

[0252] • The printing unit 50 may also exclude the maintenance unit 56. The printing unit 50 may also exclude the conveying unit 55. The printing unit 50 may also consist only of the print head 52.

[0253] • The printing device 11 is not limited to consumer or business use for individuals or families, but can also be a multifunction printer for office use.

[0254] The following will describe the technical ideas and effects learned from the above-described embodiments and their modifications.

[0255] [1] The printing system includes: an acquisition unit that acquires recording material information related to multiple recording materials with different colors; an instruction unit that instructs a generation AI module to generate an image based on the recording material information; a receiving unit that receives the generated image generated by the generation AI module; and a printing unit that uses the recording materials to print the received generated image. According to this structure, since the colors based on the recording material information related to the multiple recording materials with different colors are reflected in the generated image, the deviation in the remaining amount of recording material can be effectively reduced through the printing of the generated image.

[0256] [2] In the printing system described in [1] above, the following method can also be adopted: the indicator unit, in addition to the recorded material information, also causes the AI ​​generation module to generate an image based on the input image information containing the input image. According to this structure, since an image reflecting the color based on the recorded material information and the input image is generated, the user will be interested in the generated image, thereby making it easier to increase the printing frequency of the generated image.

[0257] [3] In the printing system described in [1] or [2] above, the indicator unit may also generate an image based on information including prompts input by the user, in addition to the information on the recorded material. According to this structure, the generating AI module can generate the image desired by the user. Therefore, since the frequency of printing the generated image by the user is increased, the deviation in the remaining amount of recorded material can be effectively reduced.

[0258] [4] In any of the printing systems described in [1] to [3] above, a display unit may be provided that displays an image generation confirmation screen confirming permission for image generation based on the recorded material information. According to this structure, since the user can only permit image generation when desired, delays in printing input images, etc., caused by unwanted image generation can be suppressed.

[0259] [5] In any of the printing systems described in [1] to [3] above, a display unit may be provided that displays the generated image and displays a printing confirmation screen confirming the printing of the generated image. With this structure, printing can be performed quickly if the user wishes to print after confirming the displayed generated image.

[0260] [6] In any of the printing systems described in [1] to [3] above, a display unit may be provided that displays a printing selection screen for selecting an image to be printed from a plurality of images, including the generated image. According to this structure, the user can select the desired image to be printed from a plurality of images, including the generated image.

[0261] [7] In any of the printing systems described in [1] to [6] above, a prompt generation unit may also be included, which generates a prompt as an image generation command based on the recorded material information. According to this structure, a generated image conforming to the image generation command can be generated using a prompt generated based on the recorded material information.

[0262] [8] In the printing system described above [7], the prompt generation unit may also have a comparison unit that compares the remaining amounts of the plurality of recording materials. According to this structure, by comparing the remaining amounts of the plurality of recording materials, the color of the recording material that should be reflected in the prompt can be determined. Therefore, since the consumption of the determined color of the recording material is promoted by printing the generated image, the deviation in the remaining amount of recording material can be effectively reduced.

[0263] [9] In the printing system described in [2] above, the input image can also be a printed image with printing instructions, and the acquisition unit acquires the recording material information after the printed image is printed. According to this structure, since the image is generated based on the latest recording material information after the printed image is printed, the deviation of the remaining amount of recording material can be effectively reduced.

[0264]

[10] In the printing system described in [6] above, the printing selection screen may also include an image generated by the AI ​​generation module based on the recording material information and an image generated by the AI ​​generation module without being based on the recording material information. According to this structure, the user can select a desired image from the image GI generated based on the recording material information and the image generated without being based on the recording material information.

[0265]

[11] In the printing system described in [4] above, the image generation confirmation screen can also be displayed based on the user's printing-related operations. According to this structure, since the user can display the generation confirmation screen at an appropriate timing based on the printing-related operations, it is less likely that the user will miss the confirmation.

[0266]

[12] In any of the printing systems described in [1] to

[11] above, the recording material information may also include information on the remaining amount of the recording material. According to this structure, since an image reflecting the color corresponding to the remaining amount of the recording material can be generated, deviations in the remaining amount of the recording material can be reduced by printing the generated image.

[0267]

[13] In any of the printing systems described in [1] to

[12] above, the recording material information may also include information on the unused period of printing using the recording material. According to this structure, in addition to the recording material information, an image reflecting the color of the unused period of printing using the recording material can be generated. Therefore, by generating an image during printing, deviations in the remaining amount of recording material can be reduced, and the frequency of printing errors caused by long unused periods of printing using the recording material due to the printing unit can be decreased.

[0268]

[14] In any of the printing systems described in [1] to

[13] above, the printing system may also choose whether to save the generated image received by the receiving unit as another file. According to this structure, users can easily save their preferred generated image as another file.

[0269]

[15] In the printing system described in [2] above, the following method can also be adopted: the input image is a file containing multiple image data, the image data including objects as editable elements and templates as non-editable elements; the input image is used as raw data and the generated image generated by the AI ​​generation module is based on the recording material information, and the color of the object in the image data is changed based on the recording material information, while the color of the template is not changed. According to this structure, since a generated image is generated based on the color of the recording material information related to multiple recording materials with different colors, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image.

[0270]

[16] A printing control device is provided in a terminal device for giving printing instructions to a printing apparatus, and includes: an acquisition unit that acquires recording material information related to multiple recording materials of different colors; an instruction unit that causes a generation AI module to generate an image based on the recording material information; a receiving unit that receives the generated image generated by the generation AI module; and an image transmission unit that transmits the received generated image to a printing apparatus that performs printing using the recording materials. According to this structure, since the colors based on the recording material information related to multiple recording materials of different colors are reflected in the generated image, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image.

[0271]

[17] A printing method includes: acquiring recording material information related to a plurality of recording materials with different colors; generating an image based on the recording material information and by a generation AI module; and printing the generated image generated by the generation AI module using the recording materials. According to this structure, since the colors based on the recording material information related to the plurality of recording materials with different colors are reflected in the generated image, the deviation of the remaining amount of recording material can be effectively reduced by printing the generated image.

[0272] Symbol Explanation 10…Printing system; 11…Printing apparatus; 12…Main body; 12A…Discharge port; 13…Image reading unit; 14…Cover; 15…Input unit; 16…Display unit; 17…Supply box; 18…Discharge tray; 19…Communication unit; 20…Terminal device; 21…Input unit; 22…Display unit; 30…Server (first server); 31…AI generation module; 40…Server (second server); 41…AI generation service providing module; 50…Printing unit; 51…Carriage; 52…Print head; 52A…Nozzle opening face; 53…Recording material storage body; 54…Nozzle; 55…Conveyor unit; 56…Maintenance unit; 57…Ejection drive element assembly; 58…Ejection drive element; 59…Ejection unit; 60…Control unit ; 61…Computer; 62…Storage unit; 63…Printing control unit; 64…Record material consumption measurement unit; 65…Image generation unit (second image generation unit); 70…Printing control device; 71…Computer; 72…Printing driver; 73…Image generation unit (first image generation unit); 81…Acquisition unit; 82…Indication unit; 83…Receiving unit; 84…Prompt generation unit; 85…Comparison unit; 86…Image transmission unit; 88…Printing condition setting screen; 90…Remaining quantity notification screen; 91-96…Remaining quantity scale; 100…Image generation confirmation screen; 101…Option; 102…Option; 103…“Yes” button; 104…“No” button; 110…Image generation method selection screen; 111…Selection Buttons; 112…Select button; 120…Image generation condition confirmation screen; 121…Options; 122…Options; 123…Input field; 124…Generate button; 125…Cancel button; 130…Composite image specification screen; 131…Options; 132…Select button; 133…Select button; 134…Input field; 140…Print confirmation screen; 141…“Yes” button; 142…“No” button; 150…Save confirmation screen; 160…File data; 161-167…Text; 170…Print selection screen; 171…Chart; 172…Bar; 173…Next button; 174…Previous button; 175…Apply button; 180…Image data (slide image); 181 …Object; 182…Object; 183…Template; 190…Print Selection Screen; 191…Image Data; 192…Selection Button; 193…Selection Button; 194…Next Button; 195…Previous Button; 196…Selection Button; NW…Network; HNW…Home Network; M…Media; X1…Width Direction; Y1…Conveyor Direction; RI…Record Material Information; II…Input Image; PI…Print Image; IS…Input Image Information; GI…Generate Image (Image); MS1…Message; MS2…Message; MS3…Message; MS4…Message; MS5…Message; MS6…Message; ΔS…Difference; ΔRn…Remaining Amount Difference; ΔR1…Remaining Amount Difference; ΔR2…Remaining Amount Difference; RS…Range;ΔS1…first difference; ΔS2…second difference; N1~N6…nozzle array.

Claims

1. A printing system, characterized in that, have: The acquisition department acquires information about multiple recording materials that are different in color from each other. The instruction unit, based on the recorded material information, causes the AI ​​generation module to generate an image; The receiving unit receives the generated image generated by the AI ​​generation module; The printing unit uses the recording material to print the received generated image.

2. The printing system as described in claim 1, characterized in that, In addition to the recorded material information, the instruction unit also causes the AI ​​generation module to generate an image based on input image information including the input image.

3. The printing system as described in claim 1 or claim 2, characterized in that, In addition to the recorded material information, the instruction unit also causes the AI ​​generation module to generate an image based on information including prompts input by the user.

4. The printing system as described in claim 1 or claim 2, characterized in that, The device includes a display unit that displays an image generation confirmation screen that confirms permission to generate an image based on the recorded material information.

5. The printing system as claimed in claim 1 or claim 2, characterized in that, The device includes a display unit that displays the generated image and displays a printing confirmation screen that confirms the printing of the generated image.

6. The printing system as claimed in claim 1 or claim 2, characterized in that, The device includes a display unit that displays a printing selection screen that selects an image from a plurality of images, including the generated image, for printing.

7. The printing system as described in claim 1 or 2, characterized in that, It also includes a prompt generation unit that generates a prompt as an image generation command based on the recorded material information.

8. The printing system as claimed in claim 7, characterized in that, The prompt generation unit has a comparison unit that compares the remaining amount of the plurality of recorded materials.

9. The printing system as claimed in claim 2, characterized in that, The input image is a printed image with printing instructions, and the acquisition unit acquires the recording material information after the printed image is printed.

10. The printing system as claimed in claim 6, characterized in that, The printing selection screen includes an image generated by the generating AI module based on the recorded material information, and an image generated by the generating AI module without being based on the recorded material information.

11. The printing system as claimed in claim 4, characterized in that, The image generation confirmation screen is displayed based on the user's printing-related actions.

12. The printing system as claimed in claim 1 or claim 2, characterized in that, The recorded material information includes the remaining quantity information of the recorded material.

13. The printing system as claimed in claim 1 or claim 2, characterized in that, The recorded material information includes information on the unused period of printing using the recorded material.

14. The printing system as claimed in claim 1 or claim 2, characterized in that, The printing system chooses whether to save the generated image received by the receiving unit as another file.

15. The printing system as claimed in claim 2, characterized in that, The input image is a file containing multiple image data, which includes objects as editable elements and templates as non-editable elements. The input image is used as raw data, and the generated image generated by the AI ​​module is based on the recorded material information. The color of the object in the image data is changed based on the recorded material information, while the color of the template is not changed.

16. A printing control device, characterized in that, It is installed in the terminal device to provide printing instructions to the printing apparatus, and includes: The acquisition department acquires information about multiple recording materials that are different in color from each other. The instruction unit, based on the recorded material information, causes the AI ​​generation module to generate an image; The receiving unit receives the generated image generated by the AI ​​generation module; The image transmitting unit transmits the received generated image to a printing apparatus that prints using the recording material.

17. A printing method, characterized in that, include: Obtain information about multiple recording materials that are different in color; An image is generated based on the recorded material information and through an AI generation module; The recorded material is used to print the generated image produced by the AI ​​generation module.

Images