Program, information processing device, and control method for the information processing device.

Abstract

This can improve the accuracy of ink level detection. [Solution] A program is provided to enable communication between an information processing device and a printing device having a window that allows a user to view the recording material stored in the storage compartment, and the information processing device being connected in a communicative manner to the printing device, the information processing device being enabled to control the imaging of the window, the imaging control means for controlling the imaging of the window, the extraction means for extracting the window from the image of the window acquired by the imaging control means, the first acquisition means for acquiring information about the storage compartment corresponding to the window, and the second acquisition means for acquiring a feature quantity indicating the proportion of the recording material that occupies the storage compartment based on a method corresponding to the color of the recording material that corresponds to the storage compartment, based on the information about the storage compartment.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing apparatus, and a control method for an information processing apparatus.

Background Art

[0002] As one of the ink remaining amount detection methods in an inkjet printer, "dot count remaining detection" is known. This method measures the amount of ejected ink and the amount of ink consumed in the ejection recovery operation by counting the number of ejection times and the number of suction recovery operations, and detects the remaining amount of ink in the ink tank.

[0003] Patent Document 1 discloses a technique for correcting the remaining amount of ink managed by a printing apparatus by resetting the ink consumption amount calculated from the ink consumption history at the time of ink replacement and the ink consumption amount temporarily stored at the time of resetting the remaining amount of ink due to ink replacement. Patent Document 2 discloses a technique for setting the remaining amount of ink in a printing apparatus based on a captured image of an ink remaining amount window of an ink tank in order to appropriately manage the remaining amount of ink replenished to the printing apparatus.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the method of estimating the remaining amount from the ink consumption history as in Patent Document 1, there is a risk that the error will become large. Also, in the method of Patent Document 2, it may be difficult to perform appropriate analysis according to the ink color on the captured image.

Means for Solving the Problems

[0006] A program according to one aspect of the present disclosure is characterized in that an information processing device, which is communicatively connected to a printing device having a window provided so that a user can view recording material contained in a storage section, functions as an imaging control means for controlling imaging of the window; an extraction means for extracting the window from an image of the window acquired by the imaging control means; a first acquisition means for acquiring information about the storage section corresponding to the window; and a second acquisition means for acquiring a feature quantity indicating the proportion of the recording material occupying the storage section based on a method corresponding to the color of the recording material corresponding to the storage section, based on the information about the storage section. [Effects of the Invention]

[0007] According to this disclosure, the accuracy of ink level detection can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of a hardware configuration. [Figure 2] This is a schematic diagram showing an example of a printing device and an information processing device. [Figure 3] An example of a software configuration. [Figure 4] This is a schematic diagram illustrating an example of refilling ink in a printing machine. [Figure 5] This is a magnified view of the ink tank section. [Figure 6] This is a schematic diagram showing the printing device and ink tank section. [Figure 7] This figure shows an example of ink tank information. [Figure 8] This figure shows an example of ink color parameter information. [Figure 9] This flowchart shows an example of a process performed by an information processing device. [Figure 10] This figure shows an example of a display screen shown on the display unit of an information processing device. [Figure 11]It is a diagram showing an example of a captured image of the obtained ink tank unit. [Figure 12] It is a diagram showing an example of a display screen displayed on the display unit of the information processing apparatus. [Figure 13] It is a flowchart showing an example of a process performed by the information processing apparatus. [Figure 14] It is a diagram showing an example of a display screen displayed on the display unit of the information processing apparatus. [Figure 15] It is a flowchart showing an example of a process performed by the information processing apparatus. [Figure 16] It is a diagram showing an example of a display screen displayed on the display unit of the information processing apparatus. [Figure 17] It is a diagram showing an example of mode information. [Figure 18] It is a diagram showing an example of a display screen displayed on the display unit of the information processing apparatus.

Mode for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the matters disclosed, and not all combinations of the features described in the following embodiments are essential for the solution means of the present disclosure. The same components will be described with the same reference numerals.

[0010] If the detection accuracy of the remaining ink amount is low, there is a possibility that a situation may occur in which the detected remaining ink amount is significantly different from the actual remaining ink amount. In order to avoid such a situation, as one method of improving the detection accuracy of the remaining ink amount, for each ink remaining amount window provided so that the user can visually recognize the remaining ink amount, a method of previously determining the target color in image analysis (color analysis) is available.

[0011] Some printing devices have multiple ink remaining amount windows. Further, there are printing devices in which some of the multiple ink remaining amount windows are arranged separately from the remaining ink remaining amount windows (for example, arranged separately on the left and right on the front of the printing device). When an information processing device detects the remaining amount of ink for such a printing device, it is difficult to determine whether the combination of the detected ink remaining amount window and the color targeted in the color analysis for the ink remaining amount window is appropriate.

[0012] As a conventional technique, when an information processing device performs color analysis to detect the remaining amount of ink in an ink tank from captured data, a method in which a user selects a mode of photographing the entire printing device and a mode of photographing each ink tank is known. By this method, the user can be made responsible for judging the validity of the combination of the detected ink remaining amount window and the color targeted in the color analysis for the ink remaining amount window by specifying the color corresponding to the ink remaining amount window to be detected.

[0013] However, in the conventional technique, it may be difficult to perform appropriate analysis according to the ink color on the captured image, and since the user needs to select a mode, the accuracy of detecting the remaining amount of ink and the usability in the setting operation of the remaining amount of ink will decrease. Hereinafter, embodiments for improving the accuracy of detecting the remaining amount of ink and the usability in the setting operation of the remaining amount of ink will be described.

[0014] <<First Embodiment>> In this embodiment, a printing system having a printing device (recording device) and an information processing device which is an external device of the printing device will be described as an example. The printing device in this embodiment will be described by taking an inkjet printer as an example. Note that the recording method of the printer may be an electrophotographic method.

[0015] In this embodiment, the information processing device will be described using a smartphone as an example. Details will be described later, but in this embodiment, an example will be described in which the remaining amount of ink (recording material, recording agent) in a printing device is managed using an image captured using the imaging function of the information processing device. For this reason, a smartphone will be used as an example of the information processing device, but the information processing device may also be a notebook PC or tablet terminal equipped with a camera. Furthermore, the information processing device may acquire an image captured by another camera device, which is a separate device from the information processing device, and perform the processing described later. In other words, the information processing device does not need to have a shooting function.

[0016] Figure 1 is a block diagram showing an example of the hardware configuration of printer 100 and smartphone 150. Figure 2 is a schematic diagram showing an example of printer 100 and smartphone 150. Figure 3 is a block diagram showing an example of the software configuration of printer 100 and smartphone 150. The configurations of printer 100 and smartphone 150 will be described below.

[0017] <Printer Configuration> Figure 1(a) is a block diagram showing an example of the hardware configuration of printer 100. Printer 100 includes a CPU 101, ROM 102, RAM 103, network communication unit 104, network connection unit 105, display unit 106, operation unit 107, scanner unit 108, printer unit 109, and ink tank unit (storage unit) 110.

[0018] The CPU 101 is a central processing unit for controlling the various parts of the printer 100. The ROM 102 stores various program codes. The RAM 103 temporarily stores image data and performs buffering, for example, when executing operations such as printing.

[0019] The network communication unit 104 connects to a router (not shown) via the network connection unit 105. In other words, the network communication unit 104 is configured to connect to a network such as the Internet via the network connection unit 105 and communicate with other devices. The RAM 103 also stores image data and other data received by the network communication unit 104.

[0020] The display unit 106 is composed of, for example, a liquid crystal display and is capable of displaying characters, graphics, and indicators. The display unit 106 is not limited to a liquid crystal display; it can also be composed of LEDs or other display devices. The information displayed by the display unit 106 may include, for example, the settings information of the printer 100 or information about the ink contained in each ink tank. The information about ink may include, for example, the remaining amount of ink contained in the ink tank or information about ink replenishment instructions. The operation of the display unit 106 is controlled by the CPU 101.

[0021] The operation unit 107 includes switches or hard keys for the user to perform various input operations. Alternatively, the display unit 106 may function as a touch panel, receiving input operations from the user.

[0022] In the printer 100, the scanner function is implemented by the scanner unit 108, and the printing function is implemented by the printer unit 109. Ink is stored in the ink tank unit 110.

[0023] As shown in Figure 2, the ink tank section 110 is configured such that at least its front surface is made of a transparent material such as plastic or glass, allowing the user to see the ink inside. A portion of the ink tank cover 401, which will be described later, is provided so as to overlap with the front portion of the ink tank section 110 from which the remaining ink level can be seen, and is also referred to as the ink level window. In this embodiment, the ink level window is made of a transparent material.

[0024] As will be described later, the ink tank section 110 is equipped with an ink inlet 402 (see Figure 4) and is configured so that the user can replenish the ink from an ink bottle 403 (see Figure 4).

[0025] The printer unit 109 has an inkjet head, and ink is supplied to the inkjet head from the ink tank unit 110 via a tube. The printer unit 109 prints an image on a recording medium such as printing paper using an inkjet method, based on image data received from an external source or image data read from the scanner unit 108. Furthermore, the printer unit 109 also manages ink information for the ink tank unit, including information on the remaining ink level (e.g., ink level value), and paper information, including information on the number of sheets of paper loaded.

[0026] Ink level information is managed to properly notify the user when it is time to replace the ink. For example, the ink level value will show 100% immediately after the ink has been replaced and 0% when the ink needs to be replaced. If the ink level is not set correctly in the printer unit 109, the printed area may appear faded even though the printer recognizes that there is still ink remaining. Conversely, if the ink consumption dot count is not reset to 0 after the ink has been replaced, the printer may issue an ink-out warning or error to the user even though there is still ink available.

[0027] Ink level information may be stored and managed in non-volatile memory (HDD) or the like (not shown). The scanner unit 108 optically reads the document placed on the document glass 201 (Figure 2), converts it into electronic data, and then transmits the image data, which has been converted into a specified file format, to an external device via the network or stores it in a storage area such as an HDD. The copy function is achieved by the scanner unit 108 reading the document placed on the document glass 201 and transferring the generated image data to the printer unit 109. The printer unit 109 then stores the image data on a storage medium based on that image data. Each of the above-mentioned units is interconnected by the bus 111, and can receive and transfer data to and from each other.

[0028] Figure 3(a) shows an example of the software configuration of printer 100. The control program of printer 100 is broadly divided into three blocks. These three blocks are the application layer 310 which manages applications, the middleware layer 320 which controls the device via various interfaces (I / F), and the operating system 330 which manages overall control.

[0029] The roles of each of the three blocks are explained below. The operating system (OS) 330 provides the basic functions for the control unit, such as the CPU 101, to execute control programs. The middleware layer 320 consists of a group of software that controls the interfaces with each physical device in the printer 100. The middleware layer 320 includes a printer control module 321, a scanner control module 322, an operation unit control module 323, and an image processing module 324.

[0030] The printer control module 321 controls the printer interface and manages printing and general operations. The scanner control module 322 controls the scanner interface. The operation unit control module 323 controls the operation unit interface. The image processing module 324 controls the image data. Printing is performed when the image processing module 324 passes the controlled image data to the printer control module 321. When using the copy function, the image processing module 324 reads the scanned image data from a scan image data memory (not shown) where the scanned image is stored, and similarly passes it to the printer control module 321 to realize the copy function.

[0031] The application layer 310 includes a job management application 311 and a sensor control application 312. The application layer 310 realizes the functions that the printer 100 provides to the user, such as printing, by operating each device via the respective middleware. For example, when the operation unit control module 323 detects a print execution instruction from the user via the operation unit 107, the print execution instruction is notified to the application layer 310. In the application layer 310, upon receiving the print execution instruction, the job management application 311 executes the print.

[0032] The job management application 311 uses the printer control module 321 of the middleware layer 320 to execute printing operations using the inkjet head. The job management application 311 controls the content displayed on the operation unit 107 via the operation unit control module 323 as needed, and controls the operation of the printer part via the printer control module 321. The sensor control application 312 controls various sensors.

[0033] <Smartphone Configuration> Figure 1(b) is a block diagram showing an example of the hardware configuration of a smartphone 150. The smartphone 150 includes a CPU 151, ROM 152, RAM 153, network communication unit 154, network connection unit 155, display unit 156, operation unit 157, imaging unit 158, control unit 159, and image processing unit 160.

[0034] The CPU 151 is a central processing unit for controlling the various parts of the smartphone 150. The ROM 152 stores various program codes. The RAM 153 is a temporary storage area used as work memory. Additionally, image data captured by the smartphone's camera and applications downloaded to the smartphone are stored in the RAM 153 or other non-volatile memory (not shown) within the smartphone 150.

[0035] The network communication unit 154 connects to a router (not shown) via the network connection unit 155. In other words, the network communication unit 154 is configured to connect to a network such as the Internet via the network connection unit 155 and communicate with other devices. Furthermore, the smartphone 150 can make calls with other mobile devices via the network communication unit 154.

[0036] The display unit 156 is composed of a liquid crystal display or the like and is capable of displaying characters, graphics, and indicators. The operation unit 157 is composed of a touch panel for the user to perform various input operations and is configured to accept user input operations. The CPU 151 performs display control to display characters, graphics, and indicators on the display unit 156.

[0037] The imaging unit 158 ​​captures an image. The CPU 151 performs imaging control to capture an image of the subject via the imaging unit 158 ​​using a printer control application described later. The display unit 156 can also display a preview image from the imaging unit 158. The control unit 159 exchanges data with the image processing unit 160 and instructs the image processing unit 160 to perform processing. The image processing unit 160 performs various image processing. For example, the image processing unit 160 can perform processing to obtain the ink level value from the image obtained when the imaging unit 158 ​​captures the ink level window of the printer 100. Note that in Figure 1(b), the CPU 151, control unit 159, and image processing unit 160 are shown as separate blocks, but all of these may be implemented by the CPU 151. Each of the above-mentioned units is interconnected by the bus 161, and can receive and pass data to each other.

[0038] In this embodiment, the smartphone 150 performs a process to capture an image of the ink level window of the printer 100 at the user's request. The smartphone 150 then estimates the ink level of the ink tank 110 from the captured image, obtains the ink level value, and transmits the obtained ink level value to the printer 100. Further details will be described later.

[0039] Figure 3(b) shows an example of the software configuration of smartphone 150. The control program of smartphone 150 is broadly divided into three blocks. These three blocks are the application layer 340, which manages applications; the middleware layer 350, which controls the device via various interfaces; and the operating system 360, which manages overall control. The role of each will be explained below.

[0040] The operating system 360 provides the basic functions for control units such as the CPU 151 to execute the smartphone control program. The middleware layer 350 consists of a set of software that controls the interfaces with each physical device in the smartphone 150. The middleware layer 350 includes a smartphone control module 351, a communication control module 352, an operation unit control module 353, an imaging unit control module 354, and an image processing module 355.

[0041] The smartphone control module 351 controls the smartphone 150 and manages all aspects of its operation. The communication control module 352 controls the communication interface. The operation control module 353 controls the operation interface. The imaging unit control module 354, which controls the imaging unit 158, is used to capture images of the ink level window of the printer 100. The image processing module 355, which controls the image data, is used for processes such as analyzing the captured image data to obtain ink level values.

[0042] The application layer 340 implements the functions of a smartphone by operating each device through its respective middleware. The application layer 340 has a smartphone application 341. The smartphone application 341 of the application layer 340 of the smartphone is launched and operated via the operation unit 157.

[0043] In this embodiment, a printer control application is used as the smartphone application 341. The printer control application has the function of sending a command to the printer 100 to print an image held by the smartphone 150. The printer control application also has the function of displaying the ink level information of the printer 100 on the smartphone application. In this embodiment, the printer control application is configured to activate the imaging unit 158 ​​(camera). For example, based on the user's operation, the printer control application performs a process to capture an image of the ink level window of the printer 100. The printer control application then performs a process to obtain the ink level value from the captured image of the ink level window via the image processing module 355. Details will be described later.

[0044] <How to refill the ink in the printer> Figure 4 is a schematic diagram showing an example of replenishing ink (liquid), a consumable, in a printer 100, which is an example of a printing device. When the ink tank cover 401 of the printer 100 is opened, the user can access the ink tank section 110.

[0045] In the example shown in Figure 4, the four ink tanks 110 are located on the right side of the front of the printer 100, as viewed from the user operating the printer 100. Specifically, the cyan ink tank 110C, magenta ink tank 110M, yellow ink tank 110Y, and black ink tank 110BK are arranged from left to right as viewed from the user.

[0046] The basic configuration of each ink tank section in this embodiment is substantially the same. Furthermore, although the ink tank section 110 and the ink tank cover 401 are located on the front of the printer 100, the installation configuration is not limited to this. For example, the ink tank cover 401 may be located elsewhere on the printer 100, as long as it allows the user to visually check the remaining ink level in the ink tank section 110. In the following description, each ink tank section will be denoted by reference numeral 110 when described comprehensively.

[0047] An ink inlet 402 is provided at the top of each ink tank section 110. The user opens the cap that is blocking the ink inlet 402, inserts the tip of the ink bottle 403 into the ink inlet, and supplies ink from the ink bottle 403 to the ink tank section 110. As mentioned above, the ink level window is part of the ink tank cover 401, but the configuration of the ink level window is not limited to this. For example, the configuration including the front surface of the ink tank section 110 may be considered the ink level window.

[0048] As will be described later, the front surface of the ink tank section 110 in this embodiment has a scale that serves as an indicator of the remaining ink level, but it is not limited to this. For example, the scale that serves as an indicator of the remaining ink level may be provided in the ink level window of the ink tank cover 401. Details of the scale will be described later.

[0049] In this embodiment, the printer 100 is described as a color printer that ejects multiple inks of different colors to record a color image on a recording medium. However, the printer 100 is not limited to this example and may be a monochrome printer that performs printing using black ink, or a printer that can be equipped with five or more inks. Furthermore, the ink tank section 110 is not limited to being concentrated on one side (right side) of the front, but may be separated and arranged on both sides (left and right) of the front.

[0050] Figure 5 is an enlarged view of the ink tank section 110 of the printer 100. The example in Figure 5 is a schematic diagram showing the external appearance of the ink tank section 110, including the ink tank cover 401. In this embodiment, the ink tank cover 401 is provided with an ink level window 501 that overlaps with the transparent front portion of the ink tank section 110. The ink level window 501 is part of the ink tank cover 401 that allows the user to visually check the ink level of each ink tank. The user can visually check the actual amount of ink stored in each ink tank section 110 through the ink level window 501. A scale 503 that shows the ink level in steps is provided so that it can be seen by the user through the ink level window 501. In this embodiment, the scale 503 is described as being provided on the ink tank section 110 side, but it is not limited to this example. The scale 503 may also be provided on the ink tank cover 401 on the printer 100 main body side. The ink level 502 represents the actual amount of ink stored in each ink tank. In other words, the user can visually check the ink level in each ink tank by looking at the ink level in each ink tank and the scale 503.

[0051] <Printer's ink tank location and ink color> Figure 6 is a schematic diagram showing the front view of the printer 100 as seen from the user facing it. In Figure 6, an ink tank section 110 is provided on the front of the printer 100 so that the user can visually check the remaining ink level. Hereafter, the external view of the printer 100 shown in Figure 6 will be referred to as the front view. The variations in the arrangement of the ink tank section 110 on the front of the printer 100 will be explained with reference to Figures 6(a) to 6(d).

[0052] Figure 6(a) is a front view showing an example where the ink tank unit 110Ra is located on the right side of the front of the printer 100. In Figure 6(a), the cyan ink tank unit 110C, the magenta ink tank unit 110M, the yellow ink tank unit 110Y, and the black ink tank unit 110BK are arranged from left to right in this order. The arrangement of the ink tank units for each color is not limited to the case shown in Figure 6(a).

[0053] Figure 6(b) is a front view showing an example where the ink tanks 110 are arranged on the left and right sides of the front of the printer 100. In Figure 6(b), the black ink tank 110BK is located in the left ink tank 110Lb of the printer 100. In Figure 6(b), the cyan ink tank 110C, magenta ink tank 110M, and yellow ink tank 110Y are arranged from left to right in the ink tank 110Rb of the printer 100.

[0054] Figure 6(c) is a front view showing an example where the ink tanks 110 are arranged on the left and right sides of the front of the printer 100. In Figure 6(c), the black ink tank 110BK and the gray ink tank 110GY are arranged from left to right in the ink tank 110Lc on the left side of the printer 100. In Figure 6(c), the cyan ink tank 110C, the magenta ink tank 110M, and the yellow ink tank 110Y are arranged from left to right in the ink tank 110Rc on the right side of the printer 100.

[0055] Figure 6(d) is a front view showing an example where the ink tanks 110 are arranged on the left and right sides of the front of the printer 100. In Figure 6(d), the cyan ink tank 110C, magenta ink tank 110M, and yellow ink tank 110Y are arranged in that order from left to right in the ink tank tank 110Ld on the left side of the printer 100. In Figure 6(d), the black ink tank 110PGBK for photo printing, black ink tank 110BK, and gray ink tank 110GY are arranged in that order from left to right in the ink tank tank 110Rd on the right side of the printer 100. However, the arrangement of the ink tanks 110 is not limited to these variations. For example, the ink tanks 110 may be arranged on the sides of the printer 100, in addition to the left and right sides of the front of the printer 100.

[0056] Figure 7 shows an example of ink tank information for managing the location of the ink tank section 110 in the printer 100 and the ink color corresponding to the placed ink tank section 110.

[0057] Ink tank information is data stored in ROM 102 or RAM 103. The position of the ink tanks and the corresponding colors are uniquely determined for each printer. While the ink tank information is stored in a table format in the printer 100, as shown in Figure 7, it can be stored in any format as long as the ink tank positions and corresponding colors can be managed. Furthermore, the ink tank information is the information that the smartphone 150 obtains from the printer 100 during the ink level acquisition process described later.

[0058] In the ink tank information shown in Figure 7, each ink tank number 701 is assigned an ink tank position 702 and an ink color 703. The ink tank information corresponding to the arrangement patterns of the ink tank sections 110 for each color shown in Figures 6(a) to 6(d) is shown in Figures 7(a) to 7(d), respectively.

[0059] For example, according to the ink tank information 704 in Figure 7(a), in the printer 100 shown in Figure 6(a), the ink tank section 110 with ink tank number (1) is located on the right side, and the ink color corresponding to this ink tank section 110 is cyan. However, the ink tank information shown in Figure 7 is just an example and is not limited to the representation shown in Figure 7.

[0060] Figure 8 shows an example of ink color parameter information, which indicates the parameters for each ink color used in the image analysis described later. The ink color parameter information is data stored in ROM 102 or RAM 103. The ink colors available for printing are uniquely determined for each printer. The ink color parameter information is explained as information that the smartphone 150 obtains from the printer 100 in the ink level acquisition process described later, but this is not the only example. For example, the smartphone 150 may store the ink color parameter information as part of the printer control application program or as an external file, without obtaining it from the printer 100.

[0061] In image analysis, a known method for obtaining features from image data based on color space parameters involves setting upper and lower limits for the color space parameters of the color corresponding to the feature, and extracting a group of pixels with a color within this range. In this embodiment, the feature is the remaining amount of ink stored in the ink tank section 110, which will be used as an example.

[0062] In the ink color parameter information shown in Figure 8, a color space parameter 802 for image analysis is assigned to each ink color 801. The color space parameter 802 has upper and lower limits for each hue 802H, saturation 802S, and lightness 802V.

[0063] For example, referring to the ink color parameter information 803 in Figure 8, the ink color 801 is cyan (C), and the hue 802H is set to a range of 0 to 180. The saturation 802S is set to a range of 155 to 255. Furthermore, the lightness 802V is set to a range of 0 to 255. In other words, when the ink color parameter information 803 in Figure 8 is used in image analysis, pixels with parameters set within these ranges for cyan are extracted. However, Figure 8 is just an example; for example, the ink color parameter information may be assigned to colors other than cyan, magenta, yellow, and black, such as photographic black, gray, white, and other colors.

[0064] <Smartphone Processing> Figure 9 is a flowchart showing an example of processing performed by the smartphone 150 in this embodiment. The processing shown in Figure 9 is achieved when the CPU 151 of the smartphone 150 reads the control program and the printer control application program stored in the ROM 152 into the RAM 153, and the CPU 151 executes them. Note that some or all of the functions of the steps in Figure 9 may be implemented by hardware such as an ASIC or electronic circuit. The symbol "S" in the description of each process means that it is a step in the flowchart (the same applies to flowcharts in this specification hereafter).

[0065] As mentioned above, various patterns are possible for the ink tank positions and corresponding ink colors in the printer body. However, in this embodiment, we will use the pattern in which the ink tanks are arranged as shown in Figure 6(b) as an example to explain the processing of the smartphone shown in Figure 9.

[0066] The process shown in Figure 9 is one of the processes executed when a user launches a printer control application, which is a smartphone application 341. Each process shown in Figure 9 is performed by the launched printer control application. For simplicity of explanation, the printer control application, which is a smartphone application 341, will be simply referred to as application 341 below. The printer control application can be any application that can exchange information with the printer 100, and it can be a dedicated application or not. The process in Figure 9 starts when application 341 is launched by the user.

[0067] In S901, the CPU 151 connects the smartphone 150 and the printer 100 via application 341. For example, the smartphone 150 and the printer 100 are wirelessly connected via a router (not shown).

[0068] In S902, the CPU 151 accepts the user's selection of the ink level imaging function via application 341. As mentioned above, application 341 has various functions, such as a printing function that sends a print command to the printer 100, but here we will explain it assuming that the user has selected the ink level imaging function.

[0069] Furthermore, when the CPU 151 receives a selection of the ink level imaging function from the user via application 341, it obtains the ink tank information shown in Figure 7 and the ink color parameter information shown in Figure 8 from the printer 100 shown in Figure 6(b). Note that the timing of obtaining the ink tank information and ink color parameter information is not limited to after the selection of the ink level imaging function from the user. For example, the ink tank information and ink color parameter information may be obtained after the smartphone 150 and the printer 100 are connected in S901, or at any of the processing steps from S903 onwards described later.

[0070] In the S903, the CPU 151 displays the ink level capture screen on the display unit 156 of the smartphone 150 via application 341.

[0071] Figure 10 shows an example of the ink level capture screen. This screen is for the user to take a picture of the ink tank. Specifically, this screen is for the user to take a picture of the ink level window, which is positioned to overlap with the front of the ink tank.

[0072] The message display area 1001 displays messages related to taking pictures of the ink level window. The ink level display section 1002 is an area that displays the ink level values ​​acquired by the image processing module 355 in S904-S912, which will be described later. As shown in Figure 10, if the image processing module 355 has not acquired the ink level values ​​corresponding to each ink tank, the ink level display section 1002 displays an initial value. The ink level display section 1002 displays a hyphen as the initial value. The ink level display section 1002 displays the remaining amount of each ink tank 110 in percentage.

[0073] The ink levels for the cyan ink tank 110C, the magenta ink tank 110M, and the yellow ink tank 110Y are displayed as 1002C, 1002M, and 1002Y, respectively. The ink level for the black ink tank 110BK is also displayed as 1002BK, similar to the cyan, magenta, and yellow ink levels.

[0074] The preview image display unit 1003 displays the image (video) acquired from the lens and image sensor of the imaging unit 158. The guideline 1004 is displayed overlaid on the preview image display unit 1003. By referring to the overlaid guideline (marker), the user can easily determine the position and orientation of the smartphone 150 when photographing the ink level windows of all colors.

[0075] The OK button 1005 is a button that accepts an instruction to set the ink level value displayed on the ink level display unit 1002 to the printer 100. The Cancel button 1006 is a button that accepts an instruction to end the capture of the ink level window. If the Cancel button 1006 is pressed by the user, the CPU 151 closes the ink level capture screen shown in Figure 10 by application 341.

[0076] In S904, the CPU 151 acquires an image (video) from the lens and image sensor of the imaging unit 158 ​​via the image processing module 355 using the application 341. S904 to S914 are processes performed each time an image (video) is acquired, and are performed at a maximum rate of one image (video) per second (FPS) of the smartphone 150.

[0077] Figure 11 shows an example of an image of the ink level window that was acquired. For example, Figure 11(a) shows an example of an image acquired when a user photographed the ink tank section 110Rb on the right side of the printer 100 shown in Figure 6(b). Also, Figure 11(b) shows an example of an image acquired when a user photographed the ink tank section 110Lb on the left side of the printer 100 shown in Figure 6(b). Figure 11(c) shows an example of an image acquired when a user photographed the entire printer 100 shown in Figure 6(b).

[0078] In S905, the CPU 151 extracts the ink level window from the captured image acquired via the image processing module 355 by application 341. Extraction of the ink level window can be achieved by detecting edges in the acquired captured image. For example, the ink level window can be extracted from the captured image by detecting edges corresponding to the exterior color of the printer 100. In this embodiment, an edge refers to the boundary line between adjacent colors in an image that has a brightness difference of a predetermined threshold or more. For edge detection, for example, the CANNY method can be used.

[0079] In S906, the CPU 151 determines, based on application 341, whether there are four ink level windows extracted in S905. If the CPU 151 determines that there are four ink level windows extracted (Yes), it proceeds to S907. If the CPU 151 determines that there are not four ink level windows extracted (No), it proceeds to S908.

[0080] In S907, the CPU 151 obtains the ink level value from the captured image via the image processing module 355 using application 341. For example, if an image like the one shown in Figure 11(c) is acquired and four ink level windows are extracted, in S907 the CPU 151 obtains the ink level value for each of the four extracted ink level windows in global mode. In this embodiment, global mode is a mode in which, when the CPU 151 refers to the ink tank number 701 shown in Figure 7 when acquiring the ink level value, it refers to all ink tank numbers 701 regardless of the ink tank position 702.

[0081] Specifically, the CPU 151 in the S907 refers to ink tank number 701 shown in Figure 7(b) in a predetermined order for the four extracted ink tank level windows. For example, in Figure 11(c), the CPU 151 refers to ink tank number 701 in ascending order for the extracted ink tank level windows, starting from the left. Then, the CPU 151 obtains the ink level value corresponding to each of the four ink tank level windows based on the ink color parameter information in Figure 8, which corresponds to the ink color 703 corresponding to ink tank number 701.

[0082] As an example of how to obtain the ink level value, CPU151 can obtain the ink level value by deriving the percentage of the area enclosed by the outline of the ink level window extracted by S905 that is occupied by ink. As mentioned above, the percentage of the area enclosed by the outline of the ink level window can be derived by extracting pixels that have a color within the range from the upper limit to the lower limit of the parameters set for each color space.

[0083] In S908, the CPU 151 determines, based on application 341, whether there is only one ink level window extracted in S905. If the CPU 151 determines that there is only one ink level window (Yes), it proceeds to S909. If the CPU 151 determines that there is more than one ink level window (No), it proceeds to S910.

[0084] In S909, the CPU 151 obtains the ink level value from the captured image via the image processing module 355 using application 341. For example, if an image like the one shown in Figure 11(b) is acquired and one ink level window is extracted, in S909 the CPU 151 obtains the ink level value for the extracted ink level window in left mode. In this implementation, left mode is a mode in which, when the CPU 151 refers to the ink tank number 701 shown in Figure 7 when obtaining the ink level value, it refers to the ink tank number 701 whose ink tank position 702 is "left".

[0085] Specifically, in the S909, the CPU 151 refers to ink tank number 701, where the ink tank position 702 shown in Figure 7(b) is "left," for one of the extracted ink tank level windows. Then, the CPU 151 obtains the ink level value based on the ink color parameter information in Figure 8, which corresponds to the Bk of ink color 703 corresponding to ink tank number 701(1).

[0086] In S910, the CPU 151 determines, based on application 341, whether there are three ink level windows extracted in S905. If the CPU 151 determines that there are three extracted ink level windows (Yes), it proceeds to S911. If the CPU 151 determines that there are not three extracted ink level windows (No), it proceeds to S904.

[0087] In S911, the CPU 151 obtains the ink level value from the captured image via the image processing module 355 using application 341. For example, if an image like the one shown in Figure 11(a) is acquired and three ink level windows are extracted, in S911 the CPU 151 obtains the ink level value for each of the three extracted ink level windows in right mode. In this embodiment, right mode is a mode in which, when the CPU 151 refers to the ink tank number 701 shown in Figure 7 when obtaining the ink level value, it refers to the ink tank number 701 whose ink tank position 702 is "right".

[0088] Specifically, the CPU 151 in S911 refers to ink tank number 701 shown in Figure 7(b) in a predetermined order for the three extracted ink tank level windows. For example, in Figure 11(c), the CPU 151 refers to ink tank number 701 in ascending order for the extracted ink tank level windows, starting from the left. Then, the CPU 151 obtains the ink level value corresponding to each of the three ink tank level windows based on the ink color parameter information in Figure 8, which corresponds to the ink color 703 corresponding to ink tank number 701.

[0089] If S910 does not determine that there are three ink level windows, it is assumed that either no ink level windows were extracted, or a number of ink level windows different from the number of ink tank numbers corresponding to the mode described above were extracted. Specifically, if the ink tank section 110 of the printer 100 shown in Figure 6(b) is photographed and the number of extracted ink level windows is 0, 2, or 5 or more, the CPU 151 does not acquire the ink level value and returns to S904 to read the next image.

[0090] In S912, the CPU 151 displays the ink level value obtained in S907, S909, or S911 via application 341. Specifically, the CPU 151 displays the obtained ink level value in percentage units on the ink level display unit 1002 shown in Figure 10 via application 341. If the CPU 151 cannot obtain an ink level value, it displays a hyphen on the corresponding ink level display unit 1002.

[0091] In S913, the CPU 151 determines whether the cancel button 1006 was pressed by application 341. If the CPU 151 determines that the cancel button 1006 was pressed (Yes), it proceeds to S917, closes the ink level capture screen shown in Figure 10, and ends this flowchart. If the CPU 151 determines that the cancel button 1006 was not pressed (No), it proceeds to S914.

[0092] In S914, the CPU 151 determines whether the OK button 1005 was pressed by the application 341. If the CPU 151 determines that the OK button 1005 was pressed (Yes), it proceeds to S915. If the CPU 151 determines that the OK button 1005 was not pressed (No), it returns to S904 and loads the next image (frame).

[0093] In S915, the CPU 151 determines, based on application 341, whether the ink level value acquisition result is correct or not. Figure 12 shows an example of the ink level capture screen after the OK button 1005 is pressed in S914. Figure 12 shows an example where the ink level value acquired in S911 is displayed.

[0094] In S915, CPU151 does not read images (frames). In S915, when CPU151 detects that the OK button 1005 has been pressed in S914, it displays a message to the user to confirm whether the ink level value acquisition result is correct, as shown in Figure 12. Specifically, CPU151 prompts the user to confirm whether the ink level value acquisition result is correct by displaying a confirmation message 1201 in the message display area 1001 that reads, "Press OK to reflect this in the printer's ink level."

[0095] In S915, if the CPU 151 detects that the OK button 1005 has been pressed, it determines that the ink level value has been obtained correctly (Yes) and proceeds to S916. If the CPU 151 detects that the Cancel button 1006 has been pressed, it determines that the ink level value has been obtained incorrectly (No) and returns to S904 to load the next image (frame).

[0096] In S916, the CPU 151 controls the printer 100 to set the ink level via application 341. For example, the CPU 151 sends the result of acquiring the ink level value to the printer 100 via application 341. In addition to the result of acquiring the ink level value, the CPU 151 may also send an instruction to the printer 100 to set the ink level.

[0097] When printer 100 receives the result of acquiring the ink level value, it replaces the ink level value it holds with the received ink level value. Alternatively, printer 100 may correct the ink level value it has stored using the received ink level value. For example, it is unlikely that the amount of ink actually stored in the ink tank will change drastically compared to the ink level value held by printer 100. Therefore, printer 100 can perform a correction process that prevents replacement if the value before replacement is very large compared to the ink level value set for itself in S916.

[0098] In S917, the CPU 151 terminates the processing of this flowchart after the ink level capture screen is closed by application 341. This concludes the explanation of the processing flow in smartphone 150.

[0099] According to this embodiment, the ink level set in the printer can be properly managed by updating the ink level using an image of the ink level window captured by a smartphone, which is an external device to the printer. Furthermore, setting the ink level in this embodiment does not require any special user operation, such as specifying the ink color corresponding to the ink tank. In other words, by photographing the ink level window, the ink color stored in the ink level window can be accurately identified, so the ink level can be obtained without impairing usability. In short, this embodiment improves the accuracy of ink level detection and the usability of setting the ink level.

[0100] Furthermore, because it can accurately acquire ink level values, it can improve ordering accuracy in systems that automatically order ink bottles (consumables) based on the remaining ink level in the ink tank. In addition, it becomes possible to accurately notify printer users of ink-out errors.

[0101] <<Second Embodiment>> This section describes an embodiment in which a warning message is displayed to the user when it is determined that the captured ink level window is small. Capturing the ink level window as large as possible improves the accuracy of ink level value acquisition, as it allows for proper extraction of the ink level window and detection of the ink portion from the captured image. Of the ink level value acquisition processes S907, S909, and S911 shown in Figure 9, described in the first embodiment, S907 is the one most likely to use an image in which the ink level window is small.

[0102] Figure 13 shows a flowchart illustrating the processing performed by the CPU 151 of the smartphone 150 in this embodiment. Figure 13 is a flowchart that adds a process for displaying a warning message to the flowchart shown in Figure 9. Specifically, Figure 13 shows an example in which a warning message is displayed when four ink level windows are extracted in S905 and it is determined that the ink level windows in the captured image are small.

[0103] In S906, if the CPU 151 determines, based on application 341, that there are four ink level windows extracted in S905, the process proceeds to S1301. In S906, if the CPU 151 determines, based on application 341, that there are not four ink level windows extracted in S905, the process proceeds to S908.

[0104] In S1301, the CPU 151 determines, based on application 341, whether the ink level window in the captured image is small or not. For example, the CPU 151 determines whether the size of the ink level window occupying the captured image is less than or equal to a predetermined threshold relative to the size of the captured image.

[0105] In S1301, if the CPU 151 determines, based on application 341, that the size of any of the ink level windows extracted in S905 is less than or equal to a predetermined threshold relative to the size of the captured image (Yes), the process proceeds to S1302. In S1301, if the CPU 151 determines, based on application 341, that the size of all of the ink level windows extracted in S905 exceeds a predetermined threshold relative to the size of the captured image (No), the process proceeds to S907.

[0106] In S1302, the CPU 151 displays a warning message via application 341 to notify the user of the correct position for photographing the ink level window, then returns to S904 to load the next image. Figure 14 shows an example of warning the user to take a close-up shot of the ink level window. By displaying the warning message 1401 "If the ink level is not displayed, move closer to the ink level window and take a picture." in the message display area 1001, the user can be prompted to adjust their shooting position.

[0107] In this embodiment, an example was described in which, if the size of any of the ink level windows extracted in S905 is less than or equal to a predetermined threshold relative to the size of the captured image, a warning message is displayed instead of displaying the ink level value. However, the system is not limited to this example. For example, even if the CPU 151 determines that the ink level windows in the captured image are small, it may proceed to S907, acquire the ink level value, and display the acquired ink level value on the ink level display unit 1002.

[0108] According to this embodiment, if the ink level window in the captured image is small, the user can be prompted to adjust the position of the ink level window during shooting. This allows for more accurate acquisition of the ink level value.

[0109] <<Third Embodiment>> This section describes an embodiment in which a warning message is displayed to the user if no ink level windows are extracted, or if a different number of ink level windows than expected are extracted. Figure 15 shows a flowchart illustrating the process executed by the CPU 151 of the smartphone 150 in this embodiment. Figure 15 is a flowchart that adds the process of displaying a warning message to the flowchart shown in Figure 13. Specifically, Figure 15 shows an example in which a warning message is displayed when two ink level windows are extracted in S905. Figure 16 shows an example in which the user is warned to take a picture of the ink level windows so that they fit within the guidelines when two ink level windows are extracted.

[0110] If two ink level windows are extracted in S905, the CPU 151 cannot determine which ink tank number in Figure 7(b) corresponds to the extracted ink level window in the flowchart shown in Figure 9 or Figure 13.

[0111] For example, when acquiring ink level values ​​under the shooting conditions shown in Figure 16(a), it is necessary to acquire the ink level values ​​corresponding to each ink color from the captured images of the ink level windows in the order of cyan (C) and magenta (M). Similarly, when acquiring ink level values ​​under the shooting conditions shown in Figure 16(b), it is necessary to acquire the ink level values ​​corresponding to each ink from the captured images of the ink level windows in the order of M and yellow (Y). In both the shooting conditions shown in Figure 16(a) and Figure 16(b), since there are only two extractable ink level windows, it is not possible to identify the ink color.

[0112] For example, when a user photographs the ink level window of printer 100 as shown in Figure 6(b), the CPU 151 will obtain the ink level value from the captured image of the ink level window based on the ink tank information shown in Figure 7(b). However, since the ink tank information in Figure 7(b) does not include a mode pattern where there are two ink tanks corresponding to the ink level window, the CPU 151 cannot identify the ink color 703 in the shooting conditions shown in Figures 16(a) and 16(b).

[0113] In S1501, if the CPU 151 determines, based on application 341, that there are two ink level windows extracted in S905 (Yes), it proceeds to S1502. In S1501, if the CPU 151 determines, based on application 341, that there are not two ink level windows extracted in S905 (No), it proceeds to S906.

[0114] In S1502, the CPU 151 displays a warning message via application 341 to notify the user about the shooting position of the ink level window, then returns to S904 to load the next image. Specifically, as shown in Figures 16(a) and 16(b), the warning message 1601 "Adjust the shooting position so that the CMY ink level windows are within the guidelines." is displayed in the message display area 1001, prompting the user to adjust the shooting position.

[0115] In this embodiment, if a different number of ink level windows than expected are extracted, the user can be prompted to adjust the position in which the ink level windows are photographed.

[0116] <<Fourth Embodiment>> In S906, S908, and S910, which determine the mode for acquiring the ink level value from the captured image of the ink level window to match the various ink tank layouts shown in Figure 6, the number of ink level windows may be changed. This can be achieved by application 341 referring to the mode information for each number of ink level windows that is stored in advance in printer 100.

[0117] Figure 17 shows the mode information used for mode switching determination for each printer 100 corresponding to Figures 6(a) to 6(d). The mode information shown in Figure 17 is stored as data in ROM 102 or RAM 103, similar to Figures 7 and 8. The layout of the ink tank section is uniquely determined for each printer. In the mode information shown in Figure 17, a mode 1702 and an ink tank number 1703 are assigned to each ink level window number 1701. For example, in the first embodiment, the mode information shown in Figure 17(b) can be applied to a printer laid out as shown in Figure 6(b).

[0118] Specifically, if S906 refers to the number of ink level windows (1704) and is able to extract four ink level windows, CPU 151, in global mode, retrieves the ink level values ​​for ink tanks numbered 1 through 4 in that order.

[0119] In the layout shown in Figure 6(d), where the number of ink tanks on the right side of the printer's front is the same as the number of ink tanks on the left side of the printer's front, the type of mode cannot be determined from the number of ink level windows. Therefore, in the layout shown in Figure 6(d), the ink level acquisition process will be in overall mode.

[0120] In this embodiment, ink level values ​​can be obtained for various printer layouts.

[0121] <<Fifth Embodiment>> In this embodiment, an example is described in which, when S912 displays the ink level value acquisition result in application 341 on the smartphone 150, the ink level window extracted in S904 and the mode identified in S906, S908, or S910 are displayed.

[0122] For example, if one Y ink level window is extracted, S908 will perform a Bk color analysis in left mode. In such a case, S916 may incorrectly set the ink level for printer 100.

[0123] Figure 18 shows an example of displaying the ink level value acquisition result in application 341 on smartphone 150. The screen shown in Figure 18 displays the ink level window display section 1801, which shows the outline of the ink level window extracted by S905, the mode display section 1802, which shows the mode identified by S908, and the ink color display section 1803, which shows the ink color identified by S909, overlaid on the captured image.

[0124] According to this embodiment, the user can easily determine whether the ink level value is being acquired correctly.

[0125] <<Other Embodiments>> In the embodiments described above, the explanation mainly focused on an example where the printing device is an inkjet printer, but the printing device may also be an electrophotographic printer. Therefore, in the embodiments described above, ink was used as an example of the recording material, but toner may also be used as the recording material. That is, in the embodiments described above, the amount of ink was used as an example of the amount of recording material, but the amount of recording material may also be the amount of toner.

[0126] This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0127] The disclosure of this embodiment includes configurations represented by the following example programs, example information processing devices, and example control methods for information processing devices.

[0128] <Configuration 1> A printing device having a window that allows the user to view the recording material stored in the storage compartment, and an information processing device that is communicatively connected to the printing device, An imaging control means for controlling the imaging of the aforementioned window, An extraction means for extracting the window from the image of the window acquired by the imaging control means, A first acquisition means for acquiring information relating to the housing section corresponding to the window, A second acquisition means for acquiring a feature quantity indicating the proportion of the recording material occupying the storage portion, based on a method corresponding to the color of the recording material corresponding to the storage portion, based on information about the storage portion. A program characterized by its ability to function in a certain way.

[0129] <Configuration 2> The program according to configuration 1, characterized in that the information relating to the storage unit includes information indicating the color of the recording material corresponding to each of the plurality of storage units.

[0130] <Structure 3> The program according to configuration 2, wherein, when the extraction means extracts a plurality of windows corresponding to the plurality of housing units, the second acquisition means acquires the feature quantities corresponding to each of the plurality of housing units corresponding to the plurality of windows based on different methods.

[0131] <Structure 4> The information indicating the color of the recording material has parameters of hue, saturation, and brightness corresponding to the color of the recording material. The program according to configuration 3, wherein the second acquisition means acquires the feature quantities corresponding to each of the plurality of storage units based on hue, saturation, and brightness parameters that are different for each of the colors of the plurality of recording materials corresponding to the plurality of storage units.

[0132] <Composition 5> The program according to configuration 1, characterized in that the information relating to the housing unit includes information indicating the order in which the plurality of housing units are arranged in the printing device.

[0133] <Composition 6> The program according to configuration 5, characterized in that the information relating to the housing unit includes information indicating the position of the housing unit in the printing device.

[0134] <Composition 7> The program according to configuration 6, characterized in that the information relating to the housing unit includes information indicating the number of housing units arranged at the position.

[0135] <Structure 8> The program according to configuration 3, characterized in that, when the extraction means extracts a plurality of windows corresponding to the plurality of storage units, the second acquisition means acquires the feature quantities corresponding to each of the plurality of storage units based on the position of the storage unit in the printing device, the order in which the storage units are arranged at the position, and the number of storage units arranged at the position.

[0136] <Composition 9> The program according to configuration 1, characterized in that the information relating to the housing units includes first mode information for acquiring the feature quantities of each of the plurality of housing units, and second mode information for acquiring the feature quantities of some of the housing units among the plurality of housing units.

[0137] <Composition 10> The second acquisition means acquires the feature quantity by referring to the first mode information or the second mode information, The program according to configuration 9, characterized in that the information processing device is further configured as a display control means for controlling the display of the first mode information and the mode information referenced by the second acquisition means among the second mode information on the display unit.

[0138] <Composition 11> The program according to configuration 10, characterized in that, if the size of the window occupying the captured image is less than or equal to a predetermined threshold relative to the size of the captured image, the display control means further performs control to display a message prompting the user to approach the window and take an image.

[0139] <Composition 12> The display control means is Further control is performed to display a marker for adjusting the shooting position of the window, superimposed on the captured image acquired by the imaging control means. The program according to configuration 10 or 11, further characterized in that, if the number of windows extracted by the extraction means differs from the number of housing units acquired by the first acquisition means, a control is performed to display a message prompting the user to adjust the shooting position of the windows on the marker.

[0140] <Composition 13> The program according to configuration 10, wherein the display control means further performs control to display a color corresponding to the feature quantity acquired by the second acquisition means.

[0141] <Composition 14> An information processing device that is communicatively connected to a printing device having a window provided so that the user can view the recording material stored in the storage section, An imaging control means for controlling the imaging of the aforementioned window, An extraction means for extracting the window from the image of the window acquired by the imaging control means, A first acquisition means for acquiring information relating to the housing section corresponding to the window, A second acquisition means for acquiring a feature quantity indicating the proportion of the recording material occupying the storage portion, based on a method corresponding to the color of the recording material corresponding to the storage portion, based on information about the storage portion. An information processing device characterized by comprising:

[0142] <Composition 15> A control method for an information processing device that is communicatively connected to a printing device having a window provided so that the user can view the recording material stored in the storage compartment, An imaging control step that controls the imaging of the aforementioned window, An extraction step for extracting the window from the image of the window acquired in the imaging control step, A first acquisition step involves acquiring information relating to the housing section corresponding to the window, A second acquisition step involves acquiring a feature quantity indicating the proportion of the recording material occupying the storage area, based on a method corresponding to the color of the recording material corresponding to the storage area, based on information regarding the storage area. A control method for an information processing device, characterized by having the following features. [Explanation of Symbols]

[0143] 100 printers 150 Smartphones

Claims

1. A printing device having a window that allows the user to view the recording material stored in the storage compartment, and an information processing device that is communicatively connected to the printing device, An imaging control means for controlling the imaging of the aforementioned window, An extraction means for extracting the window from the image of the window acquired by the imaging control means, A first acquisition means for acquiring information relating to the housing section corresponding to the window, A second acquisition means for acquiring a feature quantity indicating the proportion of the recording material occupying the storage area, based on a method corresponding to the color of the recording material corresponding to the storage area, based on information about the storage area. A program characterized by its ability to function in a certain way.

2. The program according to claim 1, characterized in that the information relating to the storage unit includes information indicating the color of the recording material corresponding to each of the plurality of storage units.

3. The program according to claim 2, wherein, when the extraction means extracts a plurality of windows corresponding to the plurality of housing units, the second acquisition means acquires the feature quantities corresponding to each of the plurality of housing units corresponding to the plurality of windows based on different methods.

4. The information indicating the color of the recording material has parameters of hue, saturation, and brightness corresponding to the color of the recording material. The program according to claim 3, wherein the second acquisition means acquires the feature quantities corresponding to each of the plurality of storage units based on hue, saturation, and brightness parameters that are different from each other for each of the plurality of recording materials corresponding to the plurality of storage units.

5. The program according to claim 1, characterized in that the information relating to the housing unit includes information indicating the order in which the plurality of housing units are arranged in the printing device.

6. The program according to claim 5, characterized in that the information relating to the housing unit includes information indicating the position of the housing unit in the printing device.

7. The program according to claim 6, characterized in that the information relating to the housing unit includes information indicating the number of housing units arranged at the position.

8. The program according to claim 3, wherein, when the extraction means extracts a plurality of windows corresponding to the plurality of storage units, the second acquisition means acquires the feature quantities corresponding to each of the plurality of storage units based on the position of the storage unit in the printing device, the order in which the storage units are arranged at the position, and the number of storage units arranged at the position.

9. The program according to claim 1, characterized in that the information relating to the housing units includes first mode information for acquiring the feature quantities of each of the plurality of housing units, and second mode information for acquiring the feature quantities of some of the housing units among the plurality of housing units.

10. The second acquisition means acquires the feature quantity by referring to the first mode information or the second mode information, The program according to claim 9, further characterized in that the information processing device functions as a display control means for controlling the display of the first mode information and the mode information referenced by the second acquisition means among the second mode information on the display unit.

11. The program according to claim 10, wherein if the size of the window occupying the captured image is less than or equal to a predetermined threshold relative to the size of the captured image, the display control means further performs control to display a message prompting the user to approach the window and take an image.

12. The display control means is Further control is performed to display a marker for adjusting the shooting position of the window, superimposed on the captured image acquired by the imaging control means. The program according to claim 10, further characterized in that, if the number of windows extracted by the extraction means differs from the number of housing units acquired by the first acquisition means, a control is performed to display a message prompting the user to adjust the shooting position of the windows on the marker.

13. The program according to claim 10, wherein the display control means further performs control to display a color corresponding to the feature quantity acquired by the second acquisition means.

14. An information processing device that is communicatively connected to a printing device having a window provided so that the user can view the recording material stored in the storage section, An imaging control means for controlling the imaging of the aforementioned window, An extraction means for extracting the window from the image of the window acquired by the imaging control means, A first acquisition means for acquiring information relating to the housing section corresponding to the window, A second acquisition means for acquiring a feature quantity indicating the proportion of the recording material occupying the storage area, based on a method corresponding to the color of the recording material corresponding to the storage area, based on information about the storage area. An information processing device characterized by comprising:

15. A control method for an information processing device that is communicatively connected to a printing device having a window provided so that the user can view the recording material stored in the storage compartment, An imaging control step that controls the imaging of the aforementioned window, An extraction step for extracting the window from the image of the window acquired in the imaging control step, A first acquisition step involves acquiring information relating to the housing section corresponding to the window, A second acquisition step involves acquiring a feature quantity indicating the proportion of the recording material occupying the storage area, based on a method corresponding to the color of the recording material corresponding to the storage area, based on information regarding the storage area. A control method for an information processing device, characterized by having the following features.

Images