Information processing device, control method for information processing device, and program

Abstract

To improve user convenience when calculating ink levels using external devices. [Solution] The information processing device comprises: a shooting means for recording a video of multiple ink windows corresponding to each of multiple types of ink tanks loaded into a recording device; an acquisition means for acquiring a first image included in the video recorded by the shooting means; an analysis means for performing image analysis on each of the ink windows included in the first image; and a storage means for storing the remaining amount information of the ink tanks for which the analysis means has succeeded in detecting the remaining amount of at least one ink window, by adding the remaining amount information of the ink tanks for which the remaining amount has been detected to the remaining amount information of the ink tanks already detected in the storage unit. The acquisition means acquires a second image included in the video if, after acquiring the first image, there are ink tanks for which the remaining amount information has not been stored in the storage unit, and the analysis means performs image analysis on the second image targeting the ink tanks for which the remaining amount information has not been stored in the storage unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a control method for the information processing apparatus, and a program.

Background Art

[0002] In a recording apparatus capable of replenishing ink in an ink tank, there is a recording apparatus provided with an ink window through which the remaining ink amount in the ink tank can be visually confirmed. In Patent Document 1, a technique has been proposed in which a recording apparatus is photographed with a camera provided in an information processing apparatus, which is an external device, and the photographed data is analyzed to derive the remaining ink amount and update the remaining ink amount setting of the recording apparatus.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Depending on the captured data, analysis of the ink window may fail and the remaining ink amount may not be derived. In particular, when there are multiple ink windows, if the analysis of even one ink window fails, the number of user operations increases, which may increase the burden on the user.

[0005] Therefore, an object of the present disclosure is to improve user convenience in deriving the remaining ink amount.

Means for Solving the Problems

[0006] An information processing apparatus according to one aspect of the present disclosure comprises: a shooting means for taking video footage of a plurality of ink windows corresponding to each of a plurality of types of ink tanks loaded into a recording device; an acquisition means for acquiring a first image included in the video footage taken by the shooting means; an analysis means for performing image analysis on each of the ink windows included in the first image; and a storage means for storing, when the analysis means successfully detects the remaining amount of at least one or more ink tanks corresponding to the ink windows, the remaining amount information of the ink tanks for which the remaining amount has been successfully detected, in addition to the remaining amount information of the ink tanks whose remaining amount has been detected in the storage unit, wherein the acquisition means acquires a second image included in the video footage if, after acquiring the first image, there are ink tanks for which the remaining amount information has not been stored in the storage unit, and the analysis means performs image analysis on the second image targeting the ink tanks for which the remaining amount information has not been stored in the storage unit. [Effects of the Invention]

[0007] According to this disclosure, user convenience can be improved in deriving the remaining ink level. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing a printer and a smartphone, which are examples of image forming apparatus. [Figure 2] This is a block diagram showing the hardware configuration of an image forming apparatus and a smartphone. [Figure 3] This is a block diagram showing the software configuration between an image forming apparatus and a smartphone. [Figure 4] This is a schematic diagram showing a method for replenishing the liquid in an image forming apparatus. [Figure 5] This is a schematic diagram showing the ink level indicator of an image forming apparatus. [Figure 6] This is a flowchart of the processes performed by a smartphone. [Figure 7] This is an explanatory diagram showing an example of how it looks on a smartphone. [Figure 8]This is a flowchart of the processes performed by a smartphone. [Figure 9] This is an explanatory diagram showing an example of how it looks on a smartphone. [Figure 10] This is a flowchart of the processes performed by a smartphone. [Figure 11] This is an explanatory diagram showing an example of how it looks on a smartphone. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described in detail below. Note that the components described in these embodiments are illustrative examples of the disclosure and do not limit the scope of this disclosure to them alone.

[0010] <Embodiment 1> First, let's describe the outline of Embodiment 1. As mentioned above, a technology has been proposed in which the ink window of a recording device is photographed by a camera provided by an external device, an information processing device, and the remaining ink level is derived by analyzing the captured data, thereby updating the ink level setting of the recording device.

[0011] However, when detecting ink levels by taking a picture of an ink window that displays the remaining ink levels of multiple colors with a mobile device's camera and analyzing the captured data, ink level detection may not be possible for colors where the shooting conditions are not appropriate. In this case, it is not possible to detect the ink levels of all colors in a single detection process.

[0012] Furthermore, generally speaking, if you try to set the ink level in the recording device for each ink color after detecting the remaining ink level, you may have to repeat the ink level detection and setting operation for each ink color.

[0013] In other words, detecting the remaining ink levels of multiple colors presented a challenge due to the large number of user steps required, resulting in a significant user burden. Below, we will explain a method for detecting ink levels that simplifies user operation and improves convenience.

[0014] (Configuration of the Printer) The configuration of an inkjet printer 100 (hereinafter referred to as the printer 100), which is an example of an image forming apparatus, will be described below with reference to FIGS. 1 to 3.

[0015] FIG. 1 is a schematic diagram showing an example of the printer 100 and the smartphone 150, which is an information processing apparatus, included in the information processing system of the present embodiment. The printer 100 described in the present embodiment includes an ink tank unit 110 for storing ink on the front surface. A plurality of types of ink tanks are loaded in the ink tank unit 110. The ink tank unit 110 is configured such that at least the front surface is made of a transmissive member such as plastic or glass, and the ink inside is visible to the user. The front surface portion of the ink tank unit 110 where the remaining amount of ink is visible is also referred to as an ink window.

[0016] FIG. 2 is a block diagram showing an example of the hardware configuration of the printer 100 and the smartphone 150. First, the hardware configuration of the printer 100 will be described.

[0017] FIG. 2(a) is a diagram showing an example of the hardware configuration of the printer 100. The printer 100 includes a CPU 101, a ROM 102, a RAM 103, a network communication unit 104, a network connection unit 105, a display unit 106, an operation unit 107, a scanner unit 108, a printer unit 109, and an ink tank unit 110. The CPU 101 is a central processing unit for controlling each unit in the printer 100. The ROM 102 stores various program codes. Image data and the like are temporarily stored or buffered in the RAM 103 when each service is executed. The ROM 102 and the RAM 103 are also referred to as a storage unit.

[0018] The network communication unit 104 connects to the router via the network connection unit 105. That is, the network communication unit 104 connects to a network such as the Internet via the network connection unit 105 and performs communication. The RAM 103 also stores image data and other data received by the network communication unit 104.

[0019] The display unit 106 is composed of, for example, a liquid crystal display and is capable of displaying characters, graphics, and indicators. The display unit 106 is not limited to a liquid crystal display; it can also be composed of LEDs or other types of displays. Information displayed by the display unit 106 includes, for example, setting information for the printer 100 and information about the ink contained in each ink tank. Further information about ink includes, for example, information about the remaining amount of ink in the ink tanks and information about ink replenishment instructions. The operation of the display unit 106 is controlled by the CPU 101. The operation unit 107 consists of switches, hard keys, etc., for the user to perform various input operations. It is also possible to configure the display unit 106 to function as an operation unit for input operations, such as a touch panel.

[0020] In printer 100, the scanner function is also provided by the scanner unit 108, and the printing function is provided by the printer unit 109. Ink is stored in the ink tank unit 110. The ink tank unit 110 is equipped with an ink inlet 402, and ink can be refilled from an ink bottle 403. The printer unit 109 has an inkjet head, and ink is supplied to the inkjet head from the ink tank unit 110 via a tube. The printer unit 109 also prints images on a recording medium such as printing paper using an inkjet method, based on image data received from an external source, image data read from the scanner unit 108, etc. Furthermore, the printer unit 109 also manages ink information for the ink tank, including ink level information, and paper information, including the number of sheets of paper loaded.

[0021] The scanner unit 108 optically reads the document placed on the document glass 201 and converts it into electronic data. It then converts the image data into a specified file format and transmits it to an external device via the network or stores it in a storage area such as an HDD. The copying function is achieved by scanning the document placed on the document glass 201 with the scanner unit 108, transferring the generated image data to the printer unit 109, and the printer unit 109 storing the image data on a storage medium. Each of the above units is interconnected by the bus 111, enabling them to send and receive data from each other.

[0022] Figure 3 shows the software configurations of printer 100 and smartphone 150. First, the software configuration of printer 100 will be described.

[0023] Figure 3(a) shows the software configuration of printer 100. Printer 100 has an application layer 320 for managing applications, a middleware layer 330 for controlling the device via various interfaces, and an operating system (OS) 340 for managing overall control.

[0024] The operation system 340 provides the basic functions for executing control programs in the control unit. The middleware layer 330 consists of software that controls the interface with the printer or each physical device. The printer control module 331, which controls the printer interface, is responsible for printing operations and overall functionality. The middleware layer 330 also includes a scanner control module 332 that controls the scanner interface, an operation unit control module 333 that controls the operation unit interface, and an image processing module 334 that controls image data. The printer 100 passes the controlled image data to the printer control module 331 and performs printing. When using functions such as copying, the image data is read from the scanned image data memory where the scanned image is stored and similarly passed to the printing unit.

[0025] The application layer 320 implements functions such as printing provided by this printing device to the user by operating each device via middleware. For example, when the operation unit control module 333 detects that the user has performed a print operation via the operation unit, the application layer 320 is notified of this. The application layer 320 then executes the print job using the job management application 321.

[0026] The job management application 321 uses the printer control module 331 of the middleware layer 330 to execute printing operations using the ink head. If necessary, it controls the content displayed on the control panel via the control panel control module 333 and the operation of the printer via the printer control module 331.

[0027] (Smartphone configuration) Figure 2(b) is a block diagram showing an example of the hardware configuration of a smartphone 150. The smartphone 150 includes a CPU 151, ROM 152, RAM 153, network communication unit 154, network connection unit 155, display unit 156, operation unit 157, imaging unit 158, and image processing unit 160. The CPU 151 is a central processing unit for controlling each part of the smartphone 150. The ROM 102 stores various program codes. The RAM 153 stores image data captured by the smartphone's camera and applications downloaded to the smartphone.

[0028] The network communication unit 154 connects to the router via the network connection unit 155. That is, the network communication unit 154 connects to a network such as the Internet via the network connection unit 155 and communicates. Furthermore, it is possible to make calls with other mobile terminals via the network communication unit 154.

[0029] The display unit 156 is composed of a liquid crystal display or the like, and is capable of displaying characters, graphics, and indicators. The operation unit 157 is a touch panel that allows the user to perform various input operations, and the display unit 156 functions as the operation unit for input operations. The imaging unit 158 ​​is capable of capturing images, and the captured images can be processed by the image processing unit 160. The imaging unit 158 ​​also captures images of the ink window, converts the captured ink level image to an ink level value, and transmits the converted ink level value to the printer body.

[0030] Figure 3(b) shows the software configuration of smartphone 150.

[0031] The smartphone 150 has an application layer 350 for managing applications, a middleware layer 360 for controlling the device via various interfaces, and an operating system (OS) 370 for managing the overall control.

[0032] The operation system 370 provides basic functions for executing the smartphone control program in the control unit. The middleware layer 360 consists of a group of software that controls the interface with the smartphone and various physical devices. The smartphone control module 361 controls the overall operation of the smartphone. It also includes an operation unit control module 363 that controls the operation unit interface and an imaging unit control module 364 that controls the camera. The imaging unit control module 364 is used to capture the ink level window of the printer body, which is a feature of the present invention. Furthermore, it also includes an image processing module 365 that controls the image data. This is used when analyzing the captured ink level image data and converting it into an ink level value, which is a feature of the present invention.

[0033] The middleware layer 360 implements the functions of the smartphone provided to the user by operating each device through each middleware. The smartphone application 351 of the smartphone application layer 350 operates via the control unit. In this embodiment, the user uses a printer control application as the smartphone application 351. The functions of the printer control application include issuing print commands for images on the smartphone. It also displays information on the printer's ink level on the smartphone application. In this embodiment, the printer control application activates the camera to take a picture of the ink window on the printer body. The captured image of the ink level of the ink window is converted into a numerical value indicating the percentage of remaining ink.

[0034] (How to refill the ink in the printer) Figure 4 is a schematic diagram showing how to replenish the liquid in printer 100. Opening the ink tank cover 301 of printer 100 provides access to the ink tank section 110. Ink is supplied to the ink tank section 110 from the ink bottle 303 by opening the cap on the ink inlet 302 after opening the ink tank cover 301. The ink tank section 110 is also called the loading section.

[0035] In this embodiment, the printer 100 is configured as a color printer that ejects multiple types of ink of different colors to record a color image on a recording medium. For this reason, four types of ink tanks are provided to store cyan, magenta, yellow, and black inks. Figure 4 shows an ink tank 110C that contains cyan liquid, an ink tank 110M that contains magenta liquid, an ink tank 110Y that contains yellow liquid, and an ink tank 110Bk that contains black liquid. The basic configuration of each ink tank in this embodiment is substantially the same. In the following description, when each ink tank is described comprehensively, the ink tank section will be denoted by reference numeral 110.

[0036] Figure 5 is a schematic diagram showing the display unit of the ink tank section 110 of the printer 100. The ink level display unit 501 is a part that displays the ink level of each ink tank, allowing the actual amount of stored ink to be visually referenced. The ink level display unit 501 also has a scale that shows the ink level in stages. In this embodiment, the scale is attached to the ink tank side, but this scale may also be attached to the printer body side. The ink level 502 shows the actual amount of ink stored in each ink tank by filling it in.

[0037] (Processing performed on a smartphone) Figure 6 is a flowchart showing the process of setting the ink level performed by the smartphone 150 in this embodiment.

[0038] Figure 7 illustrates the display screen of the smartphone 150. Screen 750 in Figure 7(f) is an application screen for checking the printer status. This application has three functions, for example: ink level capture function 752, ink level display function 753, and connection setting function 754. The title section 751 is where the screen title is written, and in this screen, the application name is written as "Remote Connection". The ink level capture function 752 is a button that starts the function of analyzing the ink level from captured data. When this button is pressed, the processing flow in Figure 6 is started, capture is performed through the imaging unit 158, and the ink level analysis is executed. The ink level display function 753 is a button that starts the function of displaying the ink level of the printer 100. When this button is pressed, the main unit information management module 335 is queried and the ink level is displayed on the display unit 156. The connection setting function 754 is a button that starts the function of changing the settings of the communication unit of the printer 100. Pressing this button allows you to change the communication settings within the main unit information management module 335 via communication.

[0039] Each process in the flowchart of Figure 6 is performed by the CPU 151 of the smartphone 150 executing a control program that has been read from the ROM 152 and loaded into the RAM 153. Therefore, the following flowchart will be explained assuming that the CPU 151 is the primary operator of the processes. Furthermore, prior to the ink level setting process, the smartphone 150 is assumed to be connected to the printer 100 via the network communication unit 154 and the network connection unit 155. In the following explanation of each process, "S" indicates a step in the sequence diagram.

[0040] In S601, the CPU 151 starts recording video. When recording video starts, the CPU 151 displays a screen 700 on the display unit 156 as shown in Figure 7(a), and instructs the user to photograph the ink windows of the printer 100. On screen 700, in order to obtain images of ink windows suitable for ink level detection during video recording, a message 701 and the guide frame 703 are displayed, indicating that all types of ink windows should be photographed so as to fit within the frame (angle of view) of the guide frame 703. The OK button 704 can be pressed when ink level detection is complete to send an ink level update request to the printer 100, but it is ineffective when ink levels have not been detected. The CANCEL button 705 is a button to interrupt and end the ink level photography function, regardless of whether ink levels have been detected or not.

[0041] In S602, the CPU 151 acquires a still image (first image) containing the ink window from the video captured by S601. This is obtained by processing the video data as a still image in any single frame. As shown in Figure 7(c), the display unit 156 displays a screen 710 containing the acquired still image 711.

[0042] In S603, the CPU 151 determines the colors to be analyzed for ink level. In this embodiment, the colors to be analyzed for ink level are determined in the order of C (cyan), M (magenta), Y (yellow), and Bk (black), starting with the colors that have not yet undergone ink level analysis.

[0043] In S604, CPU151 performs image analysis on the still image acquired by S602 to obtain the ink level analysis result. If the analysis detects that there is ink remaining, the percentage of the remaining ink is obtained, with the ink tank capacity set to 100%. On the other hand, if no ink is detected, a result indicating "not detected" is returned.

[0044] In S605, the CPU 151 determines whether it succeeded in detecting the remaining ink level for the ink color determined in S603 based on the analysis in S604. If it determines that the remaining ink level was detected, the CPU 151 proceeds to S606. If it determines that the remaining ink level was not detected, it returns to S603, changes the ink color being analyzed, and performs the ink level analysis again.

[0045] In the S606, the CPU 151 displays the screen 720 shown in Figure 7(d) on the display unit 156, which reflects the results of the ink level detection. Specifically, it displays the ink level corresponding to the ink tank near the display indicating the type of ink tank. In screen 720, the ink levels detected are 15% for M (magenta) and 78% for Y (yellow), while C (cyan) and Bk (black) indicate that no ink level has been detected.

[0046] In S607, the CPU 151 stores the ink level (remaining ink information) detected in S604 in the RAM 153 or ROM 152 (storage unit) of the smartphone 150. This process may be repeated depending on the determination made in S609, which will be described later. Therefore, when storing the ink level information in the storage unit in this process, it is possible that information on the remaining ink levels of other types of ink colors has already been stored. In this case, the remaining ink level information of the ink color whose remaining ink level analysis was successfully completed will be added to the storage unit. In S608, the CPU 151 determines whether all types of ink colors have been analyzed. If it determines that all types of ink colors have been analyzed, the CPU 151 proceeds to S609. If it determines that there are unanalyzed ink colors, it returns to S603, changes the ink color to be analyzed, and performs the ink level analysis again.

[0047] In S609, the CPU 151 determines whether there are any ink colors for which the remaining ink level information is not stored. If it determines that there are ink colors for which the remaining ink level information is not stored, the CPU 151 proceeds to S610. If it determines that there are no ink colors for which the remaining ink level information is not stored (the remaining ink level information for all types of ink colors is stored in the memory unit), it proceeds to S611.

[0048] In S610, the CPU 151 displays a screen 730 on the display unit 156, as shown in Figure 7(d). Screen 730 is a guide display corresponding to the color of the ink level that is not detected (ink level not stored). Message 731 is a message prompting the user to re-photograph the ink window so that the ink level that is not detected is within the guide frame. Guide frames 733 and 734 are displayed to make it easier to take a photograph suitable for detecting the remaining amount of the ink color that is not detected by displaying guide frames corresponding to the C (cyan) and Bk (black) inks that are not detected. Here, the guide display may consist only of a message like message 731, or it may be a display of guide frames like guide frame 733 or 734, or any other display that makes it easier to take a photograph suitable for detecting the remaining amount of the ink color that is not detected. After processing in S610, the CPU 151 returns to S602 to acquire a still image (second image) for ink level analysis again, and thereafter performs the processing described above.

[0049] In S611, the CPU 151 stops recording video. In S612, the CPU 151 determines, based on the user's selection, whether to reflect the ink level detection result in the printer 100. The CPU 151 displays a screen 740 on the display unit 156 that reflects the ink level detection result, as shown in Figure 7(e). On screen 740, the user is asked to select whether to reflect the ink level information stored in RAM 153 or ROM 152 in the ink level setting of the printer 100. If the OK button 704 is pressed, the CPU 151 proceeds to S613. If the CANCEL button 705 is pressed, the process of this flowchart ends without reflecting the ink level setting in the printer 100.

[0050] In S613, the CPU 151 communicates with the printer 100 via the communication control module 362 and sends an ink level update request saved in S607, ending the process. Upon receiving the ink level update request, the printer unit 109 in the printer 100 reflects the received ink level in the ink level setting. After that, the CPU 151 ends the process in this flowchart.

[0051] As described above, this embodiment improves user convenience when calculating ink levels using external devices. Specifically, by executing a processing flow on the smartphone 150, the printer 100 can update the ink level information of the ink tanks stored within the printer device using the ink level values ​​detected by taking a picture with the smartphone's camera. Furthermore, without repeatedly taking pictures and detecting ink levels for ink colors whose levels have already been detected, only ink colors whose levels have not been detected (not stored) are photographed and their ink levels are detected, allowing for the simultaneous setting of ink levels for multiple colors. In addition, by displaying a guide according to the ink colors whose levels have not been detected, it becomes easier to detect the levels of these ink colors, reducing the burden on the user when setting ink levels by photographing the ink window.

[0052] <Embodiment 2> In Embodiment 1, the detected ink level is saved in the smartphone 150 each time, but in this embodiment, the detected ink level is configured to be saved in the smartphone 150 upon user instruction. In this embodiment, only the differences from Embodiment 1 will be described, and the same functions and processes will not be explained.

[0053] (Processing flow performed on a smartphone) Figure 8 is a flowchart showing the processing performed by the smartphone in this embodiment. The processing flowchart in Figure 8 starts when the system detects that the ink level capture function 752 has been pressed by the user. Prior to the ink level setting process, the smartphone 150 is assumed to be connected to the printer 100 via the network communication unit 154 and the network connection unit 155. As with Figure 6, the main processor of the flowchart in Figure 8 is assumed to be the CPU 151.

[0054] Figure 9 shows the smartphone screen corresponding to each process in Figure 8. Processes S801 to S805 are the same as processes S601 to S605 in Embodiment 1, so their explanation is omitted.

[0055] In the S806, the CPU 151 displays a screen 920 on the display unit 156 that reflects the results of the ink level detection. On screen 920, the detected ink levels are 61% for C (cyan) and 15% for M (magenta), while Y (yellow) and Bk (black) indicate that no ink level has been detected. Message 921 indicates that the ink level can be stored in the smartphone 150 by pressing the button for the color whose ink level has been detected. Pressing button 922 stores the detected ink level for C (cyan). Similarly, pressing button 923 stores the detected ink level for M (magenta).

[0056] In S807, CPU151 determines whether all ink colors have been analyzed. If it determines that all ink colors have been analyzed, CPU151 proceeds to S808. On the other hand, if it determines that not all ink colors have been analyzed (that there are unanalyzed ink colors), CPU151 returns to S803, changes the ink colors to be analyzed, and performs the ink level analysis again.

[0057] In S808, CPU151 determines on screen 920 whether button 922 or button 923 has been pressed. If it determines that a button has been pressed, CPU151 proceeds to S809. If it determines that a button has not been pressed, it returns to S803, changes the ink color being analyzed, and performs the ink level analysis again.

[0058] In S809, the CPU 151 stores the remaining ink amount of the ink color corresponding to the button that was determined to have been pressed in S808 by the smartphone control module 361 in RAM 153 or ROM 152. S810 is the same as the processing in S609, so the explanation is omitted.

[0059] In S811, the CPU 151 displays a guide as shown in Figure 9(d). Specifically, the CPU 151 displays a guide frame 933 on the display unit 156 corresponding to the color for which the ink level has not been detected. The guide frame 933 corresponds to the ink colors Y (yellow) and Bk (black), which are the ink colors for which the ink level has not been detected, and facilitates taking a photograph suitable for detecting the ink levels of these two colors. However, the display of the guide frame is not limited to this; if the ink levels of three colors have not been detected, a guide frame suitable for detecting the ink levels of all three colors may be displayed, and if the ink levels of all colors have not been detected, a guide frame suitable for detecting the ink levels of all colors may be displayed. Furthermore, even if the ink levels of all colors have been detected, the guide frame 903 as shown in Figure 9(a) may remain displayed. In addition, although screen 930 shows an example of the display of guide frames when two colors with undetected ink levels are adjacent, if multiple colors that are not adjacent have undetected ink levels, multiple guide frames corresponding to the ink colors with undetected ink levels may be displayed as shown in guide frames 734 and 733. The processing in S812-S814 is identical to that in S611-S613, so the explanation is omitted.

[0060] As described above, according to this embodiment, the detected ink level can be stored at any time the user desires.

[0061] <Embodiment 3> In Embodiment 1, multiple guide frames were displayed during ink window photography according to the ink color whose remaining amount was not detected. However, in this embodiment, the guide frame for the ink with the remaining amount not detected is displayed only when there is only one ink color whose remaining amount is not detected. Note that only the differences from Embodiment 1 will be described in this embodiment, and the same functions and processes will not be explained.

[0062] (Processing flow performed on a smartphone) Figure 10 is a flowchart showing the processing performed by the smartphone in this embodiment. Figure 10 corresponds to the processing at S610 in Figure 6 described in Embodiment 1, and after the processing in Figure 10 is completed, the process returns to S602 in Figure 6 and continues. Other functions and processes are the same and are therefore omitted from the description.

[0063] In S1001, CPU151 determines whether there is one ink color with undetected remaining ink level. If it determines there is one color, CPU151 proceeds to S1002; otherwise, it proceeds to S1003.

[0064] Figure 11 shows the screen displayed on the display unit 156 in steps S1002 and S1003 of Figure 10. In step S1002, the CPU 151 displays a screen 1100 on the display unit 156, as shown in Figure 11(a), which has been updated according to the color of the undetected ink level. The screen 1100 displays a guide frame 1101 to make it easier for the user to take a picture suitable for detecting the remaining amount of cyan (C) ink, which is undetected.

[0065] In S1003, the CPU 151 displays the screen 1110 shown in Figure 11(b) on the display unit 156, corresponding to the color of the ink level that has not been detected. The screen 1110 displays a guide message 1111 to prompt the user to take a picture suitable for detecting the remaining amount of C (cyan) and BK (black) inks that have not been detected, and also displays an overall guide frame 1113.

[0066] As described above, according to this embodiment, by providing a guide display corresponding to one color whose ink level has not been detected, the burden on the user when detecting the ink level by photographing the ink window can be reduced.

[0067] <Other Embodiments> This disclosure can also be implemented by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0068] Furthermore, this disclosure includes the following components.

[0069] (Composition 1) A recording means for video recording multiple ink windows corresponding to each of the multiple types of ink tanks loaded into a recording device, An acquisition means for acquiring a first image included in a video captured by the aforementioned shooting means, An analysis means for performing image analysis on each of the ink windows included in the first image, When the analysis means successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage means adds the remaining amount information of the ink tanks whose remaining amount has been successfully detected to the remaining amount information of the ink tanks whose remaining amount has been detected in the storage unit and stores it there. An information processing device comprising, The information processing apparatus is characterized in that, after acquiring the first image, if there are ink tanks for which remaining amount information is not stored in the storage unit, the acquisition means acquires a second image included in the video, and the analysis means performs image analysis on the second image, targeting the ink tanks for which remaining amount information is not stored in the storage unit. (Configuration 2) The information processing apparatus according to configuration 1, further comprising a transmission means for transmitting the remaining ink amount information of all types of ink tanks stored in the storage unit to the recording device in response to user operation. (Composition 3) The information processing device further includes a display control means that, if there is an ink tank for which no remaining amount information is stored in the storage unit, displays a message on the display unit prompting the user to take another photograph. The information processing apparatus according to configuration 1, characterized in that the acquisition means acquires the second image after displaying a message prompting reshooting. (Composition 4) The information processing device according to configuration 3, characterized in that the display prompting reshooting is a display in which an ink window corresponding to an ink tank with no remaining ink level detected is surrounded by a frame. (Composition 5) The information processing device according to configuration 3, characterized in that the display control means displays a numerical value of the remaining ink amount near the ink tank that has successfully detected the remaining ink amount displayed on the display unit during the video recording. (Composition 6) The information processing apparatus according to any one of configurations 1 to 5, characterized in that the storage means stores the remaining amount of the ink tank whose remaining amount has been detected in the storage unit in accordance with the user's selection operation. (Composition 7) The information processing apparatus according to configuration 6, characterized in that, at the stage when the analysis means has completed the analysis of all ink windows included in the first image, the storage means stores in the storage unit the remaining amount information of the ink tanks for which the analysis means has successfully detected the remaining amount based on the first image. (Composition 8) The information processing apparatus according to any one of configurations 1 to 5, characterized in that the storage means stores the remaining amount of the ink tank for which the remaining amount has been successfully detected in the storage unit without any selection operation by the user. (Composition 9) The information processing apparatus according to configuration 3, characterized in that, at the stage when the analysis of the ink window of the first image is completed by the analysis means, if there is only one type of ink tank for which the remaining amount information is not stored in the storage unit, the display control means displays a frame around only the ink tank for which the remaining amount information is not stored in the storage unit, and displays a message prompting re-shooting. (Composition 10) The information processing device according to configuration 3, characterized in that the display control means displays a message indicating that all types of ink windows should be within the field of view when video recording begins. (Composition 11) The information processing apparatus according to any one of configurations 1 to 10, characterized in that the plurality of ink tanks are ink tanks that contain cyan, magenta, yellow, and black ink, respectively. (Composition 12) The information processing device according to any one of configurations 1 to 11, characterized in that the information processing device is a portable terminal. (Composition 13) A shooting step in which multiple ink windows corresponding to each of the multiple types of ink tanks loaded into the recording device are captured on video, An acquisition step to acquire the first image included in the video captured by the aforementioned shooting step, An analysis step in which image analysis is performed on each of the ink windows included in the first image, When the analysis step successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage step includes adding the remaining amount information of the ink tanks whose remaining amount was successfully detected to the remaining amount information of the ink tanks whose remaining amount has already been detected in the storage unit and storing it; An information processing device comprising, The method for controlling an information processing apparatus is characterized in that, after acquiring the first image, if there are ink tanks for which remaining amount information is not stored in the storage unit, a second image included in the video is acquired, and the analysis step performs image analysis on the second image, targeting the ink tanks for which remaining amount information is not stored in the storage unit. (Composition 14) A program that operates in an information processing device, wherein the information processing device A recording means for video recording multiple ink windows corresponding to each of the multiple types of ink tanks loaded into a recording device, An acquisition means for acquiring a first image included in a video captured by the aforementioned shooting means, An analysis means for performing image analysis on each of the ink windows included in the first image, When the analysis means successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage means adds the remaining amount information of the ink tanks whose remaining amount has been successfully detected to the remaining amount information of the ink tanks whose remaining amount has been detected in the storage unit and stores it there. To make it function as, The acquisition means acquires a second image included in the video if, after acquiring the first image, there are ink tanks for which the remaining amount information is not stored in the storage unit, and the analysis means performs image analysis on the second image, targeting the ink tanks for which the remaining amount information is not stored in the storage unit.

Claims

1. A recording means for video recording multiple ink windows corresponding to each of the multiple types of ink tanks loaded into a recording device, An acquisition means for acquiring a first image included in a video captured by the aforementioned shooting means, An analysis means for performing image analysis on each of the ink windows included in the first image, When the analysis means successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage means adds the remaining amount information of the ink tanks whose remaining amount was successfully detected to the remaining amount information of the ink tanks whose remaining amount has already been detected in the storage unit and stores it. An information processing device comprising, The information processing apparatus is characterized in that, after acquiring the first image, if there are ink tanks for which remaining amount information is not stored in the storage unit, the acquisition means acquires a second image included in the video, and the analysis means performs image analysis on the second image, targeting the ink tanks for which remaining amount information is not stored in the storage unit.

2. The information processing apparatus according to claim 1, further comprising a transmission means for transmitting the remaining ink amount information of all types of ink tanks stored in the storage unit to the recording device in response to user operation.

3. The information processing device further includes a display control means that, if there is an ink tank for which no remaining amount information is stored in the storage unit, displays a message on the display unit prompting the user to take another photograph. The information processing apparatus according to claim 1, characterized in that the acquisition means acquires the second image after displaying a message prompting reshooting.

4. The information processing device according to claim 3, characterized in that the display prompting reshooting is a display in which an ink window corresponding to an ink tank with no remaining ink detected is surrounded by a frame.

5. The information processing apparatus according to claim 3, characterized in that the display control means displays a numerical value of the remaining ink amount near the ink tank that has successfully detected the remaining ink amount displayed on the display unit during the video recording.

6. The information processing apparatus according to claim 1, characterized in that the storage means stores the remaining amount of the ink tank whose remaining amount has been detected in the storage unit in accordance with the user's selection operation.

7. The information processing apparatus according to claim 6, characterized in that the storage means stores in the storage unit the remaining amount information of the ink tanks for which the analysis means has successfully detected the remaining amount based on the first image, at the stage when the analysis means has completed the analysis of all ink windows included in the first image.

8. The information processing apparatus according to claim 1, characterized in that the storage means stores the remaining amount of the ink tank for which the remaining amount has been successfully detected in the storage unit without any selection operation by the user.

9. The information processing apparatus according to claim 3, characterized in that, at the stage when the analysis of the ink window of the first image is completed by the analysis means, if there is only one type of ink tank for which the remaining amount information is not stored in the storage unit, the display control means displays a frame around only the ink tank for which the remaining amount information is not stored in the storage unit, and displays a message prompting reshooting.

10. The information processing apparatus according to claim 3, characterized in that the display control means displays a message indicating that all types of ink windows should be within the field of view when video recording begins.

11. The information processing apparatus according to claim 1, characterized in that the plurality of ink tanks are ink tanks that contain cyan, magenta, yellow, and black inks, respectively.

12. The information processing device according to claim 1, characterized in that the information processing device is a portable terminal.

13. A shooting step in which multiple ink windows corresponding to each of the multiple types of ink tanks loaded into the recording device are captured on video, An acquisition step to acquire the first image included in the video captured by the aforementioned shooting step, An analysis step in which image analysis is performed on each of the ink windows included in the first image, When the analysis step successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage step includes adding the remaining amount information of the ink tanks whose remaining amount was successfully detected to the remaining amount information of the ink tanks whose remaining amount has already been detected in the storage unit and storing it; An information processing device comprising, The method for controlling an information processing apparatus is characterized in that, after acquiring the first image, if there are ink tanks for which remaining amount information is not stored in the storage unit, a second image included in the video is acquired, and the analysis step performs image analysis on the second image, targeting the ink tanks for which remaining amount information is not stored in the storage unit.

14. A program that operates in an information processing device, wherein the information processing device A recording means for video recording multiple ink windows corresponding to each of the multiple types of ink tanks loaded into a recording device, An acquisition means for acquiring a first image included in a video captured by the aforementioned shooting means, An analysis means for performing image analysis on each of the ink windows included in the first image, When the analysis means successfully detects the remaining amount of at least one ink tank corresponding to the ink window, the storage means adds the remaining amount information of the ink tanks whose remaining amount was successfully detected to the remaining amount information of the ink tanks whose remaining amount has already been detected in the storage unit and stores it. To make it function as, The acquisition means acquires a second image included in the video if, after acquiring the first image, there are ink tanks for which the remaining amount information is not stored in the storage unit, and the analysis means performs image analysis on the second image, targeting the ink tanks for which the remaining amount information is not stored in the storage unit.

Images