Information processing system, control method, and program

Abstract

This technology provides a way to easily check information regarding the expiration dates of goods. [Solution] An information processing system comprising: an analysis means for analyzing an identification code contained in a first image; a determination means for determining the expiration date of an item corresponding to the identification code analyzed by the analysis means; and a display control means for controlling the determination result by the determination means to be displayed at a predetermined position of the identification code in the first image.

Description

Technical Field

[0001] The present invention relates to an information processing system, a control method thereof, and a program.

Background Art

[0002] In recent years, there is a technology for managing articles by capturing images.

[0003] Patent Document 1 discloses a technology for managing articles using images obtained by photographing the articles.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problem to be Solved by the Invention

[0005] Patent Document 1 discloses recognizing an article included in a captured image, and outputting information for identifying the article and information indicating the position of the article on the captured image.

[0006] However, there is a predetermined deadline depending on the attribute of an article, such as the expiration date for food products and the use period for pharmaceuticals, but Patent Document 1 does not disclose a technology for allowing a user to check information related to the deadline of an article.

[0007] Accordingly, an object of the present invention is to easily check information related to the deadline of an article.

Means for Solving the Problem

[0008] analysis means for analyzing an identification code included in a first image, determination means for making a determination related to the deadline of an article corresponding to the identification code analyzed by the analysis means, A display control means that controls the display of the determination result by the determination means to a predetermined position of the identification code in the first image, An information processing system characterized by comprising the following features. [Effects of the Invention]

[0009] According to the present invention, it becomes possible to easily check information regarding the expiration date of articles. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram shows an example of the configuration of an inventory management support system. [Figure 2] A block diagram showing an example of the hardware configuration of client terminal 101. [Figure 3] This flowchart shows an example of the processing performed by the inventory support system. [Figure 4] This figure shows an example of a data table for inventory information. [Figure 5] This figure shows an example of a screen where a user selects a shelf. [Figure 6] This figure shows an example of the operation screen used for taking photos. [Figure 7] This figure shows an example of a screen that displays the judgment results. [Figure 8] This figure shows an example of the operation screen used by the user to take a second or subsequent photo. [Figure 9] This figure shows an example of a function that prevents duplicate identification code matching. [Figure 10] This is an example of a screen for checking and outputting inventory results and a list of returns. [Figure 11] This diagram shows an example of a screen used to take photos of items to be returned. [Figure 12] This figure shows an example of a data table for inventory results and a list of returns. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] Figure 1 is a diagram illustrating an example of the configuration of an inventory count support system according to an embodiment of the present invention.

[0013] In the information processing apparatus of the present invention, a client terminal 101 and a file server 102 are communicably connected via a network 100.

[0014] Note that the client terminal 101 only needs to incorporate the functions illustrated in Figure 2, and may be, for example, a personal computer (hereinafter referred to as a PC), a mobile terminal such as a smartphone, a tablet terminal, a wearable terminal such as a smart watch, a PDA, a game console, or the like.

[0015] The network 100 may take forms such as a wired LAN, a wireless LAN, or USB, depending on the physical interface included in the file server 102.

[0016] The file server 102 can store inventory information, inventory count results, returned product lists, logs of determination results, and the like. The inventory information and the like that can be stored in the file server 102 described above may be stored in the ROM 202 or the external memory 212 of the client terminal 101. Hereinafter, the hardware configuration of the information processing apparatus applicable to the client terminal 101 illustrated in Figure 1 will be described using Figure 2.

[0017] Figure 2 is a block diagram illustrating a hardware configuration applicable to the client terminal 101 and the file server 102 illustrated in Figure 1.

[0018] In Figure 2, 201 is the CPU, which comprehensively controls each device and controller connected to the system bus 204. The ROM 202 or external memory 212 stores the BIOS (Basic Input / Output System), which is the control program for the CPU 201, the operating system program (hereinafter referred to as the OS), and various programs necessary to implement the functions executed by each PC, as described later.

[0019] 203 is RAM, which functions as the main memory, work area, etc., of the CPU 201. The CPU 201 loads the necessary programs, etc., from ROM 202 or external memory 212 into RAM 203, and then executes the loaded programs to perform various operations.

[0020] 205 is an input controller that controls input from a keyboard (KB) 210 or a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger, etc.) the icons, cursors, or buttons displayed on the touch panel. The touch panel may be a multi-touch screen or other touch panel capable of detecting the positions of multiple fingers touching it.

[0021] 206 is a video controller that controls the display on a display device such as display 211. The display includes the display of a notebook PC integrated with the main unit. Note that the external output device is not limited to a display, but may be a projector, for example. Furthermore, for the aforementioned devices that can accept touch operations, an input device is also provided.

[0022] The video controller 206 can control the video memory (VRAM) used for display control. It can utilize a portion of the RAM 203 as the video memory area, or it can provide a separate, dedicated video memory.

[0023] The CPU 201 enables display on the display 211 by, for example, performing an outline font expansion (rasterization) process in the display information area of ​​RAM 203. The CPU 201 also enables user input via a mouse cursor (not shown) on the display 211.

[0024] In the flowchart-based explanation described later, unless otherwise specified, the display destination will always be display 211.

[0025] 207 is the memory controller, which controls access to external memory 212, such as external storage devices (hard disks (HDs)), flexible disks (FDs), or CompactFlash® memory connected to a PCMCIA card slot via an adapter, for storing various data. As external memory, external storage devices (hard disks), flexible disks (FDs), or CompactFlash® memory connected to a PCMCIA card slot via an adapter can be used to store boot programs, various applications, files, and various data.

[0026] 208 is a communication interface controller that controls the reception of image data from an external PC 213 via the network (TCP / IP). For example, it connects to and communicates with external devices via the network 100 shown in Figure 1. For example, it can communicate using TCP / IP, telephone lines such as ISDN, and 3G, 4G, and 5G mobile phone lines.

[0027] 209 is an image I / F controller that controls the reception of image data from camera 106 via an image transfer cable (USB, Ethernet, Camera Link, etc.).

[0028] The image interface controller may also receive image data from the external memory 212.

[0029] The various programs described later for realizing the present invention are stored in RAM 203 and executed by CPU 201.

[0030] Furthermore, the image data used when executing the above program is stored in ROM202, external memory212, external PC213, and camera106 depending on the application, and is stored in RAM203 via various controllers when the program is executed.

[0031] Next, the processes executed by the client terminal 101 in the embodiment of the present invention will be described using Figures 3 to 12.

[0032] The processes performed by the inventory support system in the embodiment of the present invention will be explained using the flowchart in Figure 3.

[0033] The flowchart in Figure 3 shows an example of the process by which the CPU 201 of the client terminal 101 reads and executes a predetermined control program, and is used to perform inventory support. Note that each step is executed by the CPU 201 of each device.

[0034] When the data table containing inventory information is stored in the client terminal 101 or the file server 102, and the client terminal 101 receives a processing start instruction, the process shown in Figure 3 begins. The data table containing inventory information will be described later using Figure 4.

[0035] In S301, CPU201 starts the inventory support system. Possible methods for starting the system include CPU201 starting the inventory support system when it receives a command from the user (for example, by clicking the application icon of the inventory support system), but other methods of starting the system are also possible.

[0036] In S302, CPU201 accepts the user's selection of a location for inventory taking (e.g., shelves, drawers, storage units). The shelves selected for inventory taking contain the items to be taken, and the item identification codes (hereinafter referred to as identification codes) are located on the appearance or packaging (e.g., boxes or bags) of the items to be taken. The locations where inventory taking is performed (e.g., shelf frames, drawer frames, storage unit frames) have codes (hereinafter referred to as location identification codes) to locate the item identification codes. The identification codes are different for each individual item or for each type of item.

[0037] In this embodiment, eight location identification codes are provided on the shelf frame, and all eight codes are different. In other words, by reading a particular location identification code, it is possible to determine the exact location on the shelf frame (for example, top left, bottom right, top center, etc.).

[0038] In this embodiment, the identification code and location identification code are QR codes (registered trademark), but the identification code and location identification code are not limited to QR codes. Other symbols may be used, such as Chameleon Code, barcode, Aztec Code, etc., as long as they are symbols that allow information corresponding to the symbols to be read by photographing an image containing the symbol and analyzing the image.

[0039] The screen for users to select shelves will be described later using Figure 5.

[0040] In S303, CPU201 displays a screen (hereinafter referred to as the "shooting screen") for taking pictures of the shelf and the items inside the shelf that were selected in S302. The user can perform the shooting described in S304 to S307 while viewing the shooting screen. Details of the shooting screen will be described later with reference to Figure 6.

[0041] In S304, CPU201 determines whether it has recognized the location identification code.

[0042] The process for recognizing the location identification code in this embodiment will now be explained. First, the CPU 201 detects the area containing the location identification code from the screen. In this embodiment, the location identification code is a QR code, and the CPU 201 detects the rectangles (cutout symbols) at the three corners of the QR code. Having detected the cutout symbols, the CPU 201 recognizes the location identification code as a QR code.

[0043] The identification code for S309 is recognized in the same way.

[0044] In this embodiment, it is assumed that it is possible to take multiple photographs of a single shelf.

[0045] In S304, the system determines whether all location identification codes (8 in the case of Figure 6) were recognized on the screen displaying the target to be photographed (live view) during the first shot, and whether a predetermined number (for example, 2 or more) of location identification codes were recognized during subsequent shots. If the CPU201 determines that the codes were recognized (Yes in S304), the system proceeds to S305; if it determines that the codes were not recognized (No in S304), the system proceeds to S303.

[0046] In step S305, CPU201 activates the shutter button on the shooting screen. In other words, while the user was unable to press the shutter button in steps S303 and S304, in step S305 the user becomes able to press the shutter button.

[0047] In S306, CPU201 determines whether the shutter button activated in S305 has been pressed by the user. If it has been pressed (Yes in S306), the process proceeds to S307; otherwise, it proceeds to S305.

[0048] In S307, CPU201 performs the shooting process. Specifically, with a single press of the shutter button by the user (S306), CPU201 changes the shooting conditions and takes multiple images in succession.

[0049] Shooting conditions may include, for example, the camera's angle of view, shutter speed, f-number, focal length, ISO sensitivity, exposure, flash, and zoom magnification. However, other conditions are also acceptable as long as they affect the analysis results of the captured image.

[0050] In other words, imaging conditions are conditions related to at least one of the following: angle of view, shutter speed, F-number, focal length, ISO sensitivity, exposure, flash, and zoom magnification.

[0051] In other words, this step is an example of a process in which an imaging operation is received, a first image is captured in response to the received imaging operation, and then the imaging conditions are changed to capture a second image.

[0052] In this embodiment, the system suppresses the user pressing the shutter button (S306) until it is determined that the location identification code is included in the live view. However, the system is not limited to this, and it is also possible to allow the user to press the shutter button (S306), but suppress shooting until it is determined that the location identification code is included in the live view.

[0053] In other words, this step is an example of a process in which a live view is displayed for capturing a second image, a second image is captured in response to an accepted capture operation, and the acceptance of capture operations or the capture of the second image is suppressed until it is determined that the location identification code contained in the first image is included in the live view.

[0054] In other words, this step is an example of a process in which the system acquires a first image, displays a live view for capturing a second image, captures the second image in response to an accepted capture operation, and then suppresses the acceptance of capture operations or suppresses the capture of the second image until it is determined that the location identification code contained in the first image is included in the live view.

[0055] After processing in S307, CPU201 may select an arbitrary image from the multiple images captured and display that selected image to the user. At this time, CPU201 can accept a choice from the user whether to execute processing from S308 onwards using that image, or to recapture the image.

[0056] This allows the user to review the captured image and retake it if there is blur or light reflection, enabling the processing from S308 onwards to be performed using the image that is clearest from the user's perspective, which is expected to improve the accuracy of the identification code acquisition process (S309).

[0057] After that, CPU201 retrieves the captured image.

[0058] In this embodiment, the CPU 201 acquires images captured in the processing steps S303 to S307, but the CPU 201 may also acquire images stored in the client terminal 101 or the file server 102.

[0059] In other words, this step is an example of the process for acquiring the first image.

[0060] In the loop processing starting from S308, steps S308 to S310 are repeated for each of the multiple images captured in S307. CPU201 terminates the loop processing when it has executed steps S308 to S310 for all captured images.

[0061] In S309, CPU201 analyzes the identification code in the image captured in S307 and obtains the coordinate information of the identification code within the image.

[0062] Specifically, CPU 201 recognizes location codes and identification codes in the image in the same manner as it recognizes location codes in S304. Next, CPU 201 performs analysis on the recognized location codes and identification codes to obtain their coordinate information. CPU 201 stores the analyzed coordinate information of location codes and identification codes in RAM 203 of client terminal 101 or file server 102.

[0063] In other words, this step is an example of a process for analyzing the identification code contained in the first image.

[0064] In S310, CPU201 converts the coordinate information of the identification code acquired in S308 within the captured image into x-axis and y-axis values ​​from the position identification code to the identification code.

[0065] Specifically, the CPU 201 uses the coordinate information of a given location identification code as the origin (x=0, y=0), corrects the captured image using the size and tilt of the location identification code (for example, by scaling the image so that the size of the location identification code within the captured image is the actual size, and by rotating the location identification code so that its orientation is correct in all directions), converts the coordinate information of a given identification code within the image into the actual distance in the x-axis direction and the actual distance in the y-axis direction from that location identification code, and stores it in the RAM 203 of the client terminal 101 or file server 102.

[0066] As described above, in the processing from S308 to S310, the CPU 201 obtains the string indicated by the identification code, the coordinate information of the identification code within the captured image, and the actual distance of the identification code from the location identification code for each of the multiple images, and stores it in the RAM 203 of the client terminal 101 or file server 102.

[0067] In S311, based on the distance from the location identification code stored in S310 to the identification code, the analysis results are aggregated by using the results obtained from multiple images that were successfully analyzed for the identification code and the coordinate information of the identification code acquired in S309.

[0068] Furthermore, by treating identification codes with the same distance from the location identification code to the identification code as the same identification code, it is possible to prevent duplicate processing of the same identification code, thereby reducing the processing time from S312 onwards.

[0069] Details of this process will be described later using Figure 9.

[0070] As described above, in steps S307 to S311, multiple images are taken by changing the shooting conditions, and the analysis results obtained from these multiple images are aggregated into a single result.

[0071] By capturing and analyzing multiple images in this way and aggregating the results, even if there is a problem analyzing the identification code or location code in one of the images, it may still be possible to analyze it correctly in other images, thus leading to improved analysis accuracy.

[0072] Furthermore, by changing the shooting conditions, it is possible to capture images with different brightness and focus levels, thereby reducing the possibility of errors in the analysis of identification codes and location codes caused by light source reflections or focus errors.

[0073] In other words, this step is an example of a process that analyzes the identification codes contained in the first image and the identification codes contained in the second image, and then aggregates the analysis results.

[0074] In S312, the CPU 201 compares the inventory information stored in the RAM 203 of the client terminal 101 or the file server 102 with the results of the analysis of the identification code that was aggregated in S311.

[0075] Specifically, CPU201 retrieves information regarding the expiration date of an item corresponding to an identification code from the inventory information. This expiration date information is based on at least one of the following: year, month, day, or time.

[0076] Furthermore, the system compares information regarding the expiration date of the acquired item with the current year, month, day, or time to determine whether the item has expired (i.e., whether it is an expired item).

[0077] CPU201 displays the results of its determination in S313 and beyond.

[0078] In this embodiment, the information regarding the expiration date is stored in a database, but the source of this information is not limited to this. For example, it may be obtained from information about the year, month, day, or time displayed near a file, storage medium, or identification code.

[0079] In other words, the expiration date for an item is either an expiration date based on year, month, day, or time information stored in a file, database, or recording medium associated with the identification code corresponding to the item, or an expiration date based on year, month, day, or time information displayed near the identification code corresponding to the item.

[0080] In this embodiment, the term "expiration date" for an item refers to the expiration date, but it is not limited to this. For example, it could refer to the shipping date, delivery date, delivery date, quality retention date, sales date, retention date, expiration date, best-before date, or use-by date of an item, or any other expiration date related to an item, such as the date based on the idea that the manufacturer, seller, and consumer each equally divide the period from the date of manufacture to the best-before date (the so-called one-third rule).

[0081] This allows users to set deadlines that are tailored to the characteristics of the items being inventoryed.

[0082] Note that S312 determines whether an item is past its expiration date, that is, whether it is an expired item or not. However, any determination related to expiration dates is not limited to determining whether an item is expired or not. For example, it could also determine whether the expiration date has been reached, or whether there are XX days or XX hours remaining until the expiration date (where XX is a number of days or hours that the user can set).

[0083] In other words, a deadline relating to goods is a deadline based on at least one of the following: the shipping deadline, the date of dispatch, the delivery deadline, the quality retention period, the sales deadline, the retention period, the use-by date, the best-before date, or the expiration date.

[0084] This allows CPU201 to perform determinations regarding expiration dates based on the characteristics of the items being inventoryed.

[0085] In other words, this step is an example of a process for determining the expiration date of an item corresponding to an analyzed identification code.

[0086] In S313, CPU201 displays the image captured in S303 to S307 and the result of processing the image in S308 to S312 (hereinafter referred to as the judgment result). CPU201 displays the judgment result corresponding to the identification code determined in S312, superimposed on the position of the identification code in the image captured in S307.

[0087] In other words, this step is an example of a process that controls the display of the determination result at a predetermined location on the identification code within the image.

[0088] In this way, the analysis results of the identification codes aggregated in S311 are compared with inventory information, and the result of determining whether the item has expired (is an expired item) is superimposed on the image and displayed.

[0089] In other words, this step is an example of a process that analyzes the identification codes contained in the first image and the identification codes contained in the second image, aggregates the analysis results, and controls the display to superimpose them on the first image and / or the second image.

[0090] If, as a result of the S312 check, an item is determined to be past its expiration date, it will be clearly marked as an item to be returned, distinguishable from other items.

[0091] Details of the screen displaying the judgment results will be described later using Figure 7.

[0092] In S314, CPU201 determines which button was pressed by the user.

[0093] If the "Additional Shooting" button is pressed, the process transitions to S303; if the "Individual Shooting" button is pressed, the process transitions to S305. The processing for additional shooting and individual shooting will be described later using Figure 8.

[0094] When the "Inventory Complete" button is pressed, the CPU 201 generates inventory result data, return list data, and a log of the judgment result based on the judgment result, and stores them in the RAM 203 of the client terminal 101 or file server 102. Then, the process transitions to S315.

[0095] In S315, CPU201 displays the inventory results confirmation screen and the return list confirmation screen. CPU201 displays information on the terminal based on the inventory results data for the inventory results confirmation screen and based on the return list data for the return list confirmation screen.

[0096] In other words, this step is an example of a process that controls the display of items that should be returned in an identifiable manner based on the determination result.

[0097] Details of these confirmation screens will be described later using Figure 10.

[0098] In S316, CPU201 outputs result files containing the inventory results data and the return list data, that is, the data displayed on the inventory results confirmation screen and the return list confirmation screen in S315.

[0099] Details about the screen for outputting the result file will be described later using Figure 11.

[0100] In this embodiment, the result file output consists of inventory result data and return list data. However, in addition to these, other data related to inventory or returns may be output, such as data with the judgment result superimposed on an image, or log data from the analysis.

[0101] In other words, this step is an example of a process that outputs information about items that have passed or have been determined to have reached a predetermined deadline.

[0102] In S317, CPU201 receives instructions from the user to photograph the items to be returned. The user uses the return list result file output in S316 as a reference to remove the items to be returned from the shelf, and CPU201 photographs the removed items.

[0103] First, when the CPU 201 receives a press of the shutter button by the user, it takes a picture of the returned item 1101 and the returned item identification code 1102. Next, the CPU 201 obtains the identification code from the captured image. The captured image is stored in the client terminal 101 or the file server 102.

[0104] In S318, CPU201 determines whether all items to be returned have been photographed without fail, that is, whether the images contain all the identification codes corresponding to the items to be returned. This process is to allow CPU201 to confirm that all items to be returned have been removed from the shelves. If they have been photographed (Yes in S318), the process in this flowchart ends; if they have not been photographed (No in S318), the process proceeds to S317. CPU201 compares the identification codes corresponding to the items to be returned contained in the images photographed in S307 with the identification codes corresponding to the items contained in the images photographed in S317, determines whether all items to be returned are included in the images photographed in S317, and displays the determination result to the user. When making the determination, all images photographed in S317 are referred to. If all items to be returned are included in the images without fail, CPU201 displays to the user that the removal of all items to be returned has been completed and exits the return item photography screen in Figure 11. If not all items to be returned are included in the image, i.e., if there are omissions, CPU201 displays a message to the user indicating that the removal of the items to be returned is incomplete, and proceeds to S317. In other words, the process is repeated until all items to be returned by the user are removed from the shelf and photographed, and the identification codes corresponding to all items to be returned are obtained.

[0105] As described above, by continuing to photograph the items to be returned until the photography is complete, users can reduce the likelihood of forgetting to remove items from the shelf. In other words, this step is an example of a process that acquires a second image, analyzes the identification codes contained in the second image, determines whether all items to be returned are included in the second image based on the analyzed identification codes, and displays the determination result. In this embodiment, the determination is made as to whether all identification codes corresponding to the items to be returned are included in the image, but the content determined in S318 is not limited to this.

[0106] In addition to the above determination, CPU201 may also determine whether the image contains any items other than those to be returned (items not to be returned). CPU201 displays the result of this determination to the user. If the image does not contain any items not to be returned, CPU201 displays to the user that the removal of items to be returned was performed correctly without any errors in the items removed. If the image contains items not to be returned, CPU201 displays to the user that there was an error in the removal of items, as items that should not have been removed were removed. This reduces the likelihood of users making mistakes when picking items from shelves.

[0107] In other words, this step is an example of a process that involves acquiring a second image, analyzing the identification code contained in the second image, determining whether items that should not be returned are included in the second image based on the analyzed identification code, and displaying the determination result.

[0108] The method for determining whether the image contains all the identification codes corresponding to the items to be returned is not limited to this method; for example, the following method may also be used. CPU201 may take a picture of the shelf after removing the items to be returned and then determine whether the items to be returned are included in the captured image. First, CPU201 removes all the items to be returned by the user and then takes a picture of the shelf in which those items were stored.

[0109] Next, CPU201 determines whether the captured image contains the identification code corresponding to the items to be returned. If it is determined that the code is not included, CPU201 displays a message indicating that all items to be returned have been successfully removed from the shelf. If it is determined that the code is included, CPU201 displays a message to the user indicating that some items to be returned have been missed from that shelf. In other words, this step is an example of a process that involves acquiring a third image, analyzing the identification code contained in the third image, determining whether the item to be returned is not included in the third image based on the analyzed identification code, and displaying the determination result.

[0110] Alternatively, the method may involve determining whether all items to be returned have been removed by comparing an image of the shelf before the items to be returned were removed with an image of the shelf after the items to be returned were removed. First, all identification codes corresponding to items that are included in the image of the shelf before the items to be returned were removed but are not included in the image of the shelf after the items to be returned were obtained. These obtained identification codes are the identification codes that represent the difference between the two images.

[0111] One possible method for CPU201 is to determine whether the difference in identification codes includes the identification code corresponding to the item to be returned. If it does not, CPU201 displays that all items to be returned have been successfully removed from the shelf. If it does include the identification code, CPU201 indicates that some items to be returned were missed from the shelf and displays that there were missing items from that shelf.

[0112] The two methods described above can reduce the occurrence of items being left behind when removed from shelves.

[0113] In other words, this step involves acquiring a third image, analyzing the identification code contained in the difference between the first and third images, determining whether the item to be returned was removed between the time the first image was captured and the third image was captured based on the analyzed identification code, and displaying the determination result.

[0114] In addition to determining whether all items to be returned have been removed by comparing an image of the shelf before the items to be returned were removed with an image of the shelf after the items to be returned were removed, it is also conceivable that CPU201 would also determine whether the identification codes in the difference include identification codes corresponding to items that are not to be returned. If they are not included, CPU201 will display a message indicating that all items to be returned have been successfully removed from the shelf without any omissions. If they are included, CPU201 will display a message to the user indicating that the removal of items to be returned was performed correctly without any errors in the items removed. If they are included, CPU201 will display a message indicating that there was an error in the user's item removal process, as items that should not have been removed were removed. This reduces the likelihood of users making mistakes when picking items from shelves.

[0115] In other words, this step involves acquiring a third image, analyzing the identification code contained in the difference between the first and third images, determining, based on the analyzed identification code contained in the difference, whether any items that should not be returned were removed between the acquisition of the first image and the acquisition of the third image, and displaying the determination result.

[0116] The above is an explanation of Figure 3.

[0117] Figure 4 shows an example of a data table for inventory information.

[0118] The data table in Figure 4 is a data table stored in advance to keep track of inventory before the inventory count is conducted, and consists of item number 401, item name 402, expiration date 403, and quantity 404.

[0119] In this embodiment, the data is stored as a data table as shown in Figures 4 and 12, but it is not limited to this. The data may be stored in any format that can store inventory information, such as Excel files or CSV files.

[0120] In this embodiment, the items subject to inventory are ink cartridges used in image forming devices such as multifunction printers. However, the system is not limited to these; it can be used for any item that can be stored in a storage location such as a shelf and for which an identification code corresponding to each item can be assigned, such as food or pharmaceuticals.

[0121] Part number 401 indicates a part number used to identify an item. In this embodiment, the part number is a combination of numbers and letters. The letter at the end represents the color of the ink cartridge; for example, in the top row, "0001C", "C" indicates a cyan ink cartridge.

[0122] Product name 402 indicates the name of the item. In this embodiment, the alphabet at the end of product number 401 that represents the color of the ink cartridge corresponds to product name 402. For example, the "C" in "0001C" and "Printer A Cyan" both indicate that the ink cartridge is cyan.

[0123] Expiry date 403 indicates the expiration date for items subject to inventory.

[0124] In inventory data tables, it is desirable that the type of item can be uniquely identified. For example, in Figure 4, the type of item can be uniquely identified by a composite key combining part number 401 and expiration date 403. Therefore, even if the part number 401 is the same, items with different expiration dates 403 will be identified as different types of items.

[0125] Quantity 404 indicates the quantity of goods. The quantity listed here represents the total number of items stored in all shelves subject to inventory.

[0126] The above is an explanation of Figure 4.

[0127] Figure 5 shows an example of a screen where a user selects a shelf.

[0128] This screen is displayed when the CPU 201 accepts the user's selection of the shelves to be inventoryed (S302), and consists of a shelf selection section 501 and a shooting start button 502.

[0129] The shelf selection unit 501 is a button that allows the user to select the shelf containing the items to be inventoryed. When the user presses the button, the available shelves are displayed, and the system accepts the selection of the shelf to be inventoryed from among the displayed shelves. In Figure 5, the user has selected the shelf named "Shelf 2".

[0130] The "Start Shooting" button is a button that can be pressed by the user when the shelf selection unit 501 has accepted the selection of a shelf. When the button is pressed, the system transitions to the screen shown in Figure 6 in order to start inventory taking of the items stored on the shelf selected by the shelf selection unit 501.

[0131] In this embodiment, the location where the items are placed is a shelf, but it may be any other location, such as a drawer, rack, or cardboard box, as long as it is a place where items can be arranged and stored, and where the arranged items and the shelf on which the items are stored can be photographed.

[0132] As described above, inventory can be conducted shelf by shelf, so even if there are multiple shelves containing the items to be inventoryed, inventory can be conducted shelf by shelf and the results can be checked.

[0133] The above is an explanation of Figure 5.

[0134] Figure 6 shows an example of a user-operated screen for taking photos.

[0135] Figure 6 shows the operation screen for when the user takes the first photograph of a shelf and the items inside the shelf (S303), and consists of an identification code 601, an item 602, a location identification code 603, and a shutter button 604.

[0136] Identification code 601 is the identification code corresponding to item 602. Figure 6 shows that a total of 25 identification codes 601, each corresponding to an item, are affixed to the packaging of each item and displayed on the screen.

[0137] In this embodiment, the identification code 601 is affixed to the package of article 602, but it is not limited to this. The location of the identification code 601 can be any location as long as it is attached near article 602 and it is visible that the identification code 601 and article 602 are associated.

[0138] Item 602 is an item subject to inventory. Figure 6 shows a total of 25 items of item 602 arranged in a 5x5 grid.

[0139] Location identification code 603 is a code used to identify the location of identification code 601. Multiple location identification codes are pre-placed on the shelf. It is mainly used in a process (S311, Figure 9) to prevent duplicate matching of the same identification code. Details of this process will be described later with reference to Figure 9.

[0140] The shutter button 604 is the shutter button used to take a picture. When the CPU 201 takes the first picture, it recognizes all of the location identification codes 603. When taking additional pictures from the second time onward, the shutter button 604 is activated when it recognizes a predetermined number (for example, two) or more of the location identification codes (S305). When taking additional pictures from the second time onward, it is sufficient to recognize at least one location identification code, but the more codes that are recognized, the more accurate the identification of the location of the identification code becomes.

[0141] When the shutter button is activated, the user can press the shutter button 604 (S306). The user then performs operations such as changing the zoom magnification or changing the shooting direction to enable recognition of the location identification code.

[0142] As described above, by preventing the shutter button from being pressed unless the location identification code is recognized, it is possible to prevent forgetting to recognize the location identification code and to accurately execute the process described later to prevent duplicate matching of the same identification code.

[0143] The above is an explanation of Figure 6.

[0144] Figure 7 shows an example of a screen displaying the judgment results.

[0145] Figure 7 consists of a figure acquisition count display unit 701, a shelf display unit 702, a mask for items within their expiration date 703, a mask for items that have expired 704, areas without masks 705a and 705b, an image display unit 706, an expired information display unit 707, an additional shooting button 708, an individual shooting button 709, an inventory completion button 710, and a shelf switching button 711.

[0146] Figure 7 shows an example screen displaying the judgment result of the first image taken, but the screen layout is the same for displaying the judgment results of images taken from the second time onward.

[0147] The acquisition count display unit 701 displays the number of identification codes obtained from the images captured in S307. In Figure 7, 21 out of 25 captured identification codes were obtained. The number of identification codes obtained is displayed for each image, i.e., for each shelf.

[0148] In other words, the system is controlled to display the number of identification codes that were successfully analyzed.

[0149] The shelf display unit 702 displays the name of the shelf corresponding to the judgment result displayed on the screen for the shelf selected by the user in S302 by the CPU 201. In Figure 7, the judgment result for the shelf named "Shelf 2" is displayed.

[0150] In other words, it accepts a selection regarding the location of an item and controls the system to display the determination result regarding the item associated with that location.

[0151] The mask 703 for items within their expiration date is a masking process that superimposes the identification code 602, which corresponds to items that were determined in S312 to not be past their expiration date and therefore do not need to be returned (items within their expiration date). In Figure 7, this is the area masked in black.

[0152] The expired item mask 704 is a masking process that superimposes the identification code 602, which corresponds to the item that was determined to be expired in S312, i.e., the item to be returned (expired item). In Figure 7, this is the part masked in gray.

[0153] As described above, the masks for items within their expiration date (703) and items that have expired (704) are identified and displayed, and these are superimposed on the identification codes in the image captured in S307. By superimposing the judgment results onto the image in this way, users can easily check which items stored in which locations have expired.

[0154] In this embodiment, the judgment result is displayed superimposed on the identification code. However, the system is not limited to this method. Any method of display is acceptable as long as it is clear that the identification code and the judgment result are associated, such as displaying the judgment result near the identification code.

[0155] In other words, based on the determination result by the determination means, the system controls the display to distinguish between items that have been determined to have expired or reached their expiration date and items that have been determined to have not expired or not reached their expiration date.

[0156] Sections 705a and 705b, which lack masking, represent identification codes that could not be analyzed in S309 for some reason (e.g., the effect of light source reflection). The analysis in these sections could not be performed because CPU 201 was unable to detect the presence of the identification codes.

[0157] In this way, by displaying in a user-friendly manner which parts of the image have been analyzed and which have not, users can easily see which parts of the image have been analyzed and which parts have not. Based on this, users can select a button to press on S314 (additional shooting, individual shooting, or inventory completion).

[0158] In other words, the system controls the display to show, in a way that distinguishes between identification codes that could be analyzed and those that could not.

[0159] The image display unit 706 overlays the image captured in S307 with the judgment result. The judgment result includes the overlaid display of the limited-time item mask 703, the expired item mask 704, and the areas without masks 705a and 705b, as shown above.

[0160] In this way, the judgment results are classified into three categories: identification codes for which identification codes could not be obtained from the image, identification codes for which expired items were obtained, and identification codes for which items within their expiration date were obtained. These results are then superimposed on the image display unit 706. This allows the user to easily see the overall picture of the judgment results at a glance. Furthermore, it is possible to see which item in the image corresponds to which result, making it easy to predict which areas should be magnified when taking additional or individual photos from S314 onwards.

[0161] In this embodiment, the expiration date is indicated by the color of the mask, but this is not the only method. Any method that allows the user to confirm whether an identification code corresponds to an expired item is acceptable, such as placing different marks near the identification code for expired items and for items within the expiration date, or making only the identification code for expired items flash.

[0162] The expired item information display unit 707 displays information corresponding to the items masked by the expired item mask 704. The information displayed on the expired item information display unit can be switched using a scroll bar or the like. In Figure 7, the item number, item name, expiration date, and quantity of the expired item are displayed. The quantity displayed on the expired item information display unit 707 is the information stored in the inventory information data table, that is, the total of all items stored on all shelves subject to inventory.

[0163] When the expired information display unit 707 is pressed, the corresponding expired item mask 704 on the image display unit 706 is displayed in a way that allows the user to identify it, such as by surrounding it with a frame or changing the color of the mask.

[0164] In other words, based on the determination result, information about items that have passed or reached their predetermined deadline is displayed, and if an instruction is received regarding the displayed information about items that have passed or reached their predetermined deadline, the system is controlled to identify and display the items that have passed or reached their predetermined deadline.

[0165] The expiration information display unit allows the user to check details about the item that is displayed as expired on the image display unit 706.

[0166] In this embodiment, the information displayed on the expiration date information display unit 707 is the information in the inventory information data table, but it is not limited to this. Other information that allows the user to understand which items have expired, such as the manufacturing date or the price of the item, may also be displayed.

[0167] In other words, the judgment result and information regarding items whose prescribed expiration date has passed are displayed in a list.

[0168] The additional shooting button 708 is used to take another picture of the shelf displayed on the shelf display unit 702 (S316).

[0169] If there are areas 705a and 705b without a mask in the image display unit 706, meaning there are identification codes that could not be obtained in the first shot, the user performs a second or subsequent shot (additional shot or individual shot).

[0170] The individual shooting button 709 is a button used to take another photograph of the shelf displayed on the shelf display unit 702 (S316).

[0171] Similar to the additional shooting function, this button is used to take a second or subsequent photo. The difference is that additional shooting uses a location identification code, while individual shooting does not use a location identification code.

[0172] The specific processing of additional and individual imaging will be described later using Figure 8.

[0173] The inventory completion button 710 is a button that the user presses when they have finished confirming the judgment results. When the CPU 201 receives confirmation that the user has pressed the inventory completion button 710, it transitions to the inventory result confirmation screen (Figure 10) (S316).

[0174] The shelf switching button 711 is a button that allows the user to switch which shelf's judgment results are displayed. When pressed by the user, it transitions to a screen displaying the judgment results for a different shelf.

[0175] Furthermore, if the judgment result for another shelf displayed by pressing the shelf switching button 711 is not displayed (i.e., the shooting in S303 to S307 and the processing in S308 to S312 have not been performed), a "New Shoot Button" may be displayed near the additional shooting button 708 or elsewhere. When the user presses the New Shoot Button, the CPU 201 transitions to the shooting screen in S303. This allows the user to take a new photo if they recognize a shelf that has not yet been photographed.

[0176] As shown in Figure 5, inventory can be taken for each shelf. The shelf switching button 711 and shelf display unit 702 allow the user to check the judgment results for each shelf. This makes it easy for the user to check which shelf's judgment results are being displayed, even if there are multiple shelves containing the items to be inventoryed.

[0177] As described above, the screen in Figure 7 allows the user to see the judgment results at a glance, check which identification codes for which items could not be obtained, and take another photo of the shelf if necessary. In addition, the user can check which items have expired from the obtained identification codes and, if necessary, remove or return the expired items.

[0178] The above is an explanation of Figure 7.

[0179] Figure 8 shows an example of the operation screen used by the user for the second and subsequent shots.

[0180] Figure 8(a) is a diagram showing the shooting range.

[0181] The shooting range 801 indicates the area captured in the first shooting. As shown in Figure 8, the shooting includes all items and their location identification codes within the shelf.

[0182] However, when photographing the entire shelf in this way, some identification codes may not be recognized properly due to light reflection or other factors.

[0183] The shooting range 802 shows the area captured in the second shot, and the shooting range 803 shows the area captured in the third shot. Subsequent shots are started when the additional shooting button 708 in Figure 7 is pressed.

[0184] Additional shooting is a method of taking close-up shots of identification codes that could not be acquired in the first shot. By doing this, instead of shooting the entire shelf from a distance, a smaller number of identification codes are focused on, allowing for adjustments to the angle and exposure to make it easier to acquire those identification codes, thus reducing the possibility of recognition failure. This increases the likelihood of acquiring identification codes that could not be acquired in the first shot.

[0185] In additional imaging, ensure that at least one location identification code is included in the frame. This allows the function to prevent duplication of identification code matching, as explained in Figure 9, to be utilized even in additional imaging.

[0186] Figure 8(b) shows the results of the image analysis performed on the second and subsequent images.

[0187] Figure 8(b) is an image cropped from a portion of Figure 7 for illustrative purposes; in reality, other data such as the expiration information display unit 707 and the inventory completion button 710 are present.

[0188] Acquired location 804 encloses part of the identification code from the second photo shoot and all of the identification code from the third photo shoot. Since the enclosed identification code is displayed with a mask overlay, it is a location that was successfully acquired by CPU 201. For example, in the image display unit 706 of Figure 7, the location was displayed as unmasked location 705a, but as a result of the second photo shoot, Figure 8(b) shows that it has been masked by the expired item mask 703.

[0189] In this way, identification codes that could not be obtained during the first scan may become obtainable during subsequent scans. Magnifying the image of the object makes it easier to obtain the identification code, reducing the likelihood of recognition errors.

[0190] The display position for the judgment results determined during the second and subsequent shots is determined based on the location identification code. Specifically, the distance from the location identification code to the identification code contained in the image taken from the second and subsequent shots is used to determine which part of the entire first image the image was magnified from. Based on the identified location, the judgment results determined from the first image and the judgment results determined from the second and subsequent images are aggregated. At this time, identification codes that have been analyzed in the past are not included in the aggregation due to a function that eliminates duplicate identification codes (described later in the explanation of Figure 9). The aggregated judgment results are displayed superimposed on the identification code corresponding to the judgment result.

[0191] In this embodiment, the judgment result is displayed superimposed on the identification code. However, the system is not limited to this method. Any method of display is acceptable as long as it is clear that the identification code and the judgment result are associated, such as displaying the judgment result near the identification code.

[0192] In other words, the system controls the determination result to be displayed in an identifiable manner at a predetermined location of the identification code within the first image, which is determined based on the location of the position identification code contained in the first image. In other words, the system controls the display of the determination result superimposed on a predetermined position of the identification code within the first image, which is determined based on the position of the location identification code contained in the first image. In other words, based on the location of the identification code identified based on the respective locations of the location identification codes contained in the first and second images, the analysis results of the identification code are aggregated and controlled to be superimposed on the first image and / or the second image. In other words, the system controls the display of an identifiable identification code in the first image at a predetermined location, determined based on the distance from the location of the localization code contained in the first image. In other words, the system controls the display to superimpose an identification code on a predetermined location within the first image, which is determined based on the distance from the location of the position identification code contained in the first image.

[0193] In other words, the analysis results of identification codes determined to be the same based on the distance from the respective locations of the identical location identification codes contained in the first image and the second image are aggregated and controlled to be superimposed on the first image and / or the second image. On the other hand, location 805, where acquisition failed, corresponds to the identification code for which acquisition failed in the second and third imaging attempts. For example, location 705a, which is not masked, is a location where masking was not performed, and even if additional imaging was performed, the identification code could not be acquired.

[0194] In this way, identification codes that cannot be obtained even after multiple attempts at photography can be acquired by taking individual photos.

[0195] Individual shooting is a method of capturing identification codes that you want to acquire in more detail than with additional shooting. Because individual shooting focuses on the identification code more than additional shooting, it is possible to adjust the angle and exposure to make the identification code easier to read, further reducing the possibility of recognition failure. As a result, identification codes that could not be acquired during the first shooting or additional shooting may become obtainable through individual shooting.

[0196] When individual images are taken, the image is taken with the identification code to be acquired enlarged, without including the location identification code. Therefore, it is difficult to determine the location of the individually acquired identification code on the entire shelf from the individually captured image alone. For this reason, the judgment result is not superimposed on the image display unit 706, and the judgment result is only displayed on the acquired count display unit 701 and the expiration information display unit 707. In addition, the processing from S308 to S310 is not executed, and S311 is simply a process of aggregating the analysis results.

[0197] Individual shooting begins when the user presses the desired identification code on the screen display unit 706, and the individual shooting of that identification code is initiated.

[0198] As described above, taking additional or individual photos increases the likelihood of obtaining identification codes that could not be acquired when photographing the entire shelf. This reduces the possibility of missing identification codes in the system, allowing users to perform inventory tasks accurately.

[0199] The above is an explanation of Figure 8.

[0200] Figure 9 shows an example of a function that prevents duplicate identification code matching.

[0201] Figure 9 is an image cropped from a portion of Figure 7 for illustrative purposes; in reality, other data exists, such as the expiration information display unit 707 and the inventory completion button 710.

[0202] In Figure 9, the duplicate location 901 is where the identification code obtained during the first scan and the identification code obtained during the third scan overlap. In such cases, to prevent the CPU 201 from matching the identification code twice in S312, processing using a location identification code (S311) is performed.

[0203] As a prerequisite for the process, it is assumed that the location identification codes are placed on the frame of the outer shelf at regular intervals. Furthermore, it is assumed that the size of the location identification codes is greater than or equal to the size that the client terminal 101 can acquire through the inventory support system.

[0204] In this embodiment, the location identification code is placed on the outer frame of the shelf, but it may also be placed inside the shelf in addition to the outer frame, as long as it is in the vicinity of the item being inventoryed and does not overlap with the identification code, other location identification codes, or the item being inventoryed. Increasing the number of locations where the location identification code can be placed in this way may improve the accuracy of identifying the location of the identification code.

[0205] This section explains the processing using location identification codes. First, CPU201 compares the distance of the acquired identification code from the location identification code converted during the first capture with the distance of the acquired identification code from the location identification code converted during the second and subsequent captures. If the comparison shows that the distance of the identification code from the location identification code converted during the second and subsequent captures is the same as the distance of the identification code from the location identification code converted during the first capture, that is, if the same identification code was acquired during the first and second captures, that identification code is excluded from aggregation as an already acquired identification code. This process is performed for all acquired identification codes.

[0206] For example, in Figure 9, a comparison of the distance from the location identification code to the identification code obtained during the first shooting and the distance from the location identification code to the identification code obtained during the third shooting revealed that the distance values ​​from the location identification code to the identification code for the four identification codes within the overlapping area 901 enclosed by the dotted line were the same. Since the acquisition of these four identification codes was duplicated between the first and third shooting, they were excluded from aggregation as already acquired identification codes.

[0207] In this way, when aggregating the analysis results (S311), the already acquired identification codes are identified based on the distance of the identification codes from the location identification code, and the analysis results related to identification codes other than those already acquired are included in the aggregation. This prevents the CPU201 from matching identification codes twice, thus reducing the number of identification codes that need to be matched in S312, and leading to an improvement in processing speed.

[0208] The above is an explanation of Figure 9.

[0209] Figure 10 shows an example of a screen for checking and outputting inventory results and a list of returns.

[0210] The inventory result confirmation screen (a) is an example of a screen that displays information about items corresponding to identification codes for items within their expiration date, that is, identification codes that have been masked by the item mask 703 in Figure 7. In other words, the inventory result confirmation screen (a) displays information about items that will not be returned.

[0211] The return list confirmation screen (b) is an example of a screen that displays information about the items to be returned. In other words, the return list confirmation screen (b) displays information about the items to be returned.

[0212] In this embodiment, items determined to be expired in S312 are included in the items to be returned. However, it is also possible to add any items that the user returns, not limited to expired items, to the return list.

[0213] In other words, the items that should be returned based on the judgment result include items whose deadline has passed or which have been determined to have reached the deadline.

[0214] The inventory result confirmation screen (a) consists of the inventory result tab 1001, the returns list tab 1002, and the inventory result information display sections 1003a and 1003b.

[0215] The return list confirmation screen (b) consists of the inventory results tab 1001, the return list tab 1002, and the return list information display section 1004.

[0216] When the user clicks on the inventory results tab 1001, it displays the inventory results confirmation screen (a). When the user clicks on the returns list tab 1002, it displays the returns list confirmation screen (b). Thus, the inventory results confirmation screen (a) can be switched between displaying the returns list confirmation screen (b).

[0217] The inventory result information display units 1003a and 1003b display detailed information about items that have been determined to be within their expiration date. For example, in Figure 10, the product number, product name, expiration date, and quantity acquired are displayed.

[0218] In this embodiment, the information displayed in the inventory result information display units 1003a and 1003b is the information in the inventory information data table. However, it is not limited to this, and other information such as the manufacturing date and item price may also be displayed, as long as it is information that allows the user to confirm what kind of inventory was conducted.

[0219] In the inventory result information display units 1003a and 1003b, detailed information about the items can be switched using methods such as scroll bars.

[0220] If there is a difference between the total quantity of items retrieved by CPU201 from all shelves subject to inventory and the quantity stored in the inventory information data table (the total quantity of items stored in all shelves subject to inventory), an alert like that shown in the inventory result information display unit 1003b will be displayed.

[0221] For example, in Figure 10, for an item with "Product Number: 0001B, Product Name: Printer B Black, Expiration Date: 2025 / 11 / 30", the quantity stored in the inventory information data table (Figure 4) is "13", while the quantity obtained from all shelves subject to inventory is "12". In this case, the inventory result information display unit 1003b displays "12", and highlights it by shading to indicate that one identification code corresponding to that item was not obtained.

[0222] In this way, by displaying an alert to the user when there is a discrepancy between the total quantity of items taken from all shelves subject to inventory and the total quantity of items stored in all shelves subject to inventory, the process of the user confirming the judgment results can be made more efficient. Specifically, the user can use the alerted area as a reference to check the actual items to verify the quantity, and to check the cause of the quantity discrepancy.

[0223] In this embodiment, the alert display method is highlighted by shading, but this is not the only method. Any method that allows the user to understand that there is a difference in quantity may be used, such as flashing the inventory result information display unit 1003 for the relevant item, or highlighting the color of the text written on the inventory result information display unit 1003 for the relevant item.

[0224] The return list information display unit 1004 displays detailed information about the items to be returned. In Figure 10, the product number, product name, expiration date, and quantity are displayed. This allows the user to confirm which items need to be returned.

[0225] In this embodiment, the information displayed on the return list information display unit 1004 is the information in the inventory information data table. However, it is not limited to this, and other information such as the manufacturing date and item price may also be displayed, as long as it is information that allows the user to confirm what items need to be returned.

[0226] The return list information display unit 1004 allows users to switch between detailed information about items using methods such as a scroll bar.

[0227] (c) The results output screen is an example of a screen that outputs the information displayed on the inventory results confirmation screen (a) and the returns list confirmation screen (b) as a file. In Figure 10, the output results display unit 1005 is superimposed on either the inventory results confirmation screen (a) or the returns list confirmation screen (b).

[0228] The output result display unit 1005 is a display unit that indicates that the output of the result files has been completed for both the inventory result confirmation screen (a) and the return list confirmation screen (b). Figure 10 shows that the result files "20250225_Inventory Results" and "20250225_Return List" have been output.

[0229] By outputting a results file in this way, users can check the output file and see which items have been inventoryed. Furthermore, the output results file can also be used to identify which items should be returned, and the results file is useful in the return process explained in Figure 11.

[0230] The above is an explanation of Figure 10.

[0231] Figure 11 shows an example of a screen used to photograph items to be returned.

[0232] Figure (a) shows the screen for taking photos of returned items. This screen is used by the user to take photos of items they have picked up from the shelf to return. When taking photos, the screen should include the items to be returned but exclude items that are not to be returned.

[0233] The returned item 1101 shown in Figure (a) is the item to be returned, and the returned item identification code 1102 is the identification code corresponding to that item.

[0234] Before taking a photograph of the items to be returned, the user removes the items to be returned from the shelf, referring to the outputted return list result file (S316).

[0235] Next, the user takes a photograph of the item to be returned. When taking the photograph, the user should ensure that the identification code of the item being returned is visible within the frame.

[0236] The above is an explanation of Figure 11.

[0237] Figure 12 shows an example of a data table for inventory results and a list of returns.

[0238] The data table in Figure 12, Inventory Results (a), is a data table that stores the information to be displayed on the Inventory Results Confirmation Screen (a) in Figure 10, and consists of part number 1201a, part name 1202a, expiration date 1203a, quantity 1204a, acquired quantity 1205a, and difference quantity 1206a.

[0239] The data table in Figure 12, the return list (b), is a data table that stores the information to be displayed on the return list confirmation screen (b) in Figure 10, and consists of part number 1201b, part name 1202b, expiration date 1203b, and quantity 1204b.

[0240] Part numbers 1201a and 1201b are part numbers used to identify items. Part number 1201a or 1201b corresponds to part number 401 in the inventory information data table in Figure 4.

[0241] Product names 1202a and 1202b indicate the names of the goods. There is a correspondence between product name 1202a or product name 1202b and product name 402 in the data table of inventory information in Figure 4.

[0242] Expiry dates 1203a and 1203b indicate the expiration dates of the items subject to inventory, in this embodiment, the ink cartridges. There is a correspondence between expiration date 1203a or 1203b and expiration date 403 in the inventory information data table in Figure 4.

[0243] Quantities 1204a and 1204b indicate the quantity of goods. The quantities listed here represent the total of all items stored in all shelves subject to inventory. Quantity 1204a or 1204b corresponds to Expiry Date 404 in the Inventory Information data table in Figure 4.

[0244] The acquisition count of 1205a represents the total quantity of items acquired by CPU201 from all shelves subject to inventory in the inventory result (a).

[0245] The difference number 1206a indicates the difference between the total quantity of items acquired by the CPU 201 from all shelves subject to inventory in the inventory result (a) and the quantity stored in the inventory information data table (the total quantity of items stored in all shelves subject to inventory). For example, for an item with "Part Number: 0001B, Item Name: Printer B Black, Expiration Date: 2025 / 11 / 30", the quantity 1204a is "13", while the acquired quantity 1205a is "12", resulting in a difference number 1206a of "1". If the difference number 1206a is a number other than 0, an alert is displayed on the inventory result information display unit 1003 in Figure 10, allowing the user to check which item has an anomaly in its quantity and reducing the burden of searching for the item with the problem.

[0246] The above is an explanation of Figure 12.

[0247] Thus, the present invention contributes to the efficient execution of inventory counting tasks. Specifically, by photographing items and their associated identification codes, and displaying the results of the analysis and determination of the identification codes, the burden on users to visually confirm the information necessary for inventory counting can be reduced. Furthermore, by outputting the determination results, it also contributes to the efficiency of return operations and other tasks performed after inventory counting.

[0248] The present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0249] Furthermore, the various controls described above, which are performed by CPU201, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0250] Furthermore, the program in this invention is a program that allows a computer to execute the processing method shown in the flowchart in Figure 3, and the storage medium of this invention stores a program that allows a computer to execute the processing method shown in Figure 3. Note that the program in this invention may also be a program for each processing method of each device shown in Figure 1.

[0251] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0252] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0253] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0254] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0255] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0256] Furthermore, the present invention may be applied to a system composed of multiple devices or to a device consisting of a single device. It goes without saying that the present invention can also be applied when the results are achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0257] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that configurations combining the above-described embodiments and their variations are all included in the present invention. [Explanation of Symbols]

[0258] 101 Client terminals 102 Servers

Claims

1. An analysis means for analyzing the identification code contained in the first image, A determination means for determining the expiration date of an item corresponding to an identification code analyzed by the aforementioned analysis means, A display control means that controls the display of the determination result by the determination means to a predetermined position of the identification code in the first image, An information processing system characterized by comprising the following features.

2. The display control means controls the display so that the identification codes that were analyzed by the analysis means and the identification codes that could not be analyzed are displayed in an identifiable manner. The information processing system according to claim 1, characterized by the following:

3. The display control means controls the display to distinguish between items that have been determined to have expired or reached their expiration date and items that have been determined to have not expired or not reached their expiration date, based on the determination result by the determination means. The information processing system according to claim 1, characterized by the following:

4. The system further includes a selection acceptance mechanism for accepting selections regarding the location of items, The display control means controls the display of the determination result for the item related to the location selected by the selection receiving means. The information processing system according to claim 1, characterized by the following:

5. The analysis means performs an analysis of the identification code contained in the second image, The determination results from the determination means are aggregated and controlled to be superimposed on the first image and / or the second image. The information processing system according to claim 1, characterized by the following:

6. The display control means is The determination result by the determination means is controlled to be superimposed and displayed at a predetermined position of the identification code in the first image, which is determined based on the position of the location identification code contained in the first image. The information processing system according to claim 1, characterized by the following:

7. The display control means is Control to superimpose the identification code on a predetermined position in the first image, which is determined based on the distance from the position of the location identification code contained in the first image. The information processing system according to claim 1, characterized by the following:

8. The system further includes an imaging means that displays a live view for capturing a second image and captures the second image in response to an acceptance of an imaging operation, The imaging means suppresses the acceptance of the imaging operation or suppresses the imaging of the second image until it is determined that the position identification code included in the first image is included in the live view. The information processing system according to claim 1, characterized by the following:

9. The system further includes a display means that displays information about articles that have been determined to have passed or reached a predetermined deadline based on the determination result by the determination means, When the display control means receives an instruction regarding information about an item that has passed or reached a predetermined deadline, as displayed by the display means, it controls the display to identify and display the item that has passed or reached a predetermined deadline. The information processing system according to claim 1, characterized by the following:

10. The system further includes output means for outputting information about items that have passed or have been determined to have reached their predetermined expiration date. The information processing system according to claim 1, characterized by the following:

11. Based on the aforementioned determination, the items to be returned include items whose deadline has passed or has been determined to have reached the deadline. The information processing system according to claim 1, characterized by the following:

12. The identification code is different for each individual item or for each type of item. The information processing system according to claim 1, characterized by the following:

13. The display control means controls the display to show the number of identification codes that were analyzed by the analysis means. The information processing system according to claim 1, characterized by the following:

14. The term "deadline" relating to the aforementioned goods is a deadline based on at least one of the following: the shipping deadline, the delivery deadline, the delivery deadline, the quality retention deadline, the sales deadline, the retention deadline, the use-by date, the best-before date, or the expiration date of the aforementioned goods. The information processing system according to claim 1, characterized by the following:

15. The expiration date for the aforementioned item is either an expiration date based on year, month, day, or time information stored in a file, database, or recording medium associated with the identification code corresponding to the item, or an expiration date based on year, month, day, or time information displayed near the identification code corresponding to the item. The information processing system according to claim 1, characterized by the following:

16. An analysis step that analyzes the identification code contained in the first image, A determination step is performed to determine the expiration date of the item corresponding to the identification code analyzed in the above analysis step, A display control step that controls the display of the determination result from the determination step to a predetermined position of the identification code in the first image, A control method for an information processing system, characterized by having the following features.

17. A program for causing at least one computer to function as one of the means described in any one of claims 1 to 15.

Images