Processing device, processing method, and recording medium

Abstract

This processing device includes a reference count specification unit and a threshold determination unit. The reference count specification unit specifies, for each product, a reference count that is the number of items detected in an image of a product shelf when the display condition satisfies a reference standard. The threshold determination unit determines, for each product, a shortage determination threshold, which is a threshold for determining a shortage condition, on the basis of the reference count.

Description

Processing device, processing method, and recording medium

[0001] The present disclosure relates to a processing device, a processing method, and a recording medium.

[0002] A technology related to this disclosure is disclosed in Patent Literature 1. The technology disclosed in Patent Literature 1 detects out-of-stock of a product based on a comparison between the out-of-stock rate of the product identified based on an image of a product shelf and a stockout threshold. In this technology, a user first sets the out-of-stock threshold for each product. Then, in one embodiment of this technology, the set out-of-stock threshold is corrected depending on the sales and time period of each product.

[0003] Japanese Patent Application Laid-Open No. 2022-139413

[0004] One example of the objective of this disclosure is to appropriately determine a threshold value for determining a shortage state for each product in a technology that uses image analysis to detect shortages of products displayed on a shelf.

[0005] The processing device disclosed herein has a standard number determination means for determining, for each product, a standard number, which is the number of products detected in an image of a product shelf when the display state meets a standard, and a threshold determination means for determining, for each product, a shortage determination threshold, which is a threshold for determining that a product is in a shortage state, based on the standard number.

[0006] The processing method disclosed herein involves one or more computers identifying, for each product, a reference number, which is the number of products that are detected in an image of a product shelf when the display state meets a standard, and determining, for each product, a shortage determination threshold, which is a threshold value for determining that the product is in a shortage state, based on the reference number.

[0007] The recording medium of this disclosure records a program that causes a computer to function as: a reference number determination means that determines, for each product, a reference number, which is the number detected in an image of a product shelf when the display state meets a standard; and a threshold determination means that determines, for each product, a shortage determination threshold, which is a threshold for determining that the product is in a shortage state, based on the reference number.

[0008] According to one aspect of the present disclosure, in a technology that uses image analysis to detect shortages of products displayed on shelves, a threshold value for determining a shortage can be appropriately determined for each product.

[0009] FIG. 1 is a diagram showing an example of a functional block diagram of a processing device. FIG. 2 is a flowchart showing an example of a processing flow of a processing device. FIG. 3 is a diagram showing an example of a hardware configuration of a processing device. FIG. 4 is a diagram showing an example of an image processed by a processing device. FIG. 5 is a diagram for explaining an example of processing executed by a processing device. FIG. 6 is a diagram showing an example of information processed by a processing device. FIG. 7 is a diagram showing an example of another example of information processed by a processing device. FIG. 8 is a flowchart showing an example of a processing flow of a processing device. FIG. 9 is a diagram showing an example of information processed by a processing device. FIG. 10 is a diagram showing an example of a functional block diagram of a processing device. FIG. 11 is a flowchart showing an example of a processing flow of a processing device.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0011] <<First Embodiment>> Fig. 1 is a functional block diagram showing an overview of a processing device 10. As shown in Fig. 1, the processing device 10 has a reference number specification unit 11 and a threshold value determination unit 12. These functional units execute the processing of the flowchart in Fig. 2.

[0012] In S10, the reference number specifying unit 11 specifies, for each product, a reference number that is the number of products that are detected in an image of the product shelf when the display state satisfies a reference number. In S11, the threshold determining unit 12 determines, for each product, a shortage determination threshold that is a threshold for determining that a product is in a shortage state, based on the reference number.

[0013] In one example of a process for detecting shortages of products displayed on shelves using image analysis, products are detected in an image of the shelves, and the shortage status is determined based on the number of detected products. The number of products detected in the image of the shelves varies depending on the display status. When the work of restocking the shelves is completed, more products are detected in the image of the shelves. Then, as products are sold and the shortage status approaches, the number of products detected in the image of the shelves decreases.

[0014] The "display state satisfies the standard" refers to a time when the product filling rate on the product shelves is equal to or greater than the standard value, preferably 100% or close to it, such as when the restocking of the product shelves is completed. The standard number, which is the number of products detected in the product shelf image at such a time, can be considered to be the maximum number of products detected in the product shelf image.

[0015] The processing device 10 appropriately determines the shortage determination threshold based on such a reference number. That is, the processing device 10 first identifies the maximum number of each product that can be detected in the image of the product shelf, and then appropriately determines the shortage determination threshold for each product based on the identification result.

[0016] Depending on such a reference number, the product display state and shortage determination threshold that can be detected by image analysis may vary.

[0017] For example, if the reference number is "3," image analysis can detect the following four product display states: - A state in which the number of products detected in the image is 3. In other words, a state in which the fill rate is 100%. - A state in which the number of products detected in the image is 2. In other words, a state in which the fill rate is 67%. - A state in which the number of products detected in the image is 1. In other words, a state in which the fill rate is 33%. - A state in which the number of products detected in the image is 0. In other words, a state in which the fill rate is 0%.

[0018] In this case, there are three possible variations of the shortage determination threshold: - A shortage is determined when the number of items detected in the image is two. In other words, a shortage is determined when the fill rate is 67%. - A shortage is determined when the number of items detected in the image is one. In other words, a shortage is determined when the fill rate is 33%. - A shortage is determined when the number of items detected in the image is zero. In other words, a shortage is determined when the fill rate is 0%.

[0019] In contrast, when the reference number is "2", the product display states that can be detected by image analysis are the following three states: - A state in which the number of products detected in the image is 2. In other words, a state in which the filling rate is 100%. - A state in which the number of products detected in the image is 1. In other words, a state in which the filling rate is 50%. - A state in which the number of products detected in the image is 0. In other words, a state in which the filling rate is 0%.

[0020] In this case, there are two possible variations of the shortage determination threshold: - A shortage is determined to have occurred when the number of products detected in the image is one. In other words, a shortage is determined when the filling rate is 50%. - A shortage is determined to have occurred when the number of products detected in the image is zero. In other words, a shortage is determined when the filling rate is 0%.

[0021] In this way, the variations that can be taken as the shortage determination threshold may differ depending on the reference number. Without taking this point into consideration, it is impossible to determine an appropriate shortage determination threshold. By using the processing device 10 that determines the shortage determination threshold based on the reference number, it is possible to appropriately determine the shortage determination threshold from the variations that can be taken for each reference number.

[0022] Furthermore, this reference number may differ for each product. For example, it may differ depending on the number of faces assigned to each product on the display shelf, the size of the display area assigned to each product, the size of each product, how each product is displayed, the display position of each product on the product shelf, the position and orientation of the camera that captures images of the product shelf, etc. Without taking this into consideration, it is impossible to determine an appropriate shortage determination threshold for each product. By using the processing device 10 that specifies a reference number for each product and determines a shortage determination threshold for each product based on the reference number for each product, it is possible to appropriately determine the shortage determination threshold.

[0023] <<Second Embodiment>> <Overview> A processing apparatus 10 according to a second embodiment is a specific embodiment of the configuration of the processing apparatus 10 according to the first embodiment. The processing apparatus 10 will be described in detail below.

[0024] <Hardware Configuration> First, an example of the hardware configuration of the processing device 10 will be described. Each functional unit of the processing device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. The software includes programs that are pre-loaded when the device is shipped, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0025] FIG. 3 is a block diagram illustrating an example of the hardware configuration of a processing device 10. As shown in FIG. 3, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0026] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a processing unit such as a CPU or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, touch panel, etc. Examples of output devices include a display, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0027] <Functional Configuration> Next, a detailed description will be given of the functional configuration of the processing device 10. Fig. 1 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 has a reference number specification unit 11 and a threshold value determination unit 12.

[0028] The reference number specification unit 11 specifies a reference number for each product.

[0029] The "reference number" is the number of products detected in the image of the product shelf when the display state meets the standard.

[0030] As mentioned above, "when the display state meets the standard" refers to when the product filling rate on the product shelves is equal to or greater than the standard value, preferably at or near 100%, such as when the restocking of the product shelves is completed. The standard number, which is the number of products detected in the product shelf image at such times, can be considered to be the maximum number of products detected in the product shelf image.

[0031] It should be noted that the maximum number of products detected in an image of a product shelf (reference number) is a different concept from the maximum number of products that can actually be displayed in the display area of ​​the product shelf. In an image of a product shelf, there are blind spots such as other products and other objects, and not all products displayed in the display area of ​​the product shelf are necessarily visible in the image. The reference number is the maximum number of products that can be detected in an image of a product shelf in such a situation, and is different from the maximum number of products that can actually be displayed in the display area of ​​the product shelf. The reference number is a value that is equal to or less than the maximum number of products that can actually be displayed in the display area of ​​the product shelf.

[0032] The "product shelf image" is an image generated by a camera that photographs product shelves to detect product shortages. Hereinafter, such a camera will be referred to as a "product surveillance camera." In other words, the reference number is the maximum number of products detected in a product shelf image generated by such a product surveillance camera.

[0033] Figure 4 shows a schematic example of an image P of a product shelf. The illustrated product shelf image P is an image taken from the front of the product shelf by a product surveillance camera. In reality, other products of the same type are displayed in a row behind each product. However, in the product shelf image P, only the product at the top of each row is detectably visible. In the example of Figure 4, the reference number of products marked with the letter "GO" is three. In this way, when a product surveillance camera takes a picture of a product shelf from the front, the number of faces assigned to each product on the product shelf can become the reference number for each product.

[0034] The product surveillance camera is installed at a predetermined position and captures images of the product shelves from a predetermined direction. In one example, as described above, the product surveillance camera is installed at a position and orientation that captures images of the product shelves from the front, but this is not limited to this. The product surveillance camera may be installed at a position and orientation that captures images of the product shelves from diagonally above, from diagonally below, or from diagonally to the side. Furthermore, if the product shelf does not have a top or is transparent, the product surveillance camera may be installed at a position and orientation that captures images of the product shelves from directly above. Furthermore, if the shelf board is transparent, the product surveillance camera may be installed at a position and orientation that captures images from directly below through the shelf board. Note that the examples given here are merely examples. The elevation angle and azimuth angle of the product surveillance camera's capture direction (optical axis direction) can be any combination that suits the installation environment, etc.

[0035] Depending on the position and orientation of the product surveillance camera, it is possible that not only the product displayed at the front (toward the product surveillance camera) on the shelf but also at least some of the other products of the same type displayed behind it will be detectably captured in the image. As is clear from the description of the first embodiment, the greater the reference number, the greater the variation in the shortage determination threshold, making it possible to determine a shortage state in a more desirable state. By devising the position and orientation of the product surveillance camera so that not only the product displayed at the front of the shelf but also other products of the same type displayed behind it will be detectably captured in the image, it becomes possible to determine a shortage state in a more desirable state.

[0036] Product surveillance cameras can capture moving or still images, and can detect visible light or other electromagnetic waves, such as infrared light, to create images.

[0037] The processing device 10 can acquire images generated by the merchandise surveillance camera from the merchandise surveillance camera. The processing device 10 and the merchandise surveillance camera are communicatively connected to each other. The processing device 10 may acquire images generated by the merchandise surveillance camera from the merchandise surveillance camera in real time processing.

[0038] "Acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request to another device and receiving a response, and accessing and reading information from another device or storage medium. An example of passive acquisition is receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0039] Next, a process for specifying the reference number for each product will be described. The reference number specifying unit 11 specifies the size of the display area allocated to each product within the image of the product shelf. Then, the reference number specifying unit 11 specifies the reference number for each product based on the size of the specified display area.

[0040] For example, the reference number determination unit 11 uses a well-known shelf allocation determination technique to determine the display area allocated to each product in the product shelf image. Then, the reference number determination unit 11 determines the size of the determined display area. The size of the display area may be represented by the number of pixels or by other methods.

[0041] By using the shelf allocation specification technology, for example, as shown in FIG. 5, the display area X １ ~X ５are identified in the image of the product shelf. The shelf allocation identification technology may be, for example, a technology that detects products in the image of the product shelf and groups the same products together to identify the display area assigned to each product as shown in Figure 5, or a technology that realizes such identification using other algorithms.

[0042] Alternatively, the reference quantity identification unit 11 may accept user input instead of using the shelf allocation identification technology. That is, the user may input information to identify the display area allocated to each product within the product shelf image. Then, the reference quantity identification unit 11 may identify the display area allocated to each product within the product shelf image based on the user input.

[0043] After determining the size of the display area within the product shelf image for each product, the reference number determination unit 11 determines a reference number for each product based on the size of the display area determined for each product and the size of the product depicted in the product shelf image. For example, the reference number determination unit 11 calculates how many products of the size depicted in the product shelf image can be accommodated within the size of the display area determined for each product. For example, the reference number determination unit 11 may calculate a quotient (with the remainder discarded) by dividing the size of the display area by the size of the product. The reference number determination unit 11 then determines the calculation result as the reference number for each product.

[0044] The sizes of the products shown in the product shelf image may be registered in advance in the processing device 10. Alternatively, the reference number identification unit 11 may analyze the product shelf image and identify the area in the product shelf image in which each product appears. The reference number identification unit 11 may then identify the size of the identified area in which each product appears as the size of the product shown in the product shelf image. The size of the product shown in the product shelf image may be indicated by the number of pixels, or may be indicated by other methods. The process of identifying the area in which each product appears in the product shelf image may be realized using technology capable of detecting the contours of an object, such as semantic segmentation, or may be realized using other image analysis technology.

[0045] Returning to FIG. 1, the threshold value determination unit 12 determines a shortage determination threshold value for each product based on the reference number.

[0046] The "shortage determination threshold" is a threshold for determining whether a product is in a shortage state. The shortage detection unit 13 described in the following embodiment detects whether a product is in a shortage state based on the number of products detected in an image of a product shelf generated by a product surveillance camera and the shortage determination threshold.

[0047] The shortage determination threshold may be defined by the number of products detected in the shelf image. In this case, the threshold determination unit 12 determines, for each product, one of the numbers smaller than the reference number as the shortage determination threshold.

[0048] Alternatively, the shortage determination threshold may be defined as a fill rate calculated based on the number of products detected in the image of the product shelf. The fill rate is calculated by dividing the number of products detected in the image of the product shelf by a reference number. In this example, if the reference number is S and the target number, which is one of the numbers smaller than the reference number, is T, the threshold determination unit 12 determines the fill rate of T / S as the shortage determination threshold for each product. The "target number" is the number of products detected in the image of the product shelf, and is the number at which a shortage is determined. In this example, the fill rate used as the shortage determination threshold is calculated based on this target number.

[0049] In the second embodiment, as shown in FIG. 6 , the shortage determination threshold for each reference quantity is determined in advance and registered in the processing device 10. That is, as described in the first embodiment, the variations of the shortage determination threshold differ depending on the reference quantity, but in the second embodiment, one of the variations is determined in advance as the shortage determination threshold for each reference quantity and registered in the processing device 10. The user can determine the shortage determination threshold for each reference quantity in advance and register it in the processing device 10. The user may also be able to update (change) the shortage determination threshold for each reference quantity registered in the processing device 10. Note that in the example of FIG. 6 , the shortage determination threshold is defined by the number of items, but as described above, the shortage determination threshold may also be defined by the fill rate.

[0050] The threshold determination unit 12 then determines a shortage determination threshold for each product based on the reference number, with reference to information such as that shown in Fig. 6. In the example of Fig. 6, the shortage determination threshold for a product with a reference number of 1 is 0. In the example of Fig. 6, the shortage determination threshold for a product with a reference number of 2 to 4 is 1. In the example of Fig. 6, the shortage determination threshold for a product with a reference number of 5 or 6 is 2.

[0051] Note that information such as that shown in FIG. 6 may be generated for each product type and registered in the processing device 10. The threshold value determination unit 12 may then determine the shortage determination threshold for each product by referring to the information on the corresponding product type. Product types can be classified in various ways. For example, they may be classified into food, daily necessities, home appliances, etc. They may also be classified more finely. For example, food may be subdivided into beverages, confectionery, vegetables, meat, seafood, etc.

[0052] The threshold value determination unit 12 can register the shortage determination threshold value determined for each product in the processing device 10. Fig. 7 schematically shows an example of information registered in the processing device 10 by the threshold value determination unit 12. The information in Fig. 7 links a column for product identification information, a column for the reference number, and a column for the shortage determination threshold value. Note that the information does not need to have a column for the reference number.

[0053] The "product identification information" field is filled in with information that distinguishes between products displayed on the product shelves. The "reference number" field is filled in with the reference number determined for each product by the reference number determination unit 11. The "shortage determination threshold" field is filled in with the shortage determination threshold determined for each product by the threshold determination unit 12.

[0054] An example of the processing flow of the processing device 10 of the second embodiment is shown in the flowchart of Fig. 2. The processing flow of the flowchart of Fig. 2 has been explained in the first embodiment, so the explanation will be omitted here.

[0055] <Operational Effects> According to the processing apparatus 10 of the second embodiment, the same operational effects as those of the processing apparatus 10 of the first embodiment are realized.

[0056] Furthermore, the processing device 10 can specify the size of the display area allocated to each product within the product shelf image, and specify a reference number based on the size of the display area within the product shelf image and the size of the product within the product shelf image. With this processing device 10, it is possible to accurately specify the reference number for each product, which is the number of products that will be detected within the product shelf image when the display state satisfies the reference number.

[0057] Furthermore, the processing device 10 can determine, for each product, a target number that is one of the numbers smaller than the reference number as the shortage determination threshold. Furthermore, if the reference number is S and the target number that is one of the numbers smaller than the reference number is T, the processing device 10 can determine, for each product, the filling rate of T / S as the shortage determination threshold. With this processing device 10, a predetermined shortage determination threshold can be determined for each product from a variety of shortage determination thresholds that correspond to the reference number of each product.

[0058] <<Third Embodiment>> A processing device 10 according to a third embodiment specifies the reference number for each product using a method different from that of the processing device 10 according to the second embodiment. This will be described in detail below.

[0059] In the third embodiment, the reference number determination unit 11 obtains a reference image, which is an image of a product shelf captured when the display state satisfies a reference, and analyzes the reference image to determine the reference number. The process performed by the reference number determination unit 11 will be described in detail below.

[0060] First, the process of acquiring the reference image will be described.

[0061] A "reference image" is an image generated by a product surveillance camera photographing a product shelf when the display state meets a standard. Product shelf image P in Figure 4 is a reference image generated by photographing a product shelf when the display state meets a standard. The reference image may be one or more still images, or may be a moving image.

[0062] The reference number specification unit 11 can acquire the reference image by either of the following acquisition process examples 1 and 2.

[0063] Acquisition Process Example 1 In acquisition process example 1, a user prepares a reference image and inputs it to the processing device 10. The reference number specification unit 11 acquires the reference image input by the user.

[0064] For example, a user may visually check images of product shelves and extract an image in which the product display state meets a standard as a reference image. Alternatively, a user may specify the timing (date and time) when the restocking of products on the product shelves was completed, and extract an image generated immediately after the specified timing (e.g., several seconds to several tens of seconds after) from images of product shelves generated by a product surveillance camera as a reference image. A user can specify the timing (date and time) when the restocking of products on the product shelves was completed based on interviews with workers, work schedules, etc.

[0065] "Acquisition Process Example 2" In acquisition process example 2, the reference number identification unit 11 detects a replenishment completion event in which replenishment of products on the product shelves is completed. Then, the reference number identification unit 11 acquires, as the reference image, an image generated by a product surveillance camera photographing the product shelves after the replenishment completion event is detected. For example, the reference number identification unit 11 acquires, as the reference image, an image generated by a product surveillance camera photographing the product shelves after a predetermined time has elapsed (e.g., several seconds to several tens of seconds) since the timing of detecting the replenishment completion event.

[0066] The reference number identification unit 11 may acquire reference images of all product shelves at the same time. Alternatively, the reference number identification unit 11 may acquire reference images of multiple product shelves at different times. For example, the timing may be different for each product, each product type, or each sales area.

[0067] Here, the process of detecting a replenishment completion event will be described.

[0068] In one example, the reference number identification unit 11 monitors the number of products detected from photographed images of product shelves and detects an increase in the number as a replenishment completion event. The reference number identification unit 11 can use any technology to detect each product from photographed images of product shelves. The reference number identification unit 11 then counts the number of each detected product to identify the number of each product detected from photographed images of product shelves at each timing. The reference number identification unit 11 detects a replenishment completion event in which the number of products has increased by comparing the number of products identified at the most recent timing with the number of products identified at previous timings.

[0069] The process of detecting each product from an image of a product shelf can be realized by detecting a pre-registered product appearance image or a feature of the product appearance from within the image. Alternatively, a classifier generated in advance by machine learning may be used.

[0070] The images of the product shelves in this process may be images generated by a product surveillance camera or may be images generated by other cameras.

[0071] As a variation of this example, the reference quantity identification unit 11 may detect, as a replenishment completion event, an increase in the number of products equal to or greater than a threshold. For example, the number of products detected from an image of the product shelf may increase due to an action such as a customer returning products to the product shelf. By detecting, as a replenishment completion event, an increase in the number of products equal to or greater than a threshold, it is possible to reduce the inconvenience of detecting, as a replenishment completion event, an increase in the number of products due to such a customer action.

[0072] In another example, the reference number identification unit 11 detects a replenishment completion event based on the action of a worker detected from an image taken around the product shelf. First, the reference number identification unit 11 detects a worker from an image taken around the product shelf. For example, the worker's appearance features (face information, gait information, physique information, clothing information, etc.) may be registered in advance. Then, the reference number identification unit 11 can use the features to detect the worker from an image taken around the product shelf. Note that the image taken around the product shelf may be an image generated by a product surveillance camera or another camera.

[0073] The reference number identification unit 11 then further analyzes the image and determines whether the detected worker is performing replenishment work. In one example, the reference number identification unit 11 can make this determination based on at least one of the worker's posture and the object being worked on that is located around the worker. Conditions for determining that the worker is performing replenishment work are determined in advance. The conditions are defined using the worker's posture during replenishment work, the object (container, etc.) used by the worker during replenishment work, etc. The reference number identification unit 11 determines that the worker is performing replenishment work when the detected posture of the worker or the object being used by the worker satisfies the conditions.

[0074] Pose detection can be achieved using well-known techniques such as OPEN POSE, while object detection can be achieved by using a classifier generated by machine learning or by matching based on appearance features (such as feature point matching).

[0075] The reference number specifying unit 11 then detects, as a replenishment completion event, that the worker determined to be performing replenishment work has left the site. Alternatively, the reference number specifying unit 11 may detect, as a replenishment completion event, that the worker determined to be performing replenishment work no longer satisfies the "condition for determining that the worker is performing replenishment work."

[0076] In another example, a scheduled time for completing the replenishment work (hereinafter referred to as "scheduled time for completing the replenishment work") is determined in advance. In some cases, the approximate time for performing the replenishment work in a store is determined in advance. Based on this, it may be possible to estimate the scheduled time for completing the replenishment work to some extent. This example can be used in such cases.

[0077] In this example, the user estimates the scheduled time of completion of the replenishment work in advance and registers it in the processing device 10. The reference number identification unit 11 monitors the arrival of the scheduled time of completion of the replenishment work that has been registered in advance in the processing device 10. Then, the reference number identification unit 11 detects the arrival of the scheduled time of completion of the replenishment work as a replenishment completion event.

[0078] The user may register the scheduled completion time of the replenishment work for each product shelf in the processing device 10. Then, the reference number specification unit 11 can detect a replenishment completion event for each product shelf when each scheduled completion time of the replenishment work arrives.

[0079] In another example, upon completion of the replenishment work, an operator makes a predetermined user input to the processing device 10. The reference number identification unit 11 detects this user input as a replenishment completion event. The predetermined user input is an input made upon completion of the replenishment work, and may be, for example, an input notifying the processing device 10 of the completion of the replenishment work.

[0080] In this example, the worker may further input information indicating the product shelves on which the replenishment work has been completed in the predetermined user input. The reference quantity identification unit 11 can detect a replenishment completion event for each product shelf based on the information indicating the product shelves on which the replenishment work has been completed.

[0081] Next, a process of acquiring a reference image through the above-described process, and then analyzing the acquired reference image to specify the reference number will be described.

[0082] The reference number specification unit 11 detects each product from within the reference image, and counts the number of each detected product to specify the reference number of each product.

[0083] The process of detecting each product from the reference image can be realized by detecting pre-registered product appearance images or product appearance features from within the image. Alternatively, a classifier generated in advance by machine learning may be used.

[0084] In the process executed by the shortage detection unit 13 described in the following embodiment to analyze images of product shelves and detect product shortages, products are detected in the product shelf images and the shortage status is determined based on the number of detected products. The process of detecting each product from the reference image executed by the reference number determination unit 11 to determine the reference number preferably uses the same method (same algorithm) as the process of detecting products in the product shelf images executed by the shortage detection unit 13. This makes it possible to more accurately determine the maximum number of products (reference number) that can be detected in the product shelf images.

[0085] Other configurations of the processing apparatus 10 of the third embodiment are similar to those of the processing apparatus 10 of the first and second embodiments.

[0086] According to the processing apparatus 10 of the third embodiment, the same effects as those of the processing apparatus 10 of the first and second embodiments are achieved.

[0087] Furthermore, the processing device 10 can acquire a reference image, which is an image of the product shelf taken when the display state meets a standard, and, based on the reference image, can identify a reference number, which is the number of items detected in the image of the product shelf when the display state meets the standard. Specifically, the processing device 10 can identify the reference number by analyzing the reference image to detect products and counting the number of products detected in the reference image for each product. With this processing device 10, it is possible to accurately identify, for each product, the reference number, which is the number of items detected in the image of the product shelf when the display state meets the standard.

[0088] Furthermore, the processing device 10 can acquire a reference image using the characteristic method described above. For example, the processing device 10 can detect a replenishment completion event when replenishment of products on product shelves is completed, and acquire an image of the product shelves captured after the replenishment completion event is detected as a reference image. With this type of processing device 10, it is possible to accurately and efficiently acquire a reference image, which is an image of the product shelves captured when the display state meets a standard.

[0089] Furthermore, the processing device 10 can detect the timing of replenishment completion using the characteristic method described above. For example, the processing device 10 can monitor the number of products detected from images of the product shelves and detect an increase in the number (e.g., an increase of the number by more than a threshold) as a replenishment completion event. Alternatively, the processing device 10 can detect a replenishment completion event based on the actions of a worker detected from images of the vicinity of the product shelves. Alternatively, the processing device 10 can detect the arrival of a predetermined scheduled time for the completion of replenishment work as a replenishment completion event. Alternatively, the processing device 10 can detect a predetermined user input made in response to the completion of replenishment work as a replenishment completion event. Such a processing device 10 can accurately detect the replenishment completion timing, which is the timing when the replenishment work is completed. Furthermore, the processing device 10 that can accurately detect the replenishment completion timing can accurately and efficiently obtain a reference image, which is an image of the product shelves taken when the display state meets a standard.

[0090] <<Fourth Embodiment>> Product allocation, i.e., the layout of products on a shelf (such as the position and number of faces of each product), can change. For example, product allocation can change depending on the season, month, or other time of year. Product allocation can also change due to the arrival of new products, the discontinuation of production of a product, the discontinuation of handling due to poor sales, or an increase in handling due to strong sales. As the allocation changes, the reference number of each product changes. The processing device 10 of the fourth embodiment can repeatedly determine the shortage determination threshold for each product. This will be described in detail below.

[0091] In the fourth embodiment, at a predetermined update timing, the reference quantity specifying unit 11 re-specifies the reference quantity for each product, and at the predetermined update timing, the threshold value determining unit 12 re-determines the shortage determination threshold for each product.

[0092] The "update timing" can be various timings. For example, the update timing is １ Every hour, L ２ Daily, L ３ Weekly, L ４ The update timing may be at a predetermined time, such as once a month. Alternatively, a predetermined time within a day may be set as the update timing. Alternatively, a predetermined time on a predetermined day of the week may be set as the update timing. Alternatively, a predetermined time on a predetermined day of each month (e.g., the 1st) may be set as the update timing.

[0093] Alternatively, the update timing may be the timing when the user inputs an update instruction to the processing device 10 .

[0094] Alternatively, the update timing may be the timing when a shelf allocation update notification is received from the shelf allocation determination system. The shelf allocation determination system is a system that automatically determines shelf allocations. There are no particular limitations on the configuration of the shelf allocation determination system. The shelf allocation determination system can automatically determine the shelf allocation of products on a product shelf using any widely known technology. Then, when the shelf allocation determination system determines a new shelf allocation, it notifies the processing device 10 of that fact.

[0095] The update timing may also be a timing after a predetermined time has elapsed since the shelf allocation determination system notifies the user of a shelf allocation update. There is a time lag between when the shelf allocation determination system determines a new shelf allocation and when the shelf allocation on the product shelf is actually changed. Taking this time lag into consideration, the update timing is set to a timing after a predetermined time has elapsed since the shelf allocation determination system notifies the user of a shelf allocation update. The predetermined time is registered in advance in the processing device 10 by the user.

[0096] In the fourth embodiment, the update timing as described above is registered in advance in the processing device 10. When the registered update timing arrives, the reference number identification unit 11 re-identifies the reference number for each product. Then, in response to the processing by the reference number identification unit 11, the threshold determination unit 12 re-determines the shortage determination threshold for each product. Note that the threshold determination unit 12 may extract products whose reference number has changed as a result of the reference number identification unit 11 re-identifying the reference number. Then, the threshold determination unit 12 may re-determine the shortage determination threshold only for products whose reference number has changed.

[0097] The update timing may differ for each product shelf. That is, the update timing may be registered in advance in the processing device 10 for each product shelf. Then, when the update timing for each registered product shelf arrives, the reference number identification unit 11 may re-identify the reference number for each product based on the latest image of each product shelf. Then, in response to this processing by the reference number identification unit 11, the threshold determination unit 12 may re-determine the shortage determination threshold for each product.

[0098] Next, an example of the processing flow of the processing device 10 will be described with reference to the flowchart of FIG.

[0099] The processing device 10 monitors the arrival of the timing for updating (S20). When the timing for updating arrives (Yes in S20), the processing device 10 identifies, for each product, a reference number, which is the number of products detected in the image of the product shelf when the display state meets the criteria (S21). Next, the processing device 10 determines, for each product, a shortage determination threshold, which is a threshold for determining a shortage state, based on the reference number (S22). The processing device 10 then updates the shortage determination threshold for each product to the determined content (S23). Thereafter, the processing device 10 repeats the same process.

[0100] Other configurations of the processing apparatus 10 of the fourth embodiment are similar to those of the processing apparatus 10 of the first to third embodiments.

[0101] According to the processing apparatus 10 of the fourth embodiment, the same effects as those of the processing apparatus 10 of the first to third embodiments are realized.

[0102] Furthermore, the processing device 10 can re-specify the reference quantity for each product when the update timing arrives and re-determine the shortage judgment threshold based on the result. The product allocation, i.e., the product layout on the product shelf (the position and number of faces of each product, etc.), can change. And as the allocation changes, the reference quantity for each product changes.

[0103] According to the processing device 10, which re-specifies the reference number for each product when the update timing arrives and re-determines the shortage judgment threshold based on the result, even if the shelf allocation of the product changes, it is possible to re-determine an appropriate shortage judgment threshold according to the changed shelf allocation.

[0104] <<Fifth Embodiment>> In the example described in the second embodiment, one shortage determination threshold was determined in advance for each reference quantity. Then, the processing device 10 determined the one shortage determination threshold corresponding to the reference quantity of each product as the shortage determination threshold for each product.

[0105] In the processing device 10 of the fifth embodiment, conditions for adopting each of a plurality of variations of the shortage determination threshold for each reference quantity are predefined. The conditions are defined based on at least one of the product attributes and the sales environment. The processing device 10 then determines a shortage determination threshold for each product that corresponds to the reference quantity of each product and satisfies at least one of the product attributes and the sales environment. This will be described in detail below.

[0106] As shown in FIG. 9, conditions for adopting each of a plurality of variations of the shortage determination threshold for each reference quantity are determined in advance and registered in the processing device 10.

[0107] The "condition" is defined based on at least one of the product attributes and the sales environment. As described in the first embodiment, the shortage determination threshold can take one or more variations depending on the number of reference quantities. The condition is a condition for adopting each of the one or more variations.

[0108] The "product attributes" include at least one of product type, popularity, and sales information.

[0109] There are various ways to categorize product types. For example, they may be categorized into food, daily necessities, home appliances, etc. Furthermore, they may be categorized more finely. For example, food may be subdivided into beverages, confectionery, vegetables, meat, seafood, etc.

[0110] The popularity level indicates the degree of popularity of a product. The popularity level is indicated by a predetermined index. For example, the popularity level may be indicated by a numerical value from 0 to 5, or by other methods. The popularity level of each product may be determined by the user. Alternatively, the processing device 10 may calculate the popularity level of each product based on the actual sales of each product. For example, a calculation method (algorithm) for calculating the popularity level from the actual sales of each product is determined in advance and registered in the processing device 10. The processing device 10 calculates the popularity level of each product based on the calculation method and the actual sales of each product. The processing device 10 can acquire data indicating the actual sales of each product from a POS (point of sales) system.

[0111] The sales information indicates the actual sales of each product (number of units sold, sales amount, etc.) The sales information may be, for example, sales for the most recent predetermined period (the most recent week, the most recent month, the most recent three months, the most recent year, etc.).

[0112] The "sales environment" includes at least one of the season, day of the week, month, time of day, weather, temperature, humidity, and events held in the vicinity.

[0113] The events held in the vicinity are events held in the vicinity of the store. Sales of each product may fluctuate depending on the events held. Information indicating the events held on each day and in each time period is registered in advance in the processing device 10.

[0114] Here, an example of a condition defined based on at least one of the product attributes and the sales environment as described above will be described.

[0115] An example of a condition is "(product attribute) beverage & popularity rating 5, (sales environment) summer & sunny & game at a nearby professional baseball stadium on the same day." A product that meets such conditions is expected to sell well. For this reason, when such conditions are met, the information in FIG. 9 is created so that a relatively high value among multiple variations is determined as the shortage determination threshold. If a high value is determined as the shortage determination threshold, a shortage state will be determined relatively early.

[0116] Another example of a condition is "(product attribute) beverage & popularity level 1, (sales environment) winter & rain & no game at the nearby professional baseball stadium on that day." A product that meets such conditions is expected to sell poorly. For this reason, when such conditions are met, the information in FIG. 9 is created so that a relatively low value among multiple variations is determined as the shortage determination threshold. If a low value is determined as the shortage determination threshold, the shortage state will be determined relatively late.

[0117] In the fifth embodiment, as in the fourth embodiment, the processing device 10 can repeatedly re-determine the shortage determination threshold in response to the arrival of the update timing. Note that in the fourth embodiment, it was sufficient to set the update timing according to the cycle in which changes occur in the shelf allocation of products. In the fifth embodiment, the update timing is set according to the cycle in which changes occur in product attributes or the sales environment.

[0118] Other configurations of the processing apparatus 10 of the fifth embodiment are similar to those of the processing apparatus 10 of the first to fourth embodiments.

[0119] According to the processing apparatus 10 of the fifth embodiment, the same effects as those of the processing apparatus 10 of the first to fourth embodiments are achieved.

[0120] Furthermore, the processing device 10 determines an appropriate shortage determination threshold based on at least one of the product attributes and the sales environment from among a plurality of variations of the shortage determination threshold determined based on the reference number. With this processing device 10, it is possible to determine a more appropriate shortage determination threshold for each product based on at least one of the product attributes and the sales environment.

[0121] <<Sixth Embodiment>> A processing device 10 according to a sixth embodiment has a function of detecting a product that is in a shortage state based on a shortage determination threshold value. This will be described in detail below.

[0122] 10 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 has a reference number specification unit 11, a threshold value determination unit 12, and a shortage detection unit 13.

[0123] The shortage detection unit 13 detects whether a product is in a shortage state based on the number of products detected in an image of a product shelf generated by a product surveillance camera and a shortage determination threshold.

[0124] That is, the shortage detection unit 13 detects products in the images of the product shelves generated by the product surveillance camera, counts the number of detected products for each product, and determines whether each product is in a shortage state based on the number of each detected product and the shortage determination threshold for each product determined by the threshold determination unit 12.

[0125] For example, the shortage detection unit 13 can determine that a product is in a shortage state when the detected number of products is equal to or less than the shortage determination threshold (number of products).The shortage detection unit 13 can also determine that a product is not in a shortage state when the detected number of products is greater than the shortage determination threshold.

[0126] Alternatively, the shortage detection unit 13 may calculate the fill rate of each product by dividing the detected number of each product by the reference number of each product. In this case, the shortage detection unit 13 can determine that a product is in a shortage state if the calculated fill rate is equal to or less than a shortage determination threshold (fill rate). Furthermore, the shortage detection unit 13 can determine that a product is not in a shortage state if the calculated fill rate is greater than the shortage determination threshold.

[0127] The process of detecting each product from an image of a product shelf can be realized based on pre-registered product appearance images or product appearance feature quantities. Alternatively, a classifier generated in advance by machine learning may be used.

[0128] Next, an example of the processing flow of the processing device 10 will be described with reference to the flowchart of FIG.

[0129] The processing device 10 monitors the arrival of the detection timing (S30). The detection timing may be every few seconds, every few minutes, or every few hours. Alternatively, the detection timing may be the timing when the user inputs a detection instruction to the processing device 10. Furthermore, the detection timing may differ for each product shelf.

[0130] When the detection timing arrives (Yes in S30), the processing device 10 acquires an image of the product shelf captured by the product surveillance camera (S31). Next, the processing device 10 detects products from the acquired product shelf image (S32) and counts (calculates) the number of detected products for each product (S33). Next, the processing device 10 executes a process to detect a shortage state for each product based on the detected number and a shortage determination threshold (S34).

[0131] The processing device 10 then executes processing according to the detection result (S35). For example, if the processing device 10 detects that a product is in short supply, it may notify an employee of this fact. In the notification, the processing device 10 may also notify the employee of information about the product that is in short supply. The information about the product that is in short supply may include the product name. The information about the product that is in short supply may further indicate the current display status (the number of products detected from the most recent display shelf image). The information about the product that is in short supply may also include the image of the display shelf that was processed when it was determined that the product was in short supply.

[0132] The notification to the worker can be realized by using any method. For example, the processing device 10 may output information related to the notification to a POS register in the store or a mobile terminal of the worker.

[0133] Other configurations of the processing apparatus 10 of the sixth embodiment are similar to those of the processing apparatus 10 of the first to fifth embodiments.

[0134] According to the processing apparatus 10 of the sixth embodiment, the same effects as those of the processing apparatuses 10 of the first to fifth embodiments are achieved.

[0135] Furthermore, the processing device 10 can determine the shortage status of a product based on a shortage determination threshold that is appropriately determined for each product using a characteristic method. With such a processing device 10, it is possible to appropriately determine the shortage status of a product.

[0136] <<Modifications>> Modifications applicable to all the embodiments will be described below. These modifications also achieve the same effects as the above-described embodiments.

[0137] In Modification 1, the number of faces on the product shelf for each product is stored in advance in a predetermined storage device. The storage device may be provided within the processing device 10 or may be provided in an external device communicably connected to the processing device 10.

[0138] Then, the reference number specifying unit 11 specifies the number of faces on the product shelf, which is registered in advance for each product, as the reference number for each product.

[0139] The face number for each product may be registered in the storage device by a user. Alternatively, the face number for each product in the shelf allocation determined by the shelf allocation determination system described in the fourth embodiment may be stored in the storage device.

[0140] In Modification 2, the reference quantity specification unit 11 accepts a user input specifying a reference quantity for each product. Then, the reference quantity specification unit 11 specifies the value specified by the user input as the reference quantity for each product.

[0141] Although this disclosure has been described above with reference to the embodiments, this disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0142] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content. Furthermore, the above-mentioned embodiments can be combined as long as the content does not contradict each other.

[0143] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. 1. A processing device having: a reference number specifying means for specifying, for each product, a reference number which is the number of products detected in an image of a product shelf when the display state meets a reference standard; and a threshold determining means for determining, for each product, a shortage determination threshold which is a threshold for determining a shortage state based on the reference number. 2. The processing device described in 1, wherein the reference number specifying means obtains a reference image which is an image of a product shelf when the display state meets a reference standard, analyzes the reference image to detect products, counts the number of products detected in the reference image for each product, and specifies the counted number as the reference number. 3. The processing device described in 2, wherein the reference number specifying means detects a replenishment completion event when replenishment of products to a product shelf is completed, and obtains an image of the product shelf after the replenishment completion event is detected as the reference image. 4. The processing device described in 3, wherein the reference number specifying means monitors the number of products detected in an image of a product shelf, and detects an increase in the number as the replenishment completion event. 5. The processing device according to 4, wherein the reference number specifying means detects an increase in the number equal to or greater than a threshold as the replenishment completion event. 6. The processing device according to 3, wherein the reference number specifying means detects the replenishment completion event based on an action of a worker detected from an image taken of the area around the product shelf. 7. The processing device according to 3, wherein the reference number specifying means detects the arrival of a predetermined scheduled time for completion of replenishment work as the replenishment completion event. 8. The processing device according to 3, wherein the reference number specifying means detects a predetermined user input made in response to the completion of replenishment work as the replenishment completion event. 9. The processing device according to 1, wherein the reference number specifying means specifies, for each product, the size of an allocated display area within the image of the product shelf, and specifies the reference number for each product based on the size of the display area within the image of the product shelf and the size of the product within the image of the product shelf. 10. The processing device according to 1, wherein the reference number specifying means specifies the number of faces on the product shelf registered in advance for each product as the reference number for each product.11. The processing device according to 1, wherein the reference number specification means accepts a user input specifying the reference number for each product. 12. The processing device according to any one of 1 to 11, wherein the threshold determination means determines, for each product, one of the numbers smaller than the reference number as the shortage determination threshold. 13. The processing device according to any one of 1 to 11, wherein, when the reference number is S and a target number that is one of the numbers smaller than the reference number is T, the threshold determination means determines, for each product, a filling rate of T / S as the shortage determination threshold. 14. The processing device according to any one of 1 to 13, wherein the threshold determination means determines, for each product, the shortage determination threshold based on at least one of product attributes and sales environment in addition to the reference number. 15. The processing device according to 14, wherein the product attributes include at least one of product type, popularity, and sales information. 16. 17. The processing device according to any one of 1 to 16, wherein the sales environment includes at least one of season, day of the week, month, time of day, weather, temperature, humidity, and events held in the vicinity. 17. The processing device according to any one of 1 to 16, wherein at a predetermined update timing, the reference number specification means re-specifies the reference number for each product, and the threshold determination means re-determines the shortage determination threshold for each product. 18. The processing device according to any one of 1 to 17, further comprising shortage detection means that detects a shortage of a product based on the number of products detected in an image of a product shelf and the shortage determination threshold. 19. A processing method in which one or more computers: specify, for each product, a reference number that is the number detected in an image of a product shelf when the display state meets a standard; and determine, for each product, a shortage determination threshold that is a threshold for determining a shortage based on the reference number. 20. A recording medium having recorded thereon a program that causes a computer to function as: a reference number determination means that determines, for each product, a reference number that is the number of products that are detected in an image of a product shelf when the display state meets a standard; and a threshold determination means that determines, for each product, a shortage determination threshold that is a threshold for determining that the product is in a shortage state, based on the reference number.

[0144] Some or all of Supplements 2 to 18 that are dependent on the processing device of Supplement 1 described above may also be dependent on the processing method of Supplement 19 and the recording medium of Supplement 20 in the same dependent relationship as Supplement 1 and Supplements 2 to 18. Furthermore, within the scope of each of the above-described embodiments, some or all of the configurations described as Supplements can be realized in various hardware, software, various recording means for recording software, or systems.

[0145] 10 Processing device 11 Reference number specification unit 12 Threshold value determination unit 13 Shortage detection unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. A processing device having a standard number determination means for determining, for each product, a standard number that is the number of products that are detected in an image of a product shelf when the display state meets a standard, and a threshold determination means for determining, for each product, a shortage determination threshold that is a threshold for determining that the product is in a shortage state based on the standard number.

2. The processing device according to claim 1, wherein the reference number determination means acquires a reference image, which is an image of a product shelf taken when the display state meets a standard, analyzes the reference image to detect products, counts the number of products detected in the reference image for each product, and determines the counted number as the reference number.

3. The processing device according to claim 2, wherein the reference number determination means detects a replenishment completion event in which replenishment of products on a product shelf is completed, and acquires an image of the product shelf taken after the replenishment completion event is detected as the reference image.

4. The processing device according to claim 3, wherein the reference number determination means monitors the number of products detected from an image of the product shelf, and detects an increase in the number as the replenishment completion event.

5. The processing device according to claim 4, wherein the reference number specifying means detects an increase in the number of items by a threshold value or more as the replenishment completion event.

6. The processing device according to claim 3, wherein the reference number determination means detects the replenishment completion event based on an action of a worker detected from an image taken around the product shelf.

7. The processing device according to claim 3, wherein the reference number specifying means detects the arrival of a predetermined scheduled time for completion of replenishment work as the replenishment completion event.

8. The processing device according to claim 3, wherein the reference number specifying means detects a predetermined user input made in response to completion of a replenishment operation as the replenishment completion event.

9. The processing device according to claim 1, wherein the reference number determination means determines the size of the display area allocated to each product within the image of the product shelf, and determines the reference number for each product based on the size of the display area within the image of the product shelf and the size of the product within the image of the product shelf.

10. The processing device according to claim 1, wherein the reference number determination means determines the number of faces on the product shelf, which is registered in advance for each product, as the reference number for each product.

11. The processing device according to claim 1, wherein the reference number specifying means accepts a user input specifying the reference number for each product.

12. A processing device according to any one of claims 1 to 11, wherein the threshold value determination means determines, for each product, one of the numbers smaller than the reference number as the shortage determination threshold value.

13. A processing device according to any one of claims 1 to 11, wherein, when the reference number is S and the target number, which is one of the numbers smaller than the reference number, is T, the threshold value determination means determines the filling rate of T / S for each product as the shortage determination threshold value.

14. A processing device according to any one of claims 1 to 13, wherein the threshold determination means determines the shortage judgment threshold for each product based on at least one of product attributes and sales environment in addition to the reference number.

15. The processing device according to claim 14, wherein the product attributes include at least one of product type, popularity, and sales information.

16. The processing device according to claim 14 or 15, wherein the sales environment includes at least one of season, day of the week, month, time of day, weather, temperature, humidity, and events held in the vicinity.

17. A processing device described in any one of claims 1 to 16, wherein, at a predetermined update timing, the reference quantity determination means re-determines the reference quantity for each product, and the threshold determination means re-determines the shortage judgment threshold for each product.

18. A processing device as claimed in any one of claims 1 to 17, further comprising a shortage detection means for detecting whether a product is in a shortage state based on the number of products detected in an image of a product shelf and the shortage determination threshold value.

19. A processing method in which one or more computers identify, for each product, a reference number, which is the number of products that are detected in an image of a product shelf when the display condition meets a standard, and determine, for each product, a shortage determination threshold, which is a threshold value for determining that the product is in shortage, based on the reference number.

20. A recording medium having recorded thereon a program that causes a computer to function as: a standard number determination means that determines, for each product, a standard number, which is the number of products that are detected in an image of a product shelf when the display state meets a standard; and a threshold determination means that determines, for each product, a shortage determination threshold, which is a threshold for determining that the product is in a shortage state, based on the standard number.

Images