Information processing device, method, and program

Abstract

The purpose of the present invention is to improve the accuracy of shelf allocation information. This information processing device has a determination unit and a generation unit. When a change in the display state of an article on an article shelf is detected, the determination unit determines whether the change in the display state has been made by a store employee. For example, the change in the display state of an article on the article shelf can be detected by a detection unit such as a weight sensor or a distance sensor. In the case of a weight sensor the change in the display state would be a change in the weight of the article shelf. The generation unit generates shelf allocation information when the change in the display state has been made by a store employee.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and the like. [Background technology]

[0002] There are techniques for creating shelf allocation information for products displayed on a shelf. For example, in the technique described in Patent Document 1, shelf allocation information is created based on images of multiple products.

[0003] In addition, in the technology described in Patent Document 2, when products are sold using an unmanned product sales shelf fixture, the product type and number of products removed from the shelf are identified based on the weight change detected by the shelf and the captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-74228 A [Patent Document 2] JP 2019-208627 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology described in Patent Document 1, there are cases where the shelf allocation information is generated when the products are not displayed on the correct product shelf or in the correct position on the product shelf. Therefore, the technology described in Patent Document 1 has a problem in that the accuracy of the shelf allocation information may be low.

[0006] An example of an objective of the present disclosure is to provide an information processing device or the like that improves the accuracy of shelf allocation information. [Means for solving the problem]

[0007] An information processing device in one aspect of the present disclosure includes a determination means for determining, when a change in the display state of products on a product shelf is detected, whether the change in the display state has been made by a store clerk, and a generation means for generating shelf allocation information if the change in the display state has been made by the store clerk.

[0008] The system in one aspect of the present disclosure includes a detection means for detecting a change in the display state of products on a product shelf, a determination means for determining whether the change in the display state has been made by a store staff member when the change in the display state is detected by the detection means, and a generation means for generating shelf allocation information when the change in the display state has been made by the store staff.

[0009] A method in one aspect of the present disclosure, when a change in the display state of products on a product shelf is detected, determines whether the change in the display state has been made by a store clerk, and generates shelf allocation information if the change in the display state has been made by the store clerk.

[0010] In one aspect of the present disclosure, a recording medium non-temporarily records a program that causes a computer to execute a process of generating shelf allocation information when a change in the display state of products on a product shelf is detected, determining whether the change in the display state has been made by a store clerk or not, and if the change in the display state has been made by the store clerk. Effect of the Invention

[0011] According to the present disclosure, it is possible to improve the accuracy of shelf allocation information. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of an example of a configuration of an information processing device according to a first embodiment. [Diagram 2] FIG. 2 is a flowchart of an operation example of the information processing device according to the first embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of a configuration example of a system according to the second embodiment. As shown in FIG. [Figure 4] FIG. 4 is a block diagram of a configuration example of a system according to the second embodiment. As shown in FIG. [Diagram 5] FIG. 5 is an explanatory diagram showing an example of a change in the display state of a product shelf. [Figure 6] FIG. 6 is a sequence diagram of an example of the operation of the system according to the second embodiment. [Figure 7] FIG. 7 is a sequence diagram showing an example of a determination process using an image. [Figure 8] FIG. 8 is a sequence diagram showing an example of a determination process using a change in weight. [Figure 9] FIG. 9 is a sequence diagram showing an example of a process for generating shelf allocation information. [Figure 10] FIG. 10 is a block diagram of a configuration example of a system according to the third embodiment. As shown in FIG. [Figure 11] FIG. 11 is a sequence diagram of an example of a process for generating shelf allocation information according to the third embodiment. [Figure 12] FIG. 12 is a block diagram of a configuration example of a system according to the fourth embodiment. As shown in FIG. [Figure 13] FIG. 13 is an explanatory diagram showing an example of changes in the display state of containers. [Figure 14] FIG. 14 shows an example of the hardware configuration of the system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an information processing device, an information processing method, and a program according to the present disclosure will be described in detail with reference to the drawings. The disclosed technology is not limited to the embodiments.

[0014] (Embodiment 1) FIG. 1 is a block diagram showing an example of a configuration of an information processing device according to a first embodiment. The information processing device 10 generates shelf allocation information for product shelves used for display in a store. The information processing device 10 is capable of communicating with an imaging device that captures images used to generate the shelf allocation information. Alternatively, the information processing device 10 is capable of accessing a storage unit that stores images captured by the imaging device. There is no particular limitation on the type or form of the store.

[0015] A product shelf is, for example, an object capable of displaying products. There is no particular limitation on the characteristics of the product shelf, such as the shape, color, and material of the product shelf. For example, a desk, a container, or the like may be a product shelf as long as it is an object capable of displaying products. The product shelf is capable of detecting a change in the display state of products on the product shelf. The information processing device 10 and the product shelf are capable of communicating with each other. When a change in the display state is detected by the detection unit, the product shelf notifies the information processing device 10 of this fact. This allows the information processing device 10 to recognize when a change in the display state has been detected.

[0016] The product shelf has, for example, a detection unit that detects a change in the display state. Specific examples of the detection unit include a weight sensor, a pressure sensor, a distance sensor, or a combination of these.

[0017] For example, the weight sensor detects a change in the weight on the product shelf, thereby detecting a change in the display state. For example, when a product is newly placed on the product shelf, the weight sensor detects an increase in weight. When a product is removed from the product shelf, the weight sensor detects a decrease in weight. Also, for example, the weight sensor may be able to detect the amount of change in weight, such as an increase or decrease. For example, in the case where allocation information is generated when a product is newly displayed on a product shelf, if the weight sensor detects an increase in weight, it may notify the information processing device 10 of the increase in weight as a change in the display state.

[0018] For example, the pressure sensor detects a change in the display state by detecting a change in pressure on the product shelf. For example, the pressure sensor may detect a change in pressure on a shelf board of the product shelf. Here, the pressure sensor may be capable of detecting the amount of change in pressure. The distance sensor detects a change in the display state by detecting a change in distance to a product displayed on the product shelf. For example, if a new product is placed on the product shelf, the distance sensor can detect the distance between the product and the distance sensor. If a product is removed from the product shelf, the distance sensor can no longer detect the distance between the product and the distance sensor.

[0019] 1, the information processing device 10 includes a determination unit 101 and a generation unit 102. When a change in the display state of products on a product shelf is detected, the determination unit 101 determines whether the change in the display state is caused by a store clerk. As described above, when a weight sensor is used on the product shelf, the change in the display state may be a change in weight. The change in weight may also be an increase in weight.

[0020] Specifically, the determination unit 101 receives a notification of a change in the display state from, for example, a product shelf. Then, the determination unit 101 determines, for example, whether the change in the display state is a change made by a store clerk. This notification may include, for example, the amount of change. There are various methods for the determination. This determination method is not particularly limited. For example, the determination method may be a method using an image, a method using a change in weight on the product shelf, or a method using a beacon device. Note that the method using the change in weight on the product shelf is implemented when a weight sensor is provided on the product shelf. Details of each determination method will be described later in the second embodiment.

[0021] The generating unit 102 generates the shelf allocation information when the change in the display state is caused by a store clerk. The method of generating the shelf allocation information may be any existing technology. The method of generating the shelf allocation information is not particularly limited. The specific database structure and storage contents of the shelf allocation information are also not particularly limited. For example, the shelf allocation information registers, for each product, product identifiers such as product name and product code, the position of the product on the product shelf, and the face of the product. The position of the product on the product shelf may be indicated by, for example, the shelf level number or slot number. For example, the shelf level number is a number indicating, for example, which level from the top of the product shelf. Also, the slot number is a number when the arrangement position of the product on a certain shelf is 1, 2, 3, etc. from the left. It is assumed that one product is arranged in one slot. For example, when the shelf level number is 1 and the slot number is 2, the product second from the left on the top shelf is indicated on the product shelf. The face of the product is, for example, the orientation of the product. The face of a product is, for example, information such as which side of a product is visible in an image of a product shelf viewed from the front. Alternatively, the face of a product is, for example, information such as whether the surface of a product is visible in an image of a product shelf viewed from the front. Furthermore, the number of products displayed may be registered in the shelf allocation information. Furthermore, the condition of a product may be registered in the shelf allocation information. The condition of a product may be the condition of the contents of the product. An example of the condition of the contents of a product is the expiration date of the product. Furthermore, the condition of a product may be the visual condition of the product. An example of the visual condition of a product is whether or not the product box is crushed, whether or not the product box is dirty, etc.

[0022] For example, the generation unit 102 identifies these pieces of information from the captured image. As a result, the generation unit 102 outputs the identification result as shelf allocation information. The image here is, for example, an image of a product shelf. The image of a product shelf is, for example, an image from which the display state of the product shelf can be determined. Specifically, for example, the image of a product shelf is an image including the entire product shelf as viewed from the front. Note that the image of the product shelf is not particularly limited as long as the shelf allocation information can be generated.

[0023] Fig. 2 is a flowchart showing an example of an operation of the information processing device 10 according to the first embodiment. Here, the processing results of each step by the information processing device 10 in Fig. 2 are stored in a storage unit accessible by the information processing device 10, such as a storage unit.

[0024] First, when a change in the display state is detected, the information processing device 10 determines whether the change in the display state is due to a store clerk (step S101). As described above, the method of determination is not particularly limited. If the change in the display state is due to a store clerk (step S101: Yes), the information processing device 10 generates shelf allocation information (step S102). Then, after step S102, the information processing device 10 ends the operation of the flow. If the change in the display state is not due to a store clerk (step S101: No), the information processing device 10 ends the operation of the flow.

[0025] Next, the effects of the first embodiment will be described. The shelf allocation information is used to check the display state, create shelf allocation proposals, analyze and predict sales, and so on. In particular, when a store does not have a fixed product registration device (register), it is important to register the shelf allocation of which products are displayed in which position on which product shelf. When the shelf allocation information is manually registered by a store clerk, it takes a lot of time for the store clerk. On the other hand, when the shelf allocation information is automatically generated, depending on the timing of generation, the shelf allocation information may be generated in a state where the products are not in the correct position. For example, the display state on the product shelf may change depending on the store clerk or the customer. For example, the store clerk displays a new product on the product shelf. Also, the store clerk removes the product from the product shelf. The removed product may be, for example, a product that has passed its expiration date or a product with a damaged box. On the other hand, when a customer purchases a product, the customer takes the product from the product shelf. Also, when a customer checks a product, the customer takes the product from the product shelf. Also, a customer may return a product that he or she once took away to the product shelf. However, customers do not necessarily return products to the positions on the product shelves where they were originally displayed. Also, when customers take products from the product shelves or return them, the display state may become disorganized. In this way, a change in the display state due to the display work of a store clerk is more likely to be accurate than a change in the display state due to a customer. Therefore, the information processing device 10 generates shelf allocation information when the change in the display state of products on the product shelves is a change made by a store clerk. This allows the information processing device 10 to generate shelf allocation information in which products are displayed more correctly on the product shelves. Therefore, the information processing device 10 can improve the accuracy of the shelf allocation information.

[0026] Furthermore, a change in the display state of a product on a product shelf is a change in weight on the product shelf. This allows the change in the display state of the products to be detected with high accuracy. As a result, the information processing device 10 can improve the accuracy of the shelf allocation information.

[0027] Furthermore, changes in weight include weight increases and weight decreases. In the case of an increase in weight, there is a possibility that a new product has been put on the shelves. When an increase in weight is detected, the information processing device 10 determines whether the increase in weight is a change made by a store clerk. In this way, the information processing device 10 can generate shelf allocation information when it is presumed that a new product has been put on the shelves by a store clerk.

[0028] (Embodiment 2) Next, the second embodiment will be described in detail with reference to the drawings. Below, the description of the second embodiment will be omitted for the sake of clarity, as long as the description is not unclear.

[0029] Fig. 3 is an explanatory diagram showing a configuration example of a system according to a second embodiment. In Fig. 3, the system 2 has a product shelf 21, an information processing device 20, and an imaging device 22. Products are displayed on the product shelf 21. The imaging device 22 is a device capable of capturing images or moving images. The information processing device 20 generates shelf allocation information. The product shelf 21, the imaging device 22, and the information processing device 20 can be connected via a communication network or the like.

[0030] A plurality of image capturing devices 22 may be provided, such as image capturing devices 22-1 to 22-3. A plurality of product shelves 21 may be provided, such as product shelves 21-1 to 21-2.

[0031] 4 is a block diagram showing a configuration example of a system 2 according to the embodiment 2. In FIG. 4, the system 2 includes an information processing device 20, a product shelf 21, and an imaging device 22.

[0032] The imaging device 22 captures an image. The product shelf 21 has a detection unit 220. As described above, the detection unit 220 is a weight sensor, a distance sensor, and a pressure sensor. A plurality of detection units 220 may be provided. Also, as described above, for example, the detection unit 220 may be a combination of at least two sensors of a weight sensor, a distance sensor, and a pressure sensor.

[0033] FIG. 5 is an explanatory diagram showing an example of a change in the display state of the product shelf 21. Two products "A1" are displayed on the product shelf 21. After that, a new product "A1" is placed on the product shelf 21. Therefore, a total of three products "A1" are displayed on the product shelf 21. The detection unit 220 detects, for example, that a new product "A1" has been placed on the product shelf 21 as a change in the display state. Then, the detection unit 220 notifies the information processing device 20 of the change in the display state via the communication network.

[0034] For example, when a weight sensor is used as the detection unit 220, the weight sensor detects a change in weight on the product shelf 21. In FIG. 5, an increase in the weight of one product "A1" is detected. The detection unit 220 notifies the information processing device 20 of the change in weight and the amount of change in weight via the communication network. Furthermore, the detection unit 220 may notify the information processing device 20 of the change in weight and the amount of change in weight via the communication network. Furthermore, the detection unit 220 may notify the change by attaching a shelf identifier or an identifier of the detection unit 220. In this way, when there are multiple product shelves 21, the information processing device 20 can identify which product shelf 21 the change in display state is on. Furthermore, when multiple detection units 220 or multiple types of detection units 220 are provided on the product shelf 21, the information processing device 20 can identify which detection unit 220 the change in display state is on.

[0035] Furthermore, the detection unit 220 may detect an increase or decrease in weight. In the case of an increase in weight, it is highly likely that a new product has been put on display. For this reason, when an increase in weight is detected, the detection unit 220 may notify the information processing device 20 of the increase in weight and the amount of increase via the communication network. For example, in FIG. 5, the detection unit 220 detects an increase in weight equivalent to the weight of a newly put on display product "A1". Then, the detection unit 220 notifies the information processing device 20 of the increase in weight and the amount of increase (the weight of the product "A1").

[0036] Furthermore, the detection unit 220 may notify the information processing device 20 of the increase in weight of the product when an increase of a predetermined weight (weight of the product) is detected. Alternatively, the weight sensor may notify the information processing device 20 when an increase in weight based on the weight of the product and a predetermined number is detected. For example, the predetermined number may be set in advance by a store clerk or a user of the information processing device 20. Alternatively, the detection unit 220 may notify the information processing device 20 of the increase in weight when, for example, the amount of increase in weight exceeds a threshold value.

[0037] Returning to the description of FIG. 4, the information processing device 20 has, for example, a determination unit 201, an identification unit 203, a generation unit 202, and a storage unit 210. The information processing device 20 according to the second embodiment is obtained by adding the identification unit 203 and the storage unit 210 to the information processing device 10 according to the first embodiment. The determination unit 201 has the function of the determination unit 101 according to the first embodiment. The generation unit 202 has the function of the generation unit 102 according to the first embodiment. First, the storage unit 210 stores the processing results of each function of the information processing device 20. The storage unit 210 also stores data used for processing by each unit. In FIG. 4, the storage unit 210 stores shelf allocation information as the storage contents. The storage contents of the storage unit 210 in FIG. 4 are an example. For example, the shelf allocation information may be stored in a storage unit of another device such as a storage device connected via a communication network or the like.

[0038] Moreover, the information processing device 20 only needs to be able to access various data. Therefore, similar to the shelf allocation information, various data may be stored in a storage device or the like that the information processing device 20 can access via a communication network. As will be described in detail with reference to an example of a hardware configuration shown in FIG. 14, the information processing device 20 loads a program including various commands for implementing each functional unit into the storage unit 210 and executes the program. Examples of the storage unit 210 include a read only memory (ROM), a random access memory (RAM), a semiconductor memory, a hard disk drive (HDD), and a solid state drive (SSD). Moreover, the storage unit 210 may be a combination of these.

[0039] When a change in the display state on the product shelf 21 is detected, the determination unit 201 determines whether the change in the display state is due to a store clerk. For example, in the case where shelf allocation information is generated when a new product is displayed, if the detection unit 220 is a weight sensor, the determination unit 201 may determine whether the change in the display state is due to a store clerk when an increase in weight on the product shelf 21 is detected. As a result, as described in the first embodiment, the information processing device 20 can generate shelf allocation information when there is a high possibility that a new product has been displayed on the product shelf. Furthermore, the determination unit 201 may determine whether the change in the display state is due to a store clerk when an increase in the weight of the product on the product shelf 21 is detected. When the detection unit 220 has a function of comparing weight, the detection unit 220 may compare the amount of increase in weight with the weight of the product, etc. Alternatively, the information processing device 20 may determine whether an increase in the weight of the product has been detected by comparing the amount of increase in weight notified by the detection unit 220 with the weight of the product.

[0040] As described in the first embodiment, there are various methods by which the determining unit 201 can determine whether the change has been caused by a store clerk. This determination method is not particularly limited. Examples of the determination method include a method using an image, a method using a change in weight on the product shelf 21, and a method using a beacon device. The method using a change in weight on the product shelf 21 is implemented when a weight sensor is provided on the product shelf 21.

[0041] <Determination method using images> First, a determination method using an image will be described. The determination unit 201 determines that the change in the display state is due to the store clerk by identifying the store clerk from the captured image. The image is captured by, for example, the imaging device 22. For example, when a change in the display state is detected, the image is captured by the imaging device 22 of the product shelf 21 or the vicinity thereof such as in front of the product shelf 21. The image may be an image of a person in the vicinity of the product shelf 21 and a part or the whole of the product shelf 21. Therefore, the image may be an image used for generating the shelf allocation information. The timing of capturing images by the imaging device 22 is not particularly limited. For example, the imaging device 22 may capture a moving image at all times. Alternatively, the imaging device 22 may capture an image when an instruction to capture images is notified from the information processing device 20. The imaging device 22 stores the captured image in the storage unit 210 or in the storage unit 210 accessible by the information processing device 20. Alternatively, the imaging device 22 may send the image to the determination unit 201. Moreover, the vicinity of the product shelf 21 means, for example, the vicinity of the front side where products can be displayed on the product shelf.

[0042] There are various methods for determining whether or not an image is used. Examples of such methods include biometric authentication, image matching of uniforms, and traffic flow. First, biometric authentication will be described. Here, biometric authentication is authentication using biometric features that can be identified from an image. An example of biometric authentication is face authentication. The determination unit 201 determines whether or not a person is a store clerk by identifying the face of the person from a captured image. Specifically, for example, face data of the store clerk is stored in advance in the storage unit 210. Then, the determination unit 201 may determine whether or not the person is a store clerk by matching face data extracted from the image with the stored face data. If the person is a store clerk, the determination unit 201 determines that the change in the display state is a change in the display state caused by the store clerk.

[0043] Next, image matching of uniforms and the like will be described. The subject of image matching may be anything that is carried or worn by a store clerk and can be identified from an image. For example, this may be a uniform, a name tag, a store clerk ID, etc. Here, a uniform is taken as an example. The determination unit 201 identifies the uniform from the captured image. If the determination unit 201 identifies the uniform, it determines that the change in the display state is a change in the display state caused by a store clerk.

[0044] Finally, a case where a flow line is used will be described. When a change in the display state is detected from a system or device that specifies the flow line of each person (herein referred to as a flow line specification system), the determination unit 201 determines whether a person who was near the product shelf 21 is a store clerk. Note that the system 2 may have the function of a flow line specification system or device that specifies the flow line of each person. For example, in a flow line specification system, images or videos are used to specify the flow line of each person. In a flow line specification system, for example, at the entrance of a store, a group passing through the entrance is treated as a person. Then, in the flow line specification system, when the group is identified as a store clerk by a store clerk ID or the like, the group is subsequently treated as a store clerk. For this reason, the determination unit 201 determines whether a person who is near the product shelf 21 is a store clerk based on the flow line of each person when a change in the display state at the product shelf 21 is detected. For example, if the person is a store clerk, the determination unit 201 determines that the change in the display state is a change in the display state caused by the store clerk.

[0045] <Weight-based determination method> Next, a method of using a change in weight will be described. This determination method is based on the premise that a weight sensor is provided as the detection unit 220 on the product shelf 21.

[0046] The determination unit 201 determines that a weight change is due to a store clerk, based on a series of weight changes on the product shelf 21 within a predetermined time from when the weight change on the product shelf 21 is detected. The predetermined time is a time designated by a store clerk or a user of the information processing device 20. Alternatively, the predetermined time may be a time learned from experience or the like.

[0047] The series of weight changes is a time-series weight change. The series of weight changes may be, for example, a transition of the amount of weight change over time. A characteristic pattern that may occur due to a change caused by a store clerk is estimated for the series of weight changes. For example, in a display operation such as stocking by a store clerk, expired products may be removed from the product shelf 21, and products within the expiration date may be displayed. For example, a case where an action such as taking two products and placing three identical products is performed within a short time (predetermined time) is taken as an example. In this case, the series of weight changes may be, for example, a decrease in weight of two products and an increase in weight of three products within the predetermined time. Therefore, for example, one or more patterns that are estimated to be a change in the display state caused by a store clerk may be prepared in advance. Then, when the series of weight changes matches the pattern, the determination unit 201 determines that the change in the display state is a change caused by a store clerk.

[0048] Furthermore, the determination unit 201 determines that the change in weight is due to a store clerk based on the cumulative change in weight on the product shelf 21 within a predetermined time from when the change in weight on the product shelf 21 is detected. For example, in a display operation such as stocking by a store clerk, products may be arranged on the product shelf 21 in units of several to several tens of products. On the other hand, it is unlikely that a customer will return a large number of products, such as several tens of products, to the product shelf 21. Therefore, when the cumulative change is a predetermined weight, the determination unit 201 determines that the change in the display state is due to a store clerk. The predetermined weight may be a weight designated by a store clerk or a user of the information processing device 20. The predetermined weight may be a multiple of the weight of the product, or a weight including an error in the multiple. The multiple of the weight of the product may be obtained, for example, by multiplying the weight of the product by a predetermined number. For example, the predetermined number is a number designated by a store clerk or a user of the information processing device 20. There may be a plurality of predetermined weights. For example, a plurality of predetermined numbers may be prepared, thereby preparing a plurality of predetermined weights. Furthermore, when the accumulated change amount is equal to or greater than a predetermined weight, the determining unit 201 may determine that the change in the display state is a change in the display state made by a store clerk.

[0049] Furthermore, the determination unit 201 may determine whether the weight change is due to a store clerk depending on whether the amount of weight change on the product shelf 21 is within a predetermined range. The predetermined range may be determined by the store clerk or the user of the information processing device 20. The predetermined range may be determined by the weight of the product displayed on the product shelf 21. For example, when canned beverages are individually displayed on the product shelf 21, if the amount of weight change is outside the range estimated as the weight of one can, there is a high possibility that the product is displayed incorrectly. For example, when the amount of weight change is 5 kilograms on the product shelf 21 where cans of about 350 ml to 500 ml are displayed, there is a high possibility that the customer placed the product on the wrong product shelf 21. Therefore, when the amount of weight change on the product shelf 21 is not within a predetermined range, the determination unit 201 determines that the weight change is not due to a store clerk. Furthermore, when the amount of weight change on the product shelf 21 is within a predetermined range, the determination unit 201 determines that the weight change is due to a store clerk. Alternatively, when the amount of change in weight on the product shelf 21 is within a predetermined range, the determination unit 201 may further determine whether the change was caused by a store clerk using another determination method.

[0050] <Determination method using beacon devices, etc.> Finally, as a method of determination by the determination unit 201, a determination method using a beacon device or the like will be described. When a change in the display state on the product shelf 21 is detected, the determination unit 201 determines whether the change in the display state is due to a store clerk, depending on whether the device that wirelessly communicates with the beacon device is the store clerk's device. For example, the beacon device is provided on the product shelf 21. There is no particular limitation as to which of the beacon device and the store clerk's device transmits a beacon signal. Here, the beacon device and the information processing device 20 are capable of communicating with each other via a network or the like. Note that the beacon device and the information processing device 20 may be the same device. For example, a case where a beacon signal is transmitted from the store clerk's device will be described. Specifically, for example, when a change in the display state on the product shelf 21 is detected, if the beacon device receives a beacon signal from the store clerk's device, the determination unit 201 determines that the change in the display state is due to a store clerk. When a change in the display state on the product shelf 21 is detected, if the beacon device does not receive a beacon signal, the determination unit 201 determines that the change in the display state is not a change made by a store clerk.

[0051] This concludes the description of each determination method by the determination unit 201. The determination methods may be implemented individually or in combination. For example, when it is determined that the change was caused by a store clerk using a plurality of determination methods, the determination unit 201 may determine that the change in the display state was caused by a store clerk. Furthermore, the determination unit 201 may switch between various determination methods. For example, when it is not possible to make a determination using one of the plurality of determination methods, the determination unit 201 may determine whether the change in the display state was caused by a store clerk using the other determination method.

[0052] Next, a description will be given of the discrimination unit 203 and the generation unit 202. The generation unit 202 has the functions of the generation unit 102 described in the first embodiment. The discrimination unit 203 is a functional unit newly added in the second embodiment.

[0053] Here, the explanation is based on the premise that only identical products are displayed on each product shelf 21 or each shelf board of the product shelf 21. Even a store clerk may mistakenly display a product. Therefore, the information processing device 20 generates shelf allocation information based on the identity of the products displayed on the product shelf 21.

[0054] First, when a change in the display state is detected, the identification unit 203 identifies the products displayed on the product shelf 21 based on the image of the display state. The method of identifying the products may use existing technology. For example, the identification unit 203 identifies the shape, name, color, barcode, and code of the product from the image of the display state. Here, for example, the product "A1" and the product "A2" are regarded as different products when they are the same type (same contents) but have different flavors or quantities. Also, here, for example, a 180 ml melon-flavored juice and a 180 ml strawberry-flavored juice manufactured by the same manufacturer are regarded as different products. Also, here, a 350 ml canned beer and a 500 ml canned beer manufactured by the same manufacturer and have the same contents are regarded as different products. The difference between these products is identified, for example, by the shape, name, and product code of the product described above. Alternatively, the same product may be defined as being identifiable, for example, by the name or code of the product.

[0055] Next, the generating unit 202 generates shelf allocation information. The generated shelf allocation information is stored in the storage unit 210 or in a storage unit of another device accessible via a communication network. Here, the generating unit 202 will be described with reference to several generation examples.

[0056] <Generation example 1> When the products identified by the identification unit 203 are the same, the generation unit 202 generates the shelf allocation information for the displayed products. On the other hand, when the products identified by the identification unit 203 are different, the generation unit 202 does not generate the shelf allocation information for the displayed products. Take the product shelf 21 on which melon-flavored juice is displayed as an example. When three melon-flavored juices are identified from an image of the display state on the product shelf 21, the generation unit 202 generates the shelf allocation information for the melon-flavored juice. On the other hand, for example, when two melon-flavored juices and one strawberry-flavored juice are identified from an image of the display state on the product shelf 21, the generation unit 202 does not generate the shelf allocation information. This allows the information processing device 20 to generate shelf allocation information when the product shelf 21 is correctly displayed by a store clerk.

[0057] <Other generation examples> For example, the generation part 2 02 is the identification part 2When a plurality of individual products are identical among the products displayed in the display unit 203, the shelf allocation information for the same product may be generated. A case will be described in which two melon-flavored juices are identified and one strawberry-flavored juice is identified. The generating unit 202 generates shelf allocation information for the melon-flavored juice. On the other hand, the generating unit 202 does not generate shelf allocation information for the strawberry-flavored juice. In this way, if the same product is displayed multiple times, the information processing device 20 can generate shelf allocation information for the product, assuming that the product is in a correct display state. On the other hand, the generating unit 202 may not generate shelf allocation information for a product that does not have an identical product among the products identified by the identifying unit 203. In this way, the information processing device 20 can not generate shelf allocation information for a product that may have been displayed incorrectly. In addition, when it is determined in advance which product is displayed on the product shelf 21 based on past shelf allocation information, product information, etc., the identifying unit 203 identifies the predetermined product, for example, from an image of the display state. Then, the generating unit 202 generates shelf allocation information for a predetermined product. An example will be described in which it has been determined that melon-flavored juice will be displayed on the product shelf 21. For example, if two melon-flavored juices and three strawberry-flavored juices are displayed on the product shelf 21, the identifying unit 203 identifies the two melon-flavored juices from an image of the display state. Then, the generating unit 202 generates shelf allocation information for the two melon-flavored juices. In this way, when it has been determined in advance which product will be displayed on the product shelf 21, the information processing device 20 can generate shelf allocation information for this product.

[0058] In addition, when it is determined which category of products is to be displayed on the product shelf 21, the information processing device 2The generating unit 202 may carry out the same processing as in each of the generation examples described above. The categories can be changed in various ways. For example, juices of the same amount produced by the same manufacturer may be placed in the same category. Or beer cans may be placed in the same category regardless of the manufacturer or the amount. Also, beverages as a broad category may be placed in the same category. When the categories of the products identified by the identifying unit 203 are the same, the generating unit 202 generates shelf allocation information for the displayed products. Also, for example, the information processing device 2 0 does not generate allocation information when the categories of the products identified by the identification unit 203 are not the same. This completes the explanation of the generation example.

[0059] 6 is a sequence diagram relating to an example of the operation of the system 2 according to the second embodiment. The product shelf 21 detects a change in the display state by the detection unit 220 (step S201). Then, the product shelf 21 notifies the information processing device 20 of the change in the display state (step S202). In step S202, the product shelf 21 may also notify the amount of change, etc. For example, if the detection unit 220 is a weight sensor, the product shelf 21 may notify the amount of change in weight.

[0060] Next, the information processing device 20 receives a notification of a change in the display state. Then, the information processing device 20 performs a determination process (step S203). The determination process may be performed using the various determination methods described above. The information processing device 20 determines whether the change in the display state was caused by a store clerk (step S204). Although step S204 is a part of the determination process, step S204 is a separate step from step S203 for ease of explanation and understanding.

[0061] If the change in the display state is due to a store clerk (step S204: Yes), the information processing device 20 performs a process of generating shelf allocation information (step S205). Here, in step S205, the process of generating shelf allocation information includes a process of identifying products by the identification unit 203. After step S205, the information processing device 20 ends the operation of the flow. If the change in the display state is not due to a store clerk (step S204: No), the information processing device 20 ends the operation of the flow.

[0062] Next, an example of the determination process (step S203) shown in Fig. 6 will be described with reference to Fig. 7 and Fig. 8. The determination process using an image will be described with reference to Fig. 7. The determination process using the cumulative change in weight will be described with reference to Fig. 8. Then, an example of the generation process (step S205) shown in Fig. 6 will be described with reference to Fig. 9.

[0063] FIG. 7 is a sequence diagram showing an example of a determination process using an image. First, in FIG. 7, the information processing device 20 notifies the imaging device 22 of an imaging instruction (step S2001). Next, the imaging device 22 captures an image including the product shelf 21 and a person or the like in front of the product shelf 21 (step S2002). For example, in step S2002, the imaging device 22 may capture images from a plurality of angles. In FIG. 9 described later, an image of the display state is used to generate shelf allocation information. For this reason, in step S2002, the imaging device 22 may capture an image of the display state and an image including the periphery of the surrounding shelves.

[0064] next, Imaging device 2 2The captured image is transmitted (step S2003). Then, the information processing device 20 acquires the image (step S2004). Next, the information processing device 20 identifies the store clerk from the image (step S2005). After step S2005, the information processing device 20 returns to the original flow. Specifically, the information processing device 20 proceeds to step S204 following the determination process (step S203) in FIG. 6. This step S2005 and step S204 shown in FIG. 6 may be one step. Also, as for the method of determination based on the image in step S2005, as described above, biometric authentication, matching of traffic flow, uniforms, etc. can be mentioned.

[0065] FIG. 8 is a sequence diagram showing an example of a determination process using a change in weight. First, in FIG. 8, the information processing device 20 determines whether a predetermined time has elapsed since a change in the display state was detected (step S2011). If the predetermined time has not elapsed (step S2011: No), the information processing device 20 returns to step S2011. In this way, a series of changes in the display state within the predetermined time from the detection of the change in the display state is obtained. If the predetermined time has elapsed (step S2011: Yes), the information processing device 20 calculates a predetermined weight based on a predetermined number and the weight of the products (step S2012). Note that step S2012 may be a step of acquiring a predetermined weight calculated in advance from the storage unit 210 or the like. Then, the information processing device 20 calculates an accumulated change amount based on a series of weight changes (step S2013). For example, in step S2013, the information processing device 20 can calculate an accumulated change amount by summing up a series of weight changes. Then, the information processing device 20 compares the accumulated change amount with a predetermined weight (step S2014). After step S2014, the information processing device 20 returns to the originating flow. Specifically, the information processing device 20 proceeds to step S204 following step S203 in FIG.

[0066] FIG. 9 is a sequence diagram showing an example of the process of generating shelf allocation information. In FIG. 9, the above-mentioned generation example 1 will be described. As described above, it is assumed that the same products are displayed. First, in FIG. 9, the information processing device 20 issues an image capture instruction to the image capture device 22 (step S2101). Next, the image capture device 22 captures an image of the display state of the product shelf 21 (step S2102). Next, Imaging device 2 2 The captured image is transmitted (step S2103). Then, the information processing device 20 acquires the image (step S2104). The information processing device 20 identifies the product displayed on the product shelf 21 from the image (step S2105). Then, the information processing device 20 determines whether different products are displayed (step S2106). If different products are displayed (step S2106: Yes), the information processing device 20 ends the operation of the flow. For example, if two melon-flavored juices and one strawberry-flavored juice are displayed, no shelf allocation information is generated. As a result, if different products are displayed on the product shelf 21, no shelf allocation information is generated. Note that since there is no step following step S205 in FIG. 6, if the answer to step S2106 is Yes, the information processing device 20 ends the operation of the flow.

[0067] If no different product is displayed (step S2106: No), the information processing device 20 generates shelf allocation information (step S2107). Then, after step S2107, the information processing device 20 ends the operation of the flow. For example, if three melon-flavored juices are displayed, shelf allocation information regarding the melon-flavored juice is generated.

[0068] When the determination method by the determination unit 201 uses the image shown in Fig. 7, steps S2101 to S2104 in Fig. 9 may be steps S2001 to S2004 shown in Fig. 7. In this case, the information processing device 20 may start the flow from S2105 in the generation process (step S205).

[0069] Next, the effects of the second embodiment will be described. As in the first embodiment, the information processing device 20 generates shelf allocation information when a change in the display state of products on the product shelves is caused by a store clerk. This allows the information processing device 20 to generate shelf allocation information in a state in which products are displayed more correctly on the product shelves. Furthermore, the information processing device 20 can suppress the generation of shelf allocation information at a time when the products are not displayed correctly. Therefore, the information processing device 20 can improve the accuracy of the shelf allocation information. First, the description in the first embodiment will be omitted, and the effects of each determination method described in the second embodiment will be described.

[0070] Next, the effect of the determination method using images will be described. The information processing device 20 determines whether a change in the display state is due to a store clerk by identifying the store clerk from the image. In addition, existing technologies such as biometric authentication, image matching of uniforms, and traffic flow can be used. This allows the information processing device 20 to accurately determine whether a change in the display state is due to a store clerk. In addition, the information processing device 20 acquires an image from the imaging device 22 when generating the shelf allocation information. Therefore, by using an image to make a determination, the information processing device 20 does not need to add new components for the determination.

[0071] Next, the effect of the determination method based on a change in weight will be described. For example, the pattern of weight changes differs between the display of products by a store clerk and the return or removal of products by a customer. Therefore, the information processing device 20 determines whether the change in the display state is due to a store clerk based on a series of weight changes within a predetermined time from when the change in the display state is detected. This makes it possible to accurately determine whether the change in the display state is due to a store clerk, based on the characteristics of the store clerk's display work. Also, when the change in the display state is detected by a weight sensor, the information processing device 20 does not need to add new components for the determination.

[0072] Next, the effect of generation example 1 in the case where only identical products are displayed on the product shelf 21 will be described. When the products identified from the image of the display state are identical, the information processing device 20 generates shelf allocation information for the displayed products. As a result, for example, when three melon-flavored juices are identified from the image of the display state on the product shelf 21, shelf allocation information for the melon-flavored juice is generated. Therefore, the information processing device 20 can generate shelf allocation information when the products are correctly displayed on the product shelf 21 by a store clerk. As a result, the information processing device 20 can generate shelf allocation information with higher accuracy.

[0073] (Embodiment 3) Next, the third embodiment will be described in detail with reference to the drawings. Below, the description of the third embodiment will be omitted for the sake of clarity, as long as the description of the third embodiment is not unclear.

[0074] FIG. 10 is a block diagram showing an example of a configuration of a system 3 according to the third embodiment. The system 3 corresponds to the system 2 according to the second embodiment. The system 3 has an information processing device 30, a product shelf 31, and an imaging device 32. The product shelf 31 has a detection unit 320. The product shelf 31 is the same as the product shelf 21 according to the second embodiment. Therefore, a description of the product shelf 31 will be omitted. Furthermore, the imaging device 32 is the same as the imaging device 22 according to the second embodiment. A description of the imaging device 32 will be omitted.

[0075] The information processing device 30 corresponds to the information processing device 10, 20 according to the first and second embodiments. The information processing device 30 includes a judgment unit 301, a classification unit 303, an output unit 304, a generation unit 302, and a storage unit 310. In the third embodiment, the information processing device 30 newly includes an output unit 304. The judgment unit 301 includes the functions of the judgment units 101, 201 according to the first and second embodiments. The generation unit 302 includes the functions of the generation units 102, 202 according to the first and second embodiments. The classification unit 303 includes the function of the classification unit 203 according to the second embodiment. The storage unit 310 may be the same as the storage unit 210 according to the second embodiment, for example.

[0076] In the third embodiment, differences from the second embodiment will be described. In the second embodiment, generation example 1 was implemented on the assumption that only identical products are displayed on the product shelf 31. In the third embodiment, in this case, if a different product is displayed on the product shelf 31, the store clerk is notified of this.

[0077] When the multiple products acquired by the identification unit 303 are different, the output unit 304 notifies the store clerk of the fact. The method of notifying the store clerk is not particularly limited. Examples of the notification method include sending an email or an electronic message, and outputting to an output device. For example, the output unit 304 outputs the fact that the products are different to an output device of the information processing device 30. Alternatively, for example, the output unit 304 may notify the fact that the products are different to a store clerk's device. The store clerk's device is a store management terminal device or a store clerk's terminal device. Alternatively, for example, the output unit 304 may notify an output device or the like provided on the product shelf 31 of the fact that the products are different. For example, when two melon-flavored juices and one strawberry-flavored juice are identified from an image of the display state of the product shelf 31, the multiple products identified by the identification unit 303 are different. Also, for example, if one melon-flavored juice and one strawberry-flavored juice are identified from the image of the display state of the product shelf 31, the multiple products identified by the identification unit 303 are different. If the identification results are as described above, the output unit 304 notifies the store clerk accordingly. This enables the information processing device 30 to inform the store clerk that the display of the product shelf 31 by the store clerk may not be accurate. On the other hand, if three melon-flavored juices are identified from the image of the display state of the product shelf 31, they are all the same. For this reason, the output unit 304 does not notify the store clerk.

[0078] Furthermore, the third embodiment can be modified in various ways. For example, the second embodiment and the third embodiment may be implemented in combination. For example, the first generation example and the notification by the store clerk via the output unit 304 may be implemented in combination. For example, if there is a different product on the product shelf 31, the information processing device 30 notifies the store clerk. On the other hand, if there is no different product on the product shelf 31, the information processing device 30 generates shelf allocation information related to the identified product. An example of implementing the combination is shown in the flowchart of FIG. 11.

[0079] In addition, in a case where it is determined that products of the same category are displayed on a product shelf for each product category, if a product of a different category is identified from the image, the output unit 304 may notify the store clerk of this.

[0080] Also, for example, when the amount of change in weight on the product shelf 21 is not within a predetermined range, the output unit 304 may notify the store clerk to that effect. In this way, when an abnormal display state is detected, the output unit 304 may also notify the store clerk to that effect.

[0081] In the third embodiment, when a plurality of individual products are the same, candidates for shelf allocation information for the same product may be generated and notified to a store clerk. Then, the store clerk may decide whether to adopt the candidates for shelf allocation information. Also, the store clerk may check the product shelves 31 by looking at the candidates for shelf allocation information. Also, the store clerk may modify the candidates for shelf allocation information. Specifically, the generating unit 302 generates candidates for shelf allocation information for the same product when a plurality of individual products are the same among the displayed products acquired by the identifying unit 303. Then, the output unit 304 notifies the store clerk of the generated candidates for shelf allocation information. The notification method here is not particularly limited. For example, a method similar to the method of informing the store clerk may be adopted as the notification method. Also, the information processing device 30 may receive from the store clerk whether or not to adopt the candidates for shelf allocation information. For example, when it is accepted that the candidates for shelf allocation information are to be adopted, the candidates for shelf allocation information are stored in the storage unit 310 as shelf allocation information. When it is accepted that the candidates for shelf allocation information are not to be adopted, the candidates for shelf allocation information may be discarded. In addition, the candidates for the shelf allocation information may be modified by a store clerk. For example, the storage unit 310 stores the modified candidates for the shelf allocation information as the shelf allocation information. To realize such a function, the information processing device 30 may further have functional units such as a selection unit and a reception unit, which are not shown. For example, when two melon-flavored juices and one strawberry-flavored juice are identified from an image of the display state, the generation unit 302 generates candidates for the shelf allocation information regarding the melon-flavored juice. On the other hand, the generation unit 202 does not generate candidates for the shelf allocation information regarding the strawberry-flavored juice. Then, the output unit 304 notifies the store clerk of the candidates for the shelf allocation information regarding the melon-flavored juice.

[0082] Next, an example of the operation of the system 3 according to the third embodiment will be described. The sequence diagram may be the same as that shown in FIG. 6 of the second embodiment, and therefore the description will be omitted. However, the process of generating the shelf allocation information will be described with reference to FIG. 11.

[0083] Fig. 11 is a sequence diagram showing an example of a process for generating shelf allocation information according to the third embodiment. Fig. 11 shows an example in which the generation example 1 described in the second embodiment is combined with the function of the output unit 304. The process for generating shelf allocation information shown in Fig. 6 (step S205) described above may be replaced with the process for generating shelf allocation information shown in Fig. 11 (step S305).

[0084] The sequence diagram shown in Fig. 11 differs from the sequence diagram shown in Fig. 9 in that step S3107 has been added. In Fig. 11, steps S3101 to S3106 are the same as steps S2101 to S2106, respectively, shown in Fig. 9. Here, the description of steps S3101 to S3105 will be omitted.

[0085] After step S3105, the information processing device 30 determines whether different products are displayed (step S3106). If different products are displayed (step S3106: Yes), the information processing device 30 notifies a store clerk that different products are displayed (step S3107). After step S3107, the information processing device 30 ends the operation of the flow. On the other hand, if different products are not displayed (step S3106: No), the information processing device 30 generates shelf allocation information (step S3108). After step S3108, the information processing device 30 ends the operation of the flow.

[0086] Next, the effects of the third embodiment will be described. For example, there are cases where only identical products are displayed on the product shelf 31. When different products are displayed, it is possible that one of the products has been displayed incorrectly by the store clerk. Therefore, when a different product is identified from an image of the display state, the information processing device 30 notifies the store clerk of the difference. This allows the information processing device 30 to urge the store clerk to display the products more correctly on the product shelf 31. For example, the store clerk can check the display state of the products through this notification. For example, the store clerk can redo the display. Ultimately, the information processing device 30 can improve the accuracy of the shelf allocation information.

[0087] Furthermore, when multiple individual products are identical among the products identified from the image of the display state, the information processing device 30 generates candidates for shelf allocation information for the same products and notifies the store clerk. As a result, when the same product is displayed multiple times, the information processing device 30 can generate candidates for shelf allocation information for the products, assuming that the products are highly likely to be displayed correctly. Then, the information processing device 30 notifies the store clerk of the candidates for shelf allocation information. As a result, the information processing device 30 can inform the store clerk of candidates for shelf allocation information that the store clerk should pay attention to. As a result, the information processing device 30 can improve the accuracy of shelf allocation information. Furthermore, the information processing device 30 can avoid generating shelf allocation information for products that may have been displayed incorrectly.

[0088] (Embodiment 4) An example will be described in which a product shelf is divided into a plurality of regions. The plurality of regions may be divided in any manner. For example, the plurality of regions may be defined by boards or containers. In the case of a board, the regions on the board where products can be placed are of In the case of a container, the area of ​​one container is considered to be one area. Here, an example in which the multiple areas are separated by multiple containers will be described.

[0089] FIG. 12 is a block diagram showing a configuration example of a system according to the fourth embodiment. The system 4 according to the fourth embodiment includes an information processing device 40, a product shelf 41, and an imaging device 42. In the fourth embodiment, a new feature is that the product shelf 41 has a plurality of containers 43-1 to 43-m. In addition, a new feature is that the detection units 420-1 to 420-p are provided for each container 43. Here, m is an integer of 2 or more. p is an integer of 1 or more. m and p may be the same. m and p may have different values. For example, the detection unit 420 may not be provided for all containers 43. In addition, a plurality of detection units 420 may be provided for some containers 43. The detection unit 420 has the same function as the detection units 220 and 320 described above. The detection unit 420 detects a change in the display state of the containers 43.

[0090] The imaging device 42 is the same as the imaging devices 22 and 32 described above. The information processing device 40 corresponds to the information processing devices 10, 20 and 30 according to the first to third embodiments. The components of the information processing device 40 are the same as those of the information processing device 30 according to the third embodiment. Each component of the information processing device 40 has the function of the corresponding component of the information processing device 30.

[0091] Next, a specific process in the fourth embodiment will be described. The generating unit 402 generates shelf allocation information for a container 43 (area) in which a change in the display state has been detected among a plurality of containers 43 (a plurality of areas). A specific example will be described with reference to FIG.

[0092] 13 is an explanatory diagram showing an example of changes in the display state of containers 43. In the example of FIG. 13, product shelf 41 is divided into multiple levels by multiple boards. Products can be placed on each board of product shelf 41. Furthermore, containers 43 are provided on each board. In this way, product shelf 41 is divided by multiple containers 43.

[0093] In the example of Fig. 13, a new product "B" is displayed in container 43-5. Specifically, detection unit 420-5 detects a change in the display state of container 43-5. Then, when a change in the display state of container 43-5 is detected, determination unit 401 determines whether the change in the display state has been made by a store clerk. If the change in the display state of container 43-5 has been made by a store clerk, generation unit 402 generates shelf allocation information for container 43-5.

[0094] Also, for example, it may be determined in advance that the same product is displayed in each of the multiple containers 43. 4 The processing by the identification unit 403 and the generation unit 402 (for example, generation example 1) may be performed for each product shelf 41.

[0095] Each of the generation examples described in the second embodiment may be implemented for each container 43. Here, an example in which the generation example 1 described in the second embodiment is applied to the fourth embodiment will be described with reference to FIG. 13. The identification unit 403 identifies the product displayed in the container 43-5 based on the image of the display state. The image of the display state may be an image of the entire product shelf 41. The image of the display state may also be an image of a part of the product shelf 41 including the container 43-5. When the displayed products acquired by the identification unit 403 are the same, the generation unit 402 generates shelf allocation information for the product "B1" displayed in the container 43-5. On the other hand, when a product different from the product "B1" is displayed in the container 43-5, the generation unit 402 does not generate shelf allocation information.

[0096] Furthermore, the notification to the store clerk described in the third embodiment may be performed for each container 43. For example, assuming that the same products are displayed in the containers 43, when a different product is identified in a container 43 for which a change in the display state has been detected from the image of the display state, the output unit 404 notifies the store clerk accordingly.

[0097] In addition, an example in which the example of notifying the store clerk of the shelf allocation information candidates described in the third embodiment is applied to the fourth embodiment will be described. For example, an example will be described in which the identification unit 403 identifies two products "B1" and one product "B2" different from the product "B1" in the container 43-5. When a plurality of individual products among the products "B1" and "B2" displayed in the container 43-5 are identical, candidates for shelf allocation information for the identical products are generated. That is, the generation unit 402 generates candidates for shelf allocation information for the product "B1" displayed in the container 43-5. On the other hand, the generation unit 402 does not generate candidates for shelf allocation information for the product "B2" displayed in the container 43-5. Then, the output unit 404 notifies the store clerk of the candidates for shelf allocation information.

[0098] Next, the effects of the fourth embodiment will be described. The information processing device 40 generates shelf allocation information for a container 43 (area) where a change in the display state has been detected among a plurality of containers 43 (a plurality of areas). In this way, the information processing device 40 can generate shelf allocation information for each small container 43 (area) that divides the product shelf 41, rather than for the entire product shelf 41. This allows the information processing device 40 to generate shelf allocation information with greater accuracy.

[0099] Furthermore, the containers 43 may be arranged in any manner on the product shelf 41. As shown in Fig. 13, one or more containers 43 may be arranged on each board of the product shelf 41. For example, the containers 43 may be attached to the product shelf 41 so as to be hooked on it.

[0100] The first to fourth embodiments may be combined. Moreover, the information processing devices 10, 20, 30, and 40 according to the first to fourth embodiments may be combined with existing technology.

[0101] FIG. 14 shows an example of a hardware configuration of the system. The system 5 includes, for example, an information processing device 50, a product shelf 51, and an imaging device 52. First, a case where the information processing devices 10, 20, 30, and 40 according to the first to fourth embodiments are realized by a computer will be described. The information processing device 50 is, for example, an example where the information processing devices 10, 20, 30, and 40 are realized by a computer. The information processing device 50 includes a CPU (Central Processing Unit) 501, a ROM 502, a RAM 503, a storage device 504, and a communication interface 505. Each component is connected to each other via a bus 506.

[0102] The CPU 501 controls the entire information processing device 50. For example, the CPU 501 may operate an operating system (OS) to control the entire information processing device 50. The CPU 501 may have multiple cores.

[0103] The information processing device 50 has a ROM 502, a RAM 503, a storage device 504, and the like as a storage unit. Examples of the storage device 504 include a semiconductor memory such as a flash memory, a HDD, and an SSD. For example, the storage device 504 stores various programs such as an OS program, an application program, and the programs according to the first to fourth embodiments. Alternatively, the ROM 502 stores an application program. The ROM 502 may also store the programs according to the first to fourth embodiments. The RAM 503 is used as a work area for the CPU 501.

[0104] Furthermore, the CPU 501 loads programs stored in the storage device 504, the ROM 502, or the like. Then, the CPU 501 executes each process coded in the program. The CPU 501 may download various programs via the communication network 510. The CPU 501 functions as a part or the whole of the information processing device 50. Then, the CPU 501 may execute processes or instructions in the illustrated flowchart based on the program.

[0105] The communication interface 505 is connected to a communication network 510 such as a LAN (Local Area Network) or a WAN (Wide Area Network) via a wireless or wired communication line. As a result, the information processing device 50 is connected to an external device or an external computer via the communication network 510. The communication interface 505 serves as an interface between the communication network 510 and the inside of the information processing device 50. The communication interface 505 controls input and output of data from the external device or the external computer.

[0106] However, the hardware configuration shown in FIG. 14 is an example, and other components may be added, or some components may not be included. For example, the information processing device 50 may have a drive device or the like. The CPU 501 may read out programs and data from a recording medium attached to the drive device or the like to the RAM 503. Examples of the recording medium include an optical disk, a flexible disk, a magneto-optical disk, and a USB (Universal Serial Bus) memory. For example, the information processing device 50 may also have input devices such as a keyboard and a mouse. The information processing device 50 may also have an output device such as a display.

[0107] Next, the hardware configuration of the product shelf 51 described in each embodiment will be described. The product shelf 51 has, for example, a sensor 511 (detection unit) and a communication interface 512. The communication interface 512 and each sensor 511 are connected via a bus 513. The sensor can be changed to various types such as a weight sensor, a distance sensor, a pressure sensor, and a human presence sensor. The communication interface 512 is connected to a communication network 510 such as a LAN or a WAN via a wireless or wired communication line.

[0108] As shown in the figure, the product shelf 51 may have a plurality of sensors 511. The number (value of p) and types of the sensors 511 are not particularly limited. The number and types of the sensors 511 can be changed in various ways depending on, for example, each embodiment or each determination method.

[0109] Without being limited to the illustrated example, the product shelf 51 may have a microcomputer, a processor, etc. This allows the information processing device 50 to cause the product shelf 51 to perform simple calculations, etc. The product shelf 51 may also have an output device, an input device, etc. The communication interface 512, the sensor 511, etc. may be built into the product shelf 51. Alternatively, the communication interface 512, the sensor 511, etc. may be externally attached to the product shelf 51. If they are externally attached, each embodiment can be realized even with an existing product shelf 51. Furthermore, these do not have to be in direct physical contact with the product shelf 51.

[0110] Next, an example of the hardware configuration of the imaging device 52 will be described. The imaging device 52 has a camera 521 and a communication interface 522. Each component is connected by a bus 523. The communication interface 522 is connected to a communication network 510 such as a LAN or a WAN through a wireless or wired communication line, for example. The imaging device 52 has a function capable of capturing images. The imaging device 52 may also be capable of capturing moving images. As described above, a plurality of imaging devices 52 may be provided.

[0111] As described above, the information processing devices 10, 20, 30, and 40 according to the first to fourth embodiments are realized by computer hardware shown in FIG. 14. The information processing devices 10, 20, 30, and 40 are realized by one device that is physically combined. Alternatively, the information processing devices 10, 20, 30, and 40 may be realized by connecting two or more physically separated devices by wire or wirelessly. An example is given of a system having a determination unit 201, a generation unit 202, and a detection unit 220. The product shelf 21 has the detection unit 220. Meanwhile, the information processing device 20 has the determination unit 201 and the generation unit 202. Alternatively, for example, the information processing device 20 has the determination unit 201. Then, a device different from the information processing device 20 may have the generation unit 202. Also, take as an example a case where a system having a determination unit 301, a recognition unit 303, an output unit 304, a generation unit 302, and a detection unit 320 is realized. The product shelf 31 has the detection unit 320. The information processing device 30 has the determination unit 301. Then, a device different from the information processing device 30 may have the recognition unit 303, the output unit 304, and the generation unit 302.

[0112] There are various variations in the method of implementing the information processing devices 10, 20, 30, and 40. For example, each device may be implemented by any combination of a different computer and program for each component. Also, multiple components included in each device may be implemented by any combination of a single computer and program.

[0113] Also, a part or all of each component of the information processing devices 10, 20, 30, 40 may be realized by a circuit for a specific application. Also, a part or all of the information processing devices 10, 20, 30, 40 may be realized by a general-purpose circuit including a processor such as an FPGA (Field Programmable Gate Array). Also, a part or all of the information processing devices 10, 20, 30, 40 may be realized by a combination of a circuit for a specific application and a general-purpose circuit. Also, these circuits may be a single integrated circuit. Or, these circuits may be divided into multiple integrated circuits. And, the multiple integrated circuits may be configured by being connected via a bus or the like.

[0114] Furthermore, when some or all of the components of each device are realized by a plurality of computers, circuits, etc., the plurality of computers, circuits, etc. may be arranged in a centralized manner or in a distributed manner.

[0115] The information processing method described in each embodiment is realized by the information processing device 10, 20, 30, 40 executing the information processing method. The information processing method is realized by the computer such as the information processing device 10, 20, 30, 40 executing a program prepared in advance. The program described in each embodiment is recorded on a computer-readable recording medium such as an HDD, SSD, flexible disk, optical disk, flexible disk, magneto-optical disk, USB memory, etc. Then, the program is executed by the computer reading it from the recording medium. The program may be distributed via the communication network 510.

[0116] The functions of each of the components of the information processing devices 10, 20, 30, and 40 in each embodiment described above may be realized by hardware, such as the information processing device 50 of a computer shown in Fig. 14. Alternatively, each component may be realized by a computer device or firmware based on program control.

[0117] Although the present disclosure has been described above with reference to each embodiment, the present disclosure is not limited to the above-mentioned embodiment. The configuration and details of the present disclosure may include embodiments to which various modifications that a person skilled in the art may understand within the scope of the present disclosure are applied. The present disclosure may include embodiments in which the matters described in this specification are appropriately combined or replaced as necessary. For example, matters described using a specific embodiment may also be applied to other embodiments to the extent that no contradiction occurs. For example, although multiple operations are described in order in the form of a flowchart, the order of description does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations can be changed to the extent that the content is not impaired.

[0118] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0119] (Appendix 1) a determination means for determining, when a change in the display state of products on the product shelf is detected, whether or not the change in the display state has been made by a store clerk; a generating means for generating shelf allocation information when the change in the display state is caused by the store clerk; An information processing device comprising:

[0120] (Appendix 2) The change in the display state is a change in weight on the product shelf. 2. An information processing device according to claim 1.

[0121] (Appendix 3) The change in weight is an increase in weight on the shelf. 3. An information processing device according to claim 2.

[0122] (Appendix 4) The determination means determines that the change in the display state is caused by the store clerk by identifying the store clerk from the captured image. 4. The information processing device according to claim 1 ,

[0123] (Appendix 5) The determination means determines that the change in weight is due to the store clerk based on a series of changes in weight at the product shelf within a predetermined time from when the change in weight at the product shelf is detected. 4. The information processing device according to claim 2 or 3.

[0124] (Appendix 6) The determination means determines that the change in weight is due to the store clerk based on an accumulated amount of change in weight at the product shelf within a predetermined time from when the change in weight at the product shelf is detected. 6. The information processing device according to claim 5.

[0125] (Appendix 7) an identification means for identifying products displayed on the product shelf based on an image of the display state when a change in the display state is detected; Equipped with the generating means generates the shelf allocation information regarding the displayed products when the displayed products acquired by the identifying means are identical. 7. An information processing device according to any one of claims 1 to 6.

[0126] (Appendix 8) an identification means for identifying products displayed on the product shelf based on an image of the display state when a change in the display state is detected; an output means for notifying the store clerk; Equipped with the generating means generates candidates for the shelf allocation information regarding the identical products when a plurality of individual products are identical among the displayed products acquired by the identifying means. The output means notifies the store clerk of the generated candidates for the shelf allocation information. 7. An information processing device according to any one of claims 1 to 6.

[0127] (Appendix 9) an identification means for identifying products displayed on the product shelf based on an image of the display state when a change in the display state is detected; an output means for notifying the store clerk when the plurality of products acquired by the identification means are different; Equipped 7. An information processing device according to any one of claims 1 to 6.

[0128] (Appendix 10) The product shelf is divided into a plurality of areas, The generating means generates shelf allocation information for a region among the plurality of regions in which the change in the display state has been detected. 10. An information processing device according to any one of claims 1 to 9.

[0129] (Appendix 11) A detection means for detecting a change in the display state of products on a product shelf; a determination means for determining whether or not the change in the display state has been made by a store clerk when the change in the display state is detected by the detection means; a generating means for generating shelf allocation information when the change in the display state is caused by the store clerk; A system comprising:

[0130] (Appendix 12) When a change in the display state of the products on the product shelf is detected, it is determined whether or not the change in the display state is caused by a store clerk; generating shelf allocation information when the change in the display state is caused by the store clerk; method.

[0131] (Appendix 13) On the computer, When a change in the display state of the products on the product shelf is detected, it is determined whether or not the change in the display state is caused by a store clerk; generating shelf allocation information when the change in the display state is caused by the store clerk; A program that executes a process.

[0132] This application claims priority based on Japanese Patent Application No. 2020-161649, filed on September 28, 2020, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0133] 2,3,5 System 10, 20, 30, 40, 50 Information processing equipment 21,31,41,51 Product shelf 22,32,42,52 Imaging device 43 Container 101,201,301,401 Judgment section 102,202,302,402 generation part 203,303,403 Identification section 210,310,410 Storage section 220,320,420 Detector 304,404 Output section

Claims

1. a determination means for determining, when a change in weight on a product shelf is detected, whether the change in weight is due to a store clerk; a generating means for generating shelf allocation information when the change in weight is caused by the store clerk; Equipped with The determination means determines that the change in weight is due to the store clerk based on a series of weight changes at the product shelf within a predetermined time from when the change in weight at the product shelf is detected. Information processing device.

2. The change in weight is an increase in weight on the shelf. The information processing device according to claim 1 .

3. The determination means determines that the change in weight is due to the store clerk by identifying the store clerk from the captured image.

3. The information processing device according to claim 1 or 2.

4. The determination means determines that the change in weight is due to the store clerk based on an accumulated amount of change in weight at the product shelf within a predetermined time from when the change in weight at the product shelf is detected. The information processing device according to claim 1 .

5. an identification means for identifying a product displayed on the product shelf based on an image of the product display state on the product shelf when the change in weight is detected; Equipped with the generating means generates the shelf allocation information regarding the displayed products when the displayed products acquired by the identifying means are identical.

5. The information processing device according to claim 1.

6. an identification means for identifying a product displayed on the product shelf based on an image of the product display state on the product shelf when the change in weight is detected; an output means for notifying the store clerk; Equipped with the generating means generates candidates for the shelf allocation information regarding the identical products when a plurality of individual products among the displayed products acquired by the identifying means are identical, The output means notifies the store clerk of the generated candidates for the shelf allocation information.

5. The information processing device according to claim 1.

7. an identification means for identifying products displayed on the product shelf based on an image of the product display state on the product shelf when the change in weight is detected; an output means for notifying the store clerk when the plurality of products acquired by the identification means are different; Equipped 5. The information processing device according to claim 1.

8. The product shelf is divided into a plurality of areas, The generating means generates shelf allocation information for a region in which the change in weight is detected among the plurality of regions.

8. The information processing device according to claim 1.

9. A computer comprising: When a change in weight of a product on a shelf is detected, it is determined whether the change in weight is due to a store clerk; generating shelf allocation information when the change in weight is due to the store clerk; Execute the process, In the determination, it is determined that the change in weight is caused by the store clerk based on a series of weight changes at the product shelf within a predetermined time from when the change in weight at the product shelf is detected. method.

10. On the computer, When a change in weight of a product on a shelf is detected, it is determined whether the change in weight is due to a store clerk; generating shelf allocation information when the change in weight is due to the store clerk; Execute the process, In the process of determining, it is determined that the change in weight is caused by the store clerk based on a series of weight changes at the store shelf within a predetermined time from when the change in weight at the store shelf is detected. program.

Images