Shelf layout data update device, shelf layout data update method, and program

Abstract

The present invention enables shelving allocation data to be updated on the basis of images acquired in time series. A region determination unit (11) performs object detection with respect to an image acquired by imaging a shelf on which objects are arranged and, on the basis of the result of the object detection, determines, for each object, a region in which the object is disposed. A non-match region identification unit (12) compares a first region determination result that is based on a first image and a second region determination result that is based on a second image, and identifies a non-match region, which is a region in which objects detected in the first and second images do not match each other. A shelving allocation data update unit (13) updates shelving allocation data on the basis of the second region determination result and the identified non-match region.

Description

Shelf layout data update device, shelf layout data update method, and computer-readable medium

[0001] The present disclosure relates to a shelf planogram data updating device, a shelf planogram data updating method, and a computer-readable medium.

[0002] As a related technique, Patent Document 1 discloses a product identification system. In the product identification system described in Patent Document 1, a management terminal performs keystone correction processing on a captured image obtained by photographing a product display shelf, and converts the captured image into an upright image, which is an image of the product shelf photographed from the front. The management terminal compares image information of a face in the second upright image with image information of a predetermined range of the face in the first upright image that corresponds to that face, thereby identifying product identification information of the product located at the face in the second upright image.

[0003] More specifically, the management terminal identifies the area where the product is placed, i.e., the face, for each shelf in the upright image information. In the first identification of product identification information, the management terminal matches the image information of the face with the product specimen image information for each face to identify the product identification information of the product displayed on that face. In the Nth identification of product identification information, the management terminal determines the similarity between the image information of the face in the Nth upright image and the image information of the face in the (N-1)th upright image information corresponding to that face. For faces with low similarity, the management terminal identifies the product identification information by performing an image matching process between the image information of the area of ​​that face and the specimen image information.

[0004] Japanese Patent Application Laid-Open No. 2019-008622

[0005] In retail stores, shelf planogram data, which digitizes the location where each product is displayed, is essential data for detecting stockouts and replenishing products. Shelf planograms are changed as needed for various reasons within the store. Whenever the shelf planogram is changed, the shelf planogram data must be updated. If the shelf planogram data were to be updated manually, it would be time-consuming and there would be concerns that changes would be overlooked.

[0006] In Patent Document 1, identification information of displayed products is specified for each face. However, although Patent Document 1 describes identifying products for each face, it does not describe converting the display location of each product into data. Therefore, in Patent Document 1, when the shelf layout is changed, the changed products can be identified, but the shelf layout data is not changed.

[0007] In view of the above circumstances, the present disclosure aims to provide a shelf layout data updating device, a shelf layout data updating method, and a computer-readable medium that are capable of updating shelf layout data based on images acquired in time series.

[0008] In order to achieve the above object, the present disclosure provides, as a first aspect, a shelf planogram data updating device. The shelf planogram data updating device includes: an area determination unit that performs object detection on an image acquired by photographing a shelf on which objects are lined up, and performs area determination that determines, for each object, an area in which the object is located based on the object detection results; an inconsistent area identification unit that compares a first area determination result that is a result of the area determination on a first image acquired by photographing the shelf at a first time with a second area determination result that is a result of the area determination on a second image acquired by photographing the shelf at a second time that is later than the first time, and identifies an inconsistent area where the objects detected in the first image and the second image do not match; and a shelf planogram data updating unit that updates shelf planogram data for the shelf based on the second area determination result and the inconsistent area.

[0009] The present disclosure provides, as a second aspect, a shelf planogram data updating method, which includes performing object detection on a first image acquired by photographing a shelf on which objects are lined up at a first time, determining, for each object, an area in which the object is located based on the object detection results, thereby obtaining a first area determination result, performing object detection on a second image acquired by photographing the shelf at a second time that is later than the first time, and determining, for each object, an area in which the object is located based on the object detection results, thereby obtaining a second area determination result, comparing the first area determination result with the second area determination result, identifying a mismatch area where objects detected in the first image and the second image do not match, and updating shelf planogram data for the shelf based on the second area determination result and the mismatch area.

[0010] In a third aspect, the present disclosure provides a computer-readable medium storing a program for causing a computer to execute processes including: performing object detection on a first image acquired by photographing a shelf on which objects are lined at a first time, determining, for each object, an area in which the object is located based on the object detection results, thereby obtaining a first area determination result; performing object detection on a second image acquired by photographing the shelf at a second time that is later than the first time, and determining, for each object, an area in which the object is located based on the object detection results, thereby obtaining a second area determination result; comparing the first area determination result with the second area determination result, identifying a mismatch area where objects detected in the first image and the second image do not match; and updating planogram data for the shelf based on the second area determination result and the mismatch area.

[0011] The shelf planogram data updating device, shelf planogram data updating method, and computer-readable medium according to the present disclosure can update shelf planogram data based on images acquired in time series.

[0012] 1 is a block diagram showing a schematic configuration example of a planogram data updating device according to the present disclosure. FIG. 1 is a block diagram showing a planogram data updating device according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram showing an example of the result of area determination performed on a first image. FIG. 3 is a schematic diagram showing an example of the result of area determination performed on a second image. FIG. 4 is a schematic diagram showing a specific example of the degree of agreement of areas. FIG. 5 is a schematic diagram showing a specific example of the result of identifying a mismatched area. FIG. 6 is a schematic diagram showing a specific example of planogram data before updating. FIG. 7 is a schematic diagram showing a specific example of planogram data after updating. A flowchart showing the operation procedure of a planogram data updating device. FIG. 8 is a block diagram showing an example of the configuration of a computer device.

[0013] Prior to describing embodiments of the present disclosure, an overview of the present disclosure will be described. Fig. 1 shows a schematic configuration example of a shelf planogram data updating device according to the present disclosure. The shelf planogram data updating device 10 includes an area determination unit 11, an inconsistent area identification unit 12, and a shelf planogram data updating unit 13.

[0014] The area determination unit 11 performs object detection on an image acquired by photographing shelves on which objects are lined up. Based on the object detection results, the area determination unit 11 determines the area in which each object is located for each object. The area determination unit 11 performs area determination on a first image acquired by photographing the shelves at a first time, and outputs a first area determination result. The area determination unit 11 also performs area determination on a second image acquired by photographing the shelves at a second time, and outputs a second area determination result. The second time is a time later than the first time.

[0015] The mismatched area identifying unit 12 compares the first area determination result with the second area determination result and identifies a mismatched area, which is an area where the objects detected in the first image and the second image do not match. The shelf planogram data updating unit 13 updates the shelf planogram data based on the second area determination result and the mismatched area identified by the mismatched area identifying unit 12.

[0016] In the present disclosure, the mismatched area identifying unit 12 compares the area determination results from two times and identifies mismatched areas, which are areas where the objects detected in the first image and the second image do not match. Areas where the objects detected in the first image and the second image, which are acquired in time series, do not match are areas where the shelf planogram may have been changed. The shelf planogram data updating unit 13 updates the shelf planogram data for such areas. In this way, the shelf planogram data updating device 10 according to the present disclosure can update the shelf planogram data based on the images acquired in time series.

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the following description and drawings have been omitted or simplified as appropriate for clarity of explanation. In addition, in the following drawings, the same or similar elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary.

[0018] 2 shows a shelf planogram data updating device according to an embodiment of the present disclosure. The shelf planogram data updating device 100 includes an area determination unit 101, an area coincidence calculation unit 102, an inconsistent area identification unit 103, a shelf planogram data updating unit 104, an area determination result storage unit 105, and a shelf planogram data storage unit 106. The shelf planogram data updating device 100 may be configured as a device including one or more processors and one or more memories. For example, at least a portion of the functional blocks within the shelf planogram data updating device 100 may be implemented by a processor executing processing in accordance with instructions read from the memory.

[0019] The planogram data updating device 100 acquires from the camera 200 images captured by the camera 200. In this embodiment, the camera 200 is assumed to be a camera that captures images of shelves on which objects such as products are displayed. The camera 200 is installed in a location such as the ceiling of a sales floor of a store, and captures images of shelves on which products are displayed in the sales floor, i.e., product shelves. The planogram data updating device 100, for example, periodically acquires images of product shelves captured by the camera 200. The images acquired by the planogram data updating device 100 include a first image captured at a first time and a second image captured at a second time that is later than the first time.

[0020] The area determination unit 101 performs object detection on the image acquired from the camera 200. For example, the area determination unit 101 performs object detection on an image of a shelf on which products are displayed, and detects the products displayed on the shelf. Based on the result of the object detection, the area determination unit 101 determines the area in which each detected object is located. For example, the area determination unit 101 determines the area on the shelf in which the product is located by merging areas or frames in which the same product is consecutive on the shelf. The area determination unit 101 performs area determination on each of the first image and the second image. The area determination unit 101 stores the results of the area determination in the area determination result storage unit 105.

[0021] FIG. 3 shows an example of the results of region determination performed on the first image. In this example, the products displayed on the shelf are bottled beverages. Six types of products, products A-F, are displayed on the shelf. The region determination unit 101, for example, performs object detection on the first image and individually detects the regions in which products A-F are arranged. The region determination unit 101 merges individual product regions in which the same products are lined up consecutively, and determines regions 210A-210F in which each product is arranged. The region determination unit 101 stores, for example, the coordinates of the regions 210A-210F in which products A-F are arranged, e.g., a pair of the coordinates of the top left and bottom right of a rectangular region, in the region determination result storage unit 105 as the first region determination result.

[0022] FIG. 4 shows an example of the results of region determination performed on the second image. After the first image was acquired, some products were sold, resulting in the existence of regions in the second image where no products were detected. Furthermore, after the first image was acquired, a product G, which was not on display at the time the first image was acquired, is present. The region determination unit 101 performs object detection on the second image and individually detects the regions in which products A-G are located. The region determination unit 101 merges individual product regions in which the same products are lined up consecutively, and determines regions 220A-220G in which each product is located. The region determination unit 101 stores, for example, the coordinates of the regions 220A-220G in which products A-G are located, e.g., a set of the coordinates of the top left and bottom right of a rectangular region, in the region determination result storage unit 105 as the second region determination result.

[0023] The region coincidence calculation unit 102 compares the first region determination result with the second region determination result and calculates the degree of coincidence of the regions for each object, i.e., each product. The region coincidence calculation unit 102 calculates, for example, the ratio of the area of ​​the overlapping portion of the regions, i.e., the common portion, to the area of ​​the union of the product region in the first region determination result and the product region in the second region determination result as the degree of coincidence. For example, the region coincidence calculation unit 102 calculates IoU (Intersection over Union), which is an index indicating the degree of overlap between the two product regions in the first region determination result and the product region in the second region determination result.

[0024] The area matching calculation unit 102 calculates the area matching degree for each of products detected from both the first image and the second image, products detected only from the first image, and products detected only from the second image. If the area matching degree for a certain product is below a predetermined threshold, it can be determined that the shelf layout has been changed, i.e., the number of faces may have increased or decreased. On the other hand, if the matching degree is equal to or greater than the threshold, it is considered that the shelf layout has not been changed. Therefore, for products with a high degree of matching, there is no need to update the shelf layout data.

[0025] 5 shows a specific example of the degree of agreement between the areas of each product in the first area determination result and the areas of each product in the second area determination result. The area agreement calculation unit 102 calculates the degree of agreement between areas 210A-F in the first area determination result and areas 220A-G in the second area determination result for each of products A-G. For product A, the area agreement calculation unit 102 calculates the ratio of the area of ​​the common part to the area of ​​the union of area 210A in the first area determination result and area 220A in the second area determination result as the degree of agreement. For other products, the area agreement calculation unit 102 also calculates the ratio of the area of ​​the common part to the area of ​​the union of the two areas as the degree of agreement.

[0026] As shown in FIG. 5 , the region coincidence calculation unit 102 calculates that the coincidence between region 210A and region 220A for product A is 64%. The region coincidence calculation unit 102 calculates that the coincidence between region 210B-F and region 220B-F for products B-F is 48%, 61%, 64%, 26%, and 97%, respectively. For product G, since the region of product G is not present in the first region determination result, the region coincidence calculation unit 102 calculates the coincidence as 0%. Among products A-G, the coincidence between the region of product F is 97%, which is equal to or greater than a predetermined threshold. In this case, it can be determined that the shelf layout of product F has not been changed during the period between the time the first image was acquired and the time the second image was acquired.

[0027] The mismatched region identifying unit 103 compares the first region determination result with the second region determination result and identifies a mismatched region, which is a region where the objects detected in the first image and the second image do not match. The mismatched region identifying unit 103 identifies a mismatched region, for example, for a product where the degree of match between the first region determination result and the second region determination result is less than a predetermined threshold. For a product where the degree of match between the first region determination result and the second region determination result is high, it is considered that no increase or decrease in face has occurred. Therefore, the mismatched region identifying unit 103 does not need to identify a mismatched region for a product where the degree of match is high.

[0028] 6 shows a specific example of the results of identifying mismatched areas. The mismatched area identification unit 103 compares areas 210A-210F of products A-F in the first area determination result (see FIG. 3) with areas 220A-G of products A-G in the second area determination result (see FIG. 4), and identifies mismatched areas. The mismatched area identification unit 103 identifies, among the areas of each product, areas that are common to both area determination results as matching areas 251-256. The mismatched area identification unit 103 identifies, among the areas of each product, areas that are not common to both area determination results as mismatched areas 261-264. The identification of mismatched areas by the mismatched area identification unit 103 and the calculation of the area coincidence by the area coincidence calculation unit 102 may be performed simultaneously.

[0029] The shelf planogram data storage unit 106 stores the shelf planogram data of the shelf photographed by the camera 200. The shelf planogram data includes, for example, a list of products to be placed on the shelf and information indicating the area of ​​the shelf where each product is placed. At the time when the second image is acquired, the shelf planogram data storage unit 106 stores the shelf planogram data that was updated at the time when the first image was acquired.

[0030] The initial values ​​of the planogram data may be automatically generated, for example, from the area determination result of the area determination unit 101 at the start of operation. For example, at the start of operation, the camera 200 acquires an image of a shelf that is filled with products without any gaps. The area determination unit 101 performs object detection on the image acquired from the camera 200 and, for each detected object, determines the area in which the object is located. In this example, the area determination result at the start of operation may be used as the initial values ​​of the planogram data. Alternatively, the initial values ​​of the planogram data may be provided manually.

[0031] The planogram data update unit 104 updates the planogram data stored in the planogram data storage unit 106 based on the second region determination result determined from the second image and the mismatched region identified by the mismatched region identification unit 103. The planogram data update unit 104 updates the planogram data, for example, depending on whether a product is detected in the mismatched region in the second image. For example, the planogram data update unit 104 determines whether a product is detected in the second image for each mismatched region identified by the mismatched region identification unit 103. Specifically, the planogram data update unit 104 determines whether an object is detected in the second image for each of the mismatched regions 261-264 shown in FIG. 6 . The planogram data update unit 104 determines which product the mismatched region corresponds to, depending on whether a product is detected in the mismatched region.

[0032] If a product is detected in the mismatched area in the second image, the planogram data update unit 104 determines that a change in the planogram or an increase or decrease in the number of faces has occurred, and updates the planogram data. If a product is detected in the mismatched area in the second image, the planogram data update unit 104 assigns the product detected in the mismatched area in the second image to the mismatched area in the updated planogram data. If no object is detected in the mismatched area in the second image, the planogram data update unit 104 determines that the mismatched area has occurred due to a product shortage. In this case, the planogram data update unit 104 does not change the product assigned to the mismatched area from the product assigned to the mismatched area in the planogram data at the time the first image was acquired. The planogram data update unit 104 stores the updated planogram data in the planogram data storage unit 106.

[0033] FIG. 7 shows a specific example of planogram data before updating. FIG. 8 shows a specific example of planogram data after updating. In FIGS. 7 and 8 , "A" to "G" indicate which rectangular area is assigned to which product, from product A to product G. In other words, "A" to "G" indicate which product, from product A to product G, each area corresponds to. The planogram data before updating shown in FIG. 7 is assumed to be planogram data updated or generated at the time the first image was acquired. For the inconsistent areas identified by the inconsistent area identification unit 103, the planogram data update unit 104 updates the products assigned to the inconsistent areas depending on whether or not a product is detected in the inconsistent areas in the second image.

[0034] In the pre-update planogram data shown in FIG. 7 , for example, the mismatched region 261 shown in FIG. 6 is assigned to product A. The planogram data update unit 104 determines whether a product is detected in the second image for the mismatched region 261. As shown in FIG. 4 , product G is detected in the mismatched region 261 in the second image. In other words, the mismatched region 261 overlaps with the region 220G of product G in the second region determination result. In this case, the planogram data update unit 104 changes the product assigned to the mismatched region 261 in the planogram data from product A before the update to product G. Because the product assigned to the mismatched region 261 has changed, the planogram data update unit 104 reduces the region of product A by one product in the updated planogram data, as shown in FIG. 8 .

[0035] 7, the mismatched area 263 shown in FIG. 6 is assigned to item C. The planogram data update unit 104 determines whether or not a product is detected in the second image for the mismatched area 263. As shown in FIG. 4, item D is detected in the mismatched area 263 in the second image. In other words, the mismatched area 263 overlaps with the area 220D of item D in the second area determination result. In this case, the planogram data update unit 104 changes the product assigned to the mismatched area 261 in the planogram data from item C before the update to item D. In this case, as shown in FIG. 8, in the updated planogram data, the area of ​​item D is enlarged by one item, and the area of ​​item C is reduced by one item.

[0036] In the pre-update planogram data shown in FIG. 7 , the mismatched areas 262 and 264 shown in FIG. 6 are assigned to product B and product E, respectively. The planogram data update unit 104 determines whether a product is detected in the second image for each of the mismatched areas 262 and 264. As shown in FIG. 4 , no product is detected in the mismatched areas 262 and 264 in the second image. In this case, the planogram data update unit 104 does not change the products assigned to the mismatched areas 262 and 264, respectively, from product B and product D before the update in the planogram data. For product F, the area coincidence calculated by the area coincidence calculation unit 102 is equal to or greater than the threshold value, so the area of ​​product F remains unchanged in the planogram data. The planogram data update unit 104 stores the updated planogram data shown in FIG. 8 in the planogram data storage unit 106.

[0037] Next, the operation procedure will be explained. Figure 9 shows the operation procedure of the shelf planogram data updating device 100. The operation procedure of the shelf planogram data updating device 100 corresponds to the shelf planogram data updating method. The area determination unit 101 acquires, from the camera 200, an image of a shelf on which products are displayed (step S1). The area determination unit 101 determines the area of ​​each product displayed on the shelf from the acquired image (step S2). In step S2, the area determination unit 101, for example, performs object detection on the image of the shelf acquired from the camera 200 to detect products, and determines the area of ​​each product by merging the detected product areas. The area determination unit 101 stores the results of the area determination in the area determination result storage unit 105.

[0038] The area coincidence calculation unit 102 compares the result of the area determination in step S2 with the result of the area determination performed in a previous planogram data update stored in the area determination result storage unit 105, and calculates the degree of area coincidence for each product present on the shelf (step S3). Here, the result of the area determination performed in the previous planogram data update is referred to as the first area determination result. The result of the area determination performed in step S1 is referred to as the second area determination result. In step S3, the area coincidence calculation unit 102 acquires, for example, the result of the area determination performed in the previous planogram data update as the first area determination result. The area coincidence calculation unit 102 calculates the IoU between the area in the first area determination result and the area in the second area determination result for each product.

[0039] The mismatched area identification unit 103 determines, for each product, whether the degree of match of the area is less than a predetermined threshold (step S4). The mismatched area identification unit 103 identifies mismatched areas for products whose degree of match of the area is less than the predetermined threshold (step S5). The planogram data update unit 104 determines, for each identified mismatched area, whether the product is present in the image acquired in step S1 (step S6). The planogram data update unit 104 updates the planogram data for the mismatched areas determined to contain a product in step S6 (step S7). In step S7, the planogram data update unit 104 assigns the mismatched areas determined to contain a product in the planogram data to the product detected in the image acquired in step S1.

[0040] The planogram data update unit 104 does not update the planogram data for products for which it is determined in step S4 that the degree of match between the areas is equal to or greater than the threshold value. Furthermore, the planogram data update unit 104 does not update the planogram data for mismatched areas for which it is determined in step S6 that no products exist. The planogram data update unit 104 stores the updated planogram data in the planogram data storage unit 106.

[0041] The user may operate the planogram data updating device 100 or another device to display the planogram data stored in the planogram data storage unit 106 on the display screen of a display device. By looking at the displayed planogram data, the user can check whether the actual planogram matches the planogram indicated by the planogram data. The user may operate the planogram data updating device 100 or another device to manually correct the planogram data stored in the planogram data storage unit 106.

[0042] When the planogram data updating unit 104 changes the products allocated to any areas in the planogram data, it may record a change history in the planogram data storage unit 106. When the planogram data stored in the planogram data storage unit 106 is displayed, the planogram data updating device 100 or another device may display frames indicating areas of changed products in a color different from the color of frames indicating areas of unchanged products. In this case, the user can know which parts have been changed in the update.

[0043] Furthermore, when displaying the planogram data, the planogram data updating device 100 or another device may display the planogram data before the update and the planogram data after the update in a manner that allows for comparison. For example, the planogram data updating device 100 may display the planogram data before the update shown in Figure 7 and the updated planogram data shown in Figure 8 side by side on the display screen. The planogram data updating device 100 or another device may display, in addition to the planogram data, an image from the camera 200 used to update the planogram data on the display screen.

[0044] The user may be able to select any area or product in the displayed planogram data. When the user selects a certain product or an area of ​​a certain product, the planogram data updating device 100 or another device uses the update history to identify the timing when the planogram data for that product or area was updated. The planogram data updating device 100 or another device may display the planogram data before the update at the identified timing and the updated planogram data in a manner that allows comparison.

[0045] In this embodiment, the area determination unit 101 performs area determination on a first image acquired at a first time and obtains a first area determination result. The area determination unit 101 also performs area determination on a second image acquired at a second time and obtains a second area determination result. The inconsistent area identification unit 103 compares the first area determination result with the second area determination result and identifies an inconsistent area. The planogram data update unit 104 updates the planogram data based on the second area determination result and the identified inconsistent area. In this embodiment, the inconsistent area identified by the inconsistent area identification unit 103 is considered to be related to an area where a change in the planogram or an increase or decrease in the number of faces has occurred. Therefore, the planogram data update device 100 can detect a change in the planogram or an increase or decrease in the number of faces based on the first image and the second image acquired in chronological order, and update the planogram data in accordance with the detected change in the planogram or increase or decrease in the number of faces.

[0046] Furthermore, in this embodiment, the planogram data update unit 104 determines whether a product is detected in the second image for each identified mismatched area. Depending on whether a product is present in the mismatched area in the second image, it can be determined whether the product is absent in the second image, i.e., whether the product is out of stock, or whether the product has been replaced. In this embodiment, the planogram data update unit 104 updates the planogram data depending on whether a product is detected in the mismatched area in the second image. In this case, the planogram data update unit 104 can appropriately update the planogram data for the out-of-stock areas and the areas where the products have been replaced.

[0047] In the above embodiment, an example has been described in which camera 200 acquires images of product display shelves arranged in a sales area of ​​a store. However, the present disclosure is not limited to this. Camera 200 does not necessarily have to be installed in a store such as a retail store. For example, camera 200 may acquire images of shelves on which some objects are placed in a location other than a store, such as a warehouse. In other words, shelving planogram data updating device 100 can be used to update shelving planogram data for objects placed on shelves in a warehouse.

[0048] In the present disclosure, the shelf planogram data updating device 100 may be configured using a computer device or a server device. Fig. 10 shows an example configuration of a computer device that may be used in the shelf planogram data updating device 100. The computer device 500 has a control unit (CPU: Central Processing Unit) 510, a storage unit 520, a ROM (Read Only Memory) 530, a RAM (Random Access Memory) 540, a communication interface (IF: Interface) 550, and a user interface 560.

[0049] The communication interface 550 is an interface for connecting the computer device 500 to a communication network via wired communication means, wireless communication means, etc. The user interface 560 includes a display unit such as a display, and an input unit such as a keyboard, a mouse, and a touch panel.

[0050] The storage unit 520 is an auxiliary storage device that can store various types of data. The storage unit 520 does not necessarily have to be a part of the computer device 500, but may be an external storage device or cloud storage connected to the computer device 500 via a network. The storage unit 520 can be used as at least one of the area determination result storage unit 105 and the shelf planogram data storage unit 106 (see FIG. 2 ).

[0051] The ROM 530 is a non-volatile storage device. For example, a semiconductor storage device with a relatively small capacity, such as a flash memory, is used for the ROM 530. The programs executed by the CPU 510 can be stored in the storage unit 520 or the ROM 530. The storage unit 520 or the ROM 530 stores various programs for realizing the functions of each unit of the shelf planogram data updating device 100, for example.

[0052] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, ROM, flash memory, solid-state drives (SSDs) or other memory technologies, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0053] The RAM 540 is a volatile storage device. Various semiconductor memory devices such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory) are used for the RAM 540. The RAM 540 can be used as an internal buffer for temporarily storing data and the like. The CPU 510 loads a program stored in the storage unit 520 or the ROM 530 into the RAM 540 and executes it. The CPU 510 executes the program, thereby realizing the functions of each unit in the shelf planogram data updating device 100. The CPU 510 may have an internal buffer for temporarily storing data and the like.

[0054] In the above embodiment, the shelf planogram data updating device 100 does not necessarily have to be configured as a single device. The shelf planogram data updating device 100 may be configured using a plurality of physically separated devices.

[0055] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the above-described embodiments, and changes and modifications to the above-described embodiments that do not deviate from the spirit of the present disclosure are also included in the present disclosure.

[0056] 10: Shelf allocation data update device 11: Area determination unit 12: Mismatched area identification unit 13: Shelf allocation data update unit 100: Shelf allocation data update device 101: Area determination unit 102: Area coincidence calculation unit 103: Mismatched area identification unit 104: Shelf allocation data update unit 105: Area determination result storage unit 106: Shelf allocation data storage unit 500: Computer device 510: Control unit 520: Storage unit 530: ROM 540: RAM 550: Communication interface 560: User interface

Claims

1. an area determination unit that performs object detection on an image acquired by photographing a shelf on which objects are lined up, and performs area determination that determines, for each object, an area in which the object is located based on the result of the object detection; a mismatched region identifying unit that compares a first region determination result, which is a result of the region determination for a first image obtained by photographing the shelf at a first time, with a second region determination result, which is a result of the region determination for a second image obtained by photographing the shelf at a second time that is later than the first time, and identifies a mismatched region, which is a region where objects detected in the first image and the second image do not match; a shelf planogram data updating unit that updates the shelf planogram data of the shelf based on the second area determination result and the mismatch area;

2. 2. The shelf planogram data updating device according to claim 1, wherein the shelf planogram data updating unit determines whether to update the shelf planogram data depending on whether an object is detected in the mismatch area in the second image.

3. 3. The shelf planogram data updating device according to claim 2, wherein, when an object is detected in the mismatched area in the second image, the shelf planogram data updating unit assigns the object detected in the mismatched area in the second image to the mismatched area in the shelf planogram data, and when an object is not detected in the mismatched area in the second image, the object assigned to the mismatched area in the shelf planogram data is not changed from the object assigned to the mismatched area in the shelf planogram data at the first time.

4. further comprising an area coincidence calculation unit that compares the first area determination result with the second area determination result and calculates, for each object, a degree of coincidence of an area in which the object is located; The shelf planogram data updating device according to claim 1 , wherein the mismatched area identifying unit identifies the mismatched area for an object whose area has a degree of match less than a predetermined value.

5. 5. The shelf planogram data updating device according to claim 4, wherein the area coincidence calculation unit calculates, for each object, the degree of coincidence as a ratio of the area of ​​the common portion between the area in the first area determination result and the area in the second area determination result to the area of ​​the union of the area in the first area determination result and the area in the second area determination result.

6. performing object detection on a first image acquired by photographing a shelf on which objects are lined up at a first time, and determining an area in which each object is located based on a result of the object detection, thereby acquiring a first area determination result; performing object detection on a second image acquired by photographing the shelf at a second time that is later than the first time, and determining, for each object, an area in which the object is located based on the object detection result, thereby acquiring a second area determination result; comparing the first region determination result with the second region determination result, and identifying a mismatch region where the objects detected in the first image and the second image do not match; A shelf planogram data updating method comprising updating the shelf planogram data of the shelf based on the second area determination result and the mismatch area.

7. 7. The planogram data updating method of claim 6, wherein updating the planogram data comprises determining whether to update the planogram data depending on whether an object is detected in the mismatch region in the second image.

8. performing object detection on a first image acquired by photographing a shelf on which objects are lined up at a first time, and determining an area in which each object is located based on a result of the object detection, thereby acquiring a first area determination result; performing object detection on a second image acquired by photographing the shelf at a second time that is later than the first time, and determining, for each object, an area in which the object is located based on the object detection result, thereby acquiring a second area determination result; comparing the first region determination result with the second region determination result, and identifying a mismatch region where the objects detected in the first image and the second image do not match; A program for causing a computer to execute a process including updating the shelf allocation data of the shelf based on the second area determination result and the mismatch area.

9. 9. The program of claim 8, wherein updating the planogram data comprises determining whether to update the planogram data depending on whether an object is detected in the mismatch region in the second image.