Display state analysis system, display state analysis method, and program

Abstract

To provide a display state analysis system capable of analyzing a display state of articles.SOLUTION: A display state analysis system includes article recognition means that recognizes articles in a display image from the display image in which the articles displayed in a display shelf are captured, analysis means that identifies a reference line connecting upper ends of each of the articles existing in forefront of one tier below the stacked articles based on a recognition result by the article recognition means, and calculates a distance from the reference line of the stacked article in a depth direction, and output means that outputs information on a display state of the display shelf based on the calculated distance.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display situation analysis device, a display situation analysis method, and a program for analyzing the display situation of items. [Background technology]

[0002] Products displayed in supermarkets and convenience stores are generally sold starting from the front of the shelves, so over time, empty spaces form at the front of the shelves. Therefore, store staff make regular rounds to replenish products and organize the shelves by appropriately performing the so-called "front-loading" work.

[0003] Patent Document 1 describes an inventory status management method that notifies a store clerk of the inventory status of a product when the product runs out on a display shelf. The management method described in Patent Document 1 monitors images of the product display shelf with a real-time camera, and when a no-product marker on the display shelf is detected, it identifies the product associated with the no-product marker and outputs a message indicating that inventory is being checked.

[0004] Furthermore, Patent Document 2 discloses a product monitoring system that uses a camera to capture images of the display state of products on display shelves, analyzes the images, and notifies the customer when to replenish products based on time-series changes in the analysis results. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-174154 [Patent Document 2] Japanese Patent Application Publication No. 05-081552 Summary of the Invention [Problem to be solved by the invention]

[0006] There is a limit to how much a store clerk can constantly monitor the display status within the store. However, if there are products in the back room but the shelves remain empty, sales opportunities will be lost. Even if there are products remaining on the shelves, if the products are not properly arranged, it will reduce consumer purchasing motivation and there is a possibility that sales opportunities will be lost.

[0007] The method described in Patent Document 1 has the problem that the display position of products cannot be flexibly changed because a product-free marker associated with each product is set on the display shelf. Also, the system described in Patent Document 2 has the problem that there are limitations on the location where the images can be taken, because accurate analysis results cannot be obtained unless the entire base of each shelf of the product shelf can be photographed.

[0008] Therefore, an object of the present invention is to provide a display situation analysis system, a display situation analysis method, and a program that can analyze the display situation of items. [Means for solving the problem]

[0009] The first display situation analysis system of the present invention comprises an item recognition means for recognizing items in a display image taken of items displayed on a row of shelves, an analysis means for identifying a reference line connecting the top ends of each of the items located at the front of the next lower row of stacked items based on the recognition results by the item recognition means, and calculating the distance of the stacked items from the reference line in the depth direction, and an output means for outputting information regarding the display situation of the display shelves based on the distance.

[0010] In a first display situation analysis method of the present invention, a computer recognizes items displayed on a display shelf from a display image taken of the items, identifies a reference line connecting the top ends of each of the items located at the front of the stacked items one level below the items based on the recognition results, calculates the distance of the stacked items from the reference line in the depth direction, and outputs information regarding the display situation of the display shelf based on the distance.

[0011] The first program of the present invention causes a computer to perform the following processes: recognize the items in a display image taken of the items displayed on a display shelf; identify a reference line connecting the top ends of the items located at the front of the stacked items one level below the items based on the recognition results; calculate the distance of the stacked items from the reference line in the depth direction; and output information regarding the display status of the display shelf based on the distance. [Effects of the Invention]

[0012] According to the present invention, the display status of items can be analyzed. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of a first embodiment of a display situation analysis device according to the present invention. [Figure 2] FIG. 10 is an explanatory diagram showing an example of processing for recognizing an article from a display image. [Figure 3] FIG. 10 is an explanatory diagram showing an example of an output result output by an analysis result processing unit. [Figure 4] 4 is a flowchart showing an example of the operation of the display situation analysis device of the first embodiment. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a reference position. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a face line. [Figure 7] FIG. 10 is an explanatory diagram showing another example of the output result output by the analysis result processing unit. [Figure 8] FIG. 10 is an explanatory diagram showing another example of processing for recognizing an article from a display image. [Figure 9] 1 is a block diagram showing an outline of each embodiment of a display situation analysis device according to the present invention. [Figure 10] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to each embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Embodiment 1. 1 is a block diagram showing an example of the configuration of a first embodiment of a display situation analysis device according to the present invention. A display situation analysis device 10 of this embodiment acquires display images of items displayed on a display shelf from an image acquisition unit 20, and outputs the display situation of the display shelf to an analysis result processing unit 30.

[0015] The image acquisition unit 20 is realized by, for example, a fixed camera for constantly monitoring the display shelves. However, the image acquisition unit 20 itself does not have to have the function of capturing images. The image acquisition unit 20 may be, for example, an interface that receives images captured by a camera (not shown) that captures images of the display shelves via a communication network.

[0016] The image acquiring unit 20 may periodically supply the display images to the display situation analysis device 10, or may supply the display images to the display situation analysis device 10 in response to an instruction from a manager or the like. The image acquiring unit 20 may also supply the display images to the display situation analysis device 10 together with the time at which the display images were captured, for example.

[0017] The analysis result processing unit 30 outputs the display situation of the items analyzed by the display situation analysis device 10. The display situation obtained by the display situation analysis device 10 of this embodiment includes not only information indicating the actual display situation of the items, such as a disordered display, but also alarms output in accordance with the analyzed display situation. In other words, the display situation in this embodiment can be said to be information linked to the display situation, such as a disordered display or an alarm. The manner in which the analysis result processing unit 30 outputs the display situation will be described later. Furthermore, the display situation analysis unit 12, which will be described later, may perform the processing of the analysis result processing unit 30 (i.e., the processing to output the display situation of the items).

[0018] In this way, the display shelf situation can be recognized through cooperation between the display situation analysis device 10, the image acquisition unit 20, and the analysis result processing unit 30. Therefore, a configuration including the display situation analysis device 10, the image acquisition unit 20, and the analysis result processing unit 30 can be called a display situation analysis system.

[0019] The display situation analysis device 10 includes an item recognition unit 11, a display situation analysis unit 12, and a storage unit 13.

[0020] The item recognition unit 11 recognizes items in the display image. Specifically, the item recognition unit 11 identifies items in the display image and recognizes the position, size, range, etc. of the items. Hereinafter, the items recognized by the item recognition unit 11 and information specifying the position, size, range, etc. of the items will be referred to as item recognition results. The item recognition results are, for example, information indicating the area of ​​the recognized item.

[0021] For example, the item recognition unit 11 may identify items by template matching items present in the display image with images of items stored in the memory unit 13. However, the method by which the item recognition unit 11 recognizes items in an image is not limited to template matching. The item recognition unit 11 may also recognize items in the display image using other well-known methods.

[0022] Fig. 2 is an explanatory diagram showing an example of a process for recognizing an item from a display image. When a display image P1 shown in Fig. 2 is input, the item recognition unit 11 recognizes an item from the display image P1. The area shown in a bold rectangle in the display image P2 shown in Fig. 2 indicates the area of ​​the item identified as the item recognition result.

[0023] The item recognition unit 11 supplies the display image and the item recognition result to the display situation analysis unit 12.

[0024] The display situation analysis unit 12 analyzes the display situation of the displayed items. Specifically, the display situation analysis unit 12 analyzes the display situation of the displayed items based on the recognized positions of the items and outputs the analysis results. Here, the display situation output by the display situation analysis unit 12 includes information linked to the display situation, such as display disorder and alerts, as described above.

[0025] Furthermore, the above-mentioned recognized position of an item includes not only the absolute position of the item recognized from the display image but also the following: That is, the position of an item includes the relative position or positional relationship with other items, points or surfaces identified from the recognized item, lines identified by connecting the positions of each recognized item, and ranges (areas) identified from these lines and other lines. For example, if a point identified from the recognized item and a reference line exist in the display image, the position of the item is recognized from this point and the reference line.

[0026] The display situation analysis unit 12 may, for example, calculate the depth at which the recognized item is located, analyze the display situation of the displayed items, and output the results. Here, the depth at which the item is located means the degree of deviation from a preferred position for displaying the item (hereinafter referred to as the reference position). For example, in the case of a display shelf, if the front row of each display shelf is defined as the reference position, the depth at which the item is located is the distance from the front row of the display shelf toward the back. In the following description, calculating the depth at which the item is located may also be simply referred to as calculating the depth.

[0027] As mentioned above, products generally sell first from the front of the display shelf. Therefore, as the number of products decreases, products tend to remain further back on the display shelf. The further back an item is located on the display shelf, the more difficult it is for consumers to see the item, and this is not a desirable display situation for the items. In other words, the shorter the depth at which the item is located (i.e., the closer the item is located on the display shelf), the better the display situation. Therefore, in the following explanation, it is assumed that the display situation analysis unit 12 analyzes the display situation from the depth at which the recognized item is located.

[0028] In the first embodiment, the display situation analysis unit 12 calculates the depth from the deviation between the recognized position of the item and a reference position. The position indicating the item is determined in advance for each individual item. Since the height and width of each item differ, for example, the center of the bottom side of the item may be defined as the position of the item.

[0029] For example, if the display images are images taken by a fixed camera, the display situation analysis unit 12 may specify in advance the position of the front row of the display shelves based on the camera's angle of view, the distance to the display shelves, etc. Furthermore, if the range in which the display images are taken is not fixed, the display situation analysis unit 12 may estimate the position of the display shelves from the display images and use the estimated position of the display shelves as the reference position.

[0030] The display situation analysis unit 12 calculates the depth at which the item is located based on the distance between the recognized position of the item and a reference position. For example, if the front row of a display shelf is defined as the reference position, the display situation analysis unit 12 may calculate the distance between the line indicating the front row and the position of the item as the depth.

[0031] The display situation analysis unit 12 may simply calculate the depth, or may determine the desirability of the display situation of the items based on the depth calculation result. For example, the display situation analysis unit 12 may determine that the smaller the sum of the distances (i.e., depths) from the reference position of each item is, the more desirable the display situation is.

[0032] The display situation analysis unit 12 supplies the analysis result to the analysis result processing unit 30. For example, the display situation analysis unit 12 may supply information indicating the positions of items whose deviation from a reference position is greater than a predetermined standard to the analysis result processing unit 30 as the analysis result. Alternatively, the display situation analysis unit 12 may supply information indicating the positions of all items and their deviation from the reference position to the analysis result processing unit 30 as the analysis result. Note that the analysis results listed here are just examples, and the display situation analysis unit 12 may supply any information necessary for the processing performed by the analysis result processing unit 30 as the analysis result.

[0033] The analysis result processing unit 30 performs any processing that indicates the display situation, using the analysis results supplied from the display situation analysis unit 12. For example, the analysis result processing unit 30 may use the analysis results to output an image in a manner that allows the calculated depth at which the item is located to be recognized.

[0034] Fig. 3 is an explanatory diagram showing an example of the output result output by the analysis result processing unit 30. The example shown in Fig. 3 shows that the analysis result processing unit 30 displays a blinking image T superimposed on a position in the display image P1 that indicates an article whose deviation from the reference position is greater than a predetermined standard.

[0035] Additionally, the analysis result processing unit 30 may send an alert to relevant parties indicating that it is time to replenish items. Furthermore, for example, if the display shelves have a function for changing their tilt, the analysis result processing unit 30 may send information indicating the positions of items to a device (not shown) that controls the display shelves so that the display shelves are tilted to automatically move items forward.

[0036] Furthermore, the analysis result processing unit 30 may calculate the sales volume of an item based on the depth at which the item is located, the size of the item itself, etc., and automatically perform ordering processing according to the sales volume, since an item that is not at the front of the display shelf is considered to have been sold.

[0037] Furthermore, the analysis result processing unit 30 may evaluate the degree of disorder in the sales floor from the analysis results of each item, and may tally the results so that they can be compared across multiple stores, for example.

[0038] The storage unit 13 stores information necessary for the item recognition process and the depth calculation process. The storage unit 13 may store, for example, images of the displayed items and the size of the items. The storage unit 13 may also store parameter information indicating the position, angle of view, and shooting range of a camera that captures the display images. The storage unit 13 may be realized, for example, by a magnetic disk or the like.

[0039] The item recognition unit 11 and the display situation analysis unit 12 are realized by a CPU (Central Processing Unit) of a computer that operates according to a program (display situation analysis program). For example, the program may be stored in the storage unit 13, and the CPU may read the program and operate as the item recognition unit 11 and the display situation analysis unit 12 according to the program.

[0040] Furthermore, the item recognition unit 11 and the display situation analysis unit 12 may each be realized by dedicated hardware. Furthermore, although the present embodiment illustrates a case where the analysis result processing unit 30 is not included in the display situation analysis device 10, the display situation analysis device 10 may be configured to include the analysis result processing unit 30.

[0041] Next, a description will be given of the operation of the display situation analysis device 10 of this embodiment. Fig. 4 is a flowchart showing an example of the operation of the display situation analysis device 10 of the first embodiment.

[0042] The item recognition unit 11 recognizes items in the display image input from the image acquisition unit 20 (step S11). Then, the display situation analysis unit 12 analyzes the display situation of the displayed items based on the positions of the recognized items (step S12). Specifically, the display situation analysis unit 12 may calculate the depth at which the recognized items are located to analyze the display situation of the displayed items, or may calculate the depth from the degree of deviation between the positions of the recognized items and a reference state.

[0043] As described above, in this embodiment, the display situation analysis unit 12 analyzes the display situation of displayed items based on the positions of the recognized items. Specifically, the display situation analysis unit 12 calculates the depth at which the recognized items are located in the display image to analyze the display situation. For example, the display situation analysis unit 12 calculates the depth at which the items are located based on the distance between the recognized position of the item and a reference position. This configuration makes it possible to analyze the display situation of items. As a result, it is possible to detect and prevent product shortages and clutter in the sales floor based on the analyzed display situation, thereby reducing lost sales opportunities and improving sales. Furthermore, this embodiment has the advantage of being easy to implement because the depth can be calculated without using special imaging equipment.

[0044] Next, a modified example of this embodiment will be described. In the first embodiment, the front row of each display shelf is defined as the reference position. Note that items may be displayed not only in one tier but also in multiple tiers. In this case, the display status analysis unit 12 may set the reference position to a line connecting the top ends of items present on the tier immediately below the tier on which the items are stacked.

[0045] FIG. 5 is an explanatory diagram showing an example of a reference position. Reference line L1 indicates the front row of the display shelf. Reference line L2 indicates the line connecting the top bases of the first tier of items displayed in the front row. By setting separate reference positions in this way, it becomes possible to judge the display status of stacked items in the same way as above.

[0046] By defining the reference position in this way, the depth at which items are stacked and displayed can also be calculated, making it possible to determine the degree of orderliness of a sales floor, even if the floor has a variety of different configurations. Embodiment 2. Next, a second embodiment of the display situation analysis device according to the present invention will be described. The configuration of the display situation analysis device of this embodiment is the same as that of the first embodiment.

[0047] In the second embodiment, the display situation analysis unit 12 calculates the depth from the line connecting the positions of each recognized item and the degree of deviation from the comparison reference line. Hereinafter, the line connecting the positions of each recognized item will be referred to as the "face line." The method for determining the positions of the items is the same as the method used in the first embodiment. The display situation analysis unit 12 may also estimate the face line using technology used in the Hough transform.

[0048] 6 is an explanatory diagram showing an example of a face line. When the position of each item is recognized as shown in the display image P2, the display situation analysis unit 12 estimates a face line L3 indicated by a dashed line in the display image P3.

[0049] The comparison reference line used in the second embodiment may be any line that can be extracted from the display image, or may be a line defined independently of the display image. For example, if the comparison reference line is defined as a simple straight line, the display situation analysis unit 12 may calculate the depth based on the degree of distortion of the face line connecting the positions of the items.

[0050] Furthermore, for example, if the straight line indicating the front row of each display shelf is defined as the comparison reference line, the display situation analysis unit 12 may calculate the depth from the distance between each point on the face line and the comparison reference line, as in the first embodiment.

[0051] When items are not in disarray, they are usually displayed side by side, so the identified face line is almost straight. In other words, if the face line is straight, it can be determined that items are being removed from the back room or brought forward. On the other hand, if the items are not in disarray, the face line will not be straight, but will be distorted or step-like. In other words, if the face line is not straight, it can be determined that there may be a shortage or out-of-stock situation, and that an alert needs to be raised.

[0052] In other words, the closer the face line is to a straight line (i.e., the smaller the difference from the comparison reference line), the better the display state of the items. Therefore, by the display state analysis unit 12 identifying such a face line, it becomes possible to analyze the display state.

[0053] 7 is an explanatory diagram showing another example of the output result output by the analysis result processing unit 30. The analysis result processing unit 30 displays the face line estimated by the display situation analysis unit 12 by superimposing it on the original display image. In the example shown in FIG. 7, the face line appears as a broken line because the displayed items are in disarray. For example, when the displayed items are not in disarray, a nearly straight face line is superimposed on the display image.

[0054] Furthermore, in the example shown in FIG. 7, the analysis result processing unit 30 superimposes an image indicating that the display is disorganized at a position with a long depth. In this way, the analysis result processing unit 30 may change the image to be superimposed depending on the degree of disorganization. Specifically, the analysis result processing unit 30 may superimpose an image that allows the degree of deviation to be recognized at a position where the degree of deviation from the comparison reference line is greater. For example, the analysis result processing unit 30 may superimpose an image that is color-coded according to the degree of disorganization of the display, like a heat map, on the display image.

[0055] While Fig. 7 shows an example in which another image is superimposed without converting the original display image, the analysis result processing unit 30 may also convert the original display image itself. For example, the analysis result processing unit 30 may perform conversion processing to change the contrast of the image of the portion showing the disordered items, thereby emphasizing the degree of disorder. By visualizing the degree of disorder on the display shelves in this way, the overall condition of the display shelves can be easily grasped.

[0056] As described above, in this embodiment, the display situation analysis unit 12 calculates the depth at which the item is located from the degree of deviation between the face line and the comparison reference line. Even with this configuration, it is possible to analyze the display situation of the items, similar to the effect of the first embodiment. Embodiment 3. Next, a display situation analysis device according to a third embodiment of the present invention will be described. The display situation analysis device of this embodiment has the same configuration as that of the first embodiment.

[0057] In the third embodiment, the display situation analysis unit 12 uses the size of each item recognized from a display image captured of items displayed in a predetermined position as a reference state, and calculates the depth at which the items are located based on the degree of discrepancy between the size of each item in the reference state and the size of each item recognized from a newly captured display image. For example, the display situation analysis unit 12 may use the size of each item recognized from the display image when the item is displayed in an appropriate predetermined position as the reference state. An appropriate position refers to a preferred display position, such as the front row of a display shelf. Hereinafter, an image of each item recognized from the display image when the item is displayed in an appropriate position will be referred to as a reference image. The reference image is stored, for example, in the memory unit 13.

[0058] Generally, it is preferable that an item be displayed at the very front of a display shelf. For example, when an item is photographed horizontally using a fixed camera, an item at the front will appear larger than an item further back. In other words, the further back an item is located, the smaller its size will be. Therefore, an image of an item at the very front of a display shelf can be used as the reference image.

[0059] The display situation analysis unit 12 may calculate the depth from the difference (deviation) between the image of the recognized item and the reference image. In this case, the display situation analysis unit 12 may calculate the depth according to the ratio between the image of the recognized item and the reference image, based on the angle of view of the camera that captures the display image, the distance to the display shelf, etc.

[0060] Fig. 8 is an explanatory diagram showing another example of a process for recognizing items from a display image. In the example shown in Fig. 8, items A1, A3, and A4 are displayed at the forefront of a display shelf, and item A2 is located further back than the other items.

[0061] 8, it is difficult to calculate the depth from the position of the recognized items alone due to the angle of the camera capturing the display image. However, item A2 located at the back appears smaller in the image compared to the reference image. Therefore, the display situation analysis unit 12 can calculate the depth from the ratio between the reference image and the captured image of item A2.

[0062] As described above, in this embodiment, the display situation analysis unit 12 calculates the depth at which an item is located from the degree of discrepancy between the size of each item in the reference state and the size of each item recognized from the display image. Therefore, even if the depth at which an item is located cannot be determined from the recognized position of the item alone, it is possible to calculate the depth.

[0063] It should be noted that items may be displayed on display shelves in a variety of orientations. Therefore, it is preferable to prepare reference images of items photographed from multiple angles so that the item recognition unit 11 can identify the items. Therefore, the storage unit 13 may store reference images of the same item photographed from multiple directions.

[0064] Furthermore, there is a preferred direction for displaying items. Therefore, information indicating a more preferred direction among reference images of the same item photographed from multiple directions may be set in each reference image. In this case, the display situation analysis unit 12 may analyze the display situation by taking into account the direction of each item in addition to the depth at which each item is located. This also applies to other embodiments. For example, the display situation analysis unit 12 may determine that the more items are displayed in a preferred direction, the better the display situation. Embodiment 4. Next, a fourth embodiment of the display situation analysis device according to the present invention will be described. The configuration of the display situation analysis device of this embodiment is also similar to that of the first embodiment.

[0065] In the fourth embodiment, the display situation analysis unit 12 uses a display image showing an appropriate display state as a reference state, and calculates the depth at which the items are located based on the degree of deviation between the position or size of each item in the reference state and the position or size of each item recognized from a newly captured display image. An appropriate display state means a state in which items are displayed in an appropriate position without any disorder. Hereinafter, the display image showing an appropriate display state will be referred to as a "reference display image." The reference display image is stored, for example, in the memory unit 13.

[0066] The display situation analysis unit 12 may calculate the depth at which an item is located based on the difference (degree of deviation) between the position of the item in the reference display image and the position of the item in the newly captured display image. Alternatively, the display situation analysis unit 12 may calculate the depth at which an item is located based on, for example, the difference (degree of deviation) between the size of the item in the reference display image and the size of the item in the newly captured display image. The display situation analysis unit 12 may determine, for example, that the smaller the difference, the closer the item is to the front, and therefore determine that the display state is good.

[0067] In this case, it is preferable to use an image captured in the past of the same range as the display image as the reference image, because it is easy to detect the difference between images captured in the same range.

[0068] In this embodiment, the depth at which the item is located is calculated by comparing each item recognized in each display image. Therefore, for example, if the range in which the item is placed is almost the same, it is possible to easily calculate the depth. Embodiment 5. Next, a fifth embodiment of the display situation analysis device according to the present invention will be described. The configuration of the display situation analysis device of this embodiment is also the same as that of the first embodiment.

[0069] Depending on the display shelf, the further an item is displayed, the darker the item may appear. Therefore, in the fifth embodiment, the display situation analysis unit 12 calculates the depth from the degree of deviation between the brightness of the recognized item and a pixel value indicating a reference brightness. The display situation analysis unit 12 may determine that an item that is darker than the reference brightness is located further back.

[0070] For example, when brightness is used as information indicating brightness, the display situation analysis unit 12 may determine that an item photographed with a pixel value closer to white is a brighter item, even if the item is the same. Alternatively, the brightness corresponding to the depth of the shelf may be measured in advance, and the display situation analysis unit 12 may calculate the depth by comparing the brightness.

[0071] In this embodiment, the brightness of the items identified from the display image is compared to calculate the depth at which the items are located. Therefore, for example, if a dark area can be recognized, the depth can be determined without identifying each individual item.

[0072] Although the method by which the display situation analysis unit 12 analyzes the display situation has been described for each embodiment, the display situation analysis unit 12 may determine the display situation by combining the methods for analyzing the display situation in each embodiment. For example, even if it is difficult to calculate the depth using one analysis method, it may be possible to calculate the depth using another method.

[0073] For example, a single display image may contain display shelves photographed from above and display shelves photographed from a horizontal direction, as in the display image P2 illustrated in Fig. 2. In this case, the display situation analysis unit 12 can analyze the display situations of both display shelves by combining, for example, the analysis method described in the first embodiment and the analysis method described in the third embodiment.

[0074] Next, an overview of each embodiment of the present invention will be described. Fig. 9 is a block diagram showing an overview of each embodiment of a display situation analysis device according to the present invention. The display situation analysis device shown in Fig. 9 includes an item recognition unit 81 (e.g., item recognition unit 11) and a display situation analysis unit 82 (e.g., display situation analysis unit 12, analysis result processing unit 30). The item recognition unit 81 recognizes items in a display image obtained by photographing the displayed items (e.g., merchandise). The display situation analysis unit 82 analyzes the display situation of the displayed items based on the positions of the recognized items (e.g., based on the depth at which the items are located).

[0075] Such a configuration allows the display status of items to be analyzed.

[0076] Specifically, the display situation analysis unit 82 may calculate the depth at which an item is located based on the distance between the recognized position of the item and a reference position in the display image (for example, the front row of a display shelf). Furthermore, the display situation analysis unit 82 may analyze the display situation of the items from the degree of deviation between a line connecting the recognized positions of each item (for example, a face line) and a comparison reference line in the display image (a straight line used as a reference or a line indicating the front row of a display shelf).

[0077] Alternatively, the display situation analysis unit 82 may analyze the display situation of items as follows. That is, the display situation analysis unit 82 sets the size of each item recognized from a display image captured of items displayed in a predetermined position as a reference state. Then, the display situation analysis unit 82 may analyze the display situation of items from the degree of deviation (for example, ratio) between the size of each item in that reference state and the size of each item recognized from a newly captured display image.

[0078] Alternatively, the display situation analysis unit 82 may analyze the display situation of the items as follows. That is, the display situation analysis unit 82 sets a display image in which an appropriate display situation is captured as a reference state. Then, the display situation analysis unit 82 may analyze the display situation of the items from the degree of deviation (for example, ratio) between the position or size of each item in the reference state and the position or size of each item recognized from the captured display image.

[0079] The display situation analysis unit 82 may also include an analysis result processing unit (for example, analysis result processing unit 30) that processes the analyzed display situation of the items. The analysis result processing unit may then output an image (for example, the image shown in FIG. 3 or FIG. 7) in which the calculated depth at which the item is located can be recognized.

[0080] Furthermore, the display situation analysis unit 82 may calculate the depth at which the recognized item is located, and output the display situation based on that depth.

[0081] 1 and 9 are realized by the hardware resources illustrated in Fig. 10. That is, the configuration illustrated in Fig. 10 includes a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, a ROM (Read Only Memory) 43, an external connection interface 44, and a storage medium 45.

[0082] In the above-described embodiments, the functions shown in the blocks of the display situation analysis device shown in Figures 1 and 9 are implemented by a software program as an example, which is executed by the CPU 10 shown in Figure 10. However, some or all of the functions shown in the blocks of Figures 1 and 9 may be implemented as hardware.

[0083] Furthermore, the present invention described using each embodiment as an example is achieved by supplying a computer program capable of realizing the functions described above to the display situation analysis device, and then having the CPU 10 read and execute the computer program into RAM 11.

[0084] Furthermore, the supplied computer program may be stored in a computer-readable storage device such as a readable / writable memory (temporary storage medium) or a hard disk drive, etc. In such a case, the present invention can be considered to be constituted by the code representing the computer program or the storage medium storing the computer program.

[0085] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0086] This application claims priority based on Japanese Patent Application No. 2014-049001, filed March 12, 2014, the disclosure of which is incorporated herein in its entirety by reference. [Industrial Applicability]

[0087] The present invention can be applied to, for example, a monitoring device for product display shelves. [Explanation of symbols]

[0088] 10 Display situation analysis device 11 Article recognition section 12 Display Status Analysis Department 13 Storage section 20 Image acquisition unit 30 Analysis result processing section P1, P2, P3 display images L1, L2 reference lines L3 Face Line

Claims

1. an item recognition means for recognizing an item in a display image obtained by photographing the item displayed on the display shelf; an analysis means for identifying a face line connecting each recognized article based on the recognition result by the article recognition means; an output means for outputting the face line by superimposing it on the display image; A display situation analysis system equipped with the above.

2. The output means outputs an alert when the face line is not linear. The display situation analysis system according to claim 1 .

3. the analyzing means analyzes whether or not a distance in a depth direction from a front row of the display shelf to each point on the face line is greater than a predetermined standard; The output means outputs an alert when the distance is greater than a predetermined standard. The display situation analysis system according to claim 1 or 2.

4. The output means outputs an image that is color-coded according to the distance from the front row of the display shelf to the recognized position of the item in the rear direction, superimposed on the display image. The display situation analysis system according to any one of claims 1 to 3.

5. The computer Recognizing the items in a display image taken of the items displayed on the display shelf; Based on the recognition results, a face line connecting each recognized object is identified, The face line is superimposed on the display image and output. Display status analysis method.

6. On the computer, A process of recognizing an item in a display image obtained by photographing the item displayed on a display shelf; A process of identifying a face line connecting each of the recognized items based on the recognition result; a process of superimposing the face line on the display image and outputting the same; A program that executes the following.

Images