Display condition analysis system, display condition analysis method, and program

Abstract

To provide a display state analysis system capable of analyzing a display state of articles.SOLUTION: A display state analysis system includes article recognition means that recognizes articles in a display image from the display image in which the articles displayed in a display shelf are captured, analysis means that identifies a reference line connecting upper ends of each of the articles existing in forefront of one tier below the stacked articles based on a recognition result by the article recognition means, and calculates a distance from the reference line of the stacked article in a depth direction, and output means that outputs information on a display state of the display shelf based on the calculated distance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an arrangement status analysis apparatus, an arrangement status analysis method, and a program for analyzing the arrangement status of articles.

Background Art

[0002] Products displayed in supermarkets and convenience stores generally sell from the products in the front row of the display shelves, so over time, the front of the shelves becomes empty. Therefore, the shelves are organized by store employees making regular rounds and appropriately replenishing products and performing so-called "front-pushing" operations.

[0003] Patent Document 1 describes an inventory status management method for notifying store employees of the inventory status of a product when the product runs out on the display shelf. In the management method described in Patent Document 1, an image of the product display shelf is monitored with a real-time camera, and when a product absence marker provided on the display shelf is detected, the product associated with the product absence marker is identified and a message indicating that the inventory is being confirmed is output.

[0004] Further, Patent Document 2 discloses a product monitoring system that photographs the display state of products on a display shelf with a camera, analyzes the image, and notifies the replenishment timing of the product based on the time-series change of the analysis result.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] There is a limit to the store staff's continuous monitoring of the display status inside the store. However, if there are products in the back yard but the display shelves are continuously empty, sales opportunities will be lost. Also, even if there are products remaining on the display shelves, if the arrangement of those products is disordered, it may dampen consumers' purchasing desire and also potentially result in the loss of sales opportunities.

[0007] In the method described in Patent Document 1, there is a problem that the position where products are displayed cannot be flexibly changed because product absence markers associated with each product are set on the display shelves. Also, in the system described in Patent Document 2, there is an issue that there are restrictions on the shooting location because correct analysis results cannot be obtained unless the bases of each step of the product shelves are all photographed.

[0008] Therefore, an object of the present invention is to provide a display status analysis system, a display status analysis method, and a program capable of analyzing the display status of articles.

Means for Solving the Problems

[0009] The first display status analysis system of the present invention includes: article recognition means for recognizing an article in the display image from a display image in which an article displayed on a display shelf is photographed; based on the recognition result by the article recognition means, specifying a reference line connecting the upper ends of each of the articles existing at the forefront of the step immediately below the stacked articles, and analysis means for calculating the distance in the depth direction from the reference line of the stacked articles; and output means for outputting information regarding the display status of the display shelf based on the distance.

[0010] The first display status analysis method of the present invention is such that a computer recognizes an article in the display image from a display image in which an article displayed on a display shelf is photographed, specifies a reference line connecting the upper ends of each of the articles existing at the forefront of the step immediately below the stacked articles based on the recognition result, calculates the distance in the depth direction from the reference line of the stacked articles, and outputs information regarding the display status of the display shelf based on the distance.

[0011] The first program of the present invention causes a computer to perform a process of recognizing an article in a display image obtained by photographing an article displayed on a display shelf, specifying a reference line connecting the upper ends of each of the articles existing at the forefront of the stage immediately below the stacked articles based on the recognition result, calculating the distance in the depth direction from the reference line of the stacked articles, and outputting information regarding the display status of the display shelf based on the distance.

Advantages of the Invention

[0012] According to the present invention, the display status of an article can be analyzed.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Embodiment 1. FIG. 1 is a block diagram showing a configuration example of a first embodiment of an arrangement status analysis apparatus according to the present invention. The arrangement status analysis apparatus 10 in the present embodiment acquires an arrangement image obtained by photographing articles arranged on a display shelf from an image acquisition unit 20 and outputs the arrangement status of the display shelf to an analysis result processing unit 30.

[0015] The image acquisition unit 20 is realized by, for example, a fixed-point camera for constantly monitoring a display shelf. However, the image acquisition unit 20 itself does not necessarily have a function of photographing an image. The image acquisition unit 20 may be an interface that receives an image photographed by a camera (not shown) that photographs a display shelf via a communication network.

[0016] The image acquisition unit 20 may supply arrangement images to the arrangement status analysis apparatus 10 periodically, or may supply arrangement images to the arrangement status analysis apparatus 10 in response to an instruction from an administrator or the like. The image acquisition unit 20 may supply, for example, the time when the arrangement image was photographed to the arrangement status analysis apparatus 10 as well.

[0017] The analysis result processing unit 30 outputs the arrangement status of the articles analyzed by the arrangement status analysis apparatus 10. The arrangement status obtained by the arrangement status analysis apparatus 10 in the present embodiment includes not only information indicating the state of the articles being arranged, such as disorder in the arrangement, but also an alarm output according to the analyzed arrangement status. That is, it can be said that the arrangement status in the present embodiment is information associated with the arrangement status, such as disorder in the arrangement or an alarm. The mode in which the analysis result processing unit 30 outputs the arrangement status will be described later. Further, an arrangement status analysis unit 12 described later may perform the processing of the analysis result processing unit 30 (that is, the processing of outputting the arrangement status of the articles).

[0018] In this way, by the collaborative operation of the display status analysis device 10, the image acquisition unit 20, and the analysis result processing unit 30, the status of the display shelf can be recognized. Therefore, a configuration including the display status analysis device 10, the image acquisition unit 20, and the analysis result processing unit 30 can be called a display status analysis system.

[0019] The display status analysis device 10 includes an article recognition unit 11, a display status analysis unit 12, and a storage unit 13.

[0020] The article recognition unit 11 recognizes articles in the display image. Specifically, the article recognition unit 11 identifies an article from the display image and recognizes the position, size, range, etc. of the article. Hereinafter, information specifying the article recognized by the article recognition unit 11, the position, size, range, etc. of the article will be referred to as an article recognition result. The article recognition result is, for example, information indicating the area of the recognized article.

[0021] The article recognition unit 11 may identify an article in the display image, for example, by performing template matching between the article in the display image and the images of articles stored in the storage unit 13. However, the method by which the article recognition unit 11 recognizes an article in the image is not limited to template matching. The article recognition unit 11 may recognize an article in the display image using other widely known methods.

[0022] FIG. 2 is an explanatory diagram showing an example of a process of recognizing an article from a display image. When the display image P1 illustrated in FIG. 2 is input, the article recognition unit 11 recognizes an article from the display image P1. The area indicated by the thick-frame rectangle in the display image P2 illustrated in FIG. 2 indicates the area of the article specified as the article recognition result.

[0023] The article recognition unit 11 supplies the display image and the article recognition result to the display status analysis unit 12.

[0024] The display status analysis unit 12 analyzes the display status of the displayed articles. Specifically, the display status analysis unit 12 analyzes the display status of the displayed articles based on the positions of the recognized articles, and outputs the analysis result. Here, as described above, the display status output by the display status analysis unit 12 includes information associated with the display status, such as display disorder and alerts.

[0025] In addition, the recognized positions of the articles described above include not only the absolute positions of the articles recognized from the display image, but also the following. That is, the positions of the articles include the relative positions and positional relationships with other articles, the points and surfaces specified from the recognized articles, the lines specified by connecting the positions of the recognized articles to each other, and the ranges (areas) specified from the lines and other lines. For example, when a point specified from a recognized article and a reference line exist in the display image, the position of the article is recognized from the point and the reference line.

[0026] The display status analysis unit 12 may, for example, calculate the depth at which the recognized article is located, analyze the display status of the displayed articles, and output the result. Here, the depth at which the article is located means the degree of deviation from a preferable position (hereinafter referred to as the reference position) as the position where the article is displayed. For example, in the case of a display shelf, when the front row of each display shelf is defined as the reference position, the depth at which the article is located is the distance in the depth direction from the front row of the display shelf. In the following description, calculating the depth at which the article is located may sometimes be simply referred to as calculating the depth.

[0027] As described above, generally, the products in the front row of the display shelf are sold first. Therefore, as the number of products decreases, the products on the display shelf tend to remain deeper. The deeper the position of the articles on the display shelf, the more difficult it is for consumers to see the products, so it cannot be said that the display status of the articles is preferable. In other words, it can be said that the better the display status is, the shorter the depth at which the article is located (that is, the closer the article is to the front of the display shelf). Therefore, in the following description, it is assumed that the display status analysis unit 12 analyzes the display status from the depth at which the recognized article is located.

[0028] In the first embodiment, the display situation analysis unit 12 calculates the depth from the degree of deviation between the recognized position of the article and the reference position. The position indicating the article is predetermined for each individual article. Since the height and width of the article vary for each article, for example, the center of the bottom side of the article may be defined as the position of the article.

[0029] For example, when the display image is an image captured by a fixed-point camera, the display situation analysis unit 12 may specify in advance the position of the frontmost row of the display shelf based on the angle of view of the camera, the distance to the display shelf, etc. Also, when the range in which the display image is captured is not fixed, the display situation analysis unit 12 may estimate the position of the display shelf from the display image and use the estimated position of the display shelf as the reference position.

[0030] The display situation analysis unit 12 calculates the depth at which the article is located based on the distance between the recognized position of the article and the reference position. For example, when the frontmost row of the display shelf is defined as the reference position, the display situation analysis unit 12 may calculate the distance between the line indicating the frontmost row and the position of the article as the depth.

[0031] The display situation analysis unit 12 may only calculate the depth, or may judge the desirability of the display situation of the article as a result of calculating the depth. For example, the display situation analysis unit 12 may judge that the display situation is more favorable as the total distance (i.e., depth) from the reference position of each article is smaller.

[0032] The display situation analysis unit 12 supplies the analysis result to the analysis result processing unit 30. For example, the display situation analysis unit 12 may supply information indicating the position of an article whose degree of deviation from the reference position is greater than a predetermined standard to the analysis result processing unit 30 as the analysis result. Also, the display situation analysis unit 12 may supply information indicating the positions of all articles and the degree of deviation from the reference position to the analysis result processing unit 30 as the analysis result. Note that the analysis results listed here are examples, and the display situation analysis unit 12 may supply any information necessary for the processing performed by the analysis result processing unit 30 as the analysis result.

[0033] The analysis result processing unit 30 performs arbitrary processing indicating the display status using the analysis result supplied from the display status analysis unit 12. For example, the analysis result processing unit 30 may output an image in a recognizable manner of the depth at which the calculated article is located using the analysis result.

[0034] FIG. 3 is an explanatory diagram showing an example of an output result output by the analysis result processing unit 30. In the example shown in FIG. 3, it shows that the analysis result processing unit 30 superimposes and displays the blinking image T at the position in the display image P1 indicating the article whose degree of deviation from the reference position is greater than a predetermined reference.

[0035] In addition, the analysis result processing unit 30 may notify a person concerned of an alert indicating that it is time to replenish the article. Also, for example, when the display shelf has a function of changing its inclination, the analysis result processing unit 30 may notify a device (not shown) that controls the display shelf so as to tilt the display shelf and automatically move the article forward, of information indicating the position of the article and the like.

[0036] Further, the analysis result processing unit 30 may calculate the sales volume of the article based on the depth at which the article is located and the size of the article itself, etc., and automatically perform an ordering process according to the sales volume. This is because an article not in front of the display shelf is considered to have been sold.

[0037] Also, the analysis result processing unit 30 may evaluate the degree of disorder in the sales floor from the analysis results of each article, and for example, aggregate them so that they can be compared among a plurality of stores.

[0038] The storage unit 13 stores information necessary for the recognition process of the article and the process of calculating the depth. The storage unit 13 may store, for example, an image of the article being displayed and the size of the article. Also, the storage unit 13 may store parameter information indicating the position, angle of view, and shooting range of the camera that shoots the display image. The storage unit 13 is realized by, for example, a magnetic disk or the like.

[0039] The article recognition unit 11 and the display status analysis unit 12 are realized by the CPU (Central Processing Unit) of a computer that operates according to a program (display status analysis program). For example, the program is stored in the storage unit 13, and the CPU may read the program and operate as the article recognition unit 11 and the display status analysis unit 12 according to the program.

[0040] Also, the article recognition unit 11 and the display status analysis unit 12 may each be realized by dedicated hardware. Further, in the present embodiment, a case where the analysis result processing unit 30 is not included in the display status analysis apparatus 10 is illustrated, but the display status analysis apparatus 10 may be configured to include the analysis result processing unit 30.

[0041] Next, the operation of the display status analysis apparatus 10 of the present embodiment will be described. FIG. 4 is a flowchart showing an operation example of the display status analysis apparatus 10 of the first embodiment.

[0042] The article recognition unit 11 recognizes articles in the input display image from the display image input from the image acquisition unit 20 (step S11). Then, the display status analysis unit 12 analyzes the display status of the articles on display based on the positions of the recognized articles (step S12). Specifically, the display status analysis unit 12 may calculate the depth in which the recognized articles are located and analyze the display status of the articles on display, or may calculate the depth from the degree of deviation between the positions of the recognized articles and the reference state.

[0043] As described above, in the present embodiment, based on the positions of the articles recognized by the display situation analysis unit 12, the display situation of the displayed articles is analyzed. Specifically, the display situation analysis unit 12 calculates the depth in which the recognized articles are located in the display image and analyzes the display situation. The display situation analysis unit 12 calculates, for example, the depth in which the article is located based on the distance between the position of the recognized article and the reference position. With such a configuration, the display situation of the articles can be analyzed. As a result, based on the analyzed display situation, out-of-stock of products and disorder in the sales floor can be detected and suppressed, so that the loss of sales opportunities can be reduced and the sales can be improved. Furthermore, in the present embodiment, since the depth can be calculated without using a special photographing device, there is also an advantage that it is easy to introduce.

[0044] Next, a modification of the present embodiment will be described. In the first embodiment, the case where the foremost row of each display shelf is defined as the reference position has been described. Note that there is also a case where articles are stacked and displayed not only in one row but also in multiple rows. In this case, the display situation analysis unit 12 may use, as the reference position, a line connecting the upper ends of the articles existing in the row one row below the row in which the articles are stacked.

[0045] FIG. 5 is an explanatory diagram showing an example of the reference position. The reference line L1 indicates the foremost row of the display shelf. The reference line L2 indicates a line connecting the upper bases of the first row of the articles displayed in the foremost row. By providing different reference positions in this way, it is possible to judge the display situation of the stacked articles in the same manner as described above.

[0046] By defining the reference position in this way, the depth in which the articles stacked and displayed in multiple rows are located can also be calculated, so that even if the form of the sales floor is diverse, it is possible to judge the degree of orderliness of the sales floor. Embodiment 2. Next, a second embodiment of the display situation analysis apparatus according to the present invention will be described. The configuration of the display situation analysis apparatus of the present embodiment is the same as that of the first embodiment.

[0047] In the second embodiment, the arrangement state analysis unit 12 calculates the depth from the degree of deviation between the line connecting the positions of the recognized items and the comparison reference line. Hereinafter, the line connecting the positions of the recognized items will be referred to as the "face line". Note that the method for determining the position of an item is the same as the method used in the first embodiment. Note that the arrangement state analysis unit 12 may estimate the face line using the technique used in the Hough transform.

[0048] FIG. 6 is an explanatory diagram showing an example of a face line. When the positions of the items are recognized as illustrated in the display image P2, the arrangement state analysis unit 12 estimates a face line L3 indicated by a broken line in the display image P3.

[0049] The comparison reference line used in the second embodiment may be any line that can be extracted from the display image, or a line defined independently of the display image. For example, when the comparison reference line is defined as a simple straight line, the arrangement state analysis unit 12 may calculate the depth based on the degree of distortion of the face line connecting the positions of the items.

[0050] Also, for example, when the straight line indicating the frontmost row of each display shelf is defined as the comparison reference line, the arrangement state analysis unit 12 may calculate the depth from the distance between each point on the face line and the comparison reference line in the same manner as in the first embodiment.

[0051] In a state where the arrangement of the items is not disordered, usually, since the items are arranged side by side, the identified face line is generally in a straight line state. That is, the fact that the face line is in a straight line state can be judged as picking from the backyard or the items being pushed forward. On the other hand, when the arrangement of the items is disordered, the face line does not become straight and is distorted or stepped. That is, the fact that the face line is not in a straight line state can be judged as there being a possibility of out-of-stock or shortage, and it is necessary to raise an alert.

[0052] In other words, it can be said that the closer the face line is to a straight line (i.e., the smaller the difference from the comparison reference line), the better the display state of the articles. Therefore, by specifying such a face line, the display situation analysis unit 12 can analyze the display situation.

[0053] FIG. 7 is an explanatory diagram showing another example of the output result output by the analysis result processing unit 30. The analysis result processing unit 30 superimposes and displays the face line estimated by the display situation analysis unit 12 on the original display image. In the example shown in FIG. 7, since the articles being displayed are disordered, the face line is in a broken line shape. For example, in a state where the articles being displayed are not disordered, a substantially straight face line is superimposed on the display image.

[0054] Furthermore, in the example shown in FIG. 7, the analysis result processing unit 30 superimposes an image indicating that the display is disordered at a position with a long depth. In this way, the analysis result processing unit 30 may change the image to be superimposed according to the degree of disorder. Specifically, the analysis result processing unit 30 may superimpose an image in which the degree of deviation is recognizable at a position with a larger degree of deviation from the comparison reference line. For example, the analysis result processing unit 30 may superimpose and display an image color-coded according to the degree of display disorder, such as a heat map, on the display image.

[0055] Note that FIG. 7 shows an example in which another image is superimposed without converting the original display image, but the analysis result processing unit 30 may also convert the original display image itself. For example, the analysis result processing unit 30 may perform a conversion process for changing the contrast of the image of the portion where the articles with disordered display are shown, so as to emphasize the degree of disorder. By visualizing the degree of disorder of the display shelf in this way, the situation of the entire display shelf can be easily grasped.

[0056] As described above, in the present embodiment, the display situation analysis unit 12 calculates the depth at which the article is located from the degree of deviation between the face line and the comparison reference line. Even with such a configuration, similar to the effects of the first embodiment, the display situation of the articles can be analyzed. Embodiment 3. Next, a third embodiment of the display situation analysis apparatus according to the present invention will be described. The configuration of the display situation analysis apparatus in this embodiment is the same as that in the first embodiment.

[0057] In the third embodiment, the display situation analysis unit 12 uses, as a reference state, the size of each article recognized from a display image in which the articles displayed at a predetermined position are photographed, and calculates the depth in which the article is located from the degree of deviation between the size of each article in that reference state and the size of each article recognized from a newly photographed display image. The display situation analysis unit 12 may use, for example, the size of each article recognized from the display image when the article is displayed at an appropriate position as the reference state. The appropriate position means a preferable position as a position for display, for example, the frontmost row of the display shelf. Hereinafter, an image of each article recognized from the display image when the article is displayed at an appropriate position will be referred to as a reference image. The reference image is stored in the storage unit 13, for example.

[0058] Generally, it is preferable that the article is displayed at the very front of the display shelf. For example, when an article is photographed horizontally with a fixed-point camera or the like, the article existing in the front is photographed larger than the article existing in the back. In other words, the farther the article is located in the back, the smaller the size of the photographed article becomes. Therefore, an image of the article in the frontmost row of the display shelf may be used as the reference image.

[0059] The display situation analysis unit 12 may calculate the depth from the difference (degree of deviation) between the recognized image of the article and the reference image. At this time, the display situation analysis unit 12 may calculate the depth according to the ratio between the recognized image of the article and the reference image based on the angle of view of the camera that photographs the display image, the distance to the display shelf, and the like.

[0060] FIG. 8 is an explanatory diagram showing another example of the process of recognizing an article from a display image. In the example shown in FIG. 8, it is assumed that the articles A1, A3, and A4 are displayed at the very front of the display shelf, and the article A2 is located deeper than the other articles.

[0061] The upper shelf illustrated in Fig. 8 makes it difficult to calculate the depth only from the position of the recognized article depending on the angle of the camera that takes the display image. However, the article A2 located in the back is photographed smaller compared to the reference image. Therefore, the display situation analysis unit 12 can calculate the depth from the ratio between the reference image and the image of the photographed article A2.

[0062] As described above, in the present embodiment, the display situation analysis unit 12 calculates the depth at which the article is located from the degree of deviation between the size of each article in the reference state and the size of each article recognized from the display image. Therefore, even when the depth at which the recognized article is located cannot be known only from the position of the recognized article, it becomes possible to calculate the depth.

[0063] Note that articles may be displayed on the display shelf in various orientations. Therefore, in order for the article recognition unit 11 to be able to identify the article, it is preferable to prepare reference images of the article taken from a plurality of angles. Thus, the storage unit 13 may store reference images of the same article taken from a plurality of directions.

[0064] Also, there are preferable orientations for displaying articles. Therefore, information indicating that it is a more preferable orientation may be set for each reference image among the reference images of the same article taken from a plurality of directions. In this case, the display situation analysis unit 12 may analyze the display situation in consideration of the orientation of the article in addition to the depth at which each article is located. This also applies to other embodiments. For example, the display situation analysis unit 12 may determine that the display situation is better as the number of articles displayed in the preferable orientation is larger. Embodiment 4. Next, a fourth embodiment of the display situation analysis apparatus according to the present invention will be described. The configuration of the display situation analysis apparatus of the present embodiment is the same as that of the first embodiment.

[0065] In the fourth embodiment, the display situation analysis unit 12 calculates the depth at which an article is located from the deviation between the position or size of each article in the reference state, which is a display image indicating an appropriate display state, and the position or size of each article recognized from the newly captured display image. An appropriate display state means a state in which articles are displayed at appropriate positions without being disordered. Hereinafter, a display image indicating an appropriate display state will be referred to as a "reference display image". The reference display image is stored in the storage unit 13, for example.

[0066] The display situation analysis unit 12 may calculate the depth at which the article is located based on the difference (deviation) in the position between the article in the reference display image and the article in the newly captured display image. Further, the display situation analysis unit 12 may calculate the depth at which the article is located based on, for example, the difference (deviation) in the size between the article in the reference display image and the article in the newly captured display image. The display situation analysis unit 12 may determine that the article is present closer as the difference is smaller and that the display state is good, for example.

[0067] At that time, it is preferable to use, as a reference image, an image that has been captured in the past and has the same range as the range shown in the display image. This is because it is easy to detect the difference if the images are captured in the same range.

[0068] In this embodiment, the depth at which the article is located is calculated by comparing each article recognized for each display image. Therefore, for example, when the range where the article is placed hardly changes, it is possible to simply calculate the depth. Embodiment 5. Next, a fifth embodiment of the display situation analysis apparatus according to the present invention will be described. The configuration of the display situation analysis apparatus of this embodiment is the same as that of the first embodiment.

[0069] Depending on the display shelf, the deeper an item is displayed, the darker it may appear. Therefore, in the fifth embodiment, the display situation analysis unit 12 calculates the depth based on the deviation degree between the brightness of the recognized item and the pixel value indicating the reference brightness. The display situation analysis unit 12 may determine that an item that is darker than the reference brightness is located deeper.

[0070] For example, when using brightness as the information indicating brightness, the display situation analysis unit 12 may determine that an item captured with a pixel value closer to white is a brighter item even if it is the same item. Also, the brightness corresponding to the depth of the item shelf may be measured in advance, and the display situation analysis unit 12 may calculate the depth by comparing it with the measured brightness.

[0071] In this embodiment, the depths at which the items are located are calculated by comparing the brightnesses of the items specified from the display image. Therefore, for example, if a dark portion can be recognized, the depth can be determined without specifying each item.

[0072] As described above, the method by which the display situation analysis unit 12 analyzes the display situation has been explained for each embodiment. However, the display situation analysis unit 12 may determine the display situation by combining the methods for analyzing the display situation in each embodiment. For example, even if it is difficult to calculate the depth by one analysis method, it may be possible to calculate the depth by another method.

[0073] For example, as in the display image P2 illustrated in FIG. 2, in one display image, there may be a display shelf photographed from above and a display shelf photographed from the horizontal direction. In this case, the display situation analysis unit 12 can analyze the display situations of both display shelves by, for example, combining the analysis method described in the first embodiment and the analysis method described in the third embodiment.

[0074] Next, an overview of each embodiment of the present invention will be described. FIG. 9 is a block diagram showing an overview of each embodiment of the display status analysis device according to the present invention. The display status analysis device shown in FIG. 9 includes an article recognition unit 81 (for example, article recognition unit 11) and a display status analysis unit 82 (for example, display status analysis unit 12, analysis result processing unit 30). The article recognition unit 81 recognizes articles in the captured display image from the display image of the displayed articles (for example, products). The display status analysis unit 82 analyzes the display status of the displayed articles based on the positions of the recognized articles (for example, based on the depth in which the articles are located).

[0075] With such a configuration, the display status of the articles can be analyzed.

[0076] Specifically, the display status analysis unit 82 may calculate the depth in which the article is located based on the distance between the position of the recognized article and a reference position (for example, the frontmost row of the display shelf) in the display image. Further, the display status analysis unit 82 may analyze the display status of the articles from the degree of deviation between a line connecting the positions of the recognized articles to each other (for example, a face line) and a comparison reference line (a straight line serving as a reference or a line indicating the frontmost row of the display shelf) in the display image.

[0077] Alternatively, the display status analysis unit 82 may analyze the display status of the articles as follows. That is, the display status analysis unit 82 sets the size of each article recognized from the display image in which the article displayed at a predetermined position is captured as a reference state. Then, the display status analysis unit 82 may analyze the display status of the articles from the degree of deviation (for example, ratio) between the size of each article in the reference state and the size of each article recognized from the newly captured display image.

[0078] In addition, the display status analysis unit 82 may analyze the display status of the articles as follows. That is, the display status analysis unit 82 sets the display image in which the appropriate display state is captured as the reference state. Then, the display status analysis unit 82 may analyze the display status of the articles from the deviation (for example, ratio) between the position or size of each article in the reference state and the position or size of each article recognized from the captured display image.

[0079] Further, the display status analysis unit 82 may include an analysis result processing unit (for example, the analysis result processing unit 30) that processes the analyzed display status of the articles. Then, the analysis result processing unit may output an image in a recognizable manner of the depth at which the calculated article is located (for example, the images illustrated in FIGS. 3 and 7).

[0080] In addition, the display status analysis unit 82 may calculate the depth at which the recognized article is located and output the display status based on the depth.

[0081] Note that each part of the display status analysis device shown in FIGS. 1 and 9 is realized in the hardware resources illustrated in FIG. 10. That is, the configuration shown in FIG. 10 includes a CPU (Central Processing Unit) 41, a RAM (Random Access Memory) 42, a ROM (Read Only Memory) 43, an external connection interface 44, and a storage medium 45.

[0082] In each of the above-described embodiments, a case where the functions indicated by the respective blocks in the display status analysis device shown in FIGS. 1 and 9 are realized by a software program as an example executed by the CPU 10 shown in FIG. 10 has been described. However, the functions indicated by the respective blocks shown in FIGS. 1 and 9 may be realized partially or entirely as hardware.

[0083] In addition, the present invention described by taking each embodiment as an example can be achieved by supplying a computer program capable of realizing the functions described above to the display situation analysis device, and then causing the CPU 10 to read and execute the computer program from the RAM 11.

[0084] In addition, the supplied computer program may be stored in a readable and writable memory (temporary storage medium) or a computer-readable storage device such as a hard disk device. And in such a case, the present invention can be regarded as being constituted by the code representing such a computer program or the storage medium storing such a computer program.

[0085] As described above, the present invention of the present application has been described with reference to the embodiments, but the present invention of the present application is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0086] This application claims the priority based on Japanese Patent Application No. 2014-049001 filed on March 12, 2014, and incorporates all of its disclosures herein.

Industrial Applicability

[0087] The present invention can be applied to, for example, a monitoring device for a display shelf of products.

Explanation of Reference Numerals

[0088] 10 Display situation analysis device 11 Article recognition unit 12 Display situation analysis unit 13 Storage unit 20 Image acquisition unit 30 Analysis result processing unit P1, P2, P3 Display images L1, L2 Reference lines L3 Face line

Claims

1. Article recognition means for recognizing an article in the display image, the display image being an image of an article displayed on a display shelf; Analysis means for specifying a reference line connecting the upper ends of the respective articles existing at the forefront in the first row based on the recognition result by the article recognition means, and calculating a distance in the depth direction from the reference line to a predetermined position of each of the articles at the forefront among the articles stacked in the second row; Output means for outputting an alert as information regarding the display status of the display shelf when the distance is greater than a predetermined reference; A display status analysis system comprising the above.

2. The output means outputs information indicating the position of the article whose distance is greater than a predetermined reference as information regarding the display status of the display shelf. The display status analysis system according to Claim 1.

3. The output means outputs, as information regarding the display status of the display shelf, an image that can recognize that the distance is greater than a predetermined reference, color-codes the image according to the distance, and superimposes the image on the display image for output. The display status analysis system according to Claim 1 or 2.

4. A computer: recognizes an article in the display image, the display image being an image of an article displayed on a display shelf; specifies a reference line connecting the upper ends of the respective articles existing at the forefront in the first row based on the recognition result, and calculates a distance in the depth direction from the reference line to a predetermined position of each of the articles at the forefront among the articles stacked in the second row; outputs an alert as information regarding the display status of the display shelf when the distance is greater than a predetermined reference. A display status analysis method.

5. A program for causing a computer to: perform a process of recognizing an article in the display image, the display image being an image of an article displayed on a display shelf; perform a process of specifying a reference line connecting the upper ends of the respective articles existing at the forefront in the first row based on the recognition result, and calculating a distance in the depth direction from the reference line to a predetermined position of each of the articles at the forefront among the articles stacked in the second row; perform a process of outputting an alert as information regarding the display status of the display shelf when the distance is greater than a predetermined reference. A program for causing the above to be executed.

Images