Shelf allocation information generation system, shelf allocation information generation method, and program

Abstract

To generate shelf allocation information with more stable accuracy that represents a commodity display state.SOLUTION: A shelf allocation information generation system comprises: display means for displaying a photographed image obtained by photographing a commodity shelf; control means for controlling the display means to display guide marks which are referenced by a photographer to determine a photographing position and which indicate a position of the commodity shelf; and generating means for generating shelf allocation information indicating a state in which commodities recognized from the photographed image are arranged on the commodity shelf. The control means estimates a position of the guide mark on the photographed image based on recommended shelf allocation information indicating an arrangement state of the commodities that is recommended for the commodity shelf for which the shelf allocation information is generated, and controls the display of the guide mark at a position on a display screen corresponding to the estimated position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a shelf allocation information generating system, a shelf allocation information generating method, and a program for generating shelf allocation information that represents the product display state. [Background technology]

[0002] In retail stores such as convenience stores and supermarkets, the display position of products has a significant impact on sales, so it is desirable to accurately grasp the display status of products in stores.

[0003] For example, Patent Document 1 discloses a technology that identifies each product displayed in a sales area from a photograph of the sales area, and based on the identification results, places the product master of the corresponding product in the corresponding position of a pre-created fixture model.

[0004] Furthermore, Patent Document 2 describes a camera that displays a composition template on a monitor display device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-187482 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-166250 Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology described in Patent Document 1 mentioned above, the photographic image is adjusted so that the size of the shelves in the photographic image is the same as the size of the shelves in a previously created fixture model, and then individual products are identified and a product master is displayed in the corresponding position on the fixture model.

[0007] However, if the quality of photography varies from photographer to photographer, the accuracy of identifying products will decrease. Therefore, the technology of Patent Document 1 may not be able to accurately reproduce the product display state.

[0008] Furthermore, the technology described in Patent Document 2 is a template for taking photographs, and is not used to reproduce the state of merchandise display.

[0009] The present invention has been made in view of the above problems, and has an object to generate shelf allocation information that represents the product display state with more stable accuracy. [Means for solving the problem]

[0010] A shelf allocation information generation system according to one aspect of the present invention comprises a display means for displaying a photographed image of a product shelf, a control means for controlling the display means to display a guide mark that indicates the position of the product shelf and is referenced by the photographer to determine the shooting position, and a generation means for generating shelf allocation information that indicates the state in which products recognized from the photographed image are arranged on the product shelf, and the control means estimates the position of the guide mark on the photographed image based on recommended shelf allocation information that indicates the recommended product arrangement state for the product shelf for which the shelf allocation information is to be generated, and controls the guide mark to be displayed at a position on the display screen that corresponds to the estimated position.

[0011] A shelf allocation information generating method according to one aspect of the present invention displays a photographed image of a product shelf, controls a display unit to display guide marks that indicate the position of the product shelf and that are referenced by the photographer to determine the photographing position, and generates shelf allocation information that indicates the state in which products recognized from the photographed image are arranged on the product shelf. Also, the shelf allocation information generating method according to one aspect of the present invention estimates the position of the guide marks on the photographed image based on recommended shelf allocation information that indicates the product arrangement state recommended for the product shelf for which the shelf allocation information is to be generated, and controls the display of the guide marks at a position on the display screen that corresponds to the estimated position.

[0012] Note that the scope of the present invention also includes a computer program for implementing each of the above systems or methods on a computer, and a computer-readable non-transitory recording medium on which the computer program is stored. [Effects of the Invention]

[0013] According to the present invention, it is possible to generate shelf allocation information with more stable accuracy. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a functional block diagram showing an example of a functional configuration of a shelf allocation information generating device according to a first embodiment of the present invention. FIG. [Figure 2] FIG. 10 is a diagram illustrating an example of the overall configuration of a shelf allocation information generating system according to a second embodiment of the present invention. [Figure 3] FIG. 10 is a diagram for explaining a usage scene of the shelf allocation information generating system according to the second embodiment of the present invention. [Figure 4] FIG. 10 is a functional block diagram showing an example of a functional configuration of a shelf allocation information generating system according to a second embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating an example of a recognition result output by a recognition unit. [Figure 6] 10 is a flowchart showing an example of a processing flow in a shelf allocation information generating system according to a second embodiment of the present invention. [Figure 7] FIG. 2 is a diagram illustrating an example of an image displayed on a display unit of the imaging device. [Figure 8] FIG. 10 is a diagram showing an example of shelf allocation information generated by a generating unit, the shelf allocation information being generated in the form of an image that can be displayed on a screen. [Figure 9] 10A and 10B are diagrams illustrating other examples of images displayed on the display unit of the imaging device. [Figure 10] FIG. 10 is a functional block diagram showing a functional configuration of a shelf allocation information generating system according to a third embodiment of the present invention. [Figure 11]FIG. 10 is a functional block diagram showing a functional configuration of a shelf allocation information generating system according to a fourth embodiment of the present invention. [Figure 12] FIG. 10 is a functional block diagram showing a functional configuration of a shelf allocation information generating system according to a fifth embodiment of the present invention. [Figure 13] 13 is a flowchart showing an example of a processing flow in a shelf allocation information generating system according to a fifth embodiment of the present invention. [Figure 14] FIG. 13 is a functional block diagram showing a functional configuration of a shelf allocation information generating system according to a sixth embodiment of the present invention. [Figure 15] FIG. 10 is a diagram illustrating an example of shelf allocation information generated by a generating unit. [Figure 16] FIG. 13 is a functional block diagram showing a functional configuration of a shelf allocation information generating system according to a seventh embodiment of the present invention. [Figure 17] FIG. 2 is a diagram illustrating an example of an image displayed on a display unit of the imaging device. [Figure 18] 10A and 10B are diagrams illustrating other examples of images displayed on the display unit of the imaging device. [Figure 19] FIG. 13 is a block diagram showing an example of a basic configuration of a shelf allocation information generating system according to a ninth embodiment of the present invention. [Figure 20] FIG. 13 is a functional block diagram showing an example of a functional configuration of a shelf allocation information generating system according to a ninth embodiment of the present invention. [Figure 21] FIG. 1 is a diagram illustrating an example of a hardware configuration of a computer (information processing device) capable of implementing each embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] First Embodiment A first embodiment of the present invention will be described with reference to the drawings. In this embodiment, a basic configuration for solving the problems of the present invention will be described. FIG. 1 is a functional block diagram showing an example of the functional configuration of a shelf allocation information generation device 10 according to this embodiment. As shown in FIG. 1, the shelf allocation information generation device 10 according to this embodiment includes a control unit 11 and a generation unit 12.

[0016] The control unit 11 controls the display of guide marks, which are referenced in determining the photographing position and indicate the position of the product shelf, on the screen of the imaging device that photographs the product shelf. For example, the control unit 11 converts the position and size of the guide marks into a position and size according to the size of the screen on which they are displayed, and displays the guide marks of the converted size at the converted position on the screen.

[0017] The generation unit 12 generates shelf allocation information indicating the state in which products recognized from a captured image taken with reference to the guide marks are arranged on a product shelf (also referred to as a product display state). At this time, the generation unit 12 generates the shelf allocation information based on position information indicating a position on the captured image corresponding to the position of the guide marks on the screen of the imaging device and the product recognition results. The position information indicating a position on the captured image is, for example, information indicating the position of a shelf board on a product shelf included in the captured image or information indicating the area of ​​the product shelf. The product recognition results are information indicating the products recognized from the captured image.

[0018] In this way, the shelf allocation information generation device 10 according to the present embodiment displays guide marks, which are used as a reference for determining the photographing position, on the screen of the imaging device that photographs the product shelves. This allows the photographer to refer to the guide marks displayed on the screen and photograph the product shelves using the guide marks as a reference. This further reduces the individuality of the photographed images, thereby making it possible to standardize the quality of the photographed images.

[0019] Furthermore, the generation unit 12 generates shelf allocation information indicating the state in which the products recognized from the captured images are arranged on the product shelves, based on position information indicating the positions on the captured images corresponding to the positions of the guide marks on the screen. These recognized products are products recognized from captured images with reduced personal dependency and uniform quality. Furthermore, because the guide marks indicate the positions of the product shelves, the product shelves are included in the positions on the captured images corresponding to the positions of the guide marks on the screen. Therefore, the display positions of the products indicated by the shelf allocation information generated by the generation unit 12 more accurately reproduce the actual display positions. Furthermore, uniforming the quality of the captured images also uniforms the accuracy of the products recognized from the captured images. Therefore, the shelf allocation information generation device 10 can generate shelf allocation information with more stable accuracy.

[0020] <Second embodiment> Next, a second embodiment of the present invention, which is based on the first embodiment described above, will be described with reference to the drawings. FIG. 2 is a diagram showing an example of the overall configuration of a shelf allocation information generation system 1 according to this embodiment. The shelf allocation information generation system 1 shown in FIG. 2 includes a shelf allocation information generation device 100 and an imaging device 200. The shelf allocation information generation device 100 includes the configuration of the shelf allocation information generation device 10 described above. FIG. 3 is a diagram for explaining a usage scene of the shelf allocation information generation system 1 according to this embodiment. FIG. 4 is a functional block diagram showing an example of the functional configuration of the shelf allocation information generation system 1 according to this embodiment. Note that FIG. 4 shows a configuration unique to the present invention, and it goes without saying that each device of the shelf allocation information generation system 1 shown in FIG. 4 may have components not shown in FIG. 4.

[0021] As shown in FIG. 3 , the shelf allocation information generation system 1 according to this embodiment is used when a photographer photographs products displayed on a product shelf 20 installed in a store using an imaging device 200 and generates shelf allocation information from the photographed image. In the shelf allocation information generation system 1 according to this embodiment, the shelf allocation information generation device 100 transmits, to the imaging device 200, marks (hereinafter referred to as guide marks) used to determine a photographing position when photographing the product shelf 20 with the imaging device 200, via a network 300. When the photographer photographs the product shelf 20 using the guide marks as a reference, data (image data) indicative of the photographed image is transmitted from the imaging device 200 to the shelf allocation information generation device 100 via the network 300. The shelf allocation information generation device 100 may be installed within a store, or may be installed in a location separate from the store, such as a data center. The shelf allocation information generation device 100 receives the image data transmitted from the imaging device 200 via the network 300, performs recognition processing using the received image data, and generates shelf allocation information.

[0022] The communication means between the imaging device 200 and the shelf allocation information generation device 100 may be either wired or wireless communication, or may be communication via any of a mobile communication network, a public line network, a LAN (Local Area Network), or a WAN (Wide Area Network). It is preferable that the communication between the imaging device 200 and the shelf allocation information generation device 100 is appropriately adopted depending on the configuration of the imaging device 200, etc. As described above, various methods of communication between the imaging device 200 and the shelf allocation information generation device 100 are conceivable, but detailed description thereof will be omitted as they are not related to the essence of this embodiment.

[0023] In this embodiment, the imaging device 200 that captures an image of the product shelf 20 is a terminal that has an imaging function and a display function, such as a mobile phone terminal, a smartphone, a digital camera, or a tablet.

[0024] In this embodiment, the product shelf 20 is a multi-tiered shelf, and from top to bottom, it is referred to as the first tier, second tier, etc. In addition, hereinafter, when simply referring to a shelf, it refers to the entire product shelf 20, and the part where products are displayed (the internal space of the product shelf 20) is referred to as a shelf level, or simply as a level. In addition, in this embodiment, a shelf board is a board on which products are placed, and includes boards provided inside the product shelf 20 and the bottom board of the lowest level of the product shelf 20.

[0025] (imaging device 200) As shown in FIG. 4, the imaging device 200 includes an imaging unit 210, a display unit 220, an input unit 230, and a control unit 240. The imaging unit 210 captures still images or moving images. The imaging unit 210 is realized by, for example, a lens (not shown) and an image sensor (not shown). If the captured image is a still image, the imaging unit 210 outputs image data representing the still image to the control unit 240. If the captured image is a moving image, the imaging unit 210 outputs image data sequentially obtained from the moving image to the control unit 240. Hereinafter, the image data of the still image or the moving image captured by the imaging unit 210 and output to the control unit 240 will also be simply referred to as image data.

[0026] The display unit 220 has a display screen and displays the image captured by the imaging unit 210 on the display screen based on instructions from the control unit 240. The display unit 220 is realized by, for example, a liquid crystal display. The display unit 220 also displays a so-called finder image. This allows the display unit 220 to function as a finder. Note that the function of displaying a finder image on the display unit 220 is common, and therefore a detailed description thereof will be omitted.

[0027] The input unit 230 detects an input operation from a user (for example, a photographer). In this embodiment, the input unit 230 and the display unit 220 are integrally formed as a touch panel. The input unit 230 may also include an input device (for example, a power button, etc.) provided in the imaging device 200. When the user performs an input operation on the touch panel screen (display unit 220), the input unit 230 detects the position (input position) at which the user's input operation was performed. Then, the input unit 230 outputs input position information indicating the detected input position to the control unit 240. When the user performs an input operation on the input device, the input unit 230 detects the content of the input operation (input content). The input unit 230 outputs the detected input content to the control unit 240.

[0028] The control unit 240 controls each unit within the imaging device 200. The control unit 240 receives image data from the photographing unit 210 and performs display control to display the image data as a photographed image on the display unit 220. As a result, the display unit 220 displays the photographed image. The control unit 240 also transmits this image data to the shelf allocation information generation device 100. The control unit 240 also controls the display unit 220 to display a viewfinder image.

[0029] Furthermore, the control unit 240 causes the display unit 220 to display an image based on the display instruction received from the shelf allocation information generation device 100. This processing will be described later using a flowchart.

[0030] The control unit 240 also receives input position information transmitted from the input unit 230. Then, the control unit 240 executes processing related to the input operation at the input position indicated by the input position information, in accordance with the image displayed on the display unit 220. For example, when information indicating a product shelf 20 (for example, the type or model number of the product shelf) or a screen for selecting a store is displayed on the display unit 220, the control unit 240 detects that a selection item displayed at the input position has been selected. Then, the control unit 240 transmits information indicated by the selection item (selection information) to the shelf allocation information generation device 100. The selection information includes, for example, information indicating a product shelf 20 for which shelf allocation information is to be generated, information indicating a store in which the product shelf 20 is located, etc.

[0031] Furthermore, for example, if an object instructing to take a photograph (for example, a button indicating a shutter) is displayed on the display unit 220 and the input position is the position of this object, the control unit 240 instructs the photographing unit 210 to perform photographing, which is the function indicated by the object. Furthermore, the control unit 240 executes processing according to the input content detected by the input unit 230.

[0032] (Planet allocation information generation device 100) Next, the shelf allocation information generation device 100 will be described. As shown in Fig. 4, the shelf allocation information generation device 100 includes a control unit 110, a generation unit 120, a first storage unit 130, a recognition unit 140, and a second storage unit 150. The first storage unit 130 and the second storage unit 150 may be realized by a single storage unit. Furthermore, the first storage unit 130 and the second storage unit 150 may each be realized by a storage device separate from the shelf allocation information generation device 100.

[0033] Information relating to guide marks is stored in the second storage unit 150. Specifically, the second storage unit 150 stores information such as the shape and position of the guide marks in association with information indicating the product shelf whose position is indicated by the guide marks. The information indicating the product shelf is, for example, an identifier indicating the product shelf. Furthermore, the information relating to the guide marks may be stored in association with an identifier indicating the store (for example, a store number, etc.).

[0034] The information about the guide mark may be an image showing the guide mark. Furthermore, the position of the guide mark in the information about the guide mark may be stored as a relative coordinate value according to the size of the screen on which the guide mark is displayed or the size of the captured image, for example. Furthermore, the information about the guide mark may include a predetermined message.

[0035] In this embodiment, the guide mark is one or more line segments that indicate the position of a shelf board of the product shelf 20. Furthermore, the information about the guide mark is an image that shows the guide mark. Hereinafter, the information about the guide mark is also referred to as guide mark information.

[0036] The control unit 110 transmits the guide mark information stored in the second storage unit 150 to the imaging device 200 that captures an image of a product shelf. At this time, when the control unit 110 receives information (for example, the above-mentioned selection information) related to the product shelf 20 for which shelf allocation information is to be generated from the imaging device 200, the control unit 110 selects guide mark information to be transmitted to the imaging device 200 from the second storage unit 150 in accordance with the selection information, and transmits the selected guide mark information. Specifically, the control unit 110 acquires, from the second storage unit 150, guide mark information associated with information indicating the product shelf 20 and / or store that matches the product shelf 20 and / or store specified by the selection information. Then, the control unit 110 transmits, to the imaging device 200, the acquired guide mark information together with a display instruction to cause the display unit 220 to display the guide mark indicated by the guide mark information. The control unit 110 also outputs the guide mark information to the generation unit 120.

[0037] The timing at which the control unit 110 transmits the guide mark information and the display instruction to the imaging device 200 may be, for example, the time when communication with the imaging device 200 becomes possible. Alternatively, the timing may be the time when the control unit 110 receives a notification from the imaging device 200 that the imaging device 200 is displaying a finder image on the display unit 220. Alternatively, in a configuration in which selection information is received from the imaging device 200, the timing at which the control unit 110 transmits the guide mark information and the display instruction to the imaging device 200 may be the time when the selection information is received. In this embodiment, a description will be given assuming that the control unit 110 is configured to receive selection information from the imaging device 200.

[0038] Furthermore, the timing at which the control unit 110 transmits the guide mark information to the imaging device 200 and the timing at which it outputs the guide mark information to the generation unit 120 may be simultaneous or different. In the case of different timings, the control unit 110 may transmit the guide mark information before the generation unit 120 generates the shelf allocation information.

[0039] Note that, when the selection information received by the control unit 110 from the imaging device 200 is location information indicating the location of a store, the control unit 110 may identify a store or product shelf that matches the received location information or that is located within a predetermined range of the received location information. In this case, the second storage unit 150 may store information identifying the store or product shelf in association with information indicating the store location. The control unit 110 may then transmit guide mark information associated with the information indicating the identified store or product shelf to the shelf allocation information generation device 100.

[0040] The first storage unit 130 stores information for recognizing a product included in an image captured by the imaging device 200 and indicated by image data transmitted from the imaging device 200. Specifically, the first storage unit 130 stores product images (also referred to as master images) and / or feature quantities included in the product images, linked to product identification information (e.g., an identifier for identifying the product, a product name, etc.). Here, the master image is part of product-related data and is an image showing the appearance of the product. In the present embodiment, product-related data is also referred to as a product master. In addition to the master image, the product master includes, for example, sales data, inventory data, product classification, manufacturer name, product name, etc. of the product. Note that the information included in the product master may be data related to the product and may include information other than the information described above. Note that the information stored in the first storage unit 130 may be information necessary for recognizing the product. Note that the information for identifying the product is not limited to the above, and may be any information that allows the product to be identified.

[0041] The recognition unit 140 receives image data of a captured image of the product shelf 20 from the imaging device 200. Then, the recognition unit 140 refers to information for recognizing products stored in the first storage unit 130, and recognizes products included in the captured image from the captured image indicated by the image data based on the received image data. The method by which the recognition unit 140 recognizes products may use, for example, local features, templates, brightness, edges, contours, shapes, color information, depth, etc., or other information. The method by which the recognition unit 140 recognizes products is not particularly limited and may be a general recognition method, so a detailed description thereof will be omitted in this specification.

[0042] The recognition unit 140 outputs, as recognition results, information indicating the captured image, information indicating products recognized from the captured image, and information indicating their positions to the generation unit 120. The information indicating the captured image may be image data itself indicating the captured image, or information identifying the captured image. Generally, multiple products are displayed on the product shelf 20. Therefore, a captured image of the product shelf 20 includes images of multiple products. Therefore, the recognition unit 140 outputs information indicating each of the multiple recognized products and information indicating the display position of each product (display position information). The display position information of each product is, for example, coordinates indicating the position of each product in the captured image. The display position information may be the center coordinates of each product or the coordinates of the bottom center of each product. In this embodiment, the display position information of each product is the coordinates of the bottom center. Furthermore, the information indicating each product may be any information capable of identifying the recognized product, such as the product name or an identifier for identifying the product. Furthermore, the information indicating each product may include a master image of each product and / or an image of the corresponding product cut out from within the photographed image.

[0043] An example of the recognition result output by the recognition unit 140 is shown in FIG. 5. FIG. 5 is a diagram showing an example of the recognition result output by the recognition unit 140. The recognition result shown in FIG. 5 includes an image name indicated by "Image 1" as information indicating the captured image. The recognition result shown in FIG. 5 also includes a product name as information indicating the recognized product. The recognition result shown in FIG. 5 also includes the coordinates of the product's bottom center as information indicating the position of the recognized product on the captured image. As shown in FIG. 5, the recognition unit 140 outputs a recognition result that a product with the product name "Product A" is located at a position where the coordinates of the bottom center are (48, 220). Note that the coordinate system of the captured image has the vertical axis as the x-axis, the horizontal axis as the y-axis, and the upper left corner as the origin.

[0044] The generation unit 120 receives guide mark information from the control unit 110. The generation unit 120 also receives a recognition result from the recognition unit 140, including information indicating a captured image. The generation unit 120 converts the display position of the guide mark indicated by the received guide mark information, depending on the size of the captured image identified by the information indicating the captured image received from the recognition unit 140. The display position of the guide mark on the captured image after this conversion corresponds to the position of the guide mark displayed on the display unit 220. The generation unit 120 may convert the display position of the guide mark before receiving the recognition result from the recognition unit 140. For example, when the recognition unit 140 is recognizing a product, the generation unit 120 may acquire, from the recognition unit 140, information indicating the captured image that is the target of product recognition. The generation unit 120 may also receive, in advance, the size of the captured image to be captured by the imaging device 200 from the imaging device 200.

[0045] The generation unit 120 generates shelf allocation information indicating the state in which the product recognized by the recognition unit 140 is arranged on the product shelf 20 based on information indicating the display position of the converted guide mark on the captured image and the product recognition result by the recognition unit 140.

[0046] As described above, in this embodiment, the guide marks are one or more line segments that indicate the positions of the shelf boards of the product shelf 20. Therefore, the generation unit 120 generates shelf allocation information that indicates a state in which the products recognized from the captured image are arranged at the display positions of the guide marks on the captured image.

[0047] (Processing flow of shelf allocation information generation system 1) Next, a description will be given of the flow of processing in the shelf allocation information generation system 1. Fig. 6 is a flowchart showing an example of the flow of processing in the shelf allocation information generation system 1 according to this embodiment. In Fig. 6, the processing of the imaging device 200 is shown on the left side, and the processing of the shelf allocation information generation device 100 is shown on the right side. Also, the dashed arrow between the flowchart for the imaging device 200 and the flowchart for the shelf allocation information generation device 100 indicates the transmission of information.

[0048] First, when a product shelf 20 for which shelf allocation information is to be generated is selected on the screen of the imaging device 200, the control unit 240 transmits the selection information to the shelf allocation information generating device 100 (step S51).

[0049] The control unit 110 of the shelf allocation information generating device 100 receives the selection information transmitted from the imaging device 200 (step S61), and acquires guide mark information associated with information about the product shelf 20 indicated by the selection information and / or information about the store from the second storage unit 150. Then, the control unit 110 transmits the acquired guide mark information together with a display instruction to the imaging device 200 to cause the display unit 220 to display the guide mark indicated by the guide mark information (step S62). At this time, the control unit 110 also outputs the acquired guide mark information to the generating unit 120.

[0050] The control unit 240 of the imaging device 200 receives the guide mark information and the display instruction transmitted from the shelf allocation information generation device 100 (step S52). Based on the display instruction, the control unit 240 converts the size and display position of the guide mark indicated by the guide mark information in accordance with the screen size, and generates a guide mark image including the converted guide mark (step S53).

[0051] Then, the control unit 240 superimposes the guide mark image generated in step S53 on the viewfinder image and causes the display unit 220 to display the image (step S54).

[0052] Fig. 7 shows an example of an image displayed on the display unit 220. As shown in Fig. 7, on the display unit 220, a guide mark image including a guide mark 9 shown with a thick line is displayed superimposed on a viewfinder image including the product shelf 20. This allows the imaging device 200 to prompt the photographer to take an image at a position where the guide mark 9 and the shelf board of the product shelf 20 coincide with each other.

[0053] Then, when the photographer determines the photographing position based on the displayed guide mark 9 and inputs an instruction to the imaging device 200 to photograph the product shelf 20 at the determined photographing position, the photographing unit 210 photographs the product shelf 20 in response to the photographing instruction (step S55). The control unit 240 transmits image data indicating the photographed image to the shelf allocation information generation device 100 (step S56). This completes the processing of the imaging device 200.

[0054] Thereafter, the recognition unit 140 of the shelf allocation information generation device 100 receives image data indicating the captured image transmitted from the imaging device 200 (step S63). Then, the recognition unit 140 uses the image data received in step S63 to recognize the product included in the captured image indicated by the image data (step S64).

[0055] The generation unit 120 converts the display position of the guide mark indicated by the guide mark information output from the control unit 110 according to the size of the photographed image in which the product was recognized (step S65). Note that step S65 may be performed before step S64 or simultaneously with step S64.

[0056] The generation unit 120 then generates shelf allocation information based on information indicating the display positions of the converted guide marks on the captured image and the recognition result obtained by the recognition unit 140 in step S64. This completes the processing of the shelf allocation information generation device 100.

[0057] The shelf allocation information generated by the generation unit 120 may be a character string in which the position of the shelf on the product shelf 20, information indicating the product, and the arrangement position of the product are arranged in order, or may be in the form of a table, or may be in some other format. For example, the generation unit 120 may generate, as the shelf allocation information, an image to be displayed on a display unit (not shown) in the shelf allocation information generation device 100. Fig. 8 is a diagram showing an example of shelf allocation information generated by the generation unit 120 in the form of an image to be displayed on a display unit.

[0058] The alphabet in each product in Fig. 8 indicates the last letter of the product name. In Fig. 8, for example, a product whose product name is "Product A" is indicated as "A". The generation unit 120 generates the shelf allocation information shown in Fig. 8 using information indicating the position on the captured image corresponding to the position of the guide mark 9 displayed on the display unit 220 of the imaging device 200 shown in Fig. 7 and the recognition result shown in Fig. 5. Note that each product displayed in Fig. 8 may be a character string indicating the product name, a master image of the product, or an image of the product cut out from the captured image.

[0059] In the present embodiment, the generation unit 120 converts the display positions of the guide marks indicated by the guide mark information in accordance with the size of the captured image, but this conversion process may be performed by the control unit 110. In this case, the control unit 110 outputs the converted display positions of the guide marks to the generation unit 120, and the generation unit 120 generates shelf allocation information based on the received display positions.

[0060] (effect) As described above, the shelf allocation information generation device 100 can generate shelf allocation information with more stable accuracy, similar to the shelf allocation information generation device 10 according to the first embodiment described above.

[0061] Furthermore, in this embodiment, the guide marks represent the positions of the shelf boards of the product shelf 20 for which the shelf allocation information is to be generated. Therefore, the position of the guide mark corresponding to the guide mark on a captured image captured with reference to the guide mark is the position of the shelf board on the product shelf 20. Then, based on the position information indicating the position of the guide mark on the captured image and the recognition result, the generation unit 120 generates shelf allocation information indicating a state in which products are arranged at the positions of the guide marks on the captured image indicated by the position information. Therefore, the display positions of products indicated by the shelf allocation information generated by the generation unit 120 more accurately reproduce the actual display positions. Therefore, the shelf allocation information generation device 100 of the shelf allocation information generation system 1 according to this embodiment can generate shelf allocation information with higher accuracy.

[0062] (Variation 1) A first modification of the present embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing another example of an image displayed on the display unit 220 of the imaging device 200.

[0063] If the product shelf 20 is too large, the entire product shelf 20 may not be captured on the display unit 220 of the imaging device 200. Also, if the aisle of the store where the product shelf 20 to be imaged is installed is narrow, the entire product shelf 20 may not be captured on the display unit 220 of the imaging device 200. Also, in the case of a product shelf 20 for displaying small products, an image may be captured from a close distance to the product shelf 20 in order to increase the recognition rate of the products. In such a case, for example, as shown in FIG. 9, the control unit 110 of the shelf allocation information generation device 100 may control the display unit 220 of the imaging device 200 to display a message 8 prompting the user to take an image.

[0064] At this time, the second storage unit 150 may include information about the top or bottom of the product shelf 20 as guide mark information. This information about the top or bottom of the product shelf 20 is stored in advance depending on the size of the product shelf 20, the store where the product shelf 20 is installed, the type of product shelf 20, etc. Then, in the above-mentioned step S61, the control unit 110 receives the selection information and acquires the guide mark information from the second storage unit 150. Then, the control unit 110 identifies the product shelf 20 to be photographed as the product shelf 20 to be photographed separately in upper and lower halves, based on the information about the top or bottom of the product shelf 20 included in the guide mark information associated with the acquired selection information. Thereafter, the control unit 110 controls the display unit 220 to display, for example, "Please photograph this product shelf separately in upper and lower halves," as a message urging the user to photograph the product shelf separately in upper and lower halves, as shown in FIG. 9.

[0065] Furthermore, the control unit 110 may cause the display unit 220 to display a further message depending on the information contained in the guide mark information about the upper or lower part of the product shelf 20. For example, if the guide mark information contains information about the upper part of the product shelf 20, the control unit 110 may perform control to display a message saying "Please take a picture of the upper part first," as shown in Fig. 9.

[0066] As described above, by controlling the control unit 110 of the shelf allocation information generation device 100 to display a message on the screen encouraging users to take photos using a different method, the subjectivity of the captured images can be further reduced, thereby making the quality of the captured images more uniform.

[0067] (Variation 2) In the second embodiment described above, recognition of each product is performed by the recognition unit 140 in the shelf allocation information generation device 100. However, product recognition may also be performed by a device separate from the shelf allocation information generation device 100. That is, the first storage unit 130 and the recognition unit 140 may be implemented by a device separate from the shelf allocation information generation device 100. In this case, the shelf allocation information generation device 100 receives the recognition results from the separate device. This allows the shelf allocation information generation device 100 to reduce the processing load imposed on the shelf allocation information generation device 100. Furthermore, even when the recognition unit 140 is implemented by a device separate from the shelf allocation information generation device 100, the shelf allocation information generation system 1 according to this embodiment can achieve the same effects as the shelf allocation information generation system 1 according to the second embodiment described above.

[0068] <Third embodiment> Next, a third embodiment of the present invention will be described with reference to the drawings. Fig. 10 is a functional block diagram showing the functional configuration of a shelf allocation information generation system 1 according to this embodiment. For ease of explanation, components having the same functions as those included in the drawings explained in the second embodiment are given the same reference numerals, and their explanation will be omitted. Furthermore, the overall configuration of the shelf allocation information generation system 1 according to this embodiment is similar to the configuration of the shelf allocation information generation system 1 in the second embodiment shown in Fig. 2, and therefore its explanation will be omitted.

[0069] In this embodiment, the guide mark is assumed to be two or more line segments indicating the position of the shelf board of the product shelf 20. In this embodiment, the shelf board of the product shelf 20 includes boards on which products are placed (boards provided inside the product shelf 20 and the bottom board of the lowest level of the product shelf 20) and the top board of the product shelf 20. In addition, the information related to the guide mark is assumed to be an image showing the guide mark.

[0070] 10, the shelf allocation information generation system 1 according to this embodiment includes a shelf allocation information generation device 101 and an imaging device 200. The configuration of the imaging device 200 is similar to that of the imaging device 200 according to the second embodiment.

[0071] The shelf allocation information generating device 101 includes a control unit 111, a generating unit 121, a first storage unit 130, a recognizing unit 141, and a second storage unit 150.

[0072] The control unit 111 has the functions of the control unit 110 described above. The control unit 111 further receives from the recognition unit 141 the size of the captured image indicated by the image data received by the recognition unit 141. Then, the control unit 111 converts the display position of the guide mark indicated by the guide mark information according to the size of the captured image. This conversion process is similar to the conversion process described for the generation unit 120 described above, and therefore a detailed description thereof will be omitted.

[0073] Then, the control unit 111 outputs, to the recognition unit 141 and the generation unit 121, information (also called position information) indicating the display position of the converted guide mark on the captured image.

[0074] The recognition unit 141 receives image data of a captured image of the product shelf 20 from the imaging device 200. The recognition unit 141 also receives information indicating the display positions of the converted guide marks on the captured image from the control unit 111. Using the image data and the display positions of the guide marks, the recognition unit 141 references the first storage unit 130 and recognizes products included in the captured image from the received image data. When recognizing products, the recognition unit 141 recognizes products within an area that includes all of the guide marks. In other words, the recognition unit 141 does not recognize products within an area that does not include the guide marks. This allows the recognition unit 141 to reduce the number of products to be subjected to the recognition process, thereby reducing the processing load and processing time of the recognition process. Furthermore, the recognition unit 141 does not recognize products displayed on product shelves other than the product shelf 20 (referred to as the target product shelf) for which shelf allocation information is to be generated. This prevents products that are not displayed on the target product shelf from being recognized as products displayed on the target product shelf.

[0075] Thereafter, the recognition unit 141 outputs the recognition result to the generation unit 121 in the same manner as the recognition unit 140 .

[0076] The generation unit 121 generates shelf allocation information indicating the state in which the product recognized by the recognition unit 141 is arranged on the product shelf 20 based on information indicating the display position of the converted guide mark on the captured image received from the control unit 111 and the product recognition result by the recognition unit 141.

[0077] As described above, in this embodiment, the guide marks are two or more line segments that indicate the positions of the shelf boards of the product shelf 20. Therefore, the generation unit 121 generates shelf allocation information that indicates a state in which the products recognized from the captured image are arranged at the display positions of the guide marks on the captured image other than the guide mark that indicates the position of the top board.

[0078] (effect) As described above, the shelf allocation information generation system 1 according to this embodiment can achieve the same effects as the shelf allocation information generation system 1 according to the second embodiment. Furthermore, in the shelf allocation information generation device 101 of the shelf allocation information generation system 1 according to this embodiment, the recognition unit 141 recognizes products based on position information indicating the display position of the guide mark on the captured image. Therefore, the generation unit 121 of the shelf allocation information generation system 1 according to this embodiment generates shelf allocation information indicating the state in which the products recognized based on the position information are arranged on the product shelf 20. This prevents the shelf allocation information generation device 101 from recognizing products that are not displayed on the target product shelf for which the shelf allocation information is generated as products that are displayed on the target product shelf. Therefore, the shelf allocation information generation device 101 can prevent products other than those included on the target product shelf from being included in the shelf allocation information, thereby generating more accurate shelf allocation information.

[0079] Furthermore, the first storage unit 130 and the recognition unit 141 of the shelf allocation information generation device 101 according to this embodiment may be realized by a device separate from the shelf allocation information generation device 101, similar to the shelf allocation information generation system 1 in the modified example of the second embodiment described above.

[0080] <Fourth embodiment> Next, a fourth embodiment of the present invention will be described with reference to the drawings. Fig. 11 is a functional block diagram showing the functional configuration of a shelf allocation information generation system 1 according to this embodiment. For ease of explanation, components having the same functions as those included in the drawings explained in the second or third embodiment are given the same reference numerals, and their explanation will be omitted. Furthermore, the overall configuration of the shelf allocation information generation system 1 according to this embodiment is similar to the configuration of the shelf allocation information generation system 1 in the second embodiment shown in Fig. 2, and therefore its explanation will be omitted.

[0081] In the second and third embodiments described above, it has been explained that the photographer selects the information indicating the product shelf 20 and / or the information indicating the store where the product shelf 20 is installed, but the present embodiment is not limited to this. In the present embodiment, a method will be described in which the imaging device transmits information indicating the store where the product shelf 20 is installed without the photographer having to input it.

[0082] 11, the shelf allocation information generation system 1 according to this embodiment includes a shelf allocation information generation device 102 and an imaging device 202. The imaging device 202 includes a photographing unit 210, a display unit 220, a control unit 240, and an acquisition unit 250. Note that the imaging device 202 may further include an input unit 230, similar to the imaging device 200 according to the second and third embodiments described above.

[0083] The acquisition unit 250 acquires the location of the own device by measuring the location of the own device. The acquisition unit 250 is realized by, for example, a GPS (Global Positioning System) sensor. The acquisition unit 250 outputs location information indicating the acquired location to the control unit 240. The control unit 240 transmits the location information acquired from the acquisition unit 250 to the shelf allocation information generation device 102.

[0084] 11 , the shelf allocation information generating device 102 includes a control unit 112, a generation unit 120, a first storage unit 130, a recognition unit 140, and a second storage unit 162. The first storage unit 130 and the second storage unit 162 may be realized by a single storage unit. Alternatively, the first storage unit 130 and the second storage unit 162 may each be realized by a storage device separate from the shelf allocation information generating device 102.

[0085] The second storage unit 162 stores information about guide marks, similar to the above-described second storage unit 150. The second storage unit 162 also stores information that identifies a store (for example, a store number) in association with information that indicates the location of the store. The second storage unit 162 may further include information that identifies a product shelf 20 in the store (for example, the type of product shelf 20, etc.).

[0086] The control unit 112 receives the location information transmitted from the imaging device 202. Then, the control unit 112 refers to the second storage unit 162 and identifies a store or a product shelf that matches the received location information or that is located within a predetermined range of the received location information. Then, the control unit 112 acquires guide mark information associated with the information indicating the identified store or product shelf from the second storage unit 162. Then, similar to the above-mentioned control unit 110, the control unit 112 transmits, to the imaging device 202, a display instruction to display, on the display unit 220, the guide mark indicated by the guide mark information together with the acquired guide mark information.

[0087] The operations of the recognition unit 140 and the generation unit 120 of the shelf allocation information generation device 102 in this embodiment are similar to the operations of the recognition unit 140 and the generation unit 120 in the second embodiment described above, and therefore a description thereof will be omitted.

[0088] As described above, the shelf allocation information generation system 1 according to this embodiment can achieve the same effects as the shelf allocation information generation system 1 according to the second embodiment described above. Furthermore, according to this embodiment, the control unit 112 receives the location of the imaging device 202, thereby specifying information about the product shelf 20 that is the subject of the image capture or the store in which the product shelf 20 is installed. This allows the shelf allocation information generation device 102 to generate shelf allocation information without the photographer having to input information indicating the store in which the product shelf 20 is installed.

[0089] Furthermore, the shelf allocation information generation device 102 according to this embodiment may be configured such that the control unit 112 outputs guide mark information to the recognition unit 140, similarly to the shelf allocation information generation device 101 according to the above-described third embodiment. The recognition unit 140 recognizes products based on position information indicating the display position of the guide mark on the captured image, and the generation unit 120 generates shelf allocation information indicating the state in which the products recognized based on the position information are arranged on the product shelf 20. As a result, the shelf allocation information generation system 1 according to this embodiment can achieve the same effects as the shelf allocation information generation system 1 according to the above-described third embodiment.

[0090] Furthermore, in the shelf allocation information generating device 102 according to this embodiment, the control unit 112 may perform control so as to display a message on the screen prompting the user to select a shooting method, similar to the first modification of the second embodiment described above.

[0091] Furthermore, the first storage unit 130 and the recognition unit 140 of the shelf allocation information generation device 102 according to this embodiment may be realized by a device separate from the shelf allocation information generation device 102, similar to the shelf allocation information generation system 1 in the second modification of the second embodiment described above.

[0092] Even with this configuration, the shelf allocation information generating system 1 according to this embodiment can achieve the same effects as the shelf allocation information generating system 1 according to the second embodiment described above.

[0093] <Fifth embodiment> Next, a fifth embodiment of the present invention will be described with reference to the drawings. Fig. 12 is a functional block diagram showing the functional configuration of a shelf allocation information generation system 1 according to this embodiment. For ease of explanation, components having the same functions as components included in the drawings explained in each of the above-mentioned embodiments are given the same reference numerals, and their explanation will be omitted. Furthermore, the overall configuration of the shelf allocation information generation system 1 according to this embodiment is similar to the configuration of the shelf allocation information generation system 1 in the second embodiment shown in Fig. 2, and therefore its explanation will be omitted. In this embodiment, the guide marks are one or more line segments indicating the positions of shelf boards on the product shelf 20.

[0094] 12, the shelf allocation information generation system 1 according to this embodiment includes a shelf allocation information generation device 103 and an imaging device 200. The configuration of the imaging device 200 is similar to that of the imaging device 200 according to the second embodiment. Note that the imaging device 200 may include an acquisition unit 250, similar to the imaging device 202 according to the fourth embodiment.

[0095] The shelf allocation information generating device 103 includes a control unit 113, a generating unit 120, a first storage unit 130, a recognition unit 140, and a second storage unit 173. The first storage unit 130 and the second storage unit 173 may be realized by a single storage unit. Alternatively, the first storage unit 130 and the second storage unit 173 may each be realized by a storage device separate from the shelf allocation information generating device 103.

[0096] After recognizing a product, the recognition unit 140 stores the product recognition result in the second storage unit 173. For example, the recognition unit 140 associates the recognition result shown in FIG. 5 with information indicating the product shelf 20 that is the target of product recognition, and stores the result in the second storage unit 173. At this time, the recognition unit 140 may include the date on which the recognition was performed or the date of the captured image in the recognition result. The recognition unit 140 may also associate information about the store with the recognition result. Hereinafter, the information including the recognition result stored in the second storage unit 173 will also be referred to as a recognition history.

[0097] The control unit 113 receives information about the product shelf 20 for which shelf allocation information is to be generated from the imaging device 200, or identifies the information from the location information. The control unit 113 then acquires, from the second storage unit 173, display position information indicating the display position of the product, which is included in the recognition result associated with the received or identified information. The control unit 113 uses this display position information to estimate the position of a shelf board on the product shelf 20 in the captured image. For example, if the information indicating the position of the recognized product in the captured image is the coordinates of the bottom center of the product, as shown in FIG. 5, the control unit 113 calculates a straight line, for example, by a Hough transform. The product is displayed on the shelf board. Therefore, the calculated straight line is estimated to be a line that includes the shelf board in the captured image. Therefore, the control unit 113 estimates this calculated straight line as the position of the shelf board.

[0098] Then, the control unit 113 transmits to the imaging device 200 an instruction to display a guide mark at a position on the screen of the imaging device 200 that corresponds to the estimated position of the shelf board on the captured image. Here, information indicating the shape of the guide mark (a line segment in this embodiment) may be stored in advance in the second storage unit 173, or may be specified from the imaging device. The shape of the guide mark may also be derived from the position estimated by the control unit 113. This causes the control unit 240 of the imaging device 200 to display the guide mark on the display unit 220. Therefore, the imaging device 200 can urge the photographer to take an image at a position where the guide mark based on the past recognition result and the shelf board of the product shelf 20 match.

[0099] (Processing flow of shelf allocation information generation system 1) Next, a process flow in the shelf allocation information generation system 1 will be described. Fig. 13 is a flowchart showing an example of a process flow in the shelf allocation information generation system 1 according to this embodiment. In Fig. 13, the process of the imaging device 200 is shown on the left side, and the process of the shelf allocation information generation device 103 is shown on the right side. The dashed arrow between the flowchart for the imaging device 200 and the flowchart for the shelf allocation information generation device 103 indicates the transmission of information. Note that the same processes as those in the flowchart shown in Fig. 6 above are assigned the same reference numerals.

[0100] First, when a product shelf 20 for which shelf allocation information is to be generated is selected on the screen of the imaging device 200, the control unit 240 transmits the selection information to the shelf allocation information generating device 103 (step S51).

[0101] The control unit 113 of the shelf allocation information generating device 103 receives the selection information transmitted from the imaging device 200 (step S61). Then, the control unit 113 acquires, from the second storage unit 173, information on the product shelf 20 indicated by the selection information and / or display position information of the recognition history associated with information on the store (step S131).

[0102] Then, the control unit 113 uses the acquired display position information to estimate the position of the shelf board of the product shelf 20 on the captured image (step S132). After that, the control unit 113 sets the estimated position of the shelf board as the display position of the guide mark, and transmits guide mark information including the display position of the guide mark to the imaging device 200 together with a display instruction (step S133). After that, the shelf allocation information generation device 103 and the imaging device 200 perform the same processes as those of the shelf allocation information generation device 100 and the imaging device 200 according to the second embodiment.

[0103] As described above, according to the shelf allocation information generating system 1 of this embodiment, the control unit 113 estimates the position of the guide mark on the captured image from the display position information indicating the display position of the product included in the past recognition result. Then, the control unit 113 controls the image capturing device 200 to display the guide mark at a position on the screen corresponding to the estimated position.

[0104] In the case of the same product shelf, the position of the shelf board estimated from the past recognition results is often the same as the position of the shelf board at the current time on the product shelf 20. Therefore, according to the shelf allocation information generation system 1 according to this embodiment, in addition to the effects of the shelf allocation information generation system 1 according to each of the above-mentioned embodiments, it is possible to generate shelf allocation information with higher accuracy.

[0105] Furthermore, in the shelf allocation information generating device 103 according to this embodiment, the control unit 113 may perform control so as to display a message on the screen prompting the user to take a photograph, similar to the first modification of the second embodiment described above.

[0106] Furthermore, the first storage unit 130 and the recognition unit 140 of the shelf allocation information generation device 103 according to this embodiment may be realized by a device separate from the shelf allocation information generation device 103, similar to the shelf allocation information generation system 1 in the second modification of the second embodiment described above.

[0107] Even with this configuration, the shelf allocation information generating system 1 according to this embodiment can achieve the same effects as the shelf allocation information generating system 1 according to the second embodiment described above.

[0108] Sixth Embodiment Next, a sixth embodiment of the present invention will be described with reference to the drawings. Fig. 14 is a functional block diagram showing the functional configuration of a shelf allocation information generation system 1 according to this embodiment. For ease of explanation, components having the same functions as components included in the drawings explained in each of the above-mentioned embodiments are given the same reference numerals, and their explanation will be omitted. Furthermore, the overall configuration of the shelf allocation information generation system 1 according to this embodiment is similar to the configuration of the shelf allocation information generation system 1 in the second embodiment shown in Fig. 2, and therefore its explanation will be omitted. In this embodiment, the guide marks are one or more line segments indicating the positions of shelf boards on the product shelf 20.

[0109] In the above-described fifth embodiment, the guide marks are estimated from the recognition history. In the present embodiment, however, the guide marks are estimated from the generation history of the shelf allocation information.

[0110] 14, the shelf allocation information generation system 1 according to this embodiment includes a shelf allocation information generation device 104 and an imaging device 200. The configuration of the imaging device 200 is similar to that of the imaging device 200 according to the second embodiment. Note that the imaging device 200 may include an acquisition unit 250, similar to the imaging device 202 according to the fourth embodiment.

[0111] The shelf allocation information generating device 104 includes a control unit 114, a generating unit 120, a first storage unit 130, a recognition unit 140, and a second storage unit 184. The first storage unit 130 and the second storage unit 184 may be realized by a single storage unit. Alternatively, the first storage unit 130 and the second storage unit 184 may each be realized by a storage device separate from the shelf allocation information generating device 104.

[0112] First, the shelf allocation information generated by the generation unit 120 will be described. The shelf allocation information generated by the generation unit 120 may be in the form of an image to be displayed on a display unit (not shown), as shown in FIG. 8 described above. Alternatively, the shelf allocation information generated by the generation unit 120 may be information indicating which product is to be placed at which position, as shown in FIG. 15. FIG. 15 is a diagram showing an example of shelf allocation information. In FIG. 15, a shelf number indicates the position of a shelf on the product shelf 20, and indicates numbers such as the first shelf, the second shelf, etc., from the top of the product shelf 20. Also, in FIG. 15, a slot number indicates the placement position of a product on a shelf, and indicates numbers such as 1, 2, 3, etc., from the left. Here, it is assumed that one product is placed in one slot. The shelf allocation information shown in FIG. 15 corresponds to the first and second shelves of the shelf allocation information shown in FIG. 8.

[0113] The generation unit 120 associates the generated shelf allocation information with information indicating the product shelf 20 for which the shelf allocation information was generated (for example, an identifier indicating the product shelf 20), and stores the information in the second storage unit 184. Hereinafter, the past shelf allocation information stored in the second storage unit 184 will also be referred to as shelf allocation history.

[0114] The control unit 114 receives information about the product shelf 20 for which shelf allocation information is to be generated from the imaging device 200, or identifies the information from the location. The control unit 114 then identifies past shelf allocation information (shelf allocation history) associated with the received or identified information. As shown in FIG. 15 , the past shelf allocation information at least reveals the number of shelves and the number of slots per shelf on the target product shelf 20. This allows the control unit 114 to estimate the position of a shelf on the product shelf 20. If the shelf allocation history includes information about the size of a product as product information, the control unit 114 may estimate the position of the shelf using this information about the size. If the shelf allocation history includes coordinates indicating the position of the product (for example, the position of the bottom center as shown in FIG. 5 ), the control unit 114 may estimate the position of the shelf using this coordinate. The control unit 114 then transmits to the imaging device 200 an instruction to display a guide mark on the screen of the imaging device 200 at a position that corresponds to the estimated position of the shelf in the captured image. The shape of the guide mark may also be derived from the position estimated by the control unit 114. As a result, the control unit 240 of the imaging device 200 causes the guide mark to be displayed on the display unit 220. Therefore, the imaging device 200 can urge the photographer to take an image at a position where the guide mark based on the past shelf allocation information matches the shelf board of the product shelf 20.

[0115] In this way, the shelf allocation information generating system 1 according to the present embodiment can generate shelf allocation information even by using past shelf allocation information.

[0116] Furthermore, in the shelf allocation information generating device 104 according to this embodiment, the control unit 114 may perform control so as to display a message on the screen prompting the user to select a shooting method, similar to the first modification of the second embodiment described above.

[0117] Furthermore, the first storage unit 130 and the recognition unit 140 of the shelf allocation information generation device 104 according to this embodiment may be realized by a device separate from the shelf allocation information generation device 104, similar to the shelf allocation information generation system 1 in the second modification of the second embodiment described above.

[0118] Even with this configuration, the shelf allocation information generating system 1 according to this embodiment can achieve the same effects as the shelf allocation information generating system 1 according to the second embodiment described above.

[0119] (Variation) A modification of the sixth embodiment will be described. In the sixth embodiment, the positions of the guide marks are estimated using past shelf allocation information. However, the control unit 114 may estimate the positions of the guide marks based on recommended shelf allocation information that indicates the arrangement of products recommended for the product shelf for which the shelf allocation information is to be generated, instead of the past shelf allocation information.

[0120] The recommended shelving allocation information is, for example, information indicating the state of shelving allocation recommended by a management company or the like that manages the store. The recommended shelving allocation information may be information such as that shown in FIG. 8, or information such as that shown in FIG. 15, as with the shelving allocation information generated by the generating unit 120, or may be in other forms. The recommended shelving allocation information includes the display positions of products. Therefore, the control unit 114 of the shelving allocation information generating system 1 according to this modification can estimate the positions of guide marks using the recommended shelving allocation information in the same manner as the control unit 114 according to the sixth embodiment.

[0121] As a result, the shelf allocation information generating system 1 according to this modification can obtain the same effects as the shelf allocation information generating system 1 according to the sixth embodiment described above.

[0122] Seventh Embodiment Next, a seventh embodiment of the present invention will be described with reference to the drawings. Fig. 16 is a functional block diagram showing the functional configuration of a shelf allocation information generation system 1 according to this embodiment. For ease of explanation, components having the same functions as components included in the drawings explained in each of the above-mentioned embodiments are given the same reference numerals, and their explanation will be omitted. Furthermore, the overall configuration of the shelf allocation information generation system 1 according to this embodiment is similar to the configuration of the shelf allocation information generation system 1 in the second embodiment shown in Fig. 2, and therefore its explanation will be omitted.

[0123] In the second to sixth embodiments described above, the guide marks are described as line segments that indicate the positions of the shelves of the product shelf 20, but the present embodiment is not limited to this. In the present embodiment, the guide marks are described as indicating the positions of the areas of the product shelf 20.

[0124] 16, the shelf allocation information generation system 1 according to this embodiment includes a shelf allocation information generation device 105 and an imaging device 200. The configuration of the imaging device 200 is similar to that of the imaging device 200 according to the second embodiment. Note that the imaging device 200 may include an acquisition unit 250, similar to the imaging device 202 according to the fourth embodiment.

[0125] The shelf allocation information generating device 105 includes a control unit 115, a generation unit 125, a first storage unit 130, a recognition unit 140, and a second storage unit 155. The first storage unit 130 and the second storage unit 155 may be realized by a single storage unit. Alternatively, the first storage unit 130 and the second storage unit 155 may each be realized by a storage device separate from the shelf allocation information generating device 105.

[0126] The second storage unit 155 stores information about guide marks, similar to the above-described second storage unit 150. In this embodiment, the guide marks indicate the positions of areas of the product shelves 20. The information about the guide marks is an image showing the guide marks.

[0127] Similar to the control unit 110 in the second embodiment described above, the control unit 115 transmits the guide mark information stored in the second storage unit 155 to the imaging device 200 that captures an image of the product shelf together with a display instruction. At this time, when the control unit 115 receives information (for example, the above-mentioned selection information) related to the product shelf 20 for which shelf allocation information is to be generated from the imaging device 200, the control unit 115 selects guide mark information to be transmitted to the imaging device 200 from the second storage unit 155 in accordance with the selection information, and transmits the selected guide mark information. The control unit 115 also outputs the guide mark information to the generation unit 125.

[0128] The control unit 240 of the imaging device 200 receives the guide mark information and display instruction transmitted from the shelf allocation information generating device 105. Then, based on the display instruction, the control unit 240 converts the size and display position of the guide mark indicated by the guide mark information in accordance with the screen size, and generates a guide mark image including the converted guide mark. Then, the control unit 240 superimposes the generated guide mark image on the viewfinder image and displays it on the display unit 220.

[0129] 17 and 18 show examples of images displayed on the display unit 220. As shown in Fig. 17 and 18, on the display unit 220, a guide mark image including guide marks 9 shown in shaded areas is superimposed on a viewfinder image including product shelves 20.

[0130] The guide marks 9 shown in Fig. 17 are guide marks that indicate the outer frame of the product shelf 20, and the guide marks 9 shown in Fig. 18 are guide marks that indicate the positions of the four corners of the product shelf 20. In either case of the guide marks 9 shown in Fig. 17 or 18, the imaging device 200 can prompt the photographer to take an image at a position where the guide marks 9 and the area of ​​the product shelf 20 coincide with each other.

[0131] The photographer determines the photographing position based on the displayed guide mark 9, and inputs an instruction to photograph the product shelf 20 at the determined photographing position to the imaging device 200. Thereafter, the photographing unit 210 of the imaging device 200 photographs the product shelf 20.

[0132] Thereafter, the recognition unit 140 of the shelf allocation information generation device 105 receives image data indicating the captured image. The recognition unit 140 recognizes products from the captured image indicated by the received image data. The recognition unit 140 then outputs information indicating the captured image, information indicating the products recognized from the captured image, and information indicating their positions to the generation unit 125 as recognition results. At this time, similar to the recognition unit 141 in the third embodiment described above, the recognition unit 140 may receive information indicating the display position of the guide mark indicated by the guide mark information on the captured image from the control unit 115. Then, when recognizing the products, the recognition unit 140 may recognize products within an area of ​​the captured image indicated by the display position of the guide mark on the captured image.

[0133] The generation unit 125 receives guide mark information from the control unit 115. The generation unit 125 also receives a recognition result from the recognition unit 140, which includes information indicating the captured image. The generation unit 125 converts the display position of the guide mark indicated by the received guide mark information, depending on the size of the captured image specified by the information indicating the captured image received from the recognition unit 140. The generation unit 125 also estimates the position of the shelf board of the product shelf 20 on the captured image, using display position information indicating the display position of the products, which is included in the received recognition result. At this time, the generation unit 125 estimates the position of the shelf board of the product shelf 20 within the area indicated by the display position of the guide mark.

[0134] Then, based on the estimated shelf position and the recognition result obtained by the recognition unit 140 recognizing the product, the generation unit 125 generates shelf allocation information that indicates the state in which the recognized product is placed at the estimated shelf position.

[0135] As described above, the control unit 115 of the shelf allocation information generating device 105 controls the display of guide marks, which indicate the position of the area of ​​the product shelf 20 on the screen of the imaging device 200, on the screen of the imaging device 200. Then, the generation unit 125 estimates the position of a shelf board of the product shelf 20 on the captured image based on the position of the recognized product on the captured image and the position of the guide mark on the captured image. Then, the generation unit 125 generates shelf allocation information that indicates a state in which the recognized product is arranged at the estimated position.

[0136] As a result, the shelf allocation information generation system 1 according to this embodiment can generate shelf allocation information with more stable accuracy even if the shape of the guide mark is not a line segment representing the position of the shelf board of the product shelf 20 but represents the position of an area of ​​the product shelf 20.

[0137] In the present embodiment, the guide marks have been described as being stored in advance in the second storage unit 155, but the present embodiment is not limited to this. As in the fifth embodiment described above, the guide marks may be estimated from the recognition history. Also, as in the sixth embodiment described above, the guide marks may be estimated from the shelf allocation history or recommended shelf allocation information.

[0138] Furthermore, in the shelf allocation information generating device 105 according to this embodiment, the control unit 115 may perform control so as to display a message on the screen prompting the user to select a shooting method, similar to the first modification of the second embodiment described above.

[0139] Furthermore, the first storage unit 130 and the recognition unit 140 of the shelf allocation information generation device 105 according to this embodiment may be realized by a device separate from the shelf allocation information generation device 105, similar to the shelf allocation information generation system 1 in the second modification of the second embodiment described above.

[0140] Even with this configuration, the shelf allocation information generating system 1 according to this embodiment can achieve the same effects as the shelf allocation information generating system 1 according to each of the above-described embodiments.

[0141] <Eighth embodiment> The input unit 230 of the imaging device 200 in the second, third, and fifth to seventh embodiments described above may have a function of accepting a change in the position of a displayed guide mark. Also, in the case where the imaging device 202 in the fourth embodiment is configured to further include the input unit 230 as described above, the input unit 230 may have a function of accepting a change in the position of a displayed guide mark. In this embodiment, a configuration will be described in which the input unit 230 has a function of accepting a change in the position of a guide mark displayed on the display unit 220. Note that this embodiment has the same configuration as the shelf allocation information generation system 1 in the second to seventh embodiments described above, and therefore description of the drawings will be omitted.

[0142] When a user performs an input operation to change the position of a guide mark using an input medium such as a finger, the input unit 230 transmits information indicating the position of the guide mark changed by the input operation to the shelf allocation information generating device (100-105) via the control unit 240. Then, the control units (110-115) receive the information indicating the changed position of the guide mark and output the received position of the guide mark to the generation units (120, 125). As a result, the generation units (120, 125) generate shelf allocation information based on the position of the changed guide mark on the captured image.

[0143] In this way, the imaging device 200 accepts changes to the guide mark displayed on the display unit 220, so that even if there is a difference between the position of the guide mark displayed on the display unit 220 and the position of the product shelf 20, the photographer can correct the difference. Therefore, the shelf allocation information generation system 1 can improve versatility. Furthermore, because the position of the guide mark can be corrected to the position of the product shelf 20, the accuracy of the shelf allocation information generated by the shelf allocation information generation device of the shelf allocation information generation system 1 can be further improved.

[0144] <Ninth embodiment> A ninth embodiment of the present invention will be described with reference to the drawings. In this embodiment, a shelving allocation information generation system that solves the problems of the present invention will be described. First, the basic configuration of the shelving allocation information generation system 2 according to this embodiment will be described. FIG. 19 is a block diagram showing an example of the basic configuration of the shelving allocation information generation system 2 according to this embodiment. The shelving allocation information generation system 2 shown in this figure can also be said to have the basic configuration of the above-mentioned shelving allocation information generation system 1. As shown in FIG. 19, the shelving allocation information generation system 2 according to this embodiment includes a control unit 21, a generation unit 22, an imaging unit 23, and a display unit 24. The control unit 21, the generation unit 22, the imaging unit 23, and the display unit 24 are connected to each other so that they can communicate with each other. The control unit 21, the generation unit 22, the imaging unit 23, and the display unit 24 may be connected by wire or wirelessly.

[0145] The display unit 24 has a display screen and displays an image on the display screen under the control of the control unit 21.

[0146] The control unit 21 has the function of the control unit 11 of the shelf allocation information generation device 10 according to the first embodiment described above. The control unit 21 also has the function of the control unit 240 of the imaging device 200 according to the second embodiment described above. The control unit 21 controls the display unit 24 to display guide marks that are referenced in determining the imaging position and that indicate the positions of product shelves.

[0147] The photographing unit 23 photographs the product shelves on which products are displayed. The photographing unit 23 corresponds to the photographing unit 210 of the imaging device 200 according to the second embodiment described above. The photographed image taken by the photographing unit 23 is photographed with reference to the guide marks displayed on the display unit 24.

[0148] The generation unit 22 corresponds to the generation unit 12 of the shelf allocation information generation device 10 according to the first embodiment described above. The generation unit 22 generates shelf allocation information indicating a state in which products recognized from a photographed image are arranged on a product shelf, based on the product recognition results and position information indicating a position on the photographed image corresponding to the position of a guide mark on the display screen.

[0149] Next, a shelf allocation information generation system 3 based on the shelf allocation information generation system 2 of FIG. 19 will be described with reference to FIG. 20. For ease of explanation, components having the same functions as components included in the drawings described in the above-described embodiments are denoted by the same reference numerals, and their description will be omitted. As shown in FIG. 20, the shelf allocation information generation system 3 according to this embodiment includes a shelf allocation information generation device 106 and an imaging device 206. The shelf allocation information generation device 106 includes a generation unit 120, a first storage unit 130, and a recognition unit 140. The imaging device 206 includes a photographing unit 210, a display unit 220, and a control unit 246. The recognition unit 140 may be implemented by a device separate from the shelf allocation information generation device 106, as in the modified example of the second embodiment described above.

[0150] The control unit 246 includes the functions of the control unit 240 of the imaging device 200 in the second embodiment described above. The control unit 246 also includes the functions of at least any one of the control units 110, 111, 112, 113, 114, and 115 described above. The control unit 246 controls the display unit 220 to display guide marks that are referenced in determining the imaging position and that indicate the positions of product shelves. The control unit 246 also transmits information related to the guide marks displayed on the display unit 220 to the shelf allocation information generation device 106.

[0151] The shelf allocation information generation device 106 generates shelf allocation information in the same manner as the shelf allocation information generation devices according to the above-described embodiments, based on the image data and information related to the guide marks transmitted from the imaging device 206. At this time, the generation unit 120 generates shelf allocation information indicating the state in which products are arranged on the product shelves, based on the recognition result and position information indicating the position on the captured image that corresponds to the position of the guide marks displayed on the display screen of the display unit 220.

[0152] In this way, in the shelf allocation information generation system 3, the function of controlling the display on the display unit 220 may be provided in the imaging device 206 rather than in the shelf allocation information generation device 106. This function may be realized, for example, by executing an application that can be run on the imaging device 206. Even with this configuration, the shelf allocation information generation system 3 according to this embodiment can obtain the same effects as the shelf allocation information generation system 1 according to each of the above-mentioned embodiments.

[0153] Furthermore, in each of the above-described embodiments, the imaging device and the shelf allocation information generation device included in the shelf allocation information generation system have been described as being realized as separate devices, but the imaging device and the shelf allocation information generation device may be formed as one unit. Furthermore, the shelf allocation information generation system described above may be configured to include a device having a photographing function (corresponding to the photographing unit), a function of displaying guide marks (corresponding to the display unit), a function of generating shelf allocation information (corresponding to the generation unit), a function of controlling the display unit (corresponding to the control unit), and another device having a recognition function (corresponding to the recognition unit).

[0154] <Tenth embodiment> A tenth embodiment of the present invention will be described below. This embodiment has the same configuration as the shelf allocation information generation system 3 according to the ninth embodiment described above, and therefore will be described with reference to FIG. 20. The basic configuration of the shelf allocation information generation system 3 according to this embodiment is the configuration described using FIG. 19. As shown in FIG. 20, the shelf allocation information generation system 3 according to this embodiment includes a shelf allocation information generation device 106 and an imaging device 206. The shelf allocation information generation device 106 includes a generation unit 120, a first storage unit 130, and a recognition unit 140. The imaging device 206 also includes a photographing unit 210, a display unit 220, and a control unit 246.

[0155] In this embodiment, a configuration will be described in which a control unit that performs display control is provided in the imaging device 206, but as in each of the second to eighth embodiments described above, the control unit that performs display control may be provided in the shelf allocation information generation device.

[0156] In the imaging device 206 in this embodiment, the photographing unit 210 outputs image data representing the photographed image to the shelf allocation information generation device 106. However, the photographing unit 210 may output image data representing the photographed image to the control unit 246, similar to the photographing unit 210 of the imaging device 200 described above. In this case, the control unit 246 transmits the received image data to the shelf allocation information generation device 106.

[0157] The control unit 246 of the imaging device 206 in this embodiment, like the control unit 240 in the second embodiment described above, superimposes a guide mark image on the viewfinder image and displays it on the display unit 220. Then, the control unit 246 calculates the match rate between the positions of the product shelves included in the viewfinder image displayed on the display unit 220 and the positions of the guide marks included in the guide mark image. Note that the method for calculating the match rate is not particularly limited. Furthermore, the calculation of the match rate may be performed at predetermined intervals or at a timing according to the magnitude of the match rate. For example, when the match rate is smaller than a predetermined value, the calculation interval may be increased, and when the match rate is equal to or greater than the predetermined value, the calculation interval may be decreased.

[0158] The control unit 246 controls the display unit 220 to change the display mode of the guide mark according to the calculated matching rate. For example, the control unit 246 displays the guide mark in blue when the matching rate is lower than a predetermined threshold, and displays the guide mark in red when the matching rate is equal to or higher than the predetermined threshold. Also, for example, the control unit 246 displays the guide mark in a flashing state when the matching rate is lower than the predetermined threshold, and displays the guide mark in a lit state when the matching rate is equal to or higher than the predetermined threshold. Also, for example, the control unit 246 displays the guide mark as a dotted line when the matching rate is lower than the predetermined threshold, and displays the guide mark as a solid line when the matching rate is equal to or higher than the predetermined threshold.

[0159] In this way, by changing the display mode of the guide mark according to the matching rate calculated by the control unit 246, the imaging device 206 can encourage the photographer to take a picture at a position where the guide mark and the shelf board of the product shelf match. This makes it possible to further reduce the personal nature of the captured image, thereby making it possible to standardize the quality of the captured image. Therefore, the shelf allocation information generation device 106, which generates shelf allocation information using such captured images, can generate shelf allocation information with more stable accuracy, just like the shelf allocation information generation devices according to the above-mentioned embodiments.

[0160] Furthermore, the control unit 246 may further instruct the photographing unit 210 to photograph the product shelves when the matching rate is equal to or greater than a predetermined threshold, and the photographing unit 210 may photograph the product shelves in accordance with the instruction. This allows the photographing unit 210 to automatically photograph the product shelves when the matching rate is equal to or greater than the predetermined threshold. This allows the imaging device 206 to further reduce the personal nature of the photographed images. Therefore, the shelf allocation information generation device 106 that generates shelf allocation information using such photographed images can generate shelf allocation information with more stable accuracy, similar to the shelf allocation information generation devices according to the above-mentioned embodiments.

[0161] <Hardware configuration example> Here, an example of a hardware configuration capable of realizing the shelf allocation information generation device (10, 100 to 106) and the image capture device (200, 202, 206) according to each of the above-described embodiments will be described. The shelf allocation information generation device (10, 100 to 106) and the image capture device (200, 202, 206) described above may be realized as a dedicated device, or may be realized using a computer (information processing device).

[0162] FIG. 21 is a diagram illustrating an example of the hardware configuration of a computer (information processing device) capable of implementing each embodiment of the present invention.

[0163] The hardware of the information processing device (computer) 90 shown in FIG. 21 includes a CPU (Central Processing Unit) 91, a communication interface (I / F) 92, an input / output user interface 93, a ROM (Read Only Memory) 94, a RAM (Random Access Memory) 95, a storage device 97, and a drive device 98 for a computer-readable storage medium 99, all of which are connected via a bus 96. The input / output user interface 93 is a man-machine interface including a keyboard as an example of an input device and a display as an output device. The communication interface 92 is a general communication means for the devices according to the above-described embodiments (FIGS. 1, 4, 10, 11, 12, 14, 16, and 19) to communicate with external devices via a communication network 80. In this hardware configuration, the CPU 91 controls the overall operation of the information processing device 90, which realizes the shelf allocation information generation devices (10, 100-106) and the imaging devices (200, 202, 206) according to the embodiments.

[0164] The present invention, which has been described using the above-mentioned embodiments as examples, is achieved, for example, by supplying a program (computer program) capable of realizing the processes described in the above-mentioned embodiments to an information processing device 90 shown in Fig. 21, and then reading and executing the program in a CPU 91. Note that such a program may be, for example, a program capable of realizing the various processes described in the flowcharts (Figs. 6 and 13) referred to in the description of the above-mentioned embodiments, or a program capable of realizing each unit (each block) shown in the device in the block diagrams shown in Figs. 1, 4, 10, 11, 12, 14, 16, and 19.

[0165] The programs provided to the information processing device 90 may be stored in a readable / writable temporary memory (95) or a non-volatile storage device (97) such as a hard disk drive. That is, in the storage device 97, the program group 97A is a program capable of implementing the functions of the units shown in the shelf allocation information generating device (10, 100-106) and the imaging device (200, 202, 206) in the above-described embodiments. The various stored information 97B is, for example, the captured images, product recognition information, recognition results, recognition history, shelf allocation history, recommended shelf allocation information, and the like in the above-described embodiments. However, when implementing the programs in the information processing device 90, the constituent units of the individual program modules are not limited to the divisions of the blocks shown in the block diagram, and may be appropriately selected by those skilled in the art.

[0166] In the above case, the method of supplying the program to the device can be any currently common method, such as installing the program in the device via a computer-readable recording medium (99) such as a CD (Compact Disk)-ROM or flash memory, or downloading the program from an external source via a communication line (80) such as the Internet. In such a case, the present invention can be considered to be constituted by the code (program group 97A) constituting the computer program or the storage medium (99) on which the code is stored.

[0167] In the above-described embodiments, the functions shown in the blocks in the block diagrams are implemented by a software program as an example, with the CPU 95 shown in Fig. 21 executing the functions. However, some or all of the functions shown in the blocks in the block diagrams may be implemented as hardware circuits.

[0168] It should be noted that the above-described embodiments are preferred embodiments of the present invention, and the scope of the present invention is not limited to the above-described embodiments alone. Those skilled in the art can modify or substitute the above-described embodiments to the extent that they do not deviate from the gist of the present invention, and can construct forms with various modifications. [Explanation of symbols]

[0169] 1, 2, 3 Shelf allocation information generation system 8 Messages 9 Guide Marks 10 Planogram information generation device 11 Control section 12 Generation part 20 product shelves 21 Control section 22 Generation part 23 Photography Department 24 Display section 100~106 Planogram information generation device 110~115 Control unit 120 Generation part 121 Generation part 125 Generation part 130 First memory unit 140 Recognition part 141 Recognition part 150, 155 Second memory unit 162, 173, 184 Second memory section 200, 202, 206 Imaging device 210 Photography Department 220 Display section 230 Input section 240, 246 Control section 250 Acquisition Department

Claims

1. a display means for displaying a photographed image of the product shelf; a control means for controlling the display means to display guide marks that indicate the positions of the product shelves and that are referenced by a photographer to determine a photographing position; a generating means for generating shelf allocation information indicating a state in which the products recognized from the photographed images are arranged on the product shelves, the control means estimates a position of the guide mark on the photographed image based on recommended allocation information indicating a product arrangement state recommended for a product shelf for which shelf allocation information is to be generated, and controls to display the guide mark at a position on the display screen corresponding to the estimated position. Shelf allocation information generation system.

2. the guide mark indicates the position of an area on the screen of the display means of a product shelf for which the shelf allocation information is to be generated, 2. The shelf allocation information generation system according to claim 1, wherein the generation means estimates a position of a shelf of the product shelf in the photographed image based on position information indicating a position of the area and a position of the recognized product in the photographed image, and generates the shelf allocation information indicating a state in which the recognized product is placed in the estimated position.

3. 3. The shelf allocation information generation system according to claim 1 or 2, wherein the control means acquires information indicating a location where the product shelf is photographed, and controls the display means to display a guide mark associated with information on at least one of the store and the product shelf, which is associated with the acquired information indicating the location.

4. the control means receives a position of the guide mark on the screen of the display means that has been changed; 4. The shelf allocation information generating system according to claim 1, wherein the generating means generates the shelf allocation information based on position information indicating a position on the captured image that corresponds to the position of the guide mark on the screen of the display means to which the change has been made, and on a result of recognizing the product.

5. 5. The shelf allocation information generation system according to claim 1, wherein the control means causes the display means to display a message prompting the user to take a photograph depending on at least one of the size of the product shelf, the store in which the product shelf is installed, the type of the product shelf, and the guide mark.

6. 6. The shelf allocation information generation system according to claim 1, wherein the recognized product is a product recognized based on position information indicating a position on the captured image corresponding to the position of the guide mark on the screen of the display means.

7. the control means causes the display means to display the guide mark superimposed on a viewfinder image before photography is started, 7. The shelf allocation information generation system according to claim 1, wherein the display mode of the guide mark is changed and displayed on the display means according to a coincidence rate between the position of the product shelf included in the viewfinder image and the position of the guide mark.

8. The shelf allocation information generation system described in Claim 7, wherein the control means determines whether the matching rate is above a predetermined threshold, and when the matching rate is above the predetermined threshold, issues an instruction to photograph the product shelf.

9. An imaging device and a shelf allocation information generating device are provided, the imaging device includes the display means and an imaging means for imaging a product shelf on which products are displayed, The shelf allocation information generating system according to claim 1 , wherein the shelf allocation information generating device includes the control means and the generating means.

10. An imaging device and a shelf allocation information generating device are provided, the imaging device includes the display means, an imaging means for imaging a product shelf on which products are displayed, and the control means, The shelf allocation information generating system according to claim 1 , wherein the shelf allocation information generating device includes the generating means.

11. A computer comprising: Displaying the photographed image of the product shelf, controlling the display means to display guide marks that are referenced by a photographer to determine a photographing position and that indicate the positions of the product shelves; generating shelf allocation information indicating a state in which the products recognized from the photographed images are arranged on the product shelves; A shelf allocation information generating method that estimates the position of the guide mark on the captured image based on recommended shelf allocation information that indicates the recommended product placement state on the product shelf for which shelf allocation information is to be generated, and controls so that the guide mark is displayed at a position on the display screen that corresponds to the estimated position.

12. A process of displaying a photographed image of the product shelf; a process of controlling a display means to display guide marks that indicate the positions of the product shelves and that are referenced by a photographer to determine a photographing position; A process of generating shelf allocation information indicating a state in which the products recognized from the photographed images are arranged on the product shelves; and a process of estimating the position of the guide mark on the captured image based on recommended allocation information that indicates the arrangement of products recommended on a product shelf for which shelf allocation information is to be generated, and controlling the display of the guide mark at a position on a display screen that corresponds to the estimated position.

Images