Furniture detection device, furniture detection method, and program

Abstract

To accurately detect each piece of furniture from an image in which multiple pieces of furniture have been captured. [Solution] The furniture detection device comprises a shelf count identification unit, a peak detection unit, and a furniture detection unit. The shelf count identification unit identifies the progression of the number of shelves in the horizontal direction of multiple furniture items included in an image, based on positional information of multiple shelves in an image captured to include multiple furniture items. The peak detection unit detects the horizontal position that is the minimum value among the progression of the number of shelves in the horizontal direction. The furniture detection unit detects furniture items from the image based on the detected horizontal position.

Description

【Technical Field】 【0001】 The present disclosure relates to a furniture detection device and the like. 【Background Art】 【0002】 In retail, manufacturing, etc., there are cases where products displayed on shelf furniture (hereinafter sometimes simply referred to as furniture) are recognized. For example, Patent Document 1 describes acquiring information indicating the display status of furniture on which products are displayed from a terminal used by a customer, and identifying missing products based on the information indicating the display status. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Unexamined Patent Application Publication No. 2023-115988 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 By the way, when a plurality of furniture are arranged side by side, in order to accurately detect the products displayed on the furniture, it is desired to accurately detect the furniture from an image in which a plurality of furniture are imaged. 【0005】 An example of the object of the present disclosure is to provide a furniture detection device and the like that can accurately detect each piece of furniture from an image in which a plurality of furniture are imaged. 【Means for Solving the Problems】 【0006】 The furniture detection device according to one aspect of the present disclosure is a shelf number specifying means for specifying a transition of the number of shelf boards in the lateral direction of the plurality of furniture included in the image based on position information regarding a plurality of shelf boards in the image imaged so as to include the plurality of furniture; a peak detection means for detecting a position in the lateral direction that is a minimum value among the transitions of the number of shelf boards in the lateral direction; A fixture detection means for detecting fixtures from the image based on the detected lateral position, It is equipped with. 【0007】 A furniture detection method in one aspect of this disclosure involves at least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. Execute the process. 【0008】 A program in one aspect of this disclosure is installed on at least one computer. Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. Execute the process. 【0009】 Each program may be stored on a non-temporary storage medium that is readable by at least one computer. [Effects of the Invention] 【0010】 According to this disclosure, it becomes possible to accurately detect each piece of furniture from an image in which multiple pieces of furniture have been captured. [Brief explanation of the drawing] 【0011】 [Figure 1] This is a block diagram showing an example configuration of a furniture detection device. [Figure 2]It is an explanatory diagram showing an image of the miscellaneous goods being imaged. [Figure 3] It is an explanatory diagram showing position information regarding the shelf board. [Figure 4] It is an explanatory diagram showing the correspondence relationship between the graph representing the transition of the end points and the number of shelf boards of the shelf board. [Figure 5] It is an explanatory diagram showing an example where the miscellaneous goods are detected. [Figure 6] It is a flowchart showing an example of an operation of the miscellaneous goods detection device. [Figure 7] It is a block diagram showing an example of a configuration of the miscellaneous goods detection device. [Figure 8] It is an explanatory diagram showing an example of selection and a straight line example of the combination of the shelf labels closest in the height direction. [Figure 9] It is a flowchart showing an example of an operation of the miscellaneous goods detection device. [Figure 10] It is an explanatory diagram showing an example of the hardware configuration of a computer. 【Mode for Carrying Out the Invention】 【0012】 Hereinafter, embodiments of a miscellaneous goods detection device, a miscellaneous goods detection method, a program, and a non-temporary recording medium for recording the program according to the present disclosure will be described in detail with reference to the drawings. This embodiment does not limit the disclosed technology. 【0013】 (First Embodiment) The first embodiment will be described in detail with reference to the drawings regarding the basic functions of the miscellaneous goods detection device. 【0014】 FIG. 1 is a block diagram showing an example of a configuration of a miscellaneous goods detection device 10. The miscellaneous goods detection device 10 includes a shelf board number specifying unit 105, a peak detection unit 107, and a miscellaneous goods detection unit 109. 【0015】 In retail, manufacturing, and warehousing industries, fixtures are installed. For example, products are displayed on these fixtures. These products may also be called merchandise. For instance, an imaging device may image multiple fixtures, and from the captured images, products, the number of product faces, and the number of displays can be detected and used for inventory management. Generally, products are displayed in the depth direction, that is, perpendicular to the fixture. To capture the product faces, it is desirable to image as perpendicular to the fixture as possible. However, imaging devices may not be able to image in the depth direction of the shelves, that is, perpendicular to the shelves. Therefore, in order to identify the area of ​​products displayed on each fixture, it is desirable to first detect the fixtures with greater accuracy. For this reason, the fixture detection device 10 detects each fixture from among the multiple fixtures included in the image, based on the positional information of multiple shelves in an image captured so as to include multiple fixtures. The image may be, for example, an RGB (Red, Green, Blue) image. 【0016】 The shelf count identification unit 105 identifies the progression of the number of shelves in the lateral direction of the multiple fixtures included in the image, based on positional information of the multiple shelves in the image captured so as to include the multiple fixtures. The multiple fixtures may be, for example, those in which products are displayed in a store. The imaging device may be, for example, a device mounted on a mobile body or a fixedly positioned device. The mobile body may be, for example, an autonomously moving robot or equipment that can move on rails, and is not particularly limited. The rails may be installed on the floor or on opposing fixtures. 【0017】 Figure 2 is an explanatory diagram showing an image of a fixture. In Figure 2, there are two fixtures, and each shelf has a shelf label attached. The image was not taken from a vertical direction, so it is not a direct front view of the fixture. Here, in the image, the horizontal direction of the shelf is the x-axis, and the height direction of the shelf is the y-axis. Here, the direction is positive in the x-axis as you move to the right of the shelf, and positive in the y-axis as you move to the bottom of the shelf. 【0018】 Figure 3 is an explanatory diagram showing positional information for shelves. The positional information for shelves is not particularly limited. For example, the positional information for shelves may be obtained from a memory unit or estimated from shelf labels placed on the shelves. In Figure 3, there are shelves a1 to a9. For example, the positional information for shelves can be represented by coordinate values ​​using the x and y axes. An example of estimating the positional information for shelves from shelf labels will be explained in the second embodiment. 【0019】 The shelf count identification unit 105 identifies the shelf area as the range from the upper horizontal limit to the lower horizontal limit of each of the multiple shelves. However, in this case, the shelf count identification unit 105 identifies the shelf area as the range of values ​​that are less than or equal to the upper horizontal limit of each of the multiple shelves and greater than the lower horizontal limit. Specifically, the following relationship holds for the shelf area. Lower limit < shelf area ≤ upper limit 【0020】 The shelf count identification unit 105 then identifies the change in the number of shelves based on the shelf area of ​​each of the multiple shelves. However, in Figure 3, there are cases where shelves are misidentified due to equipment installed in the facility being mistakenly identified as shelf labels, etc. In Figure 3, shelf a1 is misidentified because equipment installed on the ceiling was mistakenly identified as a shelf label. Furthermore, if multiple fixtures have shelves at the same height, they may be detected as a single shelf. For example, in Figure 3, shelf a5 is incorrectly detected as a long shelf in the section where the shelves of two fixtures are at the same height. The shelf count identification unit 105 may exclude shelves that are not of an appropriate size, such as shelves that are too long or too short in the horizontal direction (x-axis direction), from the shelf count. Specifically, for example, the shelf count identification unit 105 may exclude shelves from the shelf count if the size of the shelf area in the horizontal direction is outside a predetermined range. The predetermined range should be set to a range that is neither too short nor too long. The predetermined range may be set, for example, based on the size of the fixtures installed in the store. For example, in Figure 3, shelf a1 is too short and shelf a5 is too long, so shelf a1 and shelf a5 are excluded from the shelf count. 【0021】 Furthermore, if the shelf count identification unit 105 identifies the number of shelves for all horizontal (x-axis) coordinate values, the identification process is expected to take a considerable amount of time. Therefore, the shelf count identification unit 105 may identify the change in the number of shelves by using the points that represent the upper and lower limits of each of the multiple shelves on the x-axis as endpoints and identifying the number of shelves at each endpoint. 【0022】 Figure 4 is an explanatory diagram showing the correspondence between the endpoints of the shelves and a graph representing the change in the number of shelves. In the graph, the vertical axis represents the number of shelves, and the horizontal axis is the x-axis, which is the horizontal direction of the shelves. The graph plots the number of shelves at each endpoint. For ease of understanding, the upper and lower limits for each endpoint are displayed in the image. 【0023】 As mentioned above, since the relationship is Lower Limit < Shelf Area ≤ Upper Limit, the shelves at the endpoints, which are the lower limits of each shelf, are not counted. Also, as mentioned above, shelves a5 and a1 are not counted. Therefore, the number of shelves at the lower limit of shelf a3 is 0. The number of shelves at the lower limit of shelf a5 is 1 because only shelf a3 is counted. The number of shelves at the lower limit of shelf a7 is 1 because only shelf a3 is counted. The number of shelves at the lower limit of shelf a1 is 2 because shelves a3 and a7 are counted. In this way, the shelf count identification unit 105 can shorten the processing time required for identification by identifying the transition in the number of shelves using the upper and lower limits of each shelf as endpoints. 【0024】 The peak detection unit 107 detects the lateral position of the shelf number that represents the minimum value in the progression of shelf numbers in the lateral direction. Here, the peak detection unit 107 can use the endpoints at both ends as the ends of multiple fixtures and detect the endpoint where the shelf number is the minimum value. In Figure 4, the positions marked with circles on the graph are the positions where the shelf number is 1 and represents the minimum value. The x-coordinate value at this position is sometimes called the reference value. 【0025】 The peak detection unit 107 may also detect the position where the number of shelves reaches a peak (maximum value) by accumulating -1 over the change in the number of shelves. 【0026】 The furniture detection unit 109 detects furniture from the image based on the detected lateral position. In Figure 4, since the relationship Lower limit < shelf area ≤ Upper limit is assumed, the reference value is the lower limit of a certain shelf in the lateral direction (x-axis direction). In particular, when the position information regarding the shelf is estimated based on the position information regarding the shelf tag, the reference value is the x-coordinate value of the shelf tag placed at the edge of shelf a2 and shelf tag a4, i.e., the lower limit of the x-coordinates of shelf tag a2 and shelf tag a4. Therefore, it is predicted that the edge of the furniture, i.e., the boundary of the furniture, is in the negative direction in the lateral direction (x-axis direction) from that shelf tag. Accordingly, the furniture detection unit 109 identifies the boundaries of multiple pieces of furniture based on the detected lateral position, and detects each piece of furniture from the image based on the identified boundaries. As a more specific process for identifying boundaries, the furniture detection unit 109 identifies the boundaries including positions near the detected position as the boundaries of multiple pieces of furniture. To explain in more detail, the furniture detection unit 109 identifies the upper limit values ​​that are in the negative direction in the x-axis direction compared to the reference value, from the set of upper limit values ​​in the lateral direction of the shelf board. The furniture detection unit 109 then identifies the maximum value among the identified upper limit values. This maximum value is the x-coordinate value of the shelf tag installed on the edge of the shelf board of the furniture adjacent to the furniture with the reference value when multiple furniture pieces are lined up. In Figure 4, the upper limit value in the x-axis direction of shelf board a7 is this maximum value. It is estimated that there is a furniture boundary between this maximum value and the reference value. Therefore, the furniture detection unit 109 identifies the position between this maximum value and the reference value as the x-coordinate value of the furniture boundary. This position between the maximum value and the reference value may be, for example, the position that is the average value of this maximum value and the reference value. 【0027】 The fixture detection unit 109 then detects the area between the identified boundary and the ends of the aforementioned multiple fixtures as a fixture. 【0028】 Figure 5 is an explanatory diagram showing an example of furniture detection. In Figure 5, the furniture detection unit 109 detects the area between boundary b and furniture edge e1, and the area between boundary b and furniture edge e2, as furniture. Furniture edge e2 is the position where the x-coordinate of the multiple shelves is maximum. Furniture edge e1 is the position where the x-coordinate of the multiple shelves is minimum. 【0029】 Furthermore, while Figure 5 uses an image with two pieces of furniture lined up as an example, the same processing can be applied to three or more pieces. For example, in a photograph with three pieces of furniture lined up, two locations with local minimum values ​​will be detected. 【0030】 (flowchart) Figure 6 is a flowchart illustrating an example of the operation of the furniture detection device 10. The shelf count identification unit 105 identifies the progression of the number of shelves in the lateral direction of the multiple furniture pieces included in the image based on positional information of multiple shelves in an image captured to include multiple furniture pieces (step S101). Next, the peak detection unit 107 detects the lateral position of the shelf count that is the minimum value among the progression of the number of shelves in the lateral direction (step S102). The furniture detection unit 109 detects furniture from the image based on the detected lateral position (step S103), and the furniture detection device 10 completes the series of processes shown in Figure 6. 【0031】 In the first embodiment described above, the fixture detection device 10 identifies the progression of the number of shelves in the lateral direction of the multiple fixtures included in the image based on positional information of multiple shelves in an image captured to include multiple fixtures, and detects the lateral position of the shelf number that is the minimum value among the progression of the number of shelves in the lateral direction. Then, the fixture detection device 10 detects the fixture from the image based on the detected lateral position. The fixture detection device 10 makes it possible to detect fixtures with high accuracy. As mentioned above, for example, there are cases where it is not possible to capture an image perpendicular to the face surface of the fixture. Even with a field of view that is tilted relative to the face surface of the fixture, the fixture can be detected with high accuracy. 【0032】 Furthermore, for example, in order to capture an image of a fixture perpendicular to its face, it may be necessary to combine panoramic images captured by multiple imaging devices to obtain an image where the imaging direction is perpendicular to the fixture's face. Obtaining an image perpendicular to the fixture's face in this way is costly, as it requires multiple imaging devices. Therefore, even without using multiple imaging devices, the fixture detection device 10 makes it possible to detect fixtures with high accuracy. 【0033】 Furthermore, the furniture detection device 10 identifies the boundaries of multiple furniture pieces based on the detected lateral position, and detects furniture pieces from the image based on the identified boundaries. In particular, the furniture detection device 10 identifies boundaries that include positions near the detected position as the boundaries of multiple furniture pieces. In addition, the furniture detection device 10 identifies the range of values ​​that are below the upper limit in the lateral direction and above the lower limit in the lateral direction for each of the multiple shelves as the shelf area, and identifies the change in the number of shelves based on each shelf area for the multiple shelves. This makes it possible to detect the boundaries of multiple furniture pieces with greater accuracy. 【0034】 The fixture detection device 10 excludes shelf areas outside a predetermined range in size when counting the number of shelves. This makes it possible to exclude shelves that were incorrectly detected when determining the trend in the number of shelves. 【0035】 The furniture detection device 10 may require a large amount of processing power to determine the number of shelves at all horizontal coordinate values. Therefore, the furniture detection device 10 determines the change in the number of shelves by identifying the number of shelves at each of the upper and lower horizontal limits of multiple shelves indicated by the position information of multiple shelves. This reduces the amount of processing power required. 【0036】 (Second embodiment) A second embodiment will be described in detail with reference to the drawings. In the second embodiment, an example will be described in which the shelf board is estimated from the shelf label. To the extent that the description of the second embodiment does not become unclear, explanations that overlap with the above description will be omitted. 【0037】 Figure 7 is a block diagram showing one example configuration of the furniture detection device 20. The furniture detection device 20 comprises a shelf label detection unit 201, a shelf board identification unit 203, a shelf board number identification unit 205, a peak detection unit 207, a furniture detection unit 209, and an output unit 211. The furniture detection device 20 is the furniture detection device 20 shown in Figure 1, further comprising a shelf label detection unit 201, a shelf board identification unit 203, and an output unit 211. The shelf board number identification unit 205 may have the shelf board number identification unit 105 shown in Figure 1 as its basic function. The peak detection unit 207 may have the peak detection unit 107 shown in Figure 1 as its basic function. The furniture detection unit 209 may have the furniture detection unit 109 shown in Figure 1 as its basic function. 【0038】 The shelf label detection unit 201 detects multiple shelf labels from an image captured so that multiple fixtures are included. For example, the shelf label detection method may be based on the shape of the shelf label, and is not particularly limited. For example, existing technologies can be used for the shelf label detection method, and a detailed explanation is omitted. 【0039】 The shelf identification unit 203 identifies the location information of multiple shelves in the image based on the location information of the multiple shelf labels that have been detected. The shelf labels to be detected may be either paper shelf labels or electronic shelf labels. 【0040】 Specifically, for example, the shelf identification unit 203 selects the combination of shelf labels that is closest to the height direction (y-axis direction) of multiple fixtures from the set of multiple shelf labels that have been detected. Then, the shelf identification unit 203 derives a straight line passing through the combination of shelf labels. 【0041】 Figure 8 is an explanatory diagram showing examples of selection and straight line examples for the closest combination of shelf labels in the height direction. In Figure 8, shelf labels pt1 to pt8 are shown. In Figure 8, shelf labels pt1 to pt5 are at approximately the same height in the height direction (y-axis direction). In this case, shelf labels pt4 and pt5 are the closest combination. Therefore, the shelf identification unit 203 derives a straight line passing through shelf labels pt4 and pt5. Also in Figure 8, shelf labels pt6 to pt8 are at approximately the same height in the height direction (y-axis direction). In this case, shelf labels pt7 and pt8 are the closest combination. Therefore, the shelf identification unit 203 derives a straight line passing through shelf labels pt7 and pt8. 【0042】 Next, the shelf identification unit 203 identifies a subset of shelf labels from a set of multiple shelf labels whose distance from the derived line is less than or equal to a threshold. The threshold is used, for example, to detect shelf labels that are likely to be installed on the same shelf. For this reason, the threshold may be set using the size of the installed fixture. Here, the shelf identification unit 203 identifies a subset of shelf labels whose distance from the line is less than or equal to the threshold, even if the shelf labels are of different heights. In Figure 8, for example, the shelf identification unit 203 identifies a subset of shelf labels whose distance from the line passing through shelf labels pt4 and pt5 is less than or equal to the threshold. In Figure 8, for example, the shelf identification unit 203 identifies a subset of shelf labels whose distance from the line passing through shelf labels pt7 and pt8 is less than or equal to the threshold. Then, the shelf identification unit 203 further derives a line from the subset of shelf labels. For example, the shelf identification unit 203 derives a line from the subset of shelf labels using the RANSAC (Random Sample Consensus) algorithm. The RANSAC algorithm is an iterative algorithm for estimating model parameters, including outliers, from a dataset. Specifically, the shelf identification unit 203 detects the optimal straight line while excluding outliers such as falsely detected shelf tags when estimating a straight line for shelves from the positional information of shelf tags. The shelf identification unit 203 estimates the region between the maximum and minimum horizontal positions in the shelf tag subset of the derived straight line as a shelf. The maximum value is the upper limit mentioned above, and the minimum value is the lower limit mentioned above. This region includes the maximum value but does not include the minimum value, similar to the shelf region described in the first embodiment. This allows the shelf identification unit 203 to identify the positional information related to the shelves. 【0043】 Then, as described in the first embodiment, the shelf count identification unit 205 identifies the change in the number of shelves in the lateral direction of the multiple fixtures included in the image, based on the positional information of the multiple shelves in the image captured so as to include the multiple fixtures. Note that the shelf count identification unit 205, peak detection unit 207, and fixture detection unit 209 can be the same as in the first embodiment, so a detailed explanation is omitted. 【0044】 The output unit 211 may output the detection results of the fixtures. The output method is not particularly limited and may include display, audio output, or storage in the memory unit. The output destination is not particularly limited and may include a terminal. For example, the terminal may be a device used by an inventory management staff member, but is not particularly limited. The type of terminal may be a PC (Personal Computer), smartphone, tablet, etc. The output unit 211 may output the detection results of the fixtures in association with an image captured so that multiple fixtures are included. Taking display as an example, the output unit 211 may superimpose the detection results of the fixtures onto an image captured so that multiple fixtures are included. More specifically, for example, the output unit 211 may superimpose a frame on the image at the location where a fixture is detected. 【0045】 Furthermore, the output unit 211 may output, for example, the boundary identification result. Similar to the furniture detection result, the output method and output destination are not particularly limited. For example, the output unit 211 may output the boundary identification result in association with an image captured so that multiple furniture items are included. Taking display as an example, the output unit 211 may superimpose the boundary identification result onto an image captured so that multiple furniture items are included. 【0046】 (flowchart) Figure 9 is a flowchart showing an example of the operation of the fixture detection device 20. The shelf label detection unit 201 detects shelf labels from an image captured to include multiple fixtures (step S201). 【0047】 Then, the shelf identification unit 203 identifies the positional information of multiple shelves in the image based on the positional information of multiple shelf labels (step S202). 【0048】 Next, the shelf count identification unit 205 identifies the progression of the number of shelves in the lateral direction of the multiple fixtures included in the image based on positional information of multiple shelves in the image captured so as to include multiple fixtures (step S203). Next, the peak detection unit 207 detects the lateral position of the shelf count that is the minimum value among the progression of the number of shelves in the lateral direction (step S204). The fixture detection unit 209 detects the boundaries of the multiple fixtures from the image based on the detected lateral positions (step S205). Then, the fixture detection unit 209 detects the area between the boundary and each end of the multiple fixtures as a fixture (step S206), and the fixture detection device 20 completes the series of processes shown in Figure 9. 【0049】 In the second embodiment described above, the fixture detection device 20 detects multiple shelf labels from the image. Then, based on the positional information of the detected multiple shelf labels, the fixture detection device 20 identifies the positional information of multiple shelves in the image. Specifically, the fixture detection device 20 selects the combination of shelf labels that is closest to the height of the multiple fixtures from the set of detected multiple shelf labels, and derives a straight line passing through the selected combination of shelf labels. The fixture detection device 20 estimates the region between the maximum and minimum horizontal positions in the shelf label subset of the line derived from the subset of shelf labels whose distance from the derived straight line is below a threshold as a shelf. This makes it possible to indirectly estimate the positional information of shelves using the positions of the shelf labels, even if positional information of the shelves is not input. 【0050】 This concludes the description of each embodiment. Furthermore, each embodiment is not limited to the examples described above and can be modified in various ways. The first embodiment and the second embodiment may be combined as appropriate. There are no particular limitations on how the first embodiment and the second embodiment can be combined. 【0051】 <Variation> The relationship Lower Limit < Shelf Area ≤ Upper Limit was used, but the relationship Lower Limit ≤ Shelf Area < Upper Limit may also be used. In this case, the reference value is the upper limit in the lateral direction (x-axis direction) of a certain shelf. Furthermore, especially when the position information of a shelf is estimated based on the position information of the shelf label, it is predicted that the edge of the fixture, i.e., the boundary of the fixture, is in a positive direction in the lateral direction (x-axis direction) from the shelf label of the shelf containing the reference value. The fixture detection units 109 and 209 identify the boundaries of multiple fixtures based on the detected lateral position, and detect each fixture from the image based on the identified boundary. More specifically, as a process for identifying the boundary, the fixture detection units 109 and 209 identify the boundary that includes the position near the detected position as the boundary of multiple fixtures. To explain in more detail, the fixture detection units 109 and 209 identify the lower limit that exists in a positive direction in the x-axis direction from the reference value among the set of lower limits in the lateral direction of the shelf. The fixture detection units 109 and 209 identify the minimum value among the specified lower limits. This minimum value is the x-coordinate of the shelf tag installed on the edge of the shelf of the fixture adjacent to the fixture with the reference value when multiple fixtures are lined up. As described above, the fixture detection units 109 and 209 identify the position between this minimum value and the reference value as the x-coordinate of the fixture boundary. This position between the minimum value and the reference value may be, for example, the position that is the average value of this minimum value and the reference value. 【0052】 For example, each embodiment and its variations may be combined as appropriate. Furthermore, the configuration of the furniture detection devices 10 and 20 is not particularly limited. For example, the functional parts of the furniture detection devices 10 and 20 may be implemented by a single device. Alternatively, for example, each functional part or database of the furniture detection devices 10 and 20 may be implemented by different devices and configured as a system. For example, the functional parts of the furniture detection devices 10 and 20 may be made up of multiple devices. For example, a system may be realized that includes a database server containing each database and a device having each functional part. A system may be realized that includes a device equipped with some of the functional parts of the furniture detection devices 10 and 20 and a device equipped with other parts of the functional parts of the furniture detection devices 10 and 20. The number of devices is not particularly limited. 【0053】 Furthermore, the various pieces of information provided are merely examples, and may include other information or omit some of it. 【0054】 Furthermore, the process of generating information to be displayed on the terminal may be performed by the output unit. Alternatively, this process may be performed by the terminal. That is, the terminal may generate screen information to be displayed on the terminal based on the data received from the fixture detection device 20, and display the screen based on the screen information. Also, the user interface in each embodiment is just an example, and various modifications are possible. 【0055】 (Example of computer hardware configuration) Next, we will describe an example of a hardware configuration when each device, such as the furniture detection devices 10 and 20 and the terminals, is implemented using a computer. 【0056】 Figure 10 is an explanatory diagram showing an example of a computer hardware configuration. For example, some or all of each device can be implemented using any combination of computer 80 and programs as shown in Figure 10. 【0057】 Computer 80 includes, for example, a processor 801, a ROM (Read Only Memory) 802, a RAM (Random Access Memory) 803, and a storage device 804. Computer 80 also includes a communication interface 805 and an input / output interface 806. Each component is connected, for example, via a bus 807. The number of components is not particularly limited, and each component may be one or more. 【0058】 The processor 801 controls the entire computer 80. The processor 801 can be, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, or a combination thereof, and is not particularly limited. 【0059】 The computer 80 also includes a ROM 802, a RAM 803, and a storage device 804. The storage device 804 may be a semiconductor memory such as flash memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive). For example, the storage device 804 stores the OS (Operating System) program, application programs, and programs according to each embodiment. Alternatively, the ROM 802 stores the application program, programs according to the embodiment, and so on. The RAM 803 is used as the work area of ​​the processor 801. 【0060】 The processor 801 also loads programs stored in the memory device 804, ROM 802, etc. Then, the processor 801 executes each process coded in the program. The processor 801 may also download various programs via the communication network NT. Furthermore, the processor 801 functions as part or all of the computer 80. The processor 801 may also execute processes or instructions in the illustrated flowchart based on the program. 【0061】 The communication interface 805 is connected to a communication network NT, such as a LAN (Local Area Network) or WAN (Wide Area Network), via a wireless or wired communication line. The communication network NT may be composed of multiple communication networks NT. This allows the computer 80 to connect to external devices and external computers 80 via the communication network NT. The communication interface 805 manages the interface between the communication network NT and the internal workings of the computer 80. Furthermore, the communication interface 805 controls the input and output of data from external devices and external computers 80. 【0062】 Furthermore, the input / output interface 806 is connected to at least one of the input device, output device, and input / output device. The connection method may be wireless or wired. Examples of input devices include keyboards, mice, and microphones. Examples of output devices include display devices, lighting devices (such as lamps), and audio output devices that output sound. Examples of input / output devices include touch panel displays. The input devices, output devices, and input / output devices may be built into the computer 80 or attached to the computer 80 externally. That is, for example, the computer 80 may have input devices such as keyboards and mice. The computer 80 may have output devices such as displays. The computer 80 may also have input devices, output devices, and input / output devices. 【0063】 The hardware configuration of computer 80 is an example. Computer 80 may have some of the components shown in Figure 10. Computer 80 may have components other than those shown in Figure 10. For example, computer 80 may have a drive device. The processor 801 may read programs and data stored on a recording medium mounted on the drive device or the like into RAM 803. Examples of non-temporary tangible recording media include optical discs, flexible discs, magneto-optical discs, and USB (Universal Serial Bus) memory. 【0064】 Furthermore, the computer 80 may have various sensors (not shown). The type of sensor is not particularly limited. Also, the computer 80 may be equipped with an imaging device capable of capturing images or videos. 【0065】 This concludes the description of the hardware configuration of each device. Furthermore, there are various variations in how each device can be implemented. For example, each device may be implemented by any combination of different computers and programs for each component. Alternatively, the multiple components of each device may be implemented by any combination of a single computer and program. 【0066】 Furthermore, some or all of the components of each device may be implemented using application-specific circuits. Alternatively, some or all of the components of each device may be implemented using general-purpose circuits such as FPGAs (Field Programmable Gate Arrays). Furthermore, some or all of the components of each device may be implemented using a combination of application-specific circuits and general-purpose circuits. These circuits may also be a single integrated circuit, or they may be divided into multiple integrated circuits. These multiple integrated circuits may be connected via a bus or the like. 【0067】 Furthermore, if some or all of the components of each device are implemented by multiple computers or circuits, these computers or circuits may be centrally located or distributed. 【0068】 The furniture detection methods described in each embodiment may be implemented by computers such as furniture detection devices 10 and 20. 【0069】 Each program described in each embodiment is recorded on a computer-readable recording medium such as an HDD, SSD, flexible disk, optical disk, magneto-optical disk, or USB memory. Each program is then executed by being read from the recording medium by a computer. Furthermore, each program may be distributed via a communication network NT. 【0070】 Each component of the furniture detection devices 10 and 20 described above may be implemented using dedicated hardware, such as a computer. Alternatively, each component may be implemented using software. Alternatively, each component may be implemented using a combination of hardware and software. 【0071】 Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. The structure and details of the present disclosure may include embodiments that apply various modifications that can be grasped by those skilled in the art within the scope of the present disclosure. The present disclosure may include embodiments that combine or substitute the matters described herein as appropriate. For example, matters described using a particular embodiment may be applied to other embodiments to the extent that they do not cause a contradiction. For example, although several operations are described sequentially in the form of a flowchart, the order in which they are described does not limit the order in which the operations are performed. Therefore, when implementing an embodiment, the order of the operations can be changed to the extent that it does not impair the content. 【0072】 Some or all of the above embodiments may also be described as follows. However, some or all of the above embodiments are not limited to the following. 【0073】 (Note 1) A shelf count identification means that identifies the progression of the number of shelves in the lateral direction of the multiple fixtures included in an image, based on positional information of multiple shelves in an image captured so as to include multiple fixtures, A peak detection means for detecting the position in the lateral direction that is the minimum value among the changes in the number of shelves in the lateral direction, A fixture detection means for detecting fixtures from the image based on the detected lateral position, A furniture detection device equipped with [the following features]. 【0074】 (Note 2) The fixture detection means identifies the boundaries of the plurality of fixtures based on the detected lateral position, and detects the fixtures from the image based on the identified boundaries. The fixture detection device described in Appendix 1. 【0075】 (Note 3) The fixture detection means identifies a boundary including a location near the detected location as the boundary of the plurality of fixtures. The fixture detection device described in Appendix 2. 【0076】 (Note 4) The shelf number identification means identifies a range of values ​​for each of the plurality of shelves that is less than or equal to the upper limit in the horizontal direction and greater than the lower limit in the horizontal direction, and identifies the change in the number of shelves based on each of the plurality of shelves' shelf areas. A fixture detection device as described in any of the appendices 1 to 3. 【0077】 (Note 5) The shelf counting means excludes the shelf area in the shelf count if the size of the shelf area in the horizontal direction is outside a predetermined range. The fixture detection device described in Appendix 4. 【0078】 (Note 6) The shelf number identification means identifies the change in the number of shelves by identifying the number of shelves at each of the upper and lower lateral limits of the plurality of shelves indicated by the position information relating to the plurality of shelves. A fixture detection device as described in any of the appendices 1 to 5. 【0079】 (Note 7) A shelf identification means that identifies the positional information of the multiple shelves in the image based on the positional information of the multiple shelf labels, A furniture detection device as described in any of the appendices 1 to 6, further comprising the above. 【0080】 (Note 8) The shelf identification means is, From the set of multiple shelf labels detected, select the combination of shelf labels that is closest in the height direction of the multiple fixtures, and derive a straight line passing through the selected combination of shelf labels. For a line derived from a subset of shelf tags whose distance from the derived line is less than or equal to a threshold, the region between the maximum and minimum horizontal positions in the shelf tag subset is estimated to be a shelf. The fixture detection device described in Appendix 7. 【0081】 (Note 9) shelf label detection means for detecting the plurality of shelf labels from the aforementioned image, A furniture detection device as described in Appendix 7 or Appendix 8, further comprising the above. 【0082】 (Note 10) Output means for outputting the detection result of the aforementioned fixture, A furniture detection device according to any of the appendices 1 to 9, comprising the features described above. 【0083】 (Note 11) At least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. A method for detecting fixtures to execute processing. 【0084】 (Note 12) On at least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. A program that executes a process. 【0085】 (Note 13) On at least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. A non-temporary recording medium readable by the computer, which records a program that executes a process. 【0086】 Furthermore, some or all of the configurations described in Appendices 2 to 10, which are dependent on Appendice 1 above, may also be dependent on Appendices 11, 12, and 13 in the same way as those described in Appendices 2 to 10. Moreover, not limited to Appendices 1, 11, 12, and 13, some or all of the configurations described as appendices may also be dependent on various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above. [Explanation of symbols] 【0087】 10, 20 Fixture detection device 80 Computers 105, 205 shelf number specification section 107, 207 Peak detection unit 109, 209 Fixture detection unit 201 Shelf label detection unit 203 Shelf-specific section 211 Output section 801 Processor 802 ROM 803 RAM 804 Storage device 805 Communication Interface 806 Input / Output Interface 807 Bus NT communication network

Claims

[Claim 1] A shelf count identification means that identifies the progression of the number of shelves in the lateral direction of the multiple fixtures included in an image, based on positional information of multiple shelves in an image captured so as to include multiple fixtures, A peak detection means for detecting the position in the lateral direction that is the minimum value among the changes in the number of shelves in the lateral direction, A fixture detection means for detecting fixtures from the image based on the detected lateral position, A furniture detection device equipped with [the following features]. [Claim 2] The fixture detection means identifies the boundaries of the plurality of fixtures based on the detected lateral position, and detects the fixtures from the image based on the identified boundaries. The fixture detection device according to claim 1. [Claim 3] The fixture detection means identifies a boundary including a location near the detected location as the boundary of the plurality of fixtures. The fixture detection device according to claim 2. [Claim 4] The shelf number identification means identifies a range of values ​​for each of the plurality of shelves that is less than or equal to the upper limit in the horizontal direction and greater than the lower limit in the horizontal direction, and identifies the change in the number of shelves based on each of the plurality of shelves' shelf areas. The fixture detection device according to claim 1. [Claim 5] The shelf counting means excludes the shelf area in the shelf count if the size of the shelf area in the horizontal direction is outside a predetermined range. The fixture detection device according to claim 4. [Claim 6] The shelf number identification means identifies the change in the number of shelves by identifying the number of shelves at each of the upper and lower lateral limits of the plurality of shelves indicated by the position information relating to the plurality of shelves. The fixture detection device according to claim 1. [Claim 7] A shelf identification means that identifies the positional information of the multiple shelves in the image based on the positional information of the multiple shelf labels, A fixture detection device according to any one of claims 1 to 5, further comprising the above. [Claim 8] The shelf identification means is, From the set of multiple shelf labels detected, select the combination of shelf labels that is closest in the height direction of the multiple fixtures, and derive a straight line passing through the selected combination of shelf labels. For a line derived from a subset of shelf tags whose distance from the derived line is less than or equal to a threshold, the region between the maximum and minimum horizontal positions in the shelf tag subset is estimated to be a shelf. The fixture detection device according to claim 7. [Claim 9] At least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. A method for detecting fixtures to execute processing. [Claim 10] On at least one computer, Based on positional information of multiple shelves in an image captured to include multiple fixtures, the change in the number of shelves in the lateral direction of the multiple fixtures included in the image is identified. The position in the horizontal direction that is the minimum value among the changes in the number of shelves in the horizontal direction is detected. Based on the detected lateral position, the fixture is detected from the image. A program that executes a process.

Images