Monitoring device, monitoring method, and program

Abstract

Detects changes in the status of cameras installed in stores. [Solution] The monitoring device of this disclosure comprises a registration unit, a detection unit, and a determination unit. The registration unit registers at least one product group region containing a predetermined product group detected from a basic image captured by the imaging means in a basic state as a basic region. The detection unit detects product group regions containing a predetermined product group from the processing target image captured by the imaging means as comparison target regions. The determination unit determines a change in the state of the imaging means based on the number of product group regions detected from the image and the comparison result between the basic region and the comparison target region, based on at least one of the areas of each product group region.

Description

[Technical Field]

[0001] This disclosure relates to monitoring devices, monitoring methods, and programs. [Background technology]

[0002] A technology related to this disclosure is disclosed in Patent Document 1. As disclosed in Patent Document 1, there is a technology that uses a camera installed in a store to image display shelves and monitors the display status of products based on the images generated by the camera. Patent Document 1 also discloses a technology that detects when the area being imaged changes due to camera misalignment or the like. This technology sets a fixed area as a part of the display shelf that is captured in the image generated by a certain camera. Specifically, the area in which the shelf frame of the display shelf is captured is set as the fixed area. This technology then detects when the object being imaged by the camera has changed based on the similarity between the image of the fixed area and an image of the same position in another image captured by the same camera. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2016-99923 [Overview of the project] [Problems that the invention aims to solve]

[0004] When the state (orientation and position) of a camera installed in a store changes, the area captured by that camera changes. This change can sometimes prevent predetermined processing based on image analysis from being performed correctly. Therefore, there is a need for technology to detect changes in the state of the camera.

[0005] One example of the purpose of this disclosure is to provide a monitoring device, monitoring method, and program that detect changes in the status of cameras installed in stores using a new method, in light of the problems described above. [Means for solving the problem]

[0006] The monitoring device in this disclosure is A registration means that registers as a basic area an area in which a predetermined part of a display shelf is captured in a basic image captured by an imaging means in the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means for determining a change in the state of the imaging means based on the relative positional relationship between the basic region and the comparison target region, It holds.

[0007] Furthermore, the monitoring method for this disclosure is: One or more computers, In the basic image captured by the imaging means in the basic state, the area in which a predetermined part of the display shelf is captured, or the area in which the products displayed on the display shelf are captured, is registered as the basic area. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the area in which the products displayed on the display shelf are captured is detected as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the area in which a predetermined portion of the display shelf is captured is detected as the comparison area from the image to be processed, Based on the relative positional relationship between the basic area and the comparison area, a change in the state of the imaging means is determined.

[0008] Furthermore, the recording medium of this disclosure is Computers, A registration means for registering as a basic area an area in which a predetermined part of a display shelf is captured in a basic image captured by an imaging means in the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means that determines a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. It stores the program that will function as such.

[0009] Furthermore, the monitoring device described herein, A registration means that registers the area in which the product is detected from the basic image captured by the basic state imaging means as the basic area, A detection means for detecting, from the image to be processed captured by the imaging means, the region in which the product is visible as the comparison region, A determination means for determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area, It holds.

[0010] Furthermore, the monitoring method for this disclosure is: One or more computers, The imaging means in the basic state registers the area in which the detected product is visible from the basic image captured as the basic area. From the image to be processed captured by the imaging means, the region in which the product is visible is detected as the comparison region. Based on the comparison result between the basic area and the comparison target area, a change in the state of the imaging means is determined.

[0011] Furthermore, the recording medium of this disclosure is Computers, A registration means that registers the area in which the product is detected from the basic image captured by the basic state imaging means as the basic area. A detection means that detects the area in which the product is visible from the image to be processed captured by the imaging means as a comparison area. determination means for determining a change in the state of the imaging means based on a comparison result between the basic region and the comparison target region, stores a program that functions as. [Effects of the Invention]

[0012] According to one aspect of the present disclosure, a monitoring device, a monitoring method, and a program that detect a change in the state of a camera installed in a store using a new method are realized. [Brief Description of the Drawings]

[0013] [Figure 1] It is a diagram showing an example of a functional block diagram of the monitoring device according to the present disclosure. [Figure 2] It is a flowchart showing an example of a processing flow of the monitoring device according to the present disclosure. [Figure 3] It is a flowchart showing another example of the processing flow of the monitoring device according to the present disclosure. [Figure 4] It is a flowchart showing another example of the processing flow of the monitoring device according to the present disclosure. [Figure 5] It is a diagram showing an example of a hardware configuration of the monitoring device according to the present disclosure. [Figure 6] It is a diagram showing an example of an image processed by the monitoring device according to the present disclosure. [Figure 7] It is a diagram showing another example of the image processed by the monitoring device according to the present disclosure. [Figure 8] It is a diagram showing an example of information processed by the monitoring device according to the present disclosure. [Figure 9] It is a diagram showing another example of the image processed by the monitoring device according to the present disclosure. [Figure 10] It is a diagram showing another example of the image processed by the monitoring device according to the present disclosure. [Mode for Carrying Out the Invention]

[0014] Embodiments of this disclosure will be described below with reference to the drawings. In this disclosure, the drawings are associated with one or more embodiments. In all drawings, similar components are denoted by the same reference numerals, and descriptions are omitted where appropriate.

[0015] <First Embodiment> Figure 1 is a functional block diagram showing an overview of the monitoring device 10. As shown in Figure 1, the monitoring device 10 includes a registration unit 11, a detection unit 12, and a determination unit 13.

[0016] The registration unit 11 performs preliminary preparations for the "process of detecting changes in the status of cameras installed in the store." Then, the detection unit 12 and the determination unit 13 execute the "process of detecting changes in the status of cameras installed in the store."

[0017] The registration unit 11 registers as a basic area the region in which a predetermined part of the display shelf is captured in the basic image captured by the imaging means (camera) in the basic state, or the region in which the products displayed on the display shelf are captured.

[0018] The detection unit 12 and the determination unit 13 execute the process shown in the flowchart of Figure 2, or the process shown in the flowchart of Figure 3.

[0019] When the registration unit 11 registers the "area in which products displayed on the shelves are captured" as the basic area in the basic image captured by the imaging means in the basic state, the detection unit 12 and the determination unit 13 execute the process shown in the flowchart of Figure 2.

[0020] Specifically, the detection unit 12 detects a region in which a predetermined part of the display shelf is captured from the image to be processed captured by the imaging means as a comparison region (S10).

[0021] Then, the determination unit 13 determines the change in the state of the imaging means based on the relative positional relationship between the previously registered "area in which products displayed on the display shelf are captured (basic area)" and the "area in which a predetermined part of the display shelf is captured (comparison area)" detected in S10 (S11).

[0022] On the other hand, when the registration unit 11 registers the "area in which a predetermined part of the display shelf is captured" as the basic area in the basic image captured by the imaging means in the basic state, the detection unit 12 and the determination unit 13 execute the process shown in the flowchart of Figure 3.

[0023] Specifically, the detection unit 12 detects the "area in which products displayed on the shelves are shown" from the image to be processed captured by the imaging means as the comparison area (S20).

[0024] Then, the determination unit 13 determines the change in the state of the imaging means based on the relative positional relationship between the previously registered "area in which a predetermined part of the display shelf is captured (basic area)" and the "area in which the products displayed on the display shelf are captured (comparison area)" detected in S20 (S21).

[0025] Thus, the monitoring device 10 of this embodiment can pre-register the "area in which products displayed on the shelves are captured (basic area)" in the basic image captured by the imaging means in the basic state. The monitoring device 10 can then determine a change in the state of the imaging means based on the relative positional relationship between the "area in which a predetermined part of the display shelf is captured (comparison area)" detected from the processing target image captured by the imaging means and the pre-registered "area in which products displayed on the shelves are captured (basic area)".

[0026] Furthermore, the monitoring device 10 of this embodiment can pre-register a "region in which a predetermined part of the display shelf is captured (basic region)" in the basic image captured by the imaging means in its basic state. The monitoring device 10 can then determine a change in the state of the imaging means based on the relative positional relationship between the "region in which products displayed on the display shelf are captured (comparison region)" detected from the image to be processed captured by the imaging means and the pre-registered "region in which a predetermined part of the display shelf is captured (basic region)".

[0027] According to this first embodiment, a monitoring device, monitoring method, and recording medium are realized that detect changes in the status of cameras installed in stores using a new method.

[0028] <Second Embodiment> The monitoring device 10 of the second embodiment determines changes in the state of the imaging means using a different process than the monitoring device 10 of the first embodiment.

[0029] Figure 1 is a functional block diagram showing an overview of the monitoring device 10. As shown in Figure 1, the monitoring device 10 includes a registration unit 11, a detection unit 12, and a determination unit 13.

[0030] The registration unit 11 performs preliminary preparations for the "process of detecting changes in the status of cameras installed in the store." Then, the detection unit 12 and the determination unit 13 execute the "process of detecting changes in the status of cameras installed in the store."

[0031] The registration unit 11 registers the area in which the product is detected from the basic image captured by the imaging means in the basic state as the basic area.

[0032] The detection unit 12 and the determination unit 13 execute the processes shown in the flowchart of Figure 4.

[0033] Specifically, the detection unit 12 detects the area in which the product is visible (comparison area) from the image to be processed captured by the imaging means (S30).

[0034] Then, the determination unit 13 determines a change in the state of the imaging means based on the comparison result between the previously registered "area in which the product detected from the basic image is visible (basic area)" and the "area in which the product detected from the image to be processed is visible (comparison area)" (S31).

[0035] In this way, the monitoring device 10 detects the area in which the product is visible from the image to be processed, and determines the change in the state of the imaging means based on the comparison result between the "area in which the product is visible detected from the basic image" and the "area in which the product is visible detected from the image to be processed".

[0036] According to this second embodiment, a monitoring device, monitoring method, and recording medium are realized that detect changes in the status of cameras installed in stores using a new method.

[0037] <Third Embodiment> "overview" The monitoring device 10 of the third embodiment is a concrete implementation of the configuration of the monitoring device 10 of the first embodiment.

[0038] In the third embodiment, the monitoring device 10 registers the "area in which products displayed on the display shelf are captured" as the basic area in the basic image captured by the imaging means in the basic state. The monitoring device 10 then determines the change in the state of the imaging means based on the relative positional relationship between this basic area and the "area in which a predetermined part of the display shelf is captured (comparison area)" detected from the image to be processed. The configuration of the monitoring device 10 will be described in detail below.

[0039] "Hardware configuration" First, an example of the hardware configuration of the monitoring device 10 will be described. Each functional part of the monitoring device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the implementation method and the device. The software includes programs that are pre-installed at the time of shipment of the device, as well as programs downloaded from recording media such as CDs (Compact Discs) or from servers on the Internet.

[0040] Figure 5 is a block diagram illustrating the hardware configuration of the monitoring device 10. As shown in Figure 5, the monitoring device 10 includes a processor 1A, memory 2A, input / output interface 3A, peripheral circuitry 4A, and bus 5A. Peripheral circuitry 4A includes various modules. The monitoring device 10 does not necessarily have peripheral circuitry 4A. The monitoring device 10 may also be composed of multiple physically and / or logically separated devices. In this case, each of the multiple devices may have the above hardware configuration.

[0041] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuits 4A, and input / output interface 3A to send and receive data to and from each other. Processor 1A is a processing unit such as a CPU or GPU (Graphics Processing Unit). Memory 2A is a memory such as RAM (Random Access Memory) or ROM (Read Only Memory). Input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. Input / output interface 3A also includes interfaces for connecting to communication networks such as the Internet. Input devices include, for example, keyboards, mice, microphones, physical buttons, and touch panels. Output devices include, for example, displays, speakers, printers, and mailers. Processor 1A can issue commands to each module and perform calculations based on their calculation results.

[0042] "Functional Configuration" Next, the functional configuration of the monitoring device 10 will be described in detail. Figure 1 shows an example of a functional block diagram of the monitoring device 10. As shown in the figure, the monitoring device 10 has a registration unit 11, a detection unit 12, and a determination unit 13.

[0043] The registration unit 11 registers the area in the basic image captured by the imaging means in the basic state in which products displayed on the shelves are captured as the basic area.

[0044] The "imaging means" is a camera. The type of camera is not particularly limited. The imaging means may continuously capture moving images. Alternatively, the imaging means may periodically or intermittently capture still images at predetermined timings. Examples of predetermined timings include, but are not limited to, timings at set intervals (e.g., every 5 minutes, every 10 minutes, every 15 minutes, etc.) or timings when instructions are received from the operator.

[0045] The imaging means is installed in a position where it can image the display shelves installed in the store, and in a orientation that allows it to image a predetermined area of ​​the display shelf. For example, the imaging means for imaging a particular display shelf may be attached by any means to another display shelf opposite that display shelf. Alternatively, the imaging means may be attached by any means to other components positioned opposite the display shelf. Furthermore, the imaging means may be mounted on the ceiling or walls of the store.

[0046] The "basic state" is a state in which the desired area of ​​the desired display shelf can be imaged. The state of the imaging device varies depending on its installation position and orientation. When installing the imaging device in the store, the operator sets the installation position and orientation of the imaging device to a state (basic state) in which the desired area of ​​the desired display shelf can be imaged.

[0047] The "basic image" is an image captured by the imaging means in the basic state. The basic image shows the desired area of ​​the desired display shelf. Figure 6 shows the basic image P. s An example is shown. The basic image P is illustrated. s The image shows multiple products M displayed on a shelf. Products M are placed on shelf board S. Price tags L are attached to the side of shelf board S.

[0048] The "basic region" is the basic image P shown in Figure 6. s In this context, the area is the region that shows the products displayed on the shelves. The basic area is at least a portion of the area that shows the products displayed on the shelves.

[0049] Figure 7 shows the basic image P shown in Figure 6. s An example of a basic area set and registered in Figure 7 is shown. In the example shown in Figure 7, four basic areas X1 to X4 are set and registered. Specifically, for each type of product, the area in which the product is displayed (the display area assigned to each product) is the basic image P s The regions identified above are registered as basic regions X1 to X4.

[0050] FIG. 8 schematically shows an example of registered information. Said information associates a number for mutually identifying a plurality of basic regions, region information indicating each basic region in an image, and product identification information indicating a product displayed in each region with one another. The region information indicating each basic region may be represented by coordinate information of a coordinate system set in the image, or may be represented by other means. For example, when the basic region is a rectangle, the region information indicating each basic region may be represented by coordinates of four vertices of the rectangle. Such registered information may be stored in a storage device included in the monitoring device 10. Alternatively, such registered information may be stored in a storage device included in an external device configured to be able to communicate with the monitoring device 10.

[0051] As a modification, the basic image P s only a region where any one type of product among the plurality of types of products shown in is displayed may be registered as the basic region. That is, only any one of the basic regions X1 to X4 shown in FIG. 7 may be registered as the basic region.

[0052] Further, as another modification, the basic image P s a region showing any one product among the plurality of products displayed on the display shelf shown in may be registered as the basic region.

[0053] Note that the basic regions illustrated here are merely examples, and the basic image P s another partial region among the regions showing the products displayed on the display shelf shown in may be registered as the basic region.

[0054] The designation of the region to be registered as the basic region may be performed by an operator. That is, the operator may perform an operation of designating a partial region in the basic image P s via any input means, and registering the designated region as the basic region. Alternatively, the monitoring device 10 may perform image analysis on the basic image P sThe monitoring device 10 may identify the area in which various products are visible. The monitoring device 10 may then register the identified area as a basic area. For example, the characteristic quantities of the appearance of multiple types of products may be registered in advance in the monitoring device 10. The monitoring device 10 then generates a basic image P based on these characteristic quantities of appearance. s The monitoring device 10 may identify areas from which various products are visible. For example, the monitoring device 10 can identify a cluster of areas in which similar products are visible as an area where various products are displayed (a display area assigned to each product).

[0055] Returning to Figure 1, the detection unit 12 detects a region in the image to be processed that shows a predetermined part of the display shelf as the comparison region. The comparison region may be represented by coordinate information of the coordinate system set in the image, or it may be indicated by other means. For example, if the comparison region is a rectangle, the comparison region may be represented by the coordinates of the four vertices of the rectangle.

[0056] The "image to be processed" is an image captured by the imaging means described above. The image to be processed is an image captured by the imaging means at a predetermined timing after the imaging means has been set up to be in its basic state. Even if it has been set up to be in its basic state, the state of the imaging means may not be in its basic state due to subsequent unexpected events. For this reason, the image to be processed can be described as "an image captured by the imaging means whose state is unknown." Examples of unexpected events include, but are not limited to, events in which customers or store employees touch the imaging means and change its state, or events in which the state of the imaging means changes due to disasters such as earthquakes.

[0057] All images captured by the imaging means (still images, video frames) may be subject to processing by the detection unit 12 (processing target images). Alternatively, only a portion of the images captured by the imaging means (still images, video frames) may be subject to processing by the detection unit 12 (processing target images). An example of the latter is to select a portion of the images captured by the imaging means based on predetermined rules and use them as processing target images. Examples of predetermined rules include, but are not limited to, extraction at predetermined time intervals (predetermined frame intervals) or extraction at the timing of instruction input from the operator.

[0058] "A designated portion of the display shelf" includes at least one of a member that partitions in the vertical direction and a member that partitions in the horizontal direction. The area on which goods are displayed is partitioned by the member that partitions in the vertical direction and the member that partitions in the horizontal direction. The member that partitions in the vertical direction includes at least one of a shelf board and a top board. The member that partitions in the horizontal direction includes at least one of a partition board that partitions in the horizontal direction and a side wall.

[0059] The detection unit 12 can detect a region in which a predetermined part of a display shelf, including at least one of the members that divide the shelf vertically and the members that divide it horizontally, is captured as a comparison target region. For example, the detection unit 12 can detect a region in which a side portion of a shelf board, top board, partition board, side wall, etc., or more specifically, a side portion facing the imaging means, is captured as a comparison target region.

[0060] Please note that the designated section of the display shelf exemplified here is merely an example and is not limited to the examples provided.

[0061] There are various means for detecting the area in the image that shows a predetermined part of the display shelf. For example, the detection unit 12 may detect the area in the image that shows a predetermined part of the display shelf based on the characteristic features of the appearance of that predetermined part of the display shelf that have been registered in the monitoring device 10 in advance.

[0062] In addition, if a predetermined portion of the display shelf is a shelf board, the detection unit 12 may detect the area in the image where the product is visible based on the characteristic features of the product's appearance that have been previously registered in the monitoring device 10. Alternatively, the detection unit 12 may detect objects in the image using widely known object detection techniques. Furthermore, the detection unit 12 may detect the area in which an object touching the bottom of at least one detected product is visible as the area where the shelf board is visible. As another example, the detection unit 12 may create groups of products located at the same height and detect the area in which an object located between two vertically adjacent groups is visible as the area where the shelf board is visible.

[0063] Alternatively, the operator may visually inspect the image to be processed and, via any input means, specify a region showing a predetermined part of the display shelf as the comparison region.

[0064] The methods exemplified here are merely examples and are not limited to these methods.

[0065] Furthermore, when the detection unit 12 detects both an area containing a vertically dividing member and an area containing a horizontally dividing member as comparison target areas, it may distinguish between them during detection. For example, if the vertically dividing member and the horizontally dividing member have different appearances, the detection unit 12 can distinguish between them and detect them based on the difference in their appearances. Also, the vertically dividing member has a shape that extends horizontally in the image, and the horizontally dividing member has a shape that extends vertically in the image. The detection unit 12 may distinguish between them and detect them based on these shape characteristics. In addition, when the operator specifies comparison target areas, the operator may input information indicating "whether a vertically dividing member or a horizontally dividing member is visible in each specified comparison target area."

[0066] The determination unit 13 determines a change in the state of the imaging means based on the relative positional relationship between the basic region and the comparison target region.

[0067] The determination unit 13 can determine a change in the state of the imaging means based on at least one of the following: • Whether or not there is overlap between the basic domain and the comparison domain. • The degree of overlap between the basic domain and the comparison domain.

[0068] As described above, the basic area is the area in the basic image that shows the products displayed on the shelf. The comparison area is the area in the image to be processed that shows a predetermined part of the shelf.

[0069] "Determination based on whether or not there is overlap between the basic domain and the comparison domain." The determination unit 13 can determine that there has been a change in the state of the imaging means if the basic area and the comparison area overlap.

[0070] Figure 9 shows an example of a processed image P where the state of the imaging means has changed. Figure 7 shows the basic image P. s Comparing this with the processed image P in Figure 9 after the state change, it can be seen that the area being captured has shifted downwards.

[0071] Figure 9 shows the comparison regions Y1 and Y2 (regions on the image P in which a predetermined part of the display shelf is visible) detected on the image P to be processed. Figure 9 also shows the base image P from Figure 7. s Basic regions X1 to X4 (basic image P) set above s The area in which the product is captured is shown above. In Figure 9, the comparison area Y1 and the basic areas X1 and X2 overlap with each other. Also in Figure 9, the comparison area Y2 and the basic areas X3 and X4 overlap with each other. In this case, the determination unit 13 determines that there has been a change in the state of the imaging means.

[0072] Furthermore, if the basic area and the comparison area overlap, and the area overlapping with the basic area is the comparison area in which the member that divides the display shelf vertically is captured, the determination unit 13 can determine that the orientation of the imaging means has changed vertically.

[0073] Furthermore, if the basic area and the comparison area overlap, and the area overlapping with the basic area is the comparison area in which the partition separating the display shelf in the left-right direction is captured, the determination unit 13 can determine that the orientation of the imaging means has changed in the left-right direction.

[0074] As shown in Figure 10, if there is no change in the state of the imaging means, the comparison regions Y1 and Y2 and the basic regions X1 to X4 do not overlap with each other.

[0075] "Determination based on the degree of overlap between the basic domain and the comparison domain." Even if there is no change in the state of the imaging means due to unforeseen events such as detection errors, there is a possibility that the basic region and the comparison region may slightly overlap. According to the judgment in this example, it is possible to suppress the inconvenience of mistakenly determining that there has been a change in the state of the imaging means in such cases.

[0076] For example, the determination unit 13 can determine that there has been a change in the state of the imaging means if a predetermined percentage or more of the area to be compared overlaps with the basic area.

[0077] Furthermore, if a predetermined proportion or more of the comparison area overlaps with the basic area, and the overlapping area is the comparison area that shows the component that divides the display shelf vertically, the determination unit 13 can determine that the orientation of the imaging means has changed vertically.

[0078] Furthermore, if a predetermined proportion or more of the comparison area overlaps with the basic area, and the overlapping area is the comparison area that shows the partition separating the left and right sides of the display shelf, the determination unit 13 can determine that the orientation of the imaging means has changed in the left and right directions.

[0079] As another example, the determination unit 13 can determine that there has been a change in the state of the imaging means if a predetermined proportion or more of the basic area overlaps with the comparison area.

[0080] Furthermore, if a predetermined proportion or more of the basic area overlaps with the comparison area, and the overlapping area is the comparison area in which the member that divides the display shelf vertically is captured, the determination unit 13 can determine that the orientation of the imaging means has changed vertically.

[0081] Furthermore, if a predetermined proportion or more of the basic area overlaps with the comparison area, and the overlapping area is the comparison area in which the partition separating the display shelf in the left-right direction is captured, the determination unit 13 can determine that the orientation of the imaging means has changed in the left-right direction.

[0082] The above-mentioned predetermined percentages are fixed values. Specific values ​​are a matter of design.

[0083] Next, an example of the processing flow of the monitoring device 10 will be explained using the flowchart in Figure 2. First, before executing the processing in the flowchart in Figure 2, the monitoring device 10 registers the area in the basic image captured by the imaging means in the basic state in which the products displayed on the shelves are shown as the basic area.

[0084] Then, the monitoring device 10 detects a "region in which a predetermined part of the display shelf is captured" from the image to be processed captured by the imaging means as a comparison target region (S10). Next, the monitoring device 10 determines a change in the state of the imaging means based on the relative positional relationship between the "region in which products displayed on the display shelf are captured (basic region)" which has been registered in advance and the "region in which a predetermined part of the display shelf is captured (comparison target region)" detected in S10 (S11).

[0085] An example of the processing performed by the monitoring device 10 after the determination is described in the following embodiment, but for example, the monitoring device 10 may notify the operator of the determination result. The monitoring device 10 can notify the operator via any output device such as a display, speaker, projection device, or printer. The monitoring device 10 can also notify the operator via devices such as a POS (point of sales) register, a terminal device installed in the back room of the store, or a mobile terminal carried by a store employee. The monitoring device 10 may also notify a predetermined operator who has been registered in advance using means such as sending an email or push notification from a predetermined application.

[0086] Furthermore, the monitoring device 10 may notify the operator only if it determines that the state of the imaging means has changed. The notification to the operator may include the installation location of the imaging means for which a change in state has been determined. In this case, the installation locations of each of the multiple imaging means are registered in the monitoring device 10 in advance. Based on this information, the monitoring device 10 can identify the installation location of the imaging means for which a change in state has been determined. In addition, the notification to the operator may include the nature of the change in the orientation of the imaging means (change in the vertical direction / change in the horizontal direction).

[0087] Here, a modified example of this embodiment will be described. This modified example is applicable to all of the following embodiments.

[0088] The monitoring device 10 may perform processing other than determining changes in the state of the imaging means, using the image to be processed or pre-registered basic areas. For example, the monitoring device 10 may use the image to be processed to monitor the display status of products displayed on shelves. Examples of monitoring the display status include, but are not limited to, detection of foreign objects, detection of shortages in the number of displayed products, and detection of disorder in the display. In detecting shortages in the number of displayed products, the monitoring device 10 may use basic areas registered by the registration unit 11. That is, if the areas where various products are displayed (display areas assigned to various products) are registered as basic areas, the monitoring device 10 may calculate the product sufficiency rate based on the proportion of product areas that fill the basic areas. The monitoring device 10 may also use the image to be processed to monitor the behavior of store employees and customers. These processes using the image to be processed can be implemented using any technology.

[0089] In this modified example, the monitoring device 10 can further determine changes in the state of the imaging means using the processing target image and basic area used to monitor the display status of products and the behavior of store employees and customers.

[0090] "Effects and Effects" The monitoring device 10 of this embodiment achieves the same effects and advantages as the monitoring device 10 of the first embodiment.

[0091] Furthermore, the monitoring device 10 of this embodiment can determine changes in the state of the imaging means based on whether or not there is overlap between the basic area and the comparison area, and the degree of overlap. With such a monitoring device 10, changes in the state of cameras installed in stores can be detected using a new method.

[0092] Furthermore, the monitoring device 10 of this embodiment can determine the nature of any changes in the state of the imaging means. Specifically, it can determine whether the orientation of the imaging means has changed vertically or horizontally. Based on these determinations, the operator responsible for restoring the state of the imaging means can understand what actions to take.

[0093] Furthermore, the monitoring device 10 of this embodiment can determine that the orientation of the imaging means has changed in the vertical direction when the comparison area, in which the members dividing the display shelf vertically are visible, overlaps with the basic area. The monitoring device 10 of this embodiment can also determine that the orientation of the imaging means has changed in the horizontal direction when the comparison area, in which the members dividing the display shelf horizontally are visible, overlaps with the basic area. Such a monitoring device 10 can accurately determine the nature of changes in the orientation of the imaging means.

[0094] <Fourth Embodiment> The monitoring device 10 of the fourth embodiment is a concrete implementation of the configuration of the monitoring device 10 of the first embodiment.

[0095] In the fourth embodiment, the monitoring device 10 registers the "area in which a predetermined part of the display shelf is captured" as the basic area in the basic image captured by the imaging means in the basic state. The monitoring device 10 then determines the change in the state of the imaging means based on the relative positional relationship between this basic area and the "area in which products displayed on the display shelf are captured (comparison area)" detected from the image to be processed.

[0096] The monitoring device 10 of the fourth embodiment differs from the monitoring device 10 of the third embodiment in the content of the basic area and the comparison area. Specifically, the monitoring device 10 of the third embodiment registered the "area in which products displayed on the display shelf are captured" as the basic area and detected a "predetermined part of the display shelf" as the comparison area. In contrast, the monitoring device 10 of the fourth embodiment registers a "predetermined part of the display shelf" as the basic area and detects a "area in which products displayed on the display shelf are captured" as the comparison area.

[0097] The "basic region" of the fourth embodiment is a basic image P as shown in Figure 6. s In this context, the area is the region in which a predetermined part of the display shelf is captured. The basic area is at least a portion of the area in which the predetermined part of the display shelf is captured.

[0098] The other configurations of the monitoring device 10 in the fourth embodiment are the same as those of the monitoring device 10 in the third embodiment.

[0099] The monitoring device 10 of the fourth embodiment achieves the same effects and advantages as the monitoring device 10 of the third embodiment.

[0100] <Fifth Embodiment> "overview" The monitoring device 10 of the fifth embodiment is a concrete implementation of the configuration of the monitoring device 10 of the second embodiment.

[0101] In the fifth embodiment, the monitoring device 10 registers the "area in which products displayed on the shelves are captured" as the basic area in the basic image captured by the imaging means in the basic state. The monitoring device 10 then determines the change in the state of the imaging means based on the comparison result between this basic area and the "area in which products displayed on the shelves are captured (comparison area)" detected from the image to be processed. The configuration of the monitoring device 10 will be described in detail below.

[0102] "Hardware configuration" The monitoring device 10 of the fifth embodiment can adopt a configuration similar to the hardware configuration described in the third embodiment.

[0103] "Functional Configuration" Next, the functional configuration of the monitoring device 10 will be described in detail. Figure 1 shows an example of a functional block diagram of the monitoring device 10. As shown in the figure, the monitoring device 10 has a registration unit 11, a detection unit 12, and a determination unit 13.

[0104] The registration unit 11 registers the area in which products detected from the basic image captured by the imaging means in the basic state are depicted as a basic area. The registration unit 11 can register at least one product group area in which a predetermined group of products detected from the basic image is depicted as a basic area. For example, the registration unit 11 may register all of multiple product group areas detected from the basic image as basic areas.

[0105] A "specified product group" is a collection of products of the same type.

[0106] A "product group area" is the area where various products are displayed (the display area assigned to each product). In the example shown in Figure 7, four product group areas are shown, corresponding to four different types of products. These four product group areas are set up and registered as four basic areas X1 to X4.

[0107] The registration unit 11 can set and register such basic areas using the method described in the third embodiment.

[0108] The detection unit 12 detects the area containing the product from the image to be processed captured by the imaging means as the comparison area. The detection unit 12 can detect at least one product group area as the comparison area from the image to be processed captured by the imaging means. For example, the detection unit 12 can detect all of multiple product group areas captured in the image to be processed as the comparison area.

[0109] The detection unit 12 can detect the comparison area from the image to be processed using the same method as the detection of the "area in which products displayed on the shelves are captured" by the registration unit 11 described in the third embodiment.

[0110] The determination unit 13 determines a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area. The determination unit 13 can determine a change in the state of the imaging means based on at least one of the following: the number of product group areas detected from the image, the area of ​​each product group area, and the position of each product group area.

[0111] "Determination based on the number of product group regions detected from the image." When the state of the imaging device changes, the product group regions included in the image may change. And, in accordance with this change, the number of product group regions detected from the image will also change.

[0112] The determination unit 13 can determine that there has been a change in the state of the imaging means if the number of product group regions detected from the basic image is different from the number of product group regions detected from the image to be processed.

[0113] "Determination based on the area of ​​each product group region detected from the image." When the state of the imaging device changes, some product areas may become cropped. Consequently, the area of ​​those product areas within the image changes accordingly.

[0114] The determination unit 13 compares the area of ​​the product group region corresponding to each product detected from the basic image with the area of ​​the product group region corresponding to each product detected from the image to be processed. The determination unit 13 can then determine that there has been a change in the state of the imaging means if the area in the basic image and the area in the image to be processed are different in the product group region corresponding to at least one product.

[0115] "Determination based on the position of each product group region detected from the image." When the state of the imaging device changes, the position of the product group region included in the image changes within the image.

[0116] The determination unit 13 compares the position of the product group region corresponding to each product detected from the basic image with the position of the product group region corresponding to each product detected from the image to be processed. The determination unit 13 can then determine that there has been a change in the state of the imaging means if the position in the basic image and the position in the image to be processed are different for at least one product in the product group region.

[0117] Next, an example of the processing flow of the monitoring device 10 will be explained using the flowchart in Figure 4. First, before executing the processing in the flowchart in Figure 4, the monitoring device 10 registers the area in the basic image captured by the imaging means in the basic state in which the products displayed on the shelves are shown as the basic area.

[0118] The monitoring device 10 detects the area in which the product is captured (comparison area) from the image to be processed captured by the imaging means (S30).

[0119] Then, the monitoring device 10 determines the change in the state of the imaging means based on the comparison result between the previously registered "area in which the product detected from the basic image is visible (basic area)" and the "area in which the product detected from the image to be processed is visible (comparison area)" (S31).

[0120] An example of the processing performed by the monitoring device 10 after the determination will be described in the following embodiment, but for example, the monitoring device 10 may perform the processing described in the third embodiment.

[0121] "Effects and Effects" The monitoring device 10 of this embodiment achieves the same effects and advantages as the monitoring device 10 of the second embodiment.

[0122] Furthermore, the monitoring device 10 of this embodiment can determine a change in the state of the imaging means based on at least one of the following: the number of product group regions detected from the image, the area of ​​each product group region, and the position of each product group region.

[0123] In this way, the monitoring device 10 detects the area in which the product is captured from the image to be processed, and determines a change in the state of the imaging means based on the comparison result between the "area in which the product is captured detected from the basic image" and the "area in which the product is captured detected from the image to be processed." With such a monitoring device 10, it is possible to detect changes in the state of cameras installed in stores using a new method.

[0124] <Sixth Embodiment> The monitoring device 10 of the sixth embodiment can determine the degree of change when it detects a change in the state of the imaging means, and can perform appropriate processing according to that degree. This will be explained in detail below.

[0125] If the determination unit 13 determines that there has been a change in the state of the imaging means, it further determines the degree of the change in the state of the imaging means. Specifically, the determination unit 13 determines whether the degree of change is above a predetermined level. Then, the determination unit 13 executes corresponding processing according to the determined degree of change.

[0126] "Determining the degree of change" The determination unit 13 can determine that the degree of change in the state of the imaging means is above a predetermined level if it meets predetermined criteria. Conversely, the determination unit 13 can determine that the degree of change in the state of the imaging means is below a predetermined level if it does not meet predetermined criteria.

[0127] The prescribed criteria include at least one of the following: • The area exceeding the threshold percentage of the comparison area overlaps with the basic area. • The area exceeding the threshold percentage of the basic area overlaps with the comparison area. • The number of product group regions detected from the base image differs from the number of product group regions detected from the image being processed. • In a product group domain corresponding to at least one type of product, the difference or rate of change between the area in the base image and the area in the image to be processed is greater than or equal to a threshold. • In a product group domain corresponding to at least one type of product, the distance between the position in the base image and the position in the image to be processed is greater than or equal to a threshold. The threshold values ​​mentioned above are predetermined values. Specific values ​​are a matter of design.

[0128] In this way, the determination unit 13 can determine the degree of change in the state of the imaging means based on the degree of overlap between the basic region and the comparison target region. Furthermore, the determination unit 13 can determine the degree of change in the state of the imaging means based on the position of each product group region within the processing target image, the number of detected product group regions, and at least one of the areas of each product group region.

[0129] "Response processing according to the degree of change determined" The determination unit 13 can re-register the basic area if the degree of change in the state of the imaging means is below a predetermined level.

[0130] As described in the third embodiment as a modified example, the monitoring device 10 can perform various processes, such as monitoring the display status, using a pre-registered basic area. Therefore, as shown in Figure 9, if the basic area is in an inappropriate state in response to a change in the state of the imaging means, various processes, such as monitoring the display status, cannot be performed correctly. Thus, if the basic area is in an inappropriate state in response to a change in the state of the imaging means, it is necessary to take measures to bring the basic area into an appropriate state.

[0131] The measures to bring the basic area into an appropriate state can be (1) correcting the state of the imaging means and (2) re-registering the basic area. If the degree of change in the state of the imaging means is below a predetermined level, the determination unit 13 automatically re-registers the basic area to reduce the burden on the operator. If the degree of change in the state of the imaging means is below a predetermined level, there is no major change such as the product itself being changed in the image, so the determination unit 13 can handle this by automatically re-registering the basic area.

[0132] The re-registration of the basic area can be achieved by any means. For example, the characteristic quantities of the appearance of multiple types of products may be registered in advance in the monitoring device 10. The determination unit 13 may then identify the areas in the image to be processed in which various products are depicted based on these characteristic quantities of appearance. The determination unit 13 can then register a block of areas in which similar products are depicted, that is, the areas in which various products are displayed (display areas assigned to various products), as the basic area.

[0133] On the other hand, if the degree of change in the state of the imaging means is above a predetermined level, that is, if it is difficult to automatically re-register the basic area, the determination unit 13 notifies the operator of the change in the state of the imaging means. The operator checks the state of the imaging means in response to the notification and corrects the state of the imaging means or re-registers the basic area.

[0134] The determination unit 13 can notify operators via any output device such as a display, speaker, projection device, or printer. The determination unit 13 can also notify operators via devices such as a POS register, a terminal device installed in the back room of a store, or a mobile terminal carried by a store employee. Furthermore, the determination unit 13 may notify pre-registered operators using means such as sending emails or push notifications from a designated application.

[0135] Furthermore, the operator may be notified of the installation location of the imaging device in which a change in state exceeding a predetermined level has been determined. In this case, the installation locations of each of the multiple imaging devices are registered in the monitoring device 10 in advance. Based on this information, the determination unit 13 can identify the installation location of the imaging device in which a change in state exceeding a predetermined level has been determined. In addition, the operator may be notified of the nature of the change in the orientation of the imaging device (change in the vertical direction / change in the horizontal direction).

[0136] The other configurations of the monitoring device 10 in the sixth embodiment are the same as those of the monitoring device 10 in the first to fifth embodiments.

[0137] The monitoring device 10 of the sixth embodiment achieves the same effects and advantages as the monitoring devices 10 of the first to fifth embodiments. Furthermore, when the monitoring device 10 of the sixth embodiment determines that there has been a change in the state of the imaging means, it can determine the degree of the change and execute appropriate processing according to that degree.

[0138] Specifically, if the degree of change is below a predetermined level, the monitoring device 10 can automatically re-register the basic range, thereby reducing the operator's burden. If the degree of change is above the predetermined level, the monitoring device 10 can notify the operator. With such a monitoring device 10, the operator's burden can be reduced to an appropriate extent.

[0139] <Variation> The following describes variations applicable to all embodiments.

[0140] In the case of adopting the configuration of the sixth embodiment, even if the degree of change is below a predetermined level and the basic region is automatically registered, the determination unit 13 may notify the operator of this fact. In this notification, the determination unit 13 can inform the operator that there has been a change in the state of the imaging means and that the basic region will be automatically re-registered. Furthermore, the determination unit 13 may also notify the operator of the contents of the re-registered basic region. For example, the determination unit 13 may present the operator with an image in which information indicating the newly registered basic region is superimposed on the processing target image captured by the imaging means after the change in state. The contents of the image may be, for example, as shown in Figure 7. Based on the image, the operator can determine the validity of the newly registered basic region.

[0141] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0142] Furthermore, the flowcharts used in the above description show multiple steps (processes) in sequence. However, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments can be combined to the extent that their content is not contradictory.

[0143] Some or all of the above embodiments may also be described as follows, but are not limited to the following. 1. A registration means that registers as a basic area an area in which a predetermined part of the display shelf is captured in a basic image captured by the imaging means of the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means for determining a change in the state of the imaging means based on the relative positional relationship between the basic region and the comparison target region, A monitoring device having the following features. 2. The determination means is, A monitoring device according to claim 1 that determines a change in the state of the imaging means based on whether or not there is an overlap between the basic area and the comparison target area, and at least one of the degree of overlap between the basic area and the comparison target area. 3. The determination means is, The monitoring device according to claim 2, which determines that there has been a change in the state of the imaging means when the basic area and the comparison area overlap. 4. The determination means is, The monitoring device according to claim 2, which determines that there has been a change in the state of the imaging means when a predetermined percentage or more of the area of ​​the comparison target area overlaps with the basic area. 5. The determination means is, The monitoring device according to claim 2, which determines that there has been a change in the state of the imaging means when an area exceeding a predetermined proportion of the basic area overlaps with the comparison target area. 6. The determination means is a monitoring device according to any one of 2 to 5 that determines the content of a change in the state of the imaging means. 7. The designated portion of the display shelf is a member that divides the vertical direction, The determination means is, The monitoring device according to 6, which determines that the orientation of the imaging means has changed in the vertical direction when the basic area and the comparison area overlap. 8. The monitoring device according to 7, wherein the member separating the vertical direction includes at least one of a shelf board and a top board. 9. The designated portion of the display shelf is a member that divides the left and right sides, The determination means is, The monitoring device according to 6, which determines that the orientation of the imaging means has changed in the left-right direction when the basic area and the comparison area overlap. 10. The monitoring device according to 9, wherein the member separating the left and right sides includes at least one of a partition plate and a side wall. 11. The determination means is, A monitoring device according to any one of 1 to 10, which determines the degree of change in the state of the imaging means and performs corresponding processing according to the determined degree of change. 12. The determination means is, If the degree of change in the state of the imaging means exceeds a predetermined level, the operator is notified of the change in the state of the imaging means. The monitoring device according to 11, which re-registers the basic area if the degree of change in the state of the imaging means is less than the predetermined level. 13. The determination means is, A monitoring device according to 11 or 12 that determines the degree of change in the state of the imaging means based on the degree of overlap between the basic area and the comparison area. 14. The detection means detects at least one product group region in which a predetermined group of products displayed on the display shelf is shown from the image to be processed captured by the imaging means, The determination means is, A monitoring device according to 11 or 12 that determines the degree of change in the state of the imaging means based on at least one of the following: the position of each of the product group regions within the image to be processed, the number of product group regions detected, and the area of ​​each of the product group regions. 15. One or more computers, In the basic image captured by the imaging means in the basic state, the area in which a predetermined part of the display shelf is captured, or the area in which the products displayed on the display shelf are captured, is registered as the basic area. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the area in which the products displayed on the display shelf are captured is detected as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the area in which a predetermined portion of the display shelf is captured is detected as the comparison area from the image to be processed, A monitoring method for determining a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. 16. Computers, A registration means for registering as a basic area an area in which a predetermined part of a display shelf is captured in a basic image captured by an imaging means in the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means that determines a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. A recording medium that stores a program to function as such. 17. A registration means that registers the area in which the product detected from the basic image captured by the imaging means in the basic state is shown as the basic area, A detection means for detecting, from the image to be processed captured by the imaging means, the region in which the product is visible as the comparison region, A determination means for determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area, A monitoring device having the following features. 18. The registration means registers at least one product group region in which a predetermined product group detected from the basic image is depicted as the basic region. The detection means detects the product group region in which the predetermined product group is depicted from the image to be processed captured by the imaging means as the comparison target region, The determination means is, The monitoring device according to 17, which determines a change in the state of the imaging means based on at least one of the number of product group regions detected from the image, the area of ​​each product group region, and the position of each product group region. 19. One or more computers, The imaging means in the basic state registers the area in which the detected product is visible from the basic image captured as the basic area. From the image to be processed captured by the imaging means, the region in which the product is visible is detected as the comparison region. A monitoring method for determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area. 20. Computers, A registration means that registers the area in which the product is detected from the basic image captured by the basic state imaging means as the basic area. A detection means that detects the area in which the product is visible from the image to be processed captured by the imaging means as a comparison area. A determination means that determines a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area. A recording medium that stores a program to function as such. Some or all of the appendices 2 to 14, which are subordinate to the monitoring device described in appendice 1 above, may also be subordinate to the monitoring method described in appendice 15 and the recording medium described in appendice 16, in the same subordinate relationship as between appendice 1 and appendices 2 to 14. Furthermore, appendice 18, which is subordinate to the monitoring device described in appendice 17 above, may also be subordinate to the monitoring method described in appendice 19 and the recording medium described in appendice 20, in the same subordinate relationship as between appendice 17 and appendice 18. Moreover, within the scope of the embodiments described above, some or all of the configurations described as appendices can be realized in various hardware, software, various recording means for recording software, or systems. [Explanation of symbols]

[0144] 10 Monitoring equipment 11. Registration Department 12 Detection unit 13 Judgment section 1A Processor 2A Memory 3A input / output I / F 4A Peripheral Circuits 5A bus

Claims

1. A registration means that registers as a basic region at least one product group region in which a predetermined product group is captured from a basic image captured by an imaging means in the basic state, A detection means for detecting, from the image to be processed captured by the imaging means, a product group region in which the predetermined product group is captured as a comparison target region, A determination means for determining a change in the state of the imaging means based on the number of product group regions detected from the image, and the comparison result between the basic region and the comparison target region, which is based on at least one of the areas of each of the product group regions. A monitoring device having the following features.

2. The monitoring device according to claim 1, wherein the determination means further determines a change in the state of the imaging means based on the position of each of the product group regions.

3. The monitoring device according to claim 1, wherein the determination means determines that there has been a change in the state of the imaging means when the number of product group regions detected from the basic image is different from the number of product group regions detected from the image to be processed.

4. The monitoring device according to claim 1, wherein the determination means determines that there has been a change in the state of the imaging means when the area in the basic image and the area in the processing target image are different in the product group region corresponding to at least one type of product.

5. The monitoring device according to claim 2, wherein the determination means determines that there has been a change in the state of the imaging means when the position in the basic image and the position in the processing target image are different in the product group region corresponding to at least one type of product.

6. The monitoring device according to claim 1, wherein the determination means further determines the degree of change in the state of the imaging means and performs corresponding processing according to the determined degree of change.

7. The determination means is, If the degree of change in the state of the imaging means exceeds a predetermined level, the operator is notified of the change in the state of the imaging means. The monitoring device according to claim 6, wherein the basic area is re-registered if the degree of change in the state of the imaging means is less than the predetermined level.

8. The determination means is, The number of product group regions detected from the basic image is different from the number of product group regions detected from the image to be processed, and In the product group region corresponding to at least one type of product, the difference or rate of change between the area in the basic image and the area in the image to be processed is greater than or equal to a threshold. The monitoring device according to claim 7, which determines that the degree of change in the state of the imaging means is equal to or greater than the predetermined level if at least one of the following conditions is met.

9. One or more computers, The imaging means in the basic state captures a basic image in which at least one product group region containing a predetermined product group is detected is registered as the basic region. From the image to be processed captured by the imaging means, the product group region in which the predetermined product group is captured is detected as the comparison target region. A monitoring method for determining a change in the state of the imaging means based on the number of product group regions detected from the image, and the comparison result between the basic region and the comparison target region, which is based on at least one of the areas of each of the product group regions.

10. Computers, A registration means that registers as a basic region at least one product group region in which a predetermined product group is captured from a basic image captured by an imaging means in the basic state, A detection means for detecting a product group region containing the predetermined product group from the image to be processed captured by the imaging means, as a comparison target region. A determination means for determining a change in the state of the imaging means, based on the number of product group regions detected from the image, and the comparison result between the basic region and the comparison target region, which is based on at least one of the areas of each of the product group regions. A program that makes it function as such.

Images