Monitoring device, monitoring method, and program

Abstract

A monitoring device according to the present disclosure comprises a registration unit, a detection unit, and a determination unit. The registration unit registers, as a basic region, a region in which products displayed in a display shelf are shown in a basic image captured by an imaging means in a basic state. The detection unit detects, as a region to be compared, a region in which a predetermined portion of the display shelf is shown from an image to be processed. The determination unit determines a change in the state of the imaging means on the basis of a relative positional relationship between the basic region and the region to be compared.

Description

Monitoring device, monitoring method, and recording medium

[0001] The present disclosure relates to a monitoring device, a monitoring method, and a recording medium.

[0002] A technology related to the present disclosure is disclosed in Patent Document 1. Patent Document 1 discloses a technology for capturing images of display shelves with a camera installed in a store and monitoring the display status of products based on the images generated by the camera. Patent Document 1 also discloses a technology for detecting a change in the imaged area due to a misalignment of the camera or the like. This technology sets a part of the display shelf captured in an image generated by a certain camera as a fixed area. Specifically, the area capturing the shelf frame of the display shelf is set as the fixed area. This technology then detects a change in the object captured by the camera based on the similarity between the image of the fixed area in that image and images at the same position as the fixed area in other images captured by the camera.

[0003] JP 2016-99923 A

[0004] When the state (posture or position) of a camera installed in a store changes, the area captured by that camera also changes. This change can sometimes prevent the correct processing based on image analysis. Therefore, there is a need for technology to detect changes in the camera's state.

[0005] In view of the above-mentioned problems, an example of an object of the present disclosure is to provide a monitoring device, a monitoring method, and a recording medium that detect changes in the status of cameras installed in a store using a new technique.

[0006] The monitoring device of the present disclosure comprises: a registration means for registering an area in a basic image captured by an imaging means in a basic state in which a specified part of a display shelf or an area in which products displayed on the display shelf are displayed as a basic area; a detection means for, if the area in which the specified part of the display shelf is displayed is registered as the basic area, detecting an area in the image to be processed captured by the imaging means in which the products displayed on the display shelf are displayed as a comparison area, and, if the area in which the products displayed on the display shelf are displayed is registered as the basic area, detecting an area in the image to be processed in which the specified part of the display shelf is displayed as a comparison area; and a determination means for determining a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area.

[0007] In addition, the monitoring method of the present disclosure includes one or more computers registering, as a basic area, an area in a basic image captured by an imaging means in a basic state where a specified part of a display shelf or an area where products displayed on the display shelf are displayed, and if the area where the specified part of the display shelf is displayed is registered as the basic area, detecting, from the image to be processed captured by the imaging means, an area where the products displayed on the display shelf are displayed as a comparison area, and if the area where the products displayed on the display shelf are displayed is registered as the basic area, detecting, from the image to be processed, an area where the specified part of the display shelf is displayed as a comparison area, and determining a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area.

[0008] In addition, the recording medium of the present disclosure stores a program that causes a computer to function as: a registration means that registers an area in a basic image captured by an imaging means in a basic state in which a specified portion of a display shelf or an area in which products displayed on the display shelf are displayed as a basic area; a detection means that, if the area in which the specified portion of the display shelf is displayed is registered as the basic area, detects an area in the image to be processed captured by the imaging means in which the products displayed on the display shelf are displayed as a comparison area, and if the area in which the products displayed on the display shelf are displayed is registered as the basic area, detects an area in the image to be processed in which a specified portion of the display shelf is displayed as a comparison area; and a determination means that determines a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area.

[0009] The monitoring device of the present disclosure also has a registration means for registering an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area; a detection means for detecting an area in which the product is detected from an image to be processed captured by the imaging means as a comparison area; and a determination means for determining a change in the state of the imaging means based on a comparison result between the basic area and the comparison area.

[0010] In addition, the monitoring method disclosed herein involves one or more computers registering an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area, detecting an area in which the product is detected from an image to be processed captured by the imaging means as a comparison area, and determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison area.

[0011] In addition, the recording medium of the present disclosure stores a program that causes a computer to function as: a registration means that registers an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area; a detection means that detects an area in which the product is detected from an image to be processed captured by the imaging means as a comparison area; and a determination means that determines a change in the state of the imaging means based on the comparison result between the basic area and the comparison area.

[0012] According to one aspect of the present disclosure, a monitoring device, a monitoring method, and a recording medium are realized that detect changes in the status of a camera installed in a store using a new technique.

[0013] FIG. 1 is a diagram showing an example of a functional block diagram of a monitoring device according to the present disclosure. FIG. 2 is a flowchart showing an example of a processing flow of a monitoring device according to the present disclosure. FIG. 3 is a flowchart showing another example of a processing flow of a monitoring device according to the present disclosure. FIG. 4 is a diagram showing an example of a hardware configuration of a monitoring device according to the present disclosure. FIG. 5 is a diagram showing an example of an image processed by a monitoring device according to the present disclosure. FIG. 6 is a diagram showing another example of an image processed by a monitoring device according to the present disclosure. FIG. 7 is a diagram showing an example of information processed by a monitoring device according to the present disclosure. FIG. 8 is a diagram showing another example of an image processed by a monitoring device according to the present disclosure. FIG. 9 is a diagram showing another example of an image processed by a monitoring device according to the present disclosure.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are designated by similar reference numerals, and descriptions thereof will be omitted as appropriate.

[0015] 1 is a functional block diagram showing an overview of a monitoring device 10. As shown in FIG. 1, the monitoring device 10 includes a registration unit 11, a detection unit 12, and a determination unit 13.

[0016] The registration unit 11 performs advance preparations for the "process of detecting changes in the status of cameras installed in a store." Then, the detection unit 12 and the determination unit 13 execute the "process of detecting changes in the status of cameras installed in a store."

[0017] The registration unit 11 registers, as a basic area, an area in which a predetermined part of the display shelf is captured in a basic image captured by the imaging means (camera) in the basic state, or an area in which a product displayed on the display shelf is captured.

[0018] The detection unit 12 and the determination unit 13 execute the process shown in the flowchart of FIG. 2 or the process shown in the flowchart of FIG.

[0019] When the registration unit 11 registers the "area showing products displayed on a display shelf" in the basic image captured by the imaging means in the basic state as the basic area, the detection unit 12 and the judgment unit 13 execute the processing shown in the flowchart of Figure 2.

[0020] Specifically, the detection unit 12 detects, as a comparison target area, an "area in which a predetermined part of the display shelf is captured" from the processing target image captured by the imaging means (S10).

[0021] Then, the determination unit 13 determines a change in the state of the imaging means based on the relative positional relationship between the pre-registered ``area (basic area) in which products displayed on the display shelf are captured'' and the ``area (comparison area) in which a specified part of the display shelf is captured'' detected in S10 (S11).

[0022] On the other hand, when the registration unit 11 registers the "area in which a specified part of the display shelf is captured" in the basic image captured by the imaging means in the basic state as the basic area, the detection unit 12 and the judgment unit 13 execute the processing shown in the flowchart of Figure 3.

[0023] Specifically, the detection unit 12 detects, as a comparison target area, an "area showing products displayed on a display shelf" from within the processing target image captured by the imaging means (S20).

[0024] Then, the determination unit 13 determines a change in the state of the imaging means based on the relative positional relationship between the pre-registered ``area (basic area) that shows a specific part of the display shelf'' and the ``area (comparison area) that shows the products displayed on the display shelf'' detected in S20 (S21).

[0025] In this way, the monitoring device 10 of this embodiment can register in advance the "area in which products displayed on the display shelves (basic area)" in the basic image captured by the imaging means in the basic state. Then, the monitoring device 10 can determine a change in the state of the imaging means based on the relative positional relationship between the "area in which a predetermined part of the display shelves (comparison area)" detected from the processing target image captured by the imaging means and the pre-registered "area in which products displayed on the display shelves (basic area)."

[0026] Furthermore, the monitoring device 10 of this embodiment can register in advance an "area in which a predetermined part of the display shelf is captured (basic area)" in a basic image captured by the imaging means in a basic state. Then, the monitoring device 10 can determine a change in the state of the imaging means based on the relative positional relationship between the "area in which products displayed on the display shelf are captured (comparison area)" detected from the processing target image captured by the imaging means and the pre-registered "area in which a predetermined part of the display shelf is captured (basic area)."

[0027] According to the first embodiment, a monitoring device, a monitoring method, and a recording medium are realized that detect changes in the state of a camera installed in a store using a new technique.

[0028] Second Embodiment A monitoring device 10 according to a second embodiment determines a change in the state of an imaging unit using a process different from that of the monitoring device 10 according to the first embodiment.

[0029] 1 is a functional block diagram showing an overview of a monitoring device 10. As shown in FIG. 1, the monitoring device 10 includes a registration unit 11, a detection unit 12, and a determination unit 13.

[0030] The registration unit 11 performs advance preparations for the "process of detecting changes in the status of cameras installed in a store." Then, the detection unit 12 and the determination unit 13 execute the "process of detecting changes in the status of cameras installed in a store."

[0031] The registration unit 11 registers, as a basic area, an area in which the product is detected from a basic image captured by the imaging means in the basic state.

[0032] The detection unit 12 and the determination unit 13 execute the process shown in the flowchart of FIG.

[0033] Specifically, the detection unit 12 detects an area (comparison area) in which the product appears from the processing target image captured by the imaging means (S30).

[0034] Then, the judgment unit 13 judges a change in the state of the imaging means based on the comparison result between the pre-registered ``area in which the product detected from the basic image (basic area)'' and the ``area in which the product detected from the image to be processed (comparison area)'' (S31).

[0035] In this way, the monitoring device 10 detects the area in which the product appears from the image to be processed, and determines a change in the state of the imaging means based on the comparison result between the ``area in which the product appears detected from the basic image'' and the ``area in which the product appears detected from the image to be processed.''

[0036] According to the second embodiment, a monitoring device, a monitoring method, and a recording medium are realized that detect changes in the state of a camera installed in a store using a new technique.

[0037] Third Embodiment Overview A monitoring device 10 according to a third embodiment is a specific implementation of the configuration of the monitoring device 10 according to the first embodiment.

[0038] The monitoring device 10 of the third embodiment registers the "area showing products displayed on the display shelves" in the basic image captured by the imaging device in the basic state as the basic area. The monitoring device 10 then determines a change in the state of the imaging device based on the relative positional relationship between this basic area and the "area showing a predetermined part of the display shelves (comparison area)" detected from the image to be processed. The configuration of the monitoring device 10 will be described in detail below.

[0039] "Hardware Configuration" First, an example of the hardware configuration of the monitoring device 10 will be described. Each functional unit of the monitoring device 10 is realized by any combination of hardware and software. Those skilled in the art will understand that there are various variations in the realization method and device. Software includes programs that are pre-loaded in the device before shipping, and programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet.

[0040] FIG. 5 is a block diagram illustrating an example of the hardware configuration of the monitoring device 10. As shown in FIG. 5, the monitoring device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The monitoring device 10 does not necessarily have to have the peripheral circuit 4A. Note that the monitoring device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices can have the above hardware configuration.

[0041] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to mutually transmit and receive data. The processor 1A is, for example, a processing unit such as a CPU or a graphics processing unit (GPU). The memory 2A is, for example, a random access memory (RAM) or a read-only memory (ROM). The input / output interface 3A includes interfaces for acquiring information from input devices, external devices, external servers, external sensors, cameras, etc., and interfaces for outputting information to output devices, external devices, external servers, etc. The input / output interface 3A also includes an interface for connecting to a communication network such as the Internet. Examples of input devices include a keyboard, mouse, microphone, physical buttons, touch panel, etc. Examples of output devices include a display, speaker, printer, mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0042] "Functional Configuration" Next, a detailed description will be given of the functional configuration of the monitoring device 10. Fig. 1 shows an example of a functional block diagram of the monitoring device 10. As shown in the figure, the monitoring device 10 has a registration unit 11, a detection unit 12, and a determination unit 13.

[0043] The registration unit 11 registers, as a basic area, an area in which products displayed on the display shelves appear in a basic image captured by the imaging means in the basic state.

[0044] The "imaging means" is a so-called camera. There are no particular limitations on the type of camera. The imaging means may continuously capture moving images. Alternatively, the imaging means may capture still images periodically or intermittently at predetermined timings. Examples of predetermined timings include, but are not limited to, every predetermined time (e.g., every 5 minutes, every 10 minutes, every 15 minutes, etc.) or when an instruction is input by an operator.

[0045] The imaging means is installed in a position where it can image the display shelves installed in the store, in a position where it can image a predetermined area of ​​the display shelves. For example, an imaging means for imaging a certain display shelf may be attached by any means to another display shelf facing that display shelf. Also, the imaging means may be attached by any means to another member arranged facing that display shelf. Also, the imaging means may be attached to the ceiling or wall of the store.

[0046] The "basic state" is a state in which a desired area of ​​a desired display shelf can be imaged. The state of the imaging means varies depending on the installation position and posture. When installing the imaging means in a store, the operator sets the installation position and posture of the imaging means to a state (basic state) in which a desired area of ​​a desired display shelf can be imaged.

[0047] The "basic image" is an image captured by the imaging means in a basic state. The basic image captures a desired area of ​​a desired display shelf. ｓ An example of the basic image P ｓ shows a plurality of products M displayed on a display shelf. The products M are placed on a shelf board S. A price tag L is attached to the side of the shelf board S.

[0048] The "basic region" is a basic image P as shown in FIG. ｓ In the image, the basic area is an area in which products displayed on the display shelf are captured. The basic area is at least a part of the area in which products displayed on the display shelf are captured.

[0049] FIG. 7 shows the basic image P shown in FIG. ｓ In the example shown in FIG. 7, four basic regions X １ ~X ４ Specifically, for each type of product, the area in which the product is displayed (the display area assigned to the product) is set and registered as a basic image P ｓ Each of the identified regions is a basic region X １ ~X ４ It is registered as.

[0050] FIG. 8 schematically shows an example of registered information. This information links together numbers that identify multiple basic areas, area information that indicates each basic area in the image, and product identification information that indicates the products displayed in each area. The area information that indicates each basic area may be represented by coordinate information in a coordinate system set in the image, or may be represented by other means. For example, if the basic areas are rectangular, the area information that indicates each basic area may be represented by the coordinates of the four vertices of the rectangle. Such registered information may be stored in a storage device included in the monitoring device 10. Alternatively, such registered information may be stored in a storage device included in an external device configured to be able to communicate with the monitoring device 10.

[0051] As a variant, the basic image P ｓ Only the area in which any one of the multiple types of products shown in the image is displayed may be registered as the basic area. １ ~X ４ Only one of these may be registered as a basic region.

[0052] In another modification, the basic image P ｓ An area in which any one of a plurality of products displayed on a display shelf is shown may be registered as the basic area.

[0053] The basic region illustrated here is merely an example, and the basic image P ｓ Other areas within the area in which the products displayed on the display shelves are shown may be registered as basic areas.

[0054] The area to be registered as the basic area may be designated by the operator. That is, the operator may input the basic image P ｓ Alternatively, the monitoring device 10 may perform an operation of specifying a part of the basic image P by image analysis and registering the specified part as the basic image P. ｓ The monitoring device 10 may specify an area in which various products are shown from the image. The monitoring device 10 may then register the specified area as a basic area. For example, the appearance feature amounts of each of a plurality of types of products may be registered in advance in the monitoring device 10. The monitoring device 10 may then generate a basic image P based on the appearance feature amounts.ｓ The monitoring device 10 may identify an area in which various products are displayed from the image. For example, the monitoring device 10 may identify a group of areas in which the same type of product is displayed as an area in which various products are displayed (a display area allocated to various products).

[0055] 1 , the detection unit 12 detects, from the processing target image, an area in which a predetermined part of the display shelf is captured as a comparison target area. The comparison target area may be represented by coordinate information in a coordinate system set in the image, or may be represented by other means. For example, if the comparison target area is rectangular, the comparison target area may be represented by the coordinates of the four vertices of the rectangle.

[0056] The "image to be processed" is an image captured by the imaging means described above. The image to be processed is an image captured by the imaging means at a predetermined timing after the imaging means is set up to be in a basic state. Even if the imaging means is set up to be in a basic state, there is a possibility that the state of the imaging means is not in the basic state due to an unexpected event that occurs thereafter. Therefore, the image to be processed can be said to be "an image captured by imaging means that is unknown to be in the basic state." Examples of unexpected events include, but are not limited to, an event in which a customer or store clerk touches the imaging means, changing the state of the imaging means, or an event in which the state of the imaging means changes due to a disaster such as an earthquake.

[0057] All of the images (still images, video frame images) captured by the imaging means may be the targets of processing (processing target images) by the detection unit 12. Alternatively, only a portion of the images (still images, video frame images) captured by the imaging means may be the targets of processing (processing target images) by the detection unit 12. An example of the latter is when a portion of images extracted from the images captured by the imaging means based on a predetermined rule are used as the processing target images. Examples of the predetermined rule include, but are not limited to, extraction at predetermined time intervals (predetermined frame intervals) or extraction at the timing of an instruction input from the operator.

[0058] The "predetermined portion of the display shelf" includes at least one of a member that partitions the display shelf in the vertical direction and a member that partitions the display shelf in the left-right direction. The member that partitions the display shelf in the vertical direction and the member that partitions the display shelf in the left-right direction divides the display shelf. The member that partitions the display shelf in the vertical direction includes at least one of a shelf board and a top board. The member that partitions the display shelf in the left-right direction includes at least one of a partition board that partitions the display shelf in the left-right direction and a side wall.

[0059] The detection unit 12 can detect, as the comparison area, an area in which a predetermined part of the display shelf, including at least one of a member that partitions the display shelf in the vertical direction and a member that partitions the display shelf in the horizontal direction, is captured. For example, the detection unit 12 can detect, as the comparison area, an area in which a side part of a shelf board, a top board, a partition board, a side wall, etc., or more specifically, a side part facing the imaging means, is captured.

[0060] The predetermined parts of the display shelves illustrated here are merely examples, and are not limited to the examples shown here.

[0061] There are various means for detecting an area in an image in which a predetermined part of a display shelf is shown. For example, the detection unit 12 may detect an area in an image in which a predetermined part of a display shelf is shown, based on a feature amount of the appearance of the predetermined part of the display shelf that is registered in advance in the monitoring device 10.

[0062] Alternatively, if the predetermined portion of the display shelf is a shelf, the detection unit 12 may detect an area in which the product is captured from the image based on the appearance features of the product registered in advance in the monitoring device 10. Alternatively, the detection unit 12 may detect objects from the image using a well-known object detection technique. The detection unit 12 may then detect an area in which an object touching the bottom of each of at least one detected product is captured as an area in which the shelf is captured. As another example, the detection unit 12 may create groups of products located at the same height, and detect an area in which an object located between two vertically adjacent groups is captured as an area in which the shelf is captured.

[0063] Alternatively, the operator may visually check the image to be processed and specify, via any input means, an area in which a predetermined part of the display shelf is shown as the comparison area.

[0064] The means exemplified here are merely examples, and the present invention is not limited to these means.

[0065] Furthermore, when the detection unit 12 detects both an area containing a vertically partitioning component and an area containing a horizontally partitioning component as comparison target areas, the detection unit 12 may distinguish between them. For example, if the appearances of the vertically partitioning component and the horizontally partitioning component are different from each other, the detection unit 12 can distinguish between them based on the difference in appearance. Furthermore, a vertically partitioning component has a shape that extends horizontally on the image, and a horizontally partitioning component has a shape that extends vertically on the image. The detection unit 12 may distinguish between them and detect them based on such shape characteristics. Furthermore, when the operator specifies the comparison target areas, the operator may input information indicating "whether a vertically partitioning component or a horizontally partitioning component is shown in each specified comparison target area."

[0066] The determination unit 13 determines a change in the state of the imaging means based on the relative positional relationship between the base region and the comparison region.

[0067] The determination unit 13 can determine a change in the state of the imaging means based on at least one of the following: - Presence or absence of overlap between the basic region and the comparison region; - The degree of overlap between the basic region and the comparison region.

[0068] As described above, the basic region is a region in the basic image in which products displayed on the display shelves are captured, and the comparison region is a region in the processing target image in which a predetermined portion of the display shelves is captured.

[0069] "Determination Based on Whether or Not the Basic Region and the Comparison Region Overlap" When the basic region and the comparison region overlap, the determination unit 13 can determine that a change in the state of the imaging means has occurred.

[0070] 9 shows an example of the processing target image P in which the state of the imaging means has changed. ｓ When comparing the image to be processed P in FIG. 9 after the state change with the image to be processed, it can be seen that the captured area is shifted downward.

[0071] In FIG. 9, the comparison target region Y detected on the processing target image P １ and Y ２ 9 shows the area of ​​the basic image P in FIG. ｓ Basic area X set above １ ~X ４ (Basic image P ｓ In FIG. 9, the comparison area Y １ and the basic domain X １ and X ２ In addition, in FIG. 9, the comparison region Y ２ and the basic domain X ３ and X ４ In this case, the determination unit 13 determines that a change in the state of the imaging means has occurred.

[0072] In addition, if the basic area and the comparison area overlap, and the area overlapping the basic area is the comparison area that shows a component that separates the display shelf in the vertical direction, the judgment unit 13 can determine that the orientation of the imaging means has changed in the vertical direction.

[0073] Furthermore, if the basic area and the comparison area overlap, and the area overlapping the basic area is the comparison area that shows a member that separates the display shelf in the left-right direction, the judgment unit 13 can determine that the orientation of the imaging means has changed in the left-right direction.

[0074] As shown in FIG. 10, when there is no change in the state of the imaging means, the comparison region Y １ and Y ２ and the basic domain X １ ~X ４ do not overlap with each other.

[0075] "Determination based on the degree of overlap between the base region and the comparison region" Even if there is no change in the state of the imaging means due to an unforeseen event such as a detection error, there is a possibility that the base region and the comparison region may overlap slightly. The determination in this example can prevent the inconvenience of erroneously determining that there has been a change in the state of the imaging means in such cases.

[0076] For example, if a predetermined percentage or more of the comparison target region overlaps with the basic region, the determination unit 13 can determine that a change in the state of the imaging means has occurred.

[0077] In addition, if a predetermined percentage or more of the comparison area overlaps with the basic area, and the comparison area that overlaps with the basic area reflects a component that separates the display shelf in the vertical direction, the judgment unit 13 can determine that the orientation of the imaging means has changed in the vertical direction.

[0078] Furthermore, if a predetermined percentage or more of the comparison area overlaps with the basic area, and the comparison area that overlaps with the basic area is one that captures a component that separates the display shelf in the left-right direction, the judgment unit 13 can determine that the orientation of the imaging means has changed in the left-right direction.

[0079] As another example, the determination unit 13 can determine that a change in the state of the imaging means has occurred when a predetermined percentage or more of the basic region overlaps with the comparison region.

[0080] In addition, if a predetermined percentage or more of the basic area overlaps with the comparison area, and the area overlapping with the basic area is the comparison area that shows a component that separates the display shelf in the vertical direction, the judgment unit 13 can determine that the orientation of the imaging means has changed in the vertical direction.

[0081] Furthermore, if a predetermined percentage or more of the basic area overlaps with the comparison area, and the area overlapping with the basic area is the comparison area that shows a member that separates the display shelf in the left-right direction, the judgment unit 13 can judge that the orientation of the imaging means has changed in the left-right direction.

[0082] The predetermined ratio is a value that is determined in advance, and the specific value is a matter of design.

[0083] Next, an example of the processing flow of the monitoring device 10 will be described using the flowchart in Fig. 2. First, before executing the processing of the flowchart in Fig. 2, the monitoring device 10 registers, as a basic area, an area in a basic image captured by the imaging means in the basic state, in which products displayed on the display shelves are captured.

[0084] The monitoring device 10 then detects, as a comparison area, an "area showing a predetermined part of the display shelf" from the processing target image captured by the imaging device (S10). Next, the monitoring device 10 determines a change in the state of the imaging device based on the relative positional relationship between the pre-registered "area showing products displayed on the display shelf (basic area)" and the "area showing the predetermined part of the display shelf (comparison area)" detected in S10 (S11).

[0085] An example of the processing of the monitoring device 10 after the determination will be described in the following embodiment. For example, the monitoring device 10 may notify an operator of the determination result. The monitoring device 10 can notify the operator via any output device, such as a display, speaker, projector, or printer. The monitoring device 10 can also notify the operator via a device such as a point-of-sales (POS) register, a terminal device installed in the store's backroom, or a mobile device carried by a store clerk. The monitoring device 10 may also notify a pre-registered operator by sending an email, a push notification from a predetermined application, or the like.

[0086] The monitoring device 10 may notify the operator only when it is determined that the state of the imaging means has changed. The notification to the operator may include the installation location of the imaging means for which it has been determined that the state has changed. In this case, the installation locations of the multiple imaging means are registered in advance in the monitoring device 10. Based on this information, the monitoring device 10 can identify the installation location of the imaging means for which it has been determined that the state has changed. The notification to the operator may also include the change in orientation of the imaging means (change in the up / down direction / change in the left / right direction).

[0087] A modification of this embodiment will now be described, which can be applied to all of the following embodiments.

[0088] The monitoring device 10 may use the processing target image or a pre-registered basic area to perform processing other than determining changes in the state of the imaging device. For example, the monitoring device 10 may use the processing target image to monitor the display status of products displayed on a display shelf. Examples of display status monitoring include, but are not limited to, detecting foreign objects, detecting an insufficient number of displayed products, and detecting a disordered display. The monitoring device 10 may use the basic area registered by the registration unit 11 to detect an insufficient number of displayed products. In other words, if the area where various products are displayed (the display area assigned to each product) is registered as a basic area, the monitoring device 10 may calculate a product sufficiency rate based on the proportion of the product area that fills the basic area. The monitoring device 10 may also use the processing target image to monitor the behavior of store clerks and customers. Note that these processes using the processing target image can be realized using any technology.

[0089] In this variant, the monitoring device 10 can further determine changes in the state of the imaging means using the processing target image and basic area used to monitor the display status of products and the behavior of store staff and customers.

[0090] "Effects" According to the monitoring device 10 of this embodiment, the same effects as those of the monitoring device 10 of the first embodiment are achieved.

[0091] Furthermore, the monitoring device 10 of this embodiment can determine changes in the state of the imaging means based on whether or not there is overlap between the base area and the comparison area, and the degree of overlap. With this monitoring device 10, it is possible to detect changes in the state of cameras installed in a store using a new method.

[0092] Furthermore, the monitoring device 10 of this embodiment can determine the nature of the change in the state of the imaging means. Specifically, it can determine whether the orientation of the imaging means has changed vertically or horizontally. An operator who returns the state of the imaging means to its original state can understand what action to take based on such a determination result.

[0093] Furthermore, the monitoring device 10 of this embodiment can determine that the orientation of the imaging means has changed in the vertical direction when the comparison area, which shows a component that separates the display shelf vertically, overlaps with the basic area. Furthermore, the monitoring device 10 of this embodiment can determine that the orientation of the imaging means has changed in the horizontal direction when the comparison area, which shows a component that separates the display shelf horizontally, overlaps with the basic area. This monitoring device 10 can accurately determine the nature of the change in orientation of the imaging means.

[0094] Fourth Embodiment A monitoring device 10 according to a fourth embodiment is a specific implementation of the configuration of the monitoring device 10 according to the first embodiment.

[0095] The monitoring device 10 of the fourth embodiment registers, as a basic area, an "area showing a predetermined part of the display shelf" in a basic image captured by the imaging means in a basic state. The monitoring device 10 then determines a change in the state of the imaging means based on the relative positional relationship between this basic area and an "area showing products displayed on the display shelf (comparison area)" detected from the image to be processed.

[0096] The monitoring device 10 of the fourth embodiment differs from the monitoring device 10 of the third embodiment in the contents of the basic region and the comparison region. Specifically, the monitoring device 10 of the third embodiment registers "an area in which products displayed on a display shelf are captured" as the basic region and detects "a predetermined portion of the display shelf" as the comparison region. In contrast, the monitoring device 10 of the fourth embodiment registers "a predetermined portion of the display shelf" as the basic region and detects "an area in which products displayed on a display shelf are captured" as the comparison region.

[0097] The "basic region" in the fourth embodiment is a basic image P as shown in FIG. ｓ The basic area is an area in which a predetermined part of the display shelf is captured. The basic area is at least a part of the area in which the predetermined part of the display shelf is captured.

[0098] Other configurations of the monitoring device 10 of the fourth embodiment are similar to those of the monitoring device 10 of the third embodiment.

[0099] According to the monitoring device 10 of the fourth embodiment, the same effects as those of the monitoring device 10 of the third embodiment are achieved.

[0100] Fifth Embodiment Overview The monitoring device 10 of the fifth embodiment is a specific implementation of the configuration of the monitoring device 10 of the second embodiment.

[0101] The monitoring device 10 of the fifth embodiment registers the "area showing products displayed on the shelves" in the basic image captured by the imaging device in the basic state as the basic area. The monitoring device 10 then determines a change in the state of the imaging device based on the results of comparing this basic area with the "area showing products displayed on the shelves (comparison area)" detected from the image to be processed. The configuration of the monitoring device 10 will be described in detail below.

[0102] "Hardware Configuration" The monitoring device 10 of the fifth embodiment can employ a configuration similar to the hardware configuration described in the third embodiment.

[0103] "Functional Configuration" Next, a detailed description will be given of the functional configuration of the monitoring device 10. Fig. 1 shows an example of a functional block diagram of the monitoring device 10. As shown in the figure, the monitoring device 10 has a registration unit 11, a detection unit 12, and a determination unit 13.

[0104] The registration unit 11 registers, as a basic area, an area in which a product detected from a basic image captured by the imaging means in a basic state is captured. The registration unit 11 can register, as a basic area, at least one product group area in which a predetermined product group detected from the basic image is captured. For example, the registration unit 11 may register, as a basic area, all of the multiple product group areas detected from the basic image.

[0105] A "predetermined product group" is a collection of products of the same type.

[0106] The "product group area" is an area where various products are displayed (a display area allocated to each product). In the example shown in FIG. 7, four product group areas corresponding to four types of products are shown. The four product group areas are divided into four basic areas X １ ~X ４ It is set and registered as such.

[0107] The registration unit 11 can set and register such a basic region using the method described in the third embodiment.

[0108] The detection unit 12 detects an area in which a commodity appears as a comparison target area from the processing target image captured by the imaging means. The detection unit 12 can detect at least one commodity group area as a comparison target area from the processing target image captured by the imaging means. For example, the detection unit 12 can detect all of the commodity group areas appearing in the processing target image as comparison target areas.

[0109] The detection unit 12 can detect a comparison area from the image to be processed using a method similar to that used by the registration unit 11 described in the third embodiment to detect an "area showing products displayed on a display shelf."

[0110] The determination unit 13 determines a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area. The determination unit 13 can determine a change in the state of the imaging means based on at least one of the number of product group areas detected from the image, the area of ​​each product group area, and the position of each product group area.

[0111] "Determination based on the number of product group areas detected from an image" When the state of the imaging means changes, the product group areas included in the image may change. In response to this change, the number of product group areas detected from the image also changes.

[0112] If the number of product group areas detected from the basic image differs from the number of product group areas detected from the image to be processed, the determination unit 13 can determine that there has been a change in the state of the imaging means.

[0113] "Determination based on the area of ​​each product group region detected from the image" When the state of the imaging means changes, some product group regions may become cut off accordingly. In response to this change, the area of ​​the product group region in the image changes.

[0114] The determination unit 13 compares, for each type of product, the area of ​​the product group region corresponding to the product detected from the basic image with the area of ​​the product group region corresponding to the product detected from the image to be processed. If the area of ​​the product group region corresponding to at least one type of product in the basic image differs from the area in the image to be processed, the determination unit 13 can determine that a change in the state of the imaging means has occurred.

[0115] "Determination Based on the Position of Each Product Group Area Detected from an Image" When the state of the imaging means changes, the position within the image of the product group area included in the image changes.

[0116] The determination unit 13 compares, for each type of product, the position of the product group area corresponding to each type of product detected from the basic image with the position of the product group area corresponding to each type of product detected from the image to be processed. If the position in the basic image of the product group area corresponding to at least one type of product differs from the position in the image to be processed, the determination unit 13 can determine that a change in the state of the imaging means has occurred.

[0117] Next, an example of the processing flow of the monitoring device 10 will be described using the flowchart in Fig. 4. First, before executing the processing of the flowchart in Fig. 4, the monitoring device 10 registers, as a basic area, an area in a basic image captured by the imaging means in the basic state, in which products displayed on the display shelves are captured.

[0118] The monitoring device 10 detects an area (comparison area) in which the product appears from the processing target image captured by the imaging means (S30).

[0119] Then, the monitoring device 10 determines a change in the state of the imaging means based on the comparison results between the pre-registered ``area in which the product detected from the basic image (basic area)'' and the ``area in which the product detected from the image to be processed (comparison area)'' (S31).

[0120] An example of the processing of the monitoring device 10 after the determination will be described in the following embodiment, but for example, the monitoring device 10 may execute the processing described in the third embodiment.

[0121] "Effects" According to the monitoring device 10 of this embodiment, the same effects as those of the monitoring device 10 of the second embodiment are achieved.

[0122] In addition, the monitoring device 10 of this embodiment can determine changes in the state of the imaging means based on at least one of the number of product group areas detected from the image, the area of ​​each product group area, and the position of each product group area.

[0123] In this way, the monitoring device 10 detects an area in which a product appears from the image to be processed, and determines a change in the state of the imaging means based on the comparison result between the "area in which the product appears detected from the base image" and the "area in which the product appears detected from the image to be processed." Such monitoring device 10 can detect changes in the state of cameras installed in a store using a new method.

[0124] Sixth Embodiment When it is determined that a change has occurred in the state of the imaging means, the monitoring device 10 of the sixth embodiment can determine the degree of the change and execute appropriate processing according to the degree. This will be described in detail below.

[0125] If the determination unit 13 determines that the state of the imaging means has changed, the determination unit 13 further determines the degree of change in the state of the imaging means. Specifically, the determination unit 13 determines whether the degree of change is equal to or greater than a predetermined level. Then, the determination unit 13 executes a corresponding process according to the determined degree of change.

[0126] "Determination of the degree of change" The determination unit 13 can determine that the degree of change in the state of the imaging means is equal to or greater than a predetermined level if a predetermined criterion is met, and can determine that the degree of change in the state of the imaging means is less than the predetermined level if the predetermined criterion is not met.

[0127] The predetermined criteria include at least one of the following: - A threshold percentage or more of the comparison target area overlaps with the basic area. - A threshold percentage or more of the basic area overlaps with the comparison target area. - The number of product group areas detected from the basic image is different from the number of product group areas detected from the image to be processed. - In a product group area corresponding to at least one type of product, the difference or rate of change between the area in the basic image and the area in the image to be processed is greater than or equal to a threshold. - In a product group area corresponding to at least one type of product, the distance between the position in the basic image and the position in the image to be processed is greater than or equal to a threshold. The above thresholds are predetermined values. The specific values ​​are a design matter.

[0128] In this way, the determination unit 13 can determine the degree of change in the state of the imaging means based on the degree of overlap between the basic area and the comparison area. The determination unit 13 can also determine the degree of change in the state of the imaging means based on at least one of the position of each product group area within the processing target image, the number of detected product group areas, and the area of ​​each product group area.

[0129] "Response Processing According to the Determined Degree of Change" When the degree of change in the state of the imaging means is less than a predetermined level, the determination unit 13 can re-register the basic region.

[0130] As explained as a modified example in the third embodiment, the monitoring device 10 can execute various processes, such as monitoring the display status, using a basic area registered in advance. Therefore, if the basic area becomes inappropriate due to a change in the state of the imaging means, as shown in Figure 9, various processes, such as monitoring the display status, cannot be executed correctly. Therefore, if the basic area becomes inappropriate due to a change in the state of the imaging means, it is necessary to take measures to return the basic area to an appropriate state.

[0131] Possible measures to set the basic area to an appropriate state include (1) correcting the state of the imaging means and (2) re-registering the basic area. If the degree of change in the state of the imaging means is less than a predetermined level, the determination unit 13 automatically re-registers the basic area, thereby reducing the burden on the operator. If the degree of change in the state of the imaging means is less than a predetermined level, there is no major change, such as a change in the product itself shown in the image, so the determination unit 13 can handle this by automatically re-registering the basic area.

[0132] The re-registration of the basic area can be achieved by any means. For example, the appearance feature quantities of each of a plurality of types of products may be registered in advance in the monitoring device 10. Then, the determination unit 13 may identify areas in which various products appear from within the image to be processed based on the appearance feature quantities. Then, the determination unit 13 can register a group of areas in which the same type of product appears, i.e., an area where various products are displayed (a display area allocated to each type of product), as a basic area.

[0133] On the other hand, if the degree of change in the state of the imaging means is equal to or greater than a predetermined level, i.e., if the situation makes it difficult to automatically re-register the basic area, the determination unit 13 notifies the operator of the change in the state of the imaging means. The operator checks the state of the imaging means in response to the notification and corrects the state of the imaging means or re-registers the basic area.

[0134] The determination unit 13 can notify the operator via any output device, such as a display, speaker, projector, printer, etc. The determination unit 13 can also notify the operator via a device, such as a POS register, a terminal device installed in the backroom of the store, or a mobile terminal carried by a store clerk. The determination unit 13 can also notify a pre-registered operator by sending an email, a push notification of a predetermined application, or the like.

[0135] The notification to the operator may also include the installation location of the imaging means for which it has been determined that there has been a change in state of at least a predetermined level. In this case, the installation locations of the multiple imaging means are registered in advance in the monitoring device 10. Based on this information, the determination unit 13 can identify the installation location of the imaging means for which it has been determined that there has been a change in state of at least a predetermined level. The notification to the operator may also include the content of the change in orientation of the imaging means (change in the up / down direction / change in the left / right direction).

[0136] Other configurations of the monitoring device 10 of the sixth embodiment are similar to those of the monitoring device 10 of the first to fifth embodiments.

[0137] The monitoring device 10 of the sixth embodiment achieves the same effects as the monitoring devices 10 of the first to fifth embodiments. Furthermore, when the monitoring device 10 of the sixth embodiment determines that a change has occurred in the state of the imaging means, it can determine the degree of the change and execute appropriate processing according to the degree.

[0138] Specifically, if the degree of change is below a predetermined level, the monitoring device 10 automatically re-registers the basic area, thereby reducing the burden on the operator. If the degree of change is equal to or greater than the predetermined level, the monitoring device 10 can notify the operator. Such a monitoring device 10 can reduce the burden on the operator to an appropriate extent.

[0139] <Modifications> Modifications applicable to all the embodiments will be described below.

[0140] When the configuration of the sixth embodiment is adopted, even if the degree of change is below a predetermined level and the basic region is automatically registered, the determination unit 13 may notify the operator of this fact. In this notification, the determination unit 13 may notify the operator that the state of the imaging means has changed and that the basic region will be automatically re-registered. Furthermore, the determination unit 13 may notify the operator of the content of the re-registered basic region. For example, the determination unit 13 may present the operator with an image in which information indicating the newly registered basic region is superimposed on the processing target image captured by the imaging means after the state change. The content of this image may be, for example, as shown in FIG. 7 . The operator can determine the validity of the newly registered basic region based on this image.

[0141] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0142] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order. However, the order of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-described embodiments can be combined to the extent that the content is not contradictory.

[0143] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes: 1. A monitoring device comprising: registration means for registering an area in a basic image captured by imaging means in a basic state where a predetermined part of a display shelf is captured, or an area where products displayed on the display shelf are captured, as a basic area; detection means for detecting an area in an image to be processed captured by the imaging means where the products displayed on the display shelf are captured as a comparison area if the area where the products displayed on the display shelf are captured is registered as the basic area, and for detecting an area in an image to be processed where the predetermined part of the display shelf is captured as a comparison area if the area where the products displayed on the display shelf are captured is registered as the basic area; and determination means for determining a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. The monitoring device according to 1, wherein the determination means determines a change in the state of the imaging means based on at least one of whether the basic area and the comparison area overlap and the degree of overlap between the basic area and the comparison area. 3. The monitoring device according to 2, wherein the determination means determines that a change in the state of the imaging means has occurred if the basic area and the comparison area overlap. 4. The monitoring device according to 2, wherein the determination means determines that a change in the state of the imaging means has occurred if a predetermined percentage or more of the comparison area overlaps with the basic area. 5. The monitoring device according to 2, wherein the determination means determines that a change in the state of the imaging means has occurred if a predetermined percentage or more of the basic area overlaps with the comparison area. 6. The monitoring device according to any of 2 to 5, wherein the determination means determines the nature of the change in the state of the imaging means. 7. The monitoring device according to 6, wherein the predetermined portion of the display shelf is a member that separates the display shelf in the vertical direction, and the determination means determines that the orientation of the imaging means has changed in the vertical direction if the basic area and the comparison area overlap. 8. 8. The monitoring device according to 7, wherein the member that separates the upper and lower sections includes at least one of a shelf board and a top board.9. The monitoring device described in 6, wherein the predetermined portion of the display shelf is a member that partitions the display shelf in the left-right direction, and the determination means determines that the orientation of the imaging means has changed in the left-right direction if the basic area and the comparison area overlap. 10. The monitoring device described in 9, wherein the member that partitions the display shelf in the left-right direction includes at least one of a partition plate and a side wall. 11. The monitoring device described in any of 1 to 10, wherein the determination means determines the degree of change in the state of the imaging means and executes response processing according to the determined degree of change. 12. The monitoring device described in 11, wherein the determination means notifies an operator of the change in the state of the imaging means if the degree of change in the state of the imaging means is equal to or greater than a predetermined level, and re-registers the basic area if the degree of change in the state of the imaging means is less than the predetermined level. 13. The monitoring device described in 11 or 12, wherein the determination means determines the degree of change in the state of the imaging means based on the degree of overlap between the basic area and the comparison area. 14. The monitoring device described in 11 or 12, wherein the detection means detects at least one product group area from the processing target image captured by the imaging means, the product group area showing a predetermined product group displayed on the display shelf, and the determination means determines the degree of change in the state of the imaging means based on at least one of the position of each of the product group areas within the processing target image, the number of detected product group areas, and the area of ​​each of the product group areas. 15. A monitoring method in which one or more computers register an area in a basic image captured by an imaging means in a basic state, in which a specified part of a display shelf or an area in which products displayed on the display shelf are displayed, as a basic area; if the area in which the specified part of the display shelf is displayed is registered as the basic area, detect an area in the image to be processed captured by the imaging means in which the products displayed on the display shelf are displayed, as a comparison area; if the area in which the products displayed on the display shelf are displayed is registered as the basic area, detect an area in the image to be processed in which the specified part of the display shelf is displayed, as a comparison area; and determine changes in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area.16. A recording medium that stores a program that causes a computer to function as: registration means that registers an area in a basic image captured by imaging means in a basic state where a predetermined part of a display shelf is shown, or an area where products displayed on the display shelf are shown, as a basic area; detection means that, if the area where the predetermined part of the display shelf is shown is registered as the basic area, detects an area in the image to be processed captured by the imaging means where the products displayed on the display shelf are shown as a comparison area, and if the area where the products displayed on the display shelf are shown is registered as the basic area, detects an area in the image to be processed where the predetermined part of the display shelf is shown as a comparison area; and determination means that determine a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. 18. A monitoring device comprising: a registration means for registering an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area; a detection means for detecting an area in which the product is detected from an image to be processed captured by the imaging means as a comparison target area; and a determination means for determining a change in the state of the imaging means based on a comparison result between the basic area and the comparison target area. 19. The monitoring device described in 17, wherein the registration means registers at least one product group area in which a predetermined product group detected from the basic image is captured as the basic area, the detection means detects a product group area in which the predetermined product group is captured from the image to be processed captured by the imaging means as the comparison target area, and the determination means determines a change in the state of the imaging means based on at least one of the number of product group areas detected from the image, the area of ​​each of the product group areas, and the position of each of the product group areas. A monitoring method in which one or more computers register an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area, detect an area in which the product is detected from an image to be processed captured by the imaging means as a comparison area, and determine a change in the state of the imaging means based on the comparison result between the basic area and the comparison area.20. A recording medium storing a program that causes a computer to function as: a registration means that registers an area in which a product is detected from a basic image captured by an imaging means in a basic state as a basic area; a detection means that detects an area in which the product is detected from an image to be processed captured by the imaging means as a comparison area; and a determination means that determines a change in the state of the imaging means based on a comparison result between the basic area and the comparison area. Some or all of Supplements 2 to 14 that are subordinate to the monitoring device of Supplement 1 described above may also be subordinate to the monitoring method of Supplement 15 and the recording medium of Supplement 16 in a subordinate relationship similar to that between Supplement 1 and Supplements 2 to 14. Furthermore, Supplement 18 that is subordinate to the monitoring device of Supplement 17 described above may also be subordinate to the monitoring method of Supplement 19 and the recording medium of Supplement 20 in a subordinate relationship similar to that between Supplement 17 and Supplement 18. Furthermore, within the scope of each of the above-mentioned embodiments, some or all of the configurations described as appendices can be realized in various hardware, software, various recording means for recording software, or systems.

[0144] 10 Monitoring device 11 Registration unit 12 Detection unit 13 Determination unit 1A Processor 2A Memory 3A Input / output I / F 4A Peripheral circuit 5A Bus

Claims

1. A registration means that registers as a basic area an area in which a predetermined part of a display shelf is captured in a basic image captured by an imaging means in the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means for determining a change in the state of the imaging means based on the relative positional relationship between the basic region and the comparison target region, A monitoring device having the following features.

2. The determination means is, The monitoring device according to claim 1, which determines a change in the state of the imaging means based on whether or not there is an overlap between the basic area and the comparison target area, and at least one of the degree of overlap between the basic area and the comparison target area.

3. The determination means is, The monitoring device according to claim 2, which determines that there has been a change in the state of the imaging means when the basic area and the comparison area overlap.

4. The determination means is, The monitoring device according to claim 2, which determines that there has been a change in the state of the imaging means when a predetermined percentage or more of the area of ​​the comparison target area overlaps with the basic area.

5. The determination means is, A monitoring device according to any one of claims 1 to 4, which determines the degree of change in the state of the imaging means and performs corresponding processing according to the determined degree of change.

6. One or more computers, In the basic image captured by the imaging means in the basic state, the area in which a predetermined part of the display shelf is captured, or the area in which the products displayed on the display shelf are captured, is registered as the basic area. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the area in which the products displayed on the display shelf are captured is detected as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the area in which a predetermined portion of the display shelf is captured is detected as the comparison area from the image to be processed, A monitoring method for determining a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area.

7. Computers, A registration means for registering as a basic area an area in which a predetermined part of a display shelf is captured in a basic image captured by an imaging means in the basic state, or an area in which products displayed on the display shelf are captured. If the area in which a predetermined portion of the display shelf is captured is registered as the basic area, the detection means detects the area in which the products displayed on the display shelf are captured as the comparison area from the image to be processed captured by the imaging means, and if the area in which the products displayed on the display shelf are captured is registered as the basic area, the detection means detects the area in which a predetermined portion of the display shelf is captured as the comparison area from the image to be processed, A determination means that determines a change in the state of the imaging means based on the relative positional relationship between the basic area and the comparison area. A program that makes it function as such.

8. A registration means that registers the area in which the product is detected from the basic image captured by the basic state imaging means as the basic area, A detection means for detecting, from the image to be processed captured by the imaging means, the region in which the product is visible as the comparison region, A determination means for determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area, A monitoring device having the following features.

9. One or more computers, The imaging means in the basic state registers the area in which the detected product is visible from the basic image captured as the basic area. From the image to be processed captured by the imaging means, the region in which the product is visible is detected as the comparison region. A monitoring method for determining a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area.

10. Computers, A registration means that registers the area in which the product is detected from the basic image captured by the basic state imaging means as the basic area. A detection means that detects the area in which the product is visible from the image to be processed captured by the imaging means as a comparison area. A determination means that determines a change in the state of the imaging means based on the comparison result between the basic area and the comparison target area. A program that makes it function as such.