Monitoring device, monitoring method, and computer program product for monitoring

Abstract

The invention provides a monitoring device, a monitoring method, and a computer program product for monitoring. The monitoring device includes: a detection unit that detects a person from each of a plurality of chronological images generated by an imaging unit; a tracking unit that tracks the person in one or more images in which the detected person is displayed among the plurality of images; and a determination unit that, on the basis of the tracking result, determines whether or not the position of the detected person on the image is included in a first range on the image corresponding to the prohibited region over the entire first period. And determines whether or not the position of the detected person on the image is included in a second range on the image provided around the first range over a second period that is longer than the first period. When the position of the detected person on the image is included within the first range throughout the first period or within the second range throughout the second period, it is determined that the detected person has entered the prohibited area.

Description

Technical Field

[0001] The present invention relates to a monitoring device, a monitoring method, and a monitoring computer program for monitoring a predetermined area using an image representing the predetermined area. Background Art

[0002] A technology for monitoring the status of a specified area using an image showing that area has been proposed (see Japanese Patent Application Laid-Open No. 2023-3974). The in-car monitoring system disclosed in Japanese Patent Application Laid-Open No. 2023-3974 identifies people and cargo in images of the interior of a public vehicle captured by a camera mounted on the ceiling of the vehicle, and estimates whether the identified people are seated or standing. Based on the positions of the identified cargo and people, the in-car monitoring system then estimates whether there are any cargo seats occupied by cargo or any duplicate seating where a seated person has occupied multiple seats.

[0003] Sometimes, a restricted entry area is provided inside a vehicle, where passengers are not allowed to enter. Therefore, it is necessary to detect the entry of passengers into such an area. Summary of the Invention

[0004] Therefore, an object of the present invention is to provide a monitoring device that can detect the entry of a person into a prohibited area.

[0005] As one embodiment of the present invention, a surveillance device is provided. The surveillance device includes: a detection unit for detecting a person from each of a plurality of images generated in time sequence by a capturing unit, wherein the capturing unit is configured to capture a predetermined area including a prohibited area; a tracking unit for tracking the person in one or more images showing the detected person among the plurality of images; and a determination unit for determining, based on the tracking result, whether the position of the detected person on the image is included within a first range on the image corresponding to the prohibited area throughout a first period, and whether the position of the detected person on the image is included within a second range on the image surrounding the first range throughout a second period, wherein the second range is longer than the first range, and determining that the detected person has entered the prohibited area when the position of the detected person on the image is included within the first range throughout the first period or within the second range throughout the second period.

[0006] In one embodiment, the detection unit detects at least two different parts of a person from each of a plurality of images in chronological order, and the tracking unit tracks each of the at least two detected parts. For any one of the at least two detected parts, when the position of the part on the image is included in the first range throughout the first period or the position of the part on the image is included in the first range throughout the second period, the tracking unit determines that the person has entered a prohibited area.

[0007] In this case, the first range and the second range are set for each of at least two locations.

[0008] According to another embodiment, a surveillance method is provided. The method includes detecting a person from each of a plurality of images generated in time sequence by a camera unit, wherein the camera unit is configured to capture a predetermined area including a prohibited area, tracking the person in one or more images in the plurality of images in which the detected person is displayed, and determining based on the tracking result whether the position of the detected person on the image is included within a first range on the image corresponding to the prohibited area throughout a first period, and determining whether the position of the detected person on the image is included within a second range on the image surrounding the first range throughout a second period, wherein the second period is longer than the first period, and determining that the detected person has entered the prohibited area when the position of the detected person on the image is included within the first range throughout the first period or within the second range throughout the second period.

[0009] According to another embodiment, a surveillance computer program product is provided. The surveillance computer program product includes instructions for causing a computer to perform the following actions: detecting a person from each of a plurality of images generated in time sequence by a camera unit, wherein the camera unit is configured to capture a predetermined area including a prohibited area; tracking the person in one or more images showing the detected person in the plurality of images; determining, based on the tracking results, whether the position of the detected person in the image is included within a first range on the image corresponding to the prohibited area throughout a first period, and determining whether the position of the detected person in the image is included within a second range on the image surrounding the first range throughout a second period, wherein the second period is longer than the first period; and determining that the detected person has entered the prohibited area when the position of the detected person in the image is included within the first range throughout the first period or within the second range throughout the second period.

[0010] The monitoring device disclosed herein can detect the entry of a person into a prohibited area. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a schematic diagram of a vehicle equipped with a monitoring device according to one embodiment.

[0012] Figure 2 This is a picture of the interior of the vehicle.

[0013] Figure 3 This is a schematic diagram of the monitoring device.

[0014] Figure 4 This is a functional block diagram of a processor associated with monitoring processing.

[0015] Figure 5A This is a diagram explaining the outline of monitoring processing.

[0016] Figure 5B This is another diagram illustrating the outline of monitoring processing.

[0017] Figure 5C This is another diagram explaining the outline of monitoring processing.

[0018] Figure 6 This is a flowchart of the monitoring process. DETAILED DESCRIPTION

[0019] The following describes a monitoring device, a monitoring method, and a monitoring computer program with reference to the accompanying drawings. The monitoring device detects a person within a specified area from each of a plurality of images sequentially displayed in time, each of which includes a specified area including a prohibited area, and tracks the detected person across each image. Furthermore, for each person being tracked, the monitoring device determines whether the person's position on the image is included within a first range on the image corresponding to the prohibited area, and whether the person's position on the image is included within a second range on the image surrounding the first range. The monitoring device then determines that the person has entered the prohibited area if the person's position on the image is included within the first range for a first period of time or if the person's position on the image is included within the second range for a second period of time.

[0020] The following describes an example of a surveillance device being used to monitor the interior of a vehicle that can accommodate multiple passengers. Passengers are one example of a person to be detected. However, the surveillance device is not limited to this example and can also be used to monitor restricted areas within moving objects such as rail vehicles that can accommodate passengers or riders, or restricted areas within buildings or facilities.

[0021] Figure 1 This is a schematic diagram of a vehicle equipped with a monitoring device according to one embodiment. Figure 2This is a view of the interior of a vehicle equipped with a surveillance system, viewed from above. Vehicle 1 equipped with a surveillance system is a bus, for example, that can accommodate multiple passengers and has space inside for passengers to stand and move. Vehicle 1 includes a camera 2, a notification device 3, and a surveillance system 4.

[0022] Furthermore, within the vehicle 1, a restricted area 1b is defined around a doorway 1a for passengers to board or alight from the vehicle 1. The restricted area 1b is an area where passengers are prohibited from entering when the doors provided at the doorway 1a are opened or closed in order to avoid danger to the passengers.

[0023] Camera 2 is an example of a camera unit and is mounted near the interior ceiling of vehicle 1 at entrance 1a, for example, facing vertically downward, so that its imaging range includes a predetermined area 1c within the vehicle surrounding entrance 1a. In this embodiment, camera 2 is mounted so that entrance 1a is displayed at the lower end of the image generated by camera 2. The predetermined area 1c is set to include the entire restricted area 1b and be larger than it, so that passengers entering the restricted area 1b can be detected from the image generated by camera 2. Camera 2 generates an image representing the predetermined area 1c surrounding entrance 1a at predetermined imaging intervals (e.g., 1 / 30 to 1 / 10 of a second). The image captured by camera 2 can be a color image or a grayscale image. It should be noted that if vehicle 1 has multiple entrances, a camera 2 capable of capturing a predetermined area surrounding that entrance can be installed for each entrance. Each time camera 2 generates an image, it outputs the generated image to monitoring device 4 via the in-vehicle network.

[0024] Notification device 3 is a device capable of providing a prescribed notification to passengers located near boarding / exiting gate 1a. It comprises, for example, a speaker, buzzer, or display device, and is installed near boarding / exiting gate 1a within vehicle 1. In response to a notification signal from monitoring device 4, notification device 3 outputs a prescribed notification, such as a voice message warning that a passenger has entered restricted area 1b, or displays a message corresponding to the notification.

[0025] The monitoring device 4 performs monitoring processing based on the images generated by the camera 2 .

[0026] Figure 3 FIG. 4 is a diagram showing the hardware configuration of the monitoring device 4. Figure 3 As shown, the monitoring device 4 includes a communication interface 11, a memory 12, and a processor 13. The communication interface 11, the memory 12, and the processor 13 may be configured as separate circuits, or may be integrally configured as a single integrated circuit.

[0027] The communication interface 11 includes an interface circuit for connecting the monitoring device 4 to the in-vehicle network. Furthermore, each time the communication interface 11 receives an image from the camera 2, it transmits the received image to the processor 13. Furthermore, upon receiving a notification signal from the processor 13 to be output by the notification device 3, the communication interface 11 outputs the notification signal to the notification device 3.

[0028] The memory 12 is an example of a storage unit, and includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Furthermore, the memory 12 stores various programs and data used in the monitoring process executed by the processor 13 of the monitoring device 4. For example, the memory 12 stores parameters for determining the identifier used for passenger detection, the positions and ranges of various regions on the image, and the like. Furthermore, the memory 12 temporarily stores images received from the camera 2 and various data generated during the monitoring process.

[0029] The processor 13 includes one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 13 may also include other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphics processing unit. The processor 13 also performs monitoring processing.

[0030] Figure 4 This is a functional block diagram of processor 13 related to monitoring processing. Processor 13 includes a detection unit 21, a tracking unit 22, a determination unit 23, and a notification processing unit 24. These components of processor 13 are, for example, functional modules implemented by computer programs running on processor 13. Alternatively, these components of processor 13 may be dedicated arithmetic circuits provided in processor 13.

[0031] The detection unit 21 detects passengers within a predetermined area from each of a plurality of time-sequential images generated by the camera 2. In this embodiment, the detection unit 21 detects passengers from the most recent image obtained by the camera 2 at predetermined intervals. The detection unit 21 only needs to perform the same processing on each image, so the following description will focus on the processing performed on a single image.

[0032] In this embodiment, the detection unit 21 independently detects at least two of the passenger's attributes, belongings, and body parts from the image. The passenger's attributes, belongings, and body parts independently set as detection targets include "person" (i.e., the passenger's entire body), "head," "infant," "wheelchair," "stroller," and "luggage." It should be noted that the attributes, belongings, and body parts that are detection targets are not limited to the above examples. For example, if the image only shows the passenger's hands or feet, "hands" or "feet" can also be set as detection targets. Furthermore, depending on the positional relationship between the camera 2 and the prohibited area, independent detection may not be performed for any of the above attributes, belongings, and body parts. Furthermore, even if independent detection targets are not set, the detection unit 21 can simply use the passenger as the detection target, as long as the positional relationship between the passenger's position and the prohibited area can be determined. For ease of explanation, the passenger's attributes, belongings, and body parts will be referred to as "body parts."

[0033] The detection unit 21 detects these parts by inputting the image received by the monitoring device 4 from the camera 2 to a recognizer that has been pre-learned to detect these parts. As such a recognizer, a recognizer based on a so-called DNN (Deep Neural Network) is used. For example, as a recognizer, a DNN with a CNN (Convolutional Neural Network) type architecture such as Single Shot MultiBox Detector or YOLO, or a DNN with an attention mechanism such as Vision Transformer is used. Alternatively, as a recognizer, a recognizer based on other machine learning methods such as AdaBoost or support vector machine can also be used. The recognizer uses a large number of training images including images showing the parts to be detected, and is pre-learned according to a prescribed learning method such as error backpropagation.

[0034] The recognizer outputs the area showing the part to be detected (hereinafter referred to as the object area) and the reliability of each object area on the input image. Then, when there are multiple overlapping object areas showing parts of the same category, the detection unit 21 prevents multiple detections of parts of a passenger by performing NMS (Non-Maximum Suppression) or Soft NMS. That is, the detection unit 21 calculates the IoU (Intersection over Union) for multiple overlapping object areas showing the same part, and when the IoU is greater than or equal to a specified threshold, the object areas other than the object area with the highest reliability are deleted. Alternatively, the larger the IoU, the lower the reliability, and deletes the object areas whose reliability after reduction is less than the specified detection threshold.

[0035] It should be noted that different parts of the same passenger can be detected independently in a single image. For example, an object region showing the entire passenger and an object region showing the passenger's head can be detected independently in a single image. Furthermore, an object region showing the entire passenger and an object region showing a suitcase held by the passenger can be detected independently in a single image.

[0036] The detection unit 21 notifies the tracking unit 22 and the determination unit 23 of each passenger part detected from the image and the position and range of the passenger part and the object region where the passenger part is displayed.

[0037] The tracking unit 22 tracks the detected passenger in one or more images showing the detected passenger from among the multiple images generated by the camera 2 in chronological order. In this embodiment, each part is detected independently, so the tracking unit 22 performs tracking processing for each detected part of the passenger. In other words, the tracking unit 22 associates object regions showing parts of different passengers with object regions showing parts of the same passenger across multiple images. It should be noted that when detecting the entire passenger, rather than by part, the tracking unit 22 can simply perform the following processing for each detected passenger.

[0038] To this end, the tracking unit 22 applies a prescribed tracking method such as KLT tracking or ByteTrack to each object region in the latest image. Thus, the tracking unit 22 detects the part of the passenger represented by the object region in a previously acquired image (hereinafter referred to as the past image) for each object region in the latest image, and associates it with the object region showing the part of the same passenger being tracked. Each time the tracking unit 22 is notified of the detection result for the latest image from the detection unit 21, it repeats the above process, thereby tracking the part of each passenger and labeling the part of each passenger being tracked with a unique identification number (hereinafter referred to as the passenger ID). For object regions detected in the latest image that do not correspond to any of the object regions showing the passenger being tracked in the past image, the tracking unit 22 sets the part of the passenger shown in the object region as the part of the passenger who has newly entered the prescribed area and begins new tracking. On the contrary, if the object area showing any of the parts of the passenger being tracked in the past image is not associated with any of the object areas in the latest image, the tracking unit 22 sets the tracking as that part of the passenger being tracked exiting the specified area and ends the tracking.

[0039] It should be noted that when multiple different parts of the same passenger are detected, each detected part may be tracked and assigned a separate passenger ID. For example, the entire passenger may be detected as a "person," and the passenger's "head" may be detected and tracked separately. This results in separate IDs for the entire passenger and the head.

[0040] Based on the tracking results obtained by the tracking unit 22, the determination unit 23 determines whether the detected passenger's position on the image is included within a first range on the image corresponding to the prohibited area 1b throughout a first period. Furthermore, the determination unit 23 determines whether the detected passenger's position on the image is included within a second range on the image surrounding the first range throughout a second period, where the second period is longer than the first period. It should be noted that even if any part of the passenger enters the prohibited area 1b, it may sometimes appear outside the first range on the image depending on the height of that part from the floor of the vehicle. Therefore, the second range is set near the outer boundary of the prohibited area 1b to include such parts within the prohibited area 1b. For example, the second range has a size of approximately 5% to 100% of the width of the first range in either the vertical or horizontal direction.

[0041] If the detected passenger's position on the image is within the first range for the entire first period or within the second range for the entire second period, the determination unit 23 determines that the passenger has entered the restricted area 1b. On the other hand, if the duration of the passenger's position on the image being within the first range does not exceed the first period, and the duration of the passenger's position on the image being within the second range does not exceed the second period for any passenger, the determination unit 23 determines that no passenger has entered the restricted area at that time. Alternatively, if the passenger's position on the image is outside the first and second ranges for the entire predetermined period, the determination unit 23 may determine that the passenger has exited the restricted area 1b.

[0042] In this embodiment, the determination unit 23 performs the above-mentioned processing for each part of the passenger with the same passenger ID. Therefore, even if the passenger's entire body is not included in the prohibited area 1b, the determination unit 23 can accurately determine whether the passenger has entered the prohibited area 1b.

[0043] Furthermore, in this embodiment, as described above, the boarding gate 1a is displayed at the bottom of the image, and the camera 2 is mounted so as to capture images vertically downward from the vehicle's ceiling. Therefore, the lower end of the object region is presumed to represent the position of the lower end of the passenger's body part represented by the object region. Specifically, if the passenger's body part represented by the object region is a person, the lower end of the object region is presumed to represent the position of the passenger's feet. Therefore, for each passenger's body part, if the lower end of the object region is included in the first range, the determination unit 23 determines that the passenger's position is included in the first range. On the other hand, if the lower end of the object region is outside the first range, the determination unit 23 determines that the passenger's position is not included in the first range. Similarly, for each passenger's body part, if the lower end of the object region is included in the second range, the determination unit 23 determines that the passenger's position is included in the second range. On the other hand, if the lower end of the object region is outside the second range, the determination unit 23 determines that the passenger's position is not included in the second range. It should be noted that in order to determine that the passenger's position is included in the first range, it is not necessary for the entire side of the lower end of the object area to be included in the first range. It is sufficient that a specified ratio (for example, 10% to 30%) of the side of the lower end of the object area is included in the first range. The same applies to the second range. It should be noted that, without being limited to this example, the determination unit 23 may also determine that the passenger's position is included in the first range or the second range when a specified ratio of the area of ​​the object area is included in the first range or the second range. It should be noted that the specified ratio may also be set independently for each part. In this way, whether the passenger's position is included in the first range or the second range can be determined more accurately.

[0044] The first period can be, for example, a period equivalent to the image generation cycle of camera 2 or a multiple thereof, and the second period can be, for example, a multiple of the first period. By setting the periods in this manner, when a passenger is within the first range, which is assumed to indicate a passenger has indeed entered restricted area 1b, the passenger's entry into restricted area 1b can be detected quickly and quickly. In contrast, even when a passenger is within the second range, which is assumed to indicate a passenger's potential entry into restricted area 1b, detection can be performed throughout the longer second period, accurately detecting the passenger's entry into restricted area 1b.

[0045] Figures 5A to 5C , respectively, are diagrams explaining the outline of the determination of the passenger entering the prohibited area. Figures 5A to 5C In the image 500 depicting a predetermined area 1c within the vehicle 1, a boarding gate 1a is shown near the bottom end. A range corresponding to the prohibited area 1b near the boarding gate 1a on the image 500 is set as a first range 501. Furthermore, a second range 502 is set above and adjacent to the first range 501. It should be noted that the second range may also be set adjacent to the side of the first range.

[0046] exist Figure 5A In the example shown, the lower end of the object region 510 where the passenger is displayed is included in the first range 501. Therefore, if this state continues throughout the first period, it is determined that the passenger shown in the object region 510 has entered the prohibited area 1b.

[0047] exist Figure 5B In the example shown, the lower end of the object region 520 showing the passenger is not included in the first range 501 but is included in the second range 502. Therefore, if this state continues throughout the second period, it is determined that the passenger represented by the object region 520 has entered the prohibited area 1b.

[0048] exist Figure 5C In the example shown, the lower end of the object region 530 showing the passenger is not included in either the first range 501 or the second range 502. Therefore, as long as this state continues, it is determined that the passenger represented by the object region 530 has not entered the prohibited area 1b.

[0049] When determining that a passenger has entered the prohibited area 1 b , the determination unit 23 notifies the notification processing unit 24 of the fact.

[0050] When the notification processing unit 24 receives notification from the determination unit 23 that a passenger has entered the prohibited area 1b, it outputs a notification signal to the notification device 3 via the communication interface 11, warning that the passenger has entered the prohibited area 1b. The notification device 3 thus alerts the passenger to the entry into the prohibited area 1b. Alternatively, the notification processing unit 24 may output an entry warning signal to the electronic control unit (ECU) of the vehicle 1 via the communication interface 11, indicating that the passenger has entered the prohibited area 1b. The ECU may stop opening / closing the door of the boarding gate 1a while receiving the entry warning signal. It should be noted that the notification processing unit 24 may also output the notification signal to the notification device 3 or the entry warning signal to the ECU, as long as the ECU of the vehicle 1 receives the notification signal from the monitoring device 4 notifying the vehicle of the opening / closing of the door of the boarding gate 1a until the ECU of the vehicle 1 receives the opening / closing signal indicating that the door is actually opened / closed. Thus, when the gate 1a is not being opened or closed, even if a passenger enters the prohibited area 1b, no unnecessary warning is given, thereby alleviating the annoyance of the passenger.

[0051] Figure 6 The processor 13 executes the monitoring process according to the following operational flowchart.

[0052] The detection unit 21 detects a passenger from an image generated by the camera 2 (step S101 ). The tracking unit 22 tracks the detected passenger (step S102 ).

[0053] The determination unit 23 determines, for each detected passenger, whether the position of the passenger on the image is continuously included in the first range throughout the first period (step S103). If the position of any passenger is continuously included in the first range throughout the first period (step S103: Yes), the determination unit 23 determines that the passenger has entered the prohibited area 1b (step S104). The notification processing unit 24 then warns the passenger of their entry into the prohibited area 1b via the notification device 3 (step S105).

[0054] Furthermore, if the position of any passenger is not included in the first range or if the position is included in the first range for a period not continuing for the first period (step S103: No), the determination unit 23 determines whether the position of the passenger on the image is included in the second range for the entire second period for each detected passenger (step S106). If the position of any passenger is included in the second range for the entire second period (step S106: Yes), the processor 13 executes the processes of steps S104 and S105.

[0055] After step S105 , or if the passenger's position on the image is not included in the second range or the state of being included in the second range does not continue for the second period in step S106 (step S106 —No), the processor 13 ends the monitoring process.

[0056] As described above, the monitoring device determines, for each person detected and tracked from a plurality of images in chronological order, whether the person's position in the image is included within a first range on the image corresponding to a prohibited area, and whether the person's position in the image is included within a second range on the image surrounding the first range. The monitoring device then determines that the person has entered the prohibited area if the person's position in the image remains within the first range for a first period of time or if the person's position in the image remains within the second range for a second period of time. This allows the monitoring device to accurately detect a person entering a prohibited area.

[0057] It should be noted that, depending on the camera's installation position and shooting direction, the positions of various parts of the person on the image when the person is within the prohibited area may sometimes be different. For example, in the case where the camera is installed to shoot the prohibited area from an oblique angle, even if the person is standing in the prohibited area, the positions of the person's feet and head on the image may sometimes be very different. Therefore, according to a modified example, the first range and the second range can be set independently according to the part of the person. In this case, the first range on the image of each part corresponding to the prohibited area is set to the range in which the part is displayed when the person is within the prohibited area, based on the assumed height of the part. The second range of each part can be set around the first range in a manner that includes the position on the image when the part may enter the prohibited area. The determination unit 23 can determine, for each part of the person, whether the object area including the part is included in the first range or the second range corresponding to the part. According to this modified example, the monitoring device can more accurately detect people entering the prohibited area.

[0058] The computer program for causing a computer to execute the processing executed by the processor 13 of the monitoring device 4 according to the above-described embodiment or modification can be distributed as a computer program product by being recorded on a recording medium such as an optical recording medium or a magnetic recording medium.

[0059] As described above, those skilled in the art can make various modifications according to the embodiments within the scope of the present invention.

Claims

1. A monitoring device comprising: The detecting unit detects a person from each of the plurality of images generated in time sequence by the imaging unit, wherein The imaging unit is configured to image a predetermined area including a prohibited area; a tracking unit configured to track the detected person in one or more images showing the detected person among the plurality of images; as well as The determination unit determines, based on the tracking result, whether the position of the detected person on the image is included in a first range on the image corresponding to the prohibited area during an entire first period, and determines whether the position of the detected person on the image is included in a second range on the image set around the first range during an entire second period, wherein the second period is longer than the first period, and when the position of the detected person on the image is included in the first range during the entire first period or is included in the second range during the entire second period, it is determined that the detected person has entered the prohibited area.

2. The monitoring device according to claim 1, wherein The detecting unit detects at least two mutually different parts of the person from each of the plurality of images. The tracking unit tracks each of the at least two detected parts. Regarding any one of the at least two detected parts, when the position of the part on the image is included in the first range throughout the first period or the position of the part on the image is included in the first range throughout the second period, the judgment unit determines that the person has entered the prohibited area.

3. The monitoring device according to claim 2, wherein: The first range and the second range are set for each of the at least two locations.

4. A monitoring method comprising: detecting a person from each of a plurality of images generated in time sequence by an imaging unit configured to capture an image of a predetermined area including a prohibited area; tracking the detected person in one or more images of the plurality of images in which the detected person is displayed, Based on the tracking result, determining whether the position of the detected person on the image is included in a first range on the image corresponding to the prohibited area throughout a first period, and determining whether the position of the detected person on the image is included in a second range on the image set around the first range throughout a second period, wherein the second period is longer than the first period, When the position of the detected person on the image is included in the first range throughout the first period or is included in the second range throughout the second period, it is determined that the detected person has entered the prohibited area.

5. A computer program product for monitoring, configured to cause a computer to perform the following actions: A person is detected from each of a plurality of images in time sequence generated by a photographing unit, wherein The imaging unit is configured to capture an image of a predetermined area including a prohibited area. tracking the detected person in one or more images of the plurality of images in which the detected person is displayed, Based on the tracking result, determining whether the position of the detected person on the image is included in a first range on the image corresponding to the prohibited area throughout a first period, and determining whether the position of the detected person on the image is included in a second range on the image set around the first range throughout a second period, wherein the second period is longer than the first period, When the position of the detected person on the image is included in the first range throughout the first period or is included in the second range throughout the second period, it is determined that the detected person has entered the prohibited area.

Images