Monitoring device, monitoring system, and monitoring method

Abstract

To reliably detect a deviation in an angle of view and easily adjusting a deviation amount of the angle of view in a surveillance camera, to continuously perform surveillance in a constant setting state.SOLUTION: A surveillance device includes an image analysis server 102 that generates an alarm when detecting an alarm generation target that has entered an intrusion detection area, based on a camera image 301 captured by a camera 101. The image analysis server 102 acquires the camera image 301 (reference image) to set a fixed object area (reference area) corresponding to a fixed object included in the reference image, acquires a camera image 301 (live image) to extract a fixed object area (referred area) corresponding to a fixed object included in the live image, calculates a deviation amount of an angle of view based on a result of a comparison in a reference position between the reference area and the referred area, and generates a surveillance screen including an adjustment screen for guidance on an operation for cancelling the deviation amount of the angle of view to display the surveillance screen.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a monitoring device and a monitoring system capable of reliably detecting an angle deviation of a monitoring camera and easily adjusting the amount of the angle deviation.

Background Art

[0002] In recent years, in a device for monitoring whether there is an obstacle in a railroad crossing based on an imaging image of a monitoring camera that photographs the railroad crossing, without using a dedicated gyro sensor or the like that detects a change in the position of the monitoring camera, a target area for monitoring whether the monitoring camera is changing its position is specified, and the amount of change in the position of the monitoring camera is calculated from the amount of deviation of a straight line portion included in a reference image in normal times and a comparison image at the time of photographing of this target area. There is known a railroad crossing monitoring device configured as such (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The conventional technology designates a target area for a monitoring area, calculates a change in the position and posture of the monitoring camera from the relative movement amount of a point where a straight line portion included in the target area intersects the outer periphery of the target area, outputs position data, and issues a caution signal for position adjustment.

[0005] Therefore, if there is no suitable straight line within the monitoring area, or if a straight line exists but is located far away, there is a problem in that it is not possible to set a target area for calculating changes in position and orientation from the initial position of the surveillance camera. In particular, when detecting objects entering the detection area using an image recognition model (machine learning model) built using machine learning such as deep learning, it is desirable to be able to adjust the amount of angle of view shift even if the angle of view of the surveillance camera shifts, so that the already built machine learning model can be reused. However, conventional technology has not taken any consideration into account a mechanism for easily adjusting the angle of view shift.

[0006] Therefore, the present invention has been made in view of these problems, and aims to provide a monitoring device, monitoring system, and monitoring method that can reliably detect angle of view shifts of a monitoring camera regardless of the presence or absence of straight sections in the monitoring area, and can easily adjust the amount of angle of view shift so that monitoring can always be continued with the same settings. [Means for solving the problem]

[0007] The present invention provides a monitoring device equipped with a processor that issues an alarm when an object that has entered an intrusion detection area is detected based on an image captured by an imaging device, wherein the processor acquires a reference image of the image, sets a reference area corresponding to a fixed object included in the reference image, acquires a live image of the image, extracts a reference area corresponding to a fixed object included in the live image, calculates the amount of field of view shift based on the comparison result between the reference area and the reference area's reference position, and generates and displays a monitoring screen including a field of view adjustment screen that guides the user through the operation to correct the amount of field of view shift.

[0008] Furthermore, the monitoring system of the present invention detects intrusions in the intrusion detection area based on the captured images. A monitoring system that issues an alarm when an incoming alarm target is detected, comprising: a plurality of imaging devices; a server device connected to the imaging devices via a network; and a monitoring terminal that receives alarms issued by the server device via the network, wherein the server device acquires a reference image of the captured images, sets a reference area corresponding to a fixed object included in the reference image, acquires a live image of the captured images, extracts a reference area corresponding to a fixed object included in the live image, calculates the amount of field of view shift based on the comparison result between the reference area and the reference area's reference position, generates a monitoring screen including a field of view adjustment screen that guides the user through the operation to correct the field of view shift, and transmits it to the monitoring terminal.

[0009] Furthermore, the monitoring method of the present invention is a monitoring method in which, based on an image captured by an imaging device, an alarm is issued by a processor when an object that has entered an intrusion detection area is detected, and the processor is configured to acquire a reference image of the image captured, set a reference area corresponding to a fixed object included in the reference image, acquire a live image of the image captured, extract a reference area corresponding to a fixed object included in the live image, calculate the amount of field of view shift based on the comparison result between the reference area and the reference area's reference position, and generate and display a monitoring screen including a field of view adjustment screen that guides the user through the operation to correct the amount of field of view shift. [Effects of the Invention]

[0010] According to the present invention, regardless of the presence or absence of straight lines in the monitoring area, the angle of view shift of the surveillance camera can be reliably detected, and the amount of the angle of view shift can be easily adjusted, allowing monitoring to continue with the same settings at all times. [Brief explanation of the drawing]

[0011] [Figure 1] Overall configuration diagram of the monitoring system according to the first embodiment [Figure 2] Diagram illustrating the camera's shooting conditions. [Figure 3] Diagram illustrating the intrusion detection area set on the camera image. [Figure 4] Explanatory diagram showing an overview of the angle deviation detection process [Figure 5] Flow chart showing the procedure of the reference image registration process, which is a pre - process in the angle deviation detection process [Figure 6] Explanatory diagram showing the basic setting screen [Figure 7] Explanatory diagram showing the intrusion detection area setting screen [Figure 8] Explanatory diagram showing the fixed object area setting screen [Figure 9] Flow chart showing the procedure of the angle deviation notification process after registering the reference image [Figure 10] Overview diagram when the reference point is the center point of the intrusion detection area and the reference points are the center points of each fixed object area [Figure 11] Overview diagram when comparing the positional relationship between the reference point of the intrusion detection area and each reference point of the fixed object area, which are commonly set in the reference image and the live image [Figure 12] Explanatory diagram showing the adjustment 1 screen [Figure 13] Explanatory diagram showing the adjustment 2 screen [Figure 14] Explanatory diagram showing the camera list screen [Figure 15] Explanatory diagram showing the viewer screen [Figure 16] Explanatory diagram showing an overview of the luminance abnormality determination process and the angle deviation detection process [Figure 17] Flow chart showing the procedure of the luminance abnormality determination process

Mode for Carrying Out the Invention

[0012] The first invention made to solve the above problems is a monitoring device including a processor that issues an alarm when a reporting target that has intruded into an intrusion detection area is detected based on an imaging image captured by an imaging device. The processor acquires a reference image of the imaging image, sets a reference area corresponding to a fixed object included in the reference image, and the live Obtain a reference image, extract a reference area corresponding to a stationary object included in the live image, calculate the amount of angular deviation based on the comparison result between the reference area and the reference position of the reference area, and generate and display a monitoring screen including an angular adjustment screen for guiding an operation to eliminate the amount of angular deviation.

[0013] According to this, regardless of the presence or absence of a straight line portion in the monitoring area, the angular deviation of the monitoring camera can be reliably detected, the amount of angular deviation can be easily adjusted, and monitoring can be continuously performed in the same set state at all times.

[0014] Further, in the second invention, the reference area is configured to be set based on the contour of the stationary object.

[0015] According to this, since a candidate frame based on the contour of the stationary object is displayed, the user can set a reference area that surely includes the entire stationary object.

[0016] Further, in the third invention, in the comparison between the reference area and the reference position of the reference area, the absolute position of the intrusion detection area is interposed.

[0017] According to this, the processor compares the distance between the absolute position of the intrusion detection area where no position fluctuation occurs and the reference position of the reference area with the distance between the above-described absolute position and the reference position of the reference area, so that the angular deviation of the monitoring camera can be surely detected.

[0018] Further, in the fourth invention, the processor presents a plurality of candidate reference areas that are candidates for the reference area, and sets the reference area based on an operation of a user who selects any one of the candidate reference areas.

[0019] According to this, since the processor displays a plurality of candidate frames based on the contour of the automatically detected stationary object, the user can quickly confirm the stationary object in the monitoring area and can easily set the reference area.

[0020] Furthermore, the fifth invention provides that the field of view adjustment screen includes parameters relating to the orientation of the imaging device, and is configured to eliminate the field of view shift based on the user's operation to adjust the parameters.

[0021] According to this, parameters related to the orientation of the surveillance camera (pan, tilt, etc.) are displayed on the field of view adjustment screen, allowing the user to easily and quickly adjust the amount of field of view deviation of the surveillance camera by manipulating the displayed parameters.

[0022] Furthermore, the sixth invention provides that the field of view adjustment screen is configured to include a display unit for displaying the live image and an overlay button for overlaying at least one of the intrusion detection area and the reference area onto the live image.

[0023] According to this system, at least one of the intrusion detection area and the reference area is superimposed on the live image, allowing users to intuitively see how much of a field of view shift is occurring.

[0024] Furthermore, the seventh invention provides a configuration in which the field of view adjustment screen is further equipped with a redisplay button to check whether or not a discrepancy has occurred between the reference area and the reference area after the parameter adjustment operation.

[0025] According to this, by checking the degree of overlap between the reference area displayed on the field of view adjustment screen and the reference area after parameter adjustment, users can intuitively and easily confirm the degree of correction of the field of view misalignment.

[0026] Furthermore, the eighth invention is configured such that the parameters include the installation date and time of the imaging device, the name of the imaging device, the IP address, pan, tilt, or zoom as parameter items.

[0027] According to this system, the installation date and time of the surveillance camera, the name of the surveillance camera, its IP address, and information regarding pan, tilt, or zoom are displayed on the field of view adjustment screen, allowing users to easily check the installation status of the surveillance camera without requiring complex operations.

[0028] Furthermore, the ninth invention provides a configuration in which the field of view adjustment screen displays the parameter item that causes the field of view shift in a different way from the other parameter items, with a different emphasis.

[0029] According to this, because the display method for the parameter item of the surveillance camera that is causing the angle of view shift is different from that of the other parameter items, the user can easily and quickly understand which parameter of the surveillance camera is malfunctioning.

[0030] Furthermore, the tenth invention provides a configuration in which the monitoring screen is equipped with a screen switching tab that allows switching between various settings screens and the viewing angle adjustment screen.

[0031] According to this, the display of the monitoring screen can be switched to any screen, allowing users to check, correct, or change the reference area at any time they like.

[0032] Furthermore, the 11th invention is configured such that the processor determines whether or not an abnormality has occurred in the imaging device by calculating the difference between the brightness distribution of the reference image and the brightness distribution of the live image, and if an abnormality is determined, it notifies the user that a malfunction has occurred.

[0033] According to this system, the processor notifies the user that the surveillance camera is malfunctioning, allowing the user to quickly proceed with replacing the camera and restore the surveillance camera to a state where intrusion detection and field of view misalignment detection can be properly performed.

[0034] Furthermore, the twelfth invention is configured such that the processor uses the average value of the luminance signal as the luminance distribution, calculates the amount of change in the average value of the luminance signal of the reference image, and notifies the user that a malfunction has occurred in the imaging device if the amount of change exceeds a predetermined threshold.

[0035] According to this, malfunctions in surveillance cameras can be detected efficiently, thus reducing the burden on users, such as the need for visual inspection.

[0036] Furthermore, the 13th invention is a monitoring system that issues an alarm when an object that has entered an intrusion detection area is detected based on an captured image, comprising a plurality of imaging devices, a server device connected to the imaging devices via a network, and a monitoring terminal that receives alarms issued from the server device via the network, wherein the server device acquires a reference image of the captured image, sets a reference area corresponding to a fixed object included in the reference image, acquires a live image of the captured image, extracts a reference area corresponding to a fixed object included in the live image, calculates the amount of field of view shift based on the comparison result between the reference area and the reference area's reference position, generates a monitoring screen including a field of view adjustment screen that guides the user through the operation to correct the field of view shift, and transmits it to the monitoring terminal.

[0037] According to this, monitoring can be continued with the same settings at all times, similar to the first invention.

[0038] Furthermore, the fourteenth invention is configured such that the monitoring terminal is a mobile terminal equipped with a display screen that displays information sent from the server device.

[0039] According to this, users can quickly identify any distortion in the field of view, not just within a specific area, but anywhere.

[0040] Furthermore, the 15th invention is configured such that when the server device detects a shift in the field of view of at least one of the imaging devices, it transmits a list screen to the monitoring terminal that allows the user to check the live images of each of the multiple imaging devices installed within a predetermined area.

[0041] According to this system, live images from multiple surveillance cameras installed within a designated area are displayed in a list, allowing users to easily check the shooting status of multiple surveillance cameras and intuitively identify surveillance cameras experiencing field-of-view shifts.

[0042] Furthermore, the 16th invention is configured such that the server device transmits the viewer screen of the imaging device corresponding to the live image based on the user's operation of selecting any image from among the multiple live images displayed on the list screen.

[0043] According to this, by selecting the live image of any surveillance camera from among the live images of multiple surveillance cameras, users can easily and quickly check the settings and other information of that camera.

[0044] Furthermore, the 17th invention is configured such that the server device transmits the viewer screen which includes the live image of the imaging device selected by the user, the reference image of the imaging device, the parameters of the imaging device, and a comment indicating that the imaging device is experiencing a field of view shift.

[0045] According to this system, the viewer screen displays both the live image from any surveillance camera and a reference image, allowing the user to intuitively understand any field-of-view misalignment. Furthermore, if a field-of-view misalignment is detected by the server device, a comment informing the user of the misalignment will also be displayed on the viewer screen, preventing the user from leaving the misalignment uncorrected.

[0046] Furthermore, the 18th invention is a monitoring method in which a processor executes an alarm activation process when an object that has entered an intrusion detection area is detected based on an image captured by an imaging device, wherein the processor acquires a reference image of the image, sets a reference area corresponding to a fixed object included in the reference image, acquires a live image of the image, extracts a reference area corresponding to a fixed object included in the live image, calculates the amount of field of view shift based on the comparison result between the reference area and the reference area's reference position, and generates and displays a monitoring screen including a field of view adjustment screen that guides the user through the operation to correct the amount of field of view shift.

[0047] According to this, monitoring can be continued with the same settings at all times, similar to the first invention.

[0048] Embodiments of the present invention will be described below with reference to the drawings. Note that all embodiments disclosed below are illustrative and do not imply any limitations on the monitoring device and monitoring system described herein. I have no intention of doing so.

[0049] (First Embodiment) Figure 1 is an overall configuration diagram of the monitoring system according to the first embodiment.

[0050] A surveillance system typically uses an image recognition model (machine learning model), built using machine learning such as deep learning, based on images captured by an imaging device, to detect individuals appearing within an intrusion detection area and issue an alarm when an intruder is detected. Detection of an intruder is not limited to when the entire body of a person appears within the intrusion detection area; detection may also occur when only a part of the body is present. Furthermore, detection may be based on the person's posture (e.g., falling) or physical condition (e.g., body temperature or heart rate).

[0051] The monitoring system according to the first embodiment, in addition to the normal functions described above, detects deviations in the field of view of the imaging device based on the captured images taken by the imaging device and notifies the monitoring operator (user) of this fact. This monitoring system comprises multiple cameras 101 (imaging devices), an image analysis server 102 (field of view deviation detection device), and a monitoring terminal 103 (notification device). The cameras 101, the image analysis server 102, and the monitoring terminal 103 are interconnected via a network.

[0052] Camera 101 is installed in each designated monitoring area, both indoors and outdoors. Camera 101 monitors the intrusion detection area, which is the area where the intrusion of an alarm target such as a person or animal is detected, and also photographs fixed objects installed near the intrusion detection area. Camera 101 transmits the camera images of the monitoring area to the image analysis server 102 via the network.

[0053] The image analysis server 102 is installed in an equipment room or data center of a store or other facility where the camera 101 is installed. The image analysis server 102 acquires camera images of the monitored area from the camera 101 and compares them with reference images of the monitored area that are pre-registered on the server to detect any deviation in the camera's field of view (field of view deviation detection process). When the image analysis server 102 detects a field of view deviation, it also performs a process to notify the user of this fact. Specifically, it instructs the monitoring terminal 103 to perform a predetermined notification action. The functions of the image analysis server 102 may also be implemented by a cloud computer.

[0054] The monitoring terminal 103 is installed in an office or other location where the camera 101 is installed. The monitoring terminal 103 can be implemented, for example, by installing an application for detecting angle of view shift on a PC. In response to instructions from the image analysis server 102, the monitoring terminal 103 displays a notification screen to inform the store employee (user) that an angle of view shift has occurred. The monitoring terminal 103 may also be a mobile device such as a smartphone or tablet. In the case of a mobile device, the notification to inform the user that an angle of view shift has occurred may include displaying a notification screen, outputting an alarm sound, or outputting vibration.

[0055] In this embodiment, the system detects a shift in the field of view of a camera 101 installed indoors, such as in a store, and notifies the staff. However, the installation location of the camera 101 is not limited to indoors. For example, it may also detect a shift in the field of view of a camera 101 installed outdoors, such as in an amusement park, and notify the staff (user).

[0056] Next, the field of view shift detection process performed by the image analysis server 102 according to the first embodiment will be described. Figure 2 is an explanatory diagram showing the camera shooting situation. Figure 3 is an explanatory diagram showing the intrusion detection area set on the camera image. Figure 4 is an explanatory diagram showing an overview of the field of view shift detection process. This is a diagram.

[0057] As shown in Figure 2, camera 101 captures images of the indoor intrusion detection area 201 and fixed objects installed near the intrusion detection area 201 as its monitoring area. Alternatively, multiple cameras 101 may be installed to capture the entire indoor area without omission, thereby eliminating blind spots in the field of view shift detection process and preventing missed detections.

[0058] The field of view of camera 101 may shift due to various reasons. For example, if camera 101, which has a pan-tilt function, is operated incorrectly, the field of view of camera 101 may shift. Also, if an operator comes into contact with camera 101 during cleaning or inspection, the field of view of camera 101 may shift.

[0059] In this embodiment, the angle of view of the camera 101 represents the range of subjects actually photographed by the camera 101, and relates to the orientation (posture) of the camera 101 as a shooting condition. Furthermore, in the case of a camera 101 with a zoom function, the zoom magnification as a shooting condition, that is, the width of the shooting range centered on the optical axis of the lens, is also included.

[0060] Incidentally, the intrusion detection process is performed using an image recognition model (machine learning model). When the field of view of camera 101 shifts, the shooting range of camera 101 changes from the field of view 202 when the camera was installed to the field of view 203 when the field of view shift occurred. As a result, the subject set on the camera image shifts, and the accuracy of detecting an object that enters the intrusion detection area 201 decreases. To improve accuracy, engineers need to readjust the parameters of the image recognition model or collect training data over time and retrain the model. However, such methods are very time-consuming and the burden of adjustment is significant.

[0061] In this embodiment, the image analysis server 102 first acquires a camera image 301 in which the intrusion detection area 201 and the fixed object 302 installed near the intrusion detection area 201 are included within the imaging area, as shown in Figure 3. Next, the image analysis server 102 performs a process to detect the field of view shift of the camera 101 installed indoors (field of view shift detection process) based on the camera image 301, and notifies the user that a field of view shift has occurred via the monitoring terminal 103. Based on the information notified by the image analysis server 102, the user corrects the field of view shift of the camera 101 via the monitoring terminal 103. Note that the correction of the field of view shift may be performed by the user themselves, or it may be performed automatically by the system of this embodiment.

[0062] Specifically, the field of view shift detection process is performed within the image analysis server 102, as shown in Figure 4. In order to perform the field of view shift detection process, the image analysis server 102 performs a process (reference image registration process) to set a reference image to be compared with the live image of the monitored area sent from the camera 101 at regular time intervals.

[0063] The camera image acquisition unit 401, located in the image analysis server 102, acquires a camera image 301 from the camera 101. The camera image acquisition unit 401 transmits the acquired camera image 301 to the reference image setting unit 402. After acquiring the camera image 301, the reference image setting unit 402 reflects the user's operation via the monitoring terminal 103 and sets the reference image based on the camera image 301. After setting the reference image, the reference image setting unit 402 transmits the reference image to the reference image storage unit 403. The reference image storage unit 403 stores the acquired reference image as a comparison target with live images of the monitoring area sent from the camera 101 at regular time intervals. The reference image storage unit 403 may be located outside the image analysis server 102; for example, a memory (not shown) provided in the camera 101 can be used as the reference image storage unit 403.

[0064] After the reference image registration process, the image analysis server 102 acquires camera images 301 from the camera 101 at regular time intervals. The camera image acquisition unit 401 transmits the acquired camera images 301 to the fixed object area detection unit 404 provided in the image analysis server 102. After acquiring the camera images 301, the fixed object area detection unit 404 detects the fixed object areas contained in the camera images 301. Here, the fixed objects contained in the camera images 301 can be recognized using known deep learning-based object recognition technology. After detecting the fixed object areas corresponding to the recognized fixed objects, the image analysis server 102, in the field of view change determination unit 405, compares the center coordinates or coordinates of each vertex of the fixed object areas detected from the camera images 301 with the center coordinates or coordinates of each vertex of the fixed object areas set in the reference image registration process, and determines whether or not a field of view shift has occurred. Based on the determination results of the field of view change determination unit, the image analysis server 102 notifies the user via the monitoring terminal 103 of the occurrence of a field of view shift and the degree of the field of view shift (movement coordinates).

[0065] The user can check the degree of angle of view shift of camera 101 via the monitoring terminal 103. After checking the degree of angle of view shift, the user can correct the angle of view shift of camera 101 in the reference image setting unit 402 via the monitoring terminal 103, and by registering the corrected camera image 301, the corrected camera image 301 can be updated as the reference image in the reference image storage unit 403. Here, since the updated reference image is reproduced to be equivalent to the reference image before the angle of view shift occurred, the image recognition model (machine learning model) constructed by machine learning such as deep learning can continue to be used as is to detect targets that appear within the intrusion detection area.

[0066] As a result, even if the field of view of camera 101 is misaligned, the user can quickly correct the misalignment without having to readjust or retrain the parameters of the image recognition model, and consequently, intrusion detection can be performed with high accuracy using the image recognition model.

[0067] Next, the field of view shift detection process performed by the image analysis server 102 according to the first embodiment will be described.

[0068] In this embodiment, first, in order to perform the field of view shift detection process, it is necessary to set a reference image to be compared with the camera image 301 of the monitored area, which is sent from the camera 101 at regular time intervals. Figure 5 is a flowchart showing the procedure for the reference image registration process, which is a preprocessing step in the field of view shift detection process.

[0069] The reference image is registered by the user via the monitoring terminal 103. When registering a reference image, the monitoring terminal 103 displays the basic settings screen 601, as shown in Figure 6, as the user operation screen.

[0070] The basic settings screen 601 includes a live image display unit 602. The live image display unit 602 displays the camera image 301 of the monitoring area captured by the camera 101. The image analysis server 102 acquires the camera image 301 from the camera 101 and displays it to the user via the monitoring terminal 103. (ST101)

[0071] Next, the user checks the camera image 301 displayed on the live image display unit 602 and determines whether it is an image that can be registered as a reference image. (ST102)

[0072] In Figure 6, the basic settings screen 601 is provided with a camera information acquisition button 604. By pressing the camera information acquisition button 604, the user can acquire camera parameters 603. The camera parameters 603 include the date and time when the camera 101 was installed, and This includes information such as the name of the Mera101, its IP address, and pan / tilt / zoom settings.

[0073] If the user determines that camera image 301 is not suitable for registration as a reference image, they can adjust the pan, tilt, etc. of camera 101 by manipulating camera parameters 603. (ST103)

[0074] By adjusting the pan and tilt of camera 101 in ST103, the image analysis server 102 acquires the camera image 301 after the parameter adjustment, so the user repeats the operation in ST102. Note that the operations from ST101 to ST103 can be repeated until the user determines that the image acquired from camera 101 is an image suitable for registration as a reference image.

[0075] If the camera image 301 is deemed worthy of being registered as a reference image, the user can save the camera image 301 as a reference image, linked to the camera parameters 603, by pressing the save settings button 605 located on the basic settings screen 601.

[0076] Furthermore, the basic settings screen 601 is provided with an overlay button 606. By pressing the overlay button 606, the user can overlay a frame image corresponding to at least one of the intrusion detection area 201 and the fixed object area 608 (reference area) onto the camera image 301 displayed on the live image display unit 602. The settings for the intrusion detection area 201 and the fixed object area 608 will be described later.

[0077] The user can switch to the desired screen on the monitoring terminal 103 by pressing the screen switching tab 607 located on the basic settings screen 601. For example, if the user presses the intrusion detection area tab on the screen switching tab 607, the screen will switch to the intrusion detection area settings screen 701, as shown in Figure 7. Alternatively, if the user presses the save settings button 605 while the basic settings screen 601 is displayed, the monitoring terminal 103 screen may automatically switch to the intrusion detection area settings screen 701.

[0078] The intrusion detection area setting screen 701 is equipped with a live image display unit 602. The live image display unit 602 displays the reference image registered on the basic settings screen 601.

[0079] On the intrusion detection area setting screen 701, the user can set an intrusion detection area 201 on the reference image displayed on the live image display unit 602 to detect targets such as people or animals that trigger an alarm. (ST104)

[0080] The user, Intrusion detection The intrusion detection area setting unit 702 located on the area setting screen 701 can be used to set the intrusion detection area 201. Specifically, the user can use the intrusion detection area setting unit 702. 702 After pressing the add button 703 located in the intrusion detection area section, a rectangular intrusion detection area 201 can be set by pressing the reference image displayed on the live image display unit 602 to set any four points. Note that the method for setting the intrusion detection area 201 is not limited to the user setting any four points on the reference image; various methods can be employed. Furthermore, the shape of the intrusion detection area 201 is not limited to a rectangle; it can also be an ellipse, a star, or other shape. Multiple intrusion detection areas 201 may also be set.

[0081] When the user has finished setting up the intrusion detection area 201, they can complete the setting of the intrusion detection area 201 by pressing the setting button 704 located in the intrusion detection area field of the intrusion detection area setting unit 702. The user can also complete the setting of the configured intrusion detection area 2 If you want to delete 01, you can delete the previously configured intrusion detection area 201 by pressing the delete button 705 located in the intrusion detection area field of the intrusion detection area setting unit 702.

[0082] Furthermore, the user can set a detection mask area by operating the intrusion detection area setting unit 702. In this embodiment, the detection mask area is an area within the intrusion detection area 201 where detection processing is not performed. An example of a detection mask area is when a poster or the like containing the object to trigger an alarm is present within the intrusion detection area 201, in which case the area of ​​the poster or the like is excluded from the detection processing area.

[0083] After the user has finished setting the intrusion detection area 201, detection mask area, and person detection sensitivity by operating the intrusion detection area setting unit 702, they can save the settings for the intrusion detection area 201, etc., linked to the reference image displayed on the live image display unit 602 by pressing the setting save button 706 provided on the intrusion detection area setting screen 701. In addition, store personnel and others who frequently enter and exit the intrusion detection area 201 should be excluded from the alarm target, so it is advisable to add machine learning data for excluding them from the alarm target.

[0084] Furthermore, the intrusion detection area setting screen 701 is provided with a fixed object area overlay button 707. By pressing the fixed object area overlay button 707, the user can overlay a frame image corresponding to the fixed object area 608 onto the camera image 301 displayed on the live image display unit 602. The settings for the fixed object area 608 will be described later.

[0085] The user can switch to the desired screen on the monitoring terminal 103 by pressing the screen switching tab 607 located on the intrusion detection area setting screen 701. For example, pressing the fixed object area tab switches to the fixed object area setting screen 801, as shown in Figure 8. Alternatively, if the user presses the save settings button 706 while the intrusion detection area setting screen 701 is displayed, the monitoring terminal 103 screen may automatically switch to the fixed object area setting screen 801.

[0086] The fixed object area setting screen 801 is equipped with a live image display unit 602. The live image display unit 602 displays the reference image registered on the basic settings screen 601.

[0087] On the fixed object area setting screen 801, the user can set fixed object areas 608 on the reference image displayed on the live image display unit 602, indicating areas such as clocks, doors, and floor tiles installed within the monitoring area. (ST105)

[0088] The fixed object area 608 can be set by operating the fixed object area setting unit 802 located on the fixed object area setting screen 801. Specifically, the fixed object area 608 can be set by pressing the add button 803 located in the fixed object area field of the fixed object area setting unit 802 and performing the same operation as when setting the intrusion detection area 201. The method for setting the fixed object area 608 is not limited to the user setting four arbitrary points; the image analysis server 102 may automatically detect fixed objects and display candidate frames based on their outlines, and the system may automatically select them in order of proximity to the intrusion detection area 201, or the user may select from the candidate frames. The outlines of fixed objects may be rectangles, polygons, or curved shapes. The candidate frames should be frames that include the outlines of fixed objects.

[0089] When a user determines that they have finished configuring the fixed object area 608, they can complete the configuration by pressing the setting button 804 located in the fixed object area section of the fixed object area setting unit 802. Furthermore, if a user wishes to delete a configured fixed object area 608, they can do so by pressing the setting button 804 located in the fixed object area section of the fixed object area setting unit 802. By pressing the delete button 805, you can delete the fixed object area 608 that you have already configured. Note that movable objects (such as flowerpots) that appear in the live image can be excluded from the fixed object area 608 at the user's discretion.

[0090] Furthermore, users can select multiple fixed objects from the reference image, and it is possible to select three or more. While it is also possible to select only one fixed object, selecting multiple fixed objects and understanding the amount of angle of view shift from multiple viewpoints can reduce false detections.

[0091] Once the user has finished setting the fixed object area 608, etc., they can save the settings for the detection area, etc., linked to the reference image by pressing the save settings button 806 located on the fixed object area setting screen 801.

[0092] Furthermore, the fixed object area setting screen 801 is provided with an intrusion detection area overlay button 807. By pressing the intrusion detection area overlay button 807, the user can overlay a frame image corresponding to the intrusion detection area 201 onto the camera image 301 displayed on the live image display unit 602.

[0093] In this way, by performing the processes from ST102 to ST105, the user completes the registration of the reference image used for the field of view shift detection process. (ST106)

[0094] As shown in Figures 6, 7, and 8, the user can switch to the screen they wish to display on the monitoring terminal 103 by pressing the screen switching tab 607 located on the various setting screens, including the basic settings screen 601, the intrusion detection area setting screen 701, and the fixed object area setting screen 801. This allows the user to modify or change the reference image at any time they like.

[0095] In this embodiment, the ST105 process is performed after the ST104 process, but this is not limited to this, and the order of the steps may be changed.

[0096] Next, we will explain the field-of-view shift detection process, which is performed using the reference image registered in the reference image registration process. Figure 9 is a flowchart showing the procedure for field-of-view shift notification processing after reference image registration.

[0097] The angle of view shift detection process after registration of the reference image is performed within the image analysis server 102. The image analysis server 102 acquires camera images 301 of the monitoring area captured by the camera 101 at regular time intervals. (ST201)

[0098] The image analysis server 102 detects multiple (for example, three) fixed object areas 608' (reference areas) from the acquired camera image 301 and calculates the distance between the center point or each vertex of the intrusion detection area set in the reference image and the center point or vertex of the detected fixed object areas 608'. (ST202)

[0099] Although multiple fixed object areas 608' are detected, the system is not limited to this, and only one fixed object area 608' may be detected. Furthermore, by detecting people or luggage moving within the monitoring area, the system can determine whether each vertex of the fixed object area 608 is concealed, and the center point and vertices of concealed fixed object areas 608 may not be used in distance calculation.

[0100] After ST202, the image analysis server 102 calculates the distance between the center point or each vertex of the intrusion detection area 201 set in the reference image and the center point or vertex of the fixed object area 608 set in the reference image. (ST203)

[0101] Furthermore, the fixed object selected during the calculation of ST203 shall be the same as the fixed object detected from the camera image 301 during the calculation of ST202, through a known object recognition identification process.

[0102] Here, we will explain how to determine whether or not a field of view shift has occurred. In this embodiment, the distance between the center point (P4) of the intrusion detection area 201, which is set in common for both the reference image and the live image, and the center points of multiple (in this case, three) fixed object areas 608 is calculated, and by detecting the change in this distance, it is determined whether or not a field of view shift has occurred. Figure 10 is a schematic diagram when the reference point (absolute position) is the center point of the intrusion detection area, and the reference points (reference positions) are the center points of each fixed object area.

[0103] Figure 10(a) shows the camera image 301 (live image) acquired by the image analysis server 102 from the camera 101. The image analysis server 102 detects multiple (in this case, three) fixed object areas 608' (dashed lines) from the camera image 301. Note that the detection of fixed object areas 608' (dashed lines) may be performed by selecting them in order of proximity to the intrusion detection area 201.

[0104] After detecting multiple fixed object areas 608' (dashed lines), the image analysis server 102 sets a center point for each fixed object area 608' (dashed lines). For example, if each fixed object area 608' (dashed lines) is designated as P1, P2, and P3, the center points of each fixed object area P1, P2, and P3 are set as P1A, P2A, and P3A.

[0105] The image analysis server 102 calculates the distance between the center point of each fixed object area 608' (dashed line frame) and the center point of the intrusion detection area 201, which is set by the user during the reference image registration process. Specifically, with P4 as the center point of the intrusion detection area 201, the image analysis server 102 calculates the distance L1A between P1A and P4, the distance L2A between P2A and P4, and the distance L3A between P3A and P4.

[0106] Figure 10(b) shows the reference image 1001, which was set and registered by the user during the reference image registration process. In the reference image 1001, the user has set an intrusion detection area 201 and multiple fixed object areas 608 during the reference image registration process. In addition, the center point of each fixed object area 608 (solid line frame) is saved as a reference point. For example, if the center point of the intrusion detection area 201 is set to P4, and each fixed object area 608 (solid line frame) is set to P1', P2', and P3', then the center points of each fixed object area P1', P2', and P3' are saved as reference points such as P1A', P2A', and P3A'.

[0107] The image analysis server 102 selects from the fixed object areas 608 (solid line frames) set in the reference image registration process that are the same as the fixed object area 608' (dashed line frame) detected from the camera image 301, and calculates the distance between the center point of each fixed object area 608 (solid line frame) and the center point of the intrusion detection area 201. Specifically, the image analysis server 102 calculates the distance L1A' between P1A' and P4, the distance L2A between P2A' and P4, and the distance L3A' between P3A' and P4.

[0108] The image analysis server 102 detects changes in the distances between three points by comparing the distances between three points (L1A, L2A, and L3A) calculated based on the camera image 301 with the distances between three points (L1A', L2A', and L3A') calculated based on the reference image 1001, thereby detecting whether a field of view shift has occurred.

[0109] In this embodiment, three points are selected from the center point of each fixed object area 608, but three points may also be selected from the vertices of each fixed object area 608.

[0110] Figure 11 is a schematic diagram comparing the positional relationship between the reference points (absolute positions) (P5~P7) of the intrusion detection area, which are set in common for both the reference image 1001 and the camera image 301 (live image), and each reference point (reference position) of the fixed object area.

[0111] Figure 11(a) is a camera image 301 acquired by the image analysis server 102 from the camera 101. The image analysis server 102 detects at least one fixed object area 608' (dashed line frame) from the camera image 301. After detecting the fixed object areas 608' (dashed line frame), the image analysis server 102 sets the vertices of each fixed object area 608' (dashed line frame). For example, if each fixed object area 608' (dashed line frame) is P1, P2, and P3, the reference points of each fixed object area P1, P2, and P3 are set as P1B, P2B, and P3B.

[0112] In this embodiment, one point is selected from each of the fixed object areas P1, P2, and P3 to set the reference points, but it is also possible to select three points from a single fixed object area to set the reference points. Furthermore, the method of selecting each reference point may be to select them in order of proximity to the intrusion detection area 201.

[0113] The image analysis server 102 calculates the distance between the vertices of each fixed object area 608' (dashed line frame) and the vertices of the intrusion detection area set by the user during the reference image registration process. Specifically, with P5, P6, and P7 as the reference points of the intrusion detection area 201, the image analysis server 102 calculates the distance L1B between P1B and P5, the distance L2B between P2B and P6, and the distance L3B between P3B and P7. In this embodiment, the distance between the reference point of each fixed object area 608' (dashed line frame) and the reference point of the intrusion detection area 201 is calculated, but the distance between the reference point of each fixed object area 608' (dashed line frame) and the center point of the intrusion detection area 201 may also be calculated.

[0114] Figure 11(b) shows the reference image 1001, which was set and registered by the user during the reference image registration process. In the reference image 1001, the user has set an intrusion detection area 201 and multiple fixed object areas 608 (solid line frames). In addition, the vertices of each fixed object area 608 (solid line frames) are saved as reference points. For example, if the reference points of the intrusion detection area 201 are set to P5, P6, and P7, and the fixed object areas 608 (solid line frames) are set to P1', P2', and P3', then the reference points of each fixed object area P1', P2', and P3' are saved as P1B', P2B', and P3B', respectively.

[0115] The image analysis server 102 selects from the fixed object areas 608 (solid line frame) set in the reference image registration process that are the same as the fixed object area 608' (dashed line frame) detected from the camera image 301, and calculates the distance between the reference point of the fixed object area 608 (solid line frame) and the reference point of the intrusion detection area 201. Specifically, the image analysis server 102 calculates the distance L1B' between P1B' and P5, the distance L2B' between P2B' and P6, and the distance L3B' between P3B' and P7.

[0116] The image analysis server 102 detects changes in the distances between three points by comparing the distances between three points (L1B, L2B, and L3B) calculated based on the camera image 301 (live image) with the distances between three points (L1B', L2B', and L3B') calculated based on the reference image 1001, thereby detecting whether a field of view shift has occurred.

[0117] Returning to Figure 9, the image analysis server 102 compares the calculation results of ST202 and ST203 (ST204) to check whether the distance between the three selected points (calculation result) has changed, thereby detecting whether a field of view shift has occurred (ST205).

[0118] Furthermore, the method for selecting the three points (center point or reference point of the fixed object area 608') in ST202 and ST203 may be configured to switch depending on whether or not two or more fixed object areas 608' are detected.

[0119] The image analysis server 102 terminates the field of view shift detection process if the distance between the three selected points (calculated result) has not changed.

[0120] Next, the image analysis server 102 detects the occurrence of a field of view shift if the distance between the three selected points (calculated result) has changed, and calculates the amount of shift of the camera 101 based on the coordinates of the fixed object area 608'. After calculating the amount of shift, the image analysis server 102 notifies the user of the result via the monitoring terminal 103. (ST206) If the amount of change based on the comparison result of the three points is not constant, the average value of the amount of change can be obtained, or a representative value can be obtained from an approximate amount of change, and this can be reflected in the field of view shift detection process.

[0121] In this embodiment, the angle of view shift is detected based on the comparison result between the reference point (reference position) of the fixed object area 608 and the reference point (reference position) of the fixed object area 608', with the reference point (absolute position) of the intrusion detection area 201 in between. However, the system is not limited to this, and it is also possible to detect whether an angle of view shift has occurred by checking whether the area of ​​the overlapping portion of the aforementioned fixed object area 608 (reference area) and fixed object area 608' (reference area) exceeds a predetermined threshold.

[0122] Furthermore, in this embodiment, the field of view shift is detected via a reference point (absolute position) of the intrusion detection area 201. However, this is not limited to this, and the field of view shift may also be detected based on a comparison result between the reference point of the fixed area 608 set in the reference image 1001 and the reference point (reference position) of the fixed area 608' detected from the camera image 301, using the reference point (absolute position) of the fixed area 608 as the reference point (absolute position).

[0123] In the notification process, the image analysis server 102 notifies the user of the results of the field of view shift detection process via the monitoring terminal 103. Specifically, the monitoring terminal 103 is instructed to perform a notification operation to inform the user that a field of view shift has occurred in the camera 101.

[0124] Next, the notification screen displayed on the monitoring terminal 103 according to this embodiment will be described.

[0125] On the monitoring terminal 103, in response to instructions from the image analysis server 102, an adjustment screen 1201, as shown in Figure 12, is displayed as a notification operation to inform the user that a misalignment of the camera 101's field of view has occurred.

[0126] The adjustment screen 1201 is equipped with a live image display unit 602. The live image display unit 602 displays the camera image 301 at the moment when the angle of view shift of the camera 101 was detected.

[0127] Furthermore, the adjustment screen 1201 is equipped with camera parameters 1202. By pressing the camera information acquisition button 1203 on the adjustment screen 1201, the user can display the camera parameters 1202, which include the parameters of camera 101 when the field of view shift was detected and the parameters of camera 101 when the reference image was registered. The camera parameters 1202 include the date and time when camera 101 was installed, the name of camera 101, its IP address, and pan, tilt, and zoom information.

[0128] Camera parameter 1202 is the parameter of camera 101 that is causing the angle of view shift. Since the background color of the item is displayed in red, the user can quickly understand which parameter of camera 101 is malfunctioning. In this embodiment, the background color is changed, but this is not the only method, and other highlighting methods such as shading, patterns, or flashing may also be used.

[0129] Furthermore, camera parameter 1202 displays the pan-tilt setting value calculated from the amount of camera 101 displacement obtained by the image analysis server 102. Therefore, the user can quickly correct the field of view displacement of camera 101 by intuitively operating camera parameter 1202. A conversion table between the amount of camera 101 displacement and the pan-tilt setting value of camera 101 is pre-stored in the image analysis server 102. Based on this conversion table, the pan-tilt setting value of camera 101 is calculated and displayed in camera parameter 603.

[0130] Furthermore, the adjustment screen 1201 is provided with an overlay button 1204. By pressing the overlay button 1204, the user can overlay a frame image corresponding to at least one of the intrusion detection area 201 or the fixed object area 608 onto the camera image 301 displayed on the live image display unit 602.

[0131] Furthermore, the Adjustment 1 screen 1201 is equipped with a setting save button 1205. After the user checks or corrects the camera parameters 1202 of the camera 101 where the field of view shift has been detected, they can press the setting save button 1205 to proceed to the Adjustment 2 screen 1301, as shown in Figure 13. Note that the screen will not proceed to the Adjustment 2 screen 1301 unless the user presses the setting save button 1205 on the Adjustment 1 screen 1201.

[0132] The adjustment screen 2 1301 is provided with a live image display unit 602 and a reference image display unit 1302. The live image display unit 602 displays the camera image 301 when a field of view shift of the camera 101 is detected. The reference image display unit 1302 displays the reference image registered in the reference image registration process. On the camera image 301 displayed in the live image display unit 602, in addition to the intrusion detection area 201 and fixed object area 608 (solid line frame) set in the reference image registration process, the fixed object area 608' (dashed line frame) detected from the camera image 301 is superimposed and displayed.

[0133] Furthermore, the Adjustment 2 screen 1301 is provided with a redisplay button 1303. By pressing the redisplay button 1303, the user can check the degree of overlap (degree of angle of view shift correction) between the fixed object area 608 (solid line frame) and the fixed object area 608' (dashed line frame) where the corrected camera parameters 1202 are reflected. The user can check the degree of angle of view shift correction by checking the degree of overlap between the fixed object area 608 (solid line frame) and the fixed object area 608' (dashed line frame). If the user determines that the redisplay result is not appropriate, they can press the Adjustment 1 button 1304 provided on the Adjustment 2 screen 1301 to switch the display screen of the monitoring terminal 103 to Adjustment 1 screen 1201 and correct the angle of view shift again by operating the camera parameters 1202.

[0134] Furthermore, the adjustment screen 1301 is equipped with a setting save button 1305. The user can check the degree of correction of the field of view shift, and if they determine that the correction is appropriate, they can press the setting save button 1305 to store the corrected camera image 301 as a new reference image, linked to the camera parameters 1202, in the reference image storage unit 403, and update it.

[0135] As shown in Figures 12 and 13, the user adjusts screen 1201 and screen 230 By pressing the screen switching tab 607 located on the field of view adjustment screen (1), the user can switch to the screen they wish to display on the monitoring terminal 103 (for example, various settings screens).

[0136] Next, we will describe the notification screen displayed on the monitoring terminal 103 when multiple cameras 101 are configured according to this embodiment.

[0137] If the monitoring terminal 103 detects a camera 101 with a misaligned field of view among the multiple cameras 101 installed, it will display a camera list screen 1401, grouped by area, as shown in Figure 14, as a notification action to inform the user, in accordance with instructions from the image analysis server 102.

[0138] The camera list screen 1401 is provided with live image display units 602 corresponding to each camera 101 included in each group (predetermined area). Each live image display unit 602 displays the camera image 301 captured by each camera 101 when the field of view shift is detected.

[0139] The background of the group icon for camera 101 where a field of view shift has been detected is displayed in a different color (for example, red) than the other group icons, allowing the user to quickly identify which of the multiple installed cameras 101 is experiencing the malfunction. In this embodiment, the background color is changed, but this is not the only method; other highlighting methods such as shading, patterns, or flashing may also be used.

[0140] Furthermore, users can switch between and view the list screens of cameras 101 assigned to each group by clicking the camera list tab 1402 located on the camera list screen 1401. While examples of groups include those set for specific spaces such as hotel lobbies and dining areas, the system is not limited to these.

[0141] By pressing one of the live image display units 602 corresponding to each camera 101 on the camera list screen 1401, the user can transition to a viewer screen 1501 that displays the camera image 301 of the pressed camera 101 and a reference image, as shown in Figure 15.

[0142] The viewer screen 1501 is equipped with a live image display unit 602 and a reference image display unit 1302 for camera 101. The live image display unit 602 displays the camera image 301 at the time of field of view shift detection, corresponding to camera 101 selected by the user on the camera list screen 1401. The reference image display unit 1302 displays the reference image corresponding to camera 101 selected by the user on the camera list screen 1401.

[0143] The viewer screen 1501 also includes camera parameters 1502, a comment display section 1503, and an adjustment button 1504. The comment display section 912 displays a comment prompting the user to press the adjustment button 1504.

[0144] By pressing the Adjustment 1 button 1504, the user can switch the display screen of the monitoring terminal 103 to the Adjustment 1 screen 1201, as shown in Figure 12.

[0145] After switching the display screen of the monitoring terminal 103 to the adjustment screen 1201, the user can quickly correct the misalignment of the camera 101's field of view by performing the same process as described above.

[0146] Furthermore, if multiple cameras 101 are experiencing angle-of-view misalignment, the user can complete the correction of one of the cameras 101 experiencing the misalignment, and then proceed to the adjustment screen 1201. By clicking the camera list tab in the screen switching tab 607, the camera list screen 1401 can be displayed again. This allows the user to quickly move on to correcting the remaining cameras 101 that are experiencing angle-of-view misalignment.

[0147] In this embodiment, pressing the camera list tab on the screen switching tab 607 displays the camera list screen 1401 again, but this is not limited to this. For example, pressing the settings save button 1305 on the adjustment 2 screen 1301 may automatically switch the display screen of the monitoring terminal 103 to the camera list screen 1401.

[0148] (Second Embodiment) Next, a second embodiment will be described. Points not specifically mentioned here are the same as in the previous embodiment. Figure 16 is an explanatory diagram showing an overview of the brightness anomaly detection process and the field of view shift detection process.

[0149] In this embodiment, as shown in Figure 16, a brightness anomaly detection unit 1601 is provided within the image analysis server 102. The image analysis server 102 detects whether or not the camera 101 is malfunctioning based on the brightness distribution of the camera image 301 acquired from the camera 101 at regular time intervals and the brightness distribution of the reference image registered in the reference image registration process (brightness anomaly determination process).

[0150] If camera 101 is malfunctioning, the intrusion detection process cannot be properly performed even if the field of view misalignment is adjusted, so the user is notified that camera 101 is malfunctioning. On the other hand, if camera 101 is not malfunctioning, the intrusion detection process can be properly performed, so the intrusion detection process and field of view misalignment detection process continue.

[0151] Incidentally, when the user is notified that camera 101 is malfunctioning, the operator replaces camera 101. This restores camera 101 to a state where it can continue intrusion detection processing and field of view shift detection processing. At this time, the image analysis server 102 determines whether camera 101 is malfunctioning or not, and if it determines that camera 101 is in a state where it can continue intrusion detection processing and field of view shift detection processing, it notifies the staff that it has returned to a state where it can resume intrusion detection processing and field of view shift detection processing.

[0152] Next, the brightness anomaly detection process performed by the image analysis server 102 according to the second embodiment will be described. Figure 17 is a flowchart showing the procedure for the brightness anomaly detection process.

[0153] Brightness anomaly detection processing is performed within the image analysis server 102. The image analysis server 102 acquires camera images 301 of the monitoring area captured by the camera 101 at regular time intervals. (ST301)

[0154] The image analysis server 102 calculates the difference between the brightness distribution of the acquired camera image 301 and the brightness distribution of the reference image registered in the reference image registration process. (ST302)

[0155] The image analysis server 102 detects whether the camera 101 is malfunctioning by checking whether there is a change in the average value of the brightness signal of the reference image that exceeds a predetermined threshold. (ST303)

[0156] If the difference in brightness is within a predetermined threshold, the image analysis server 102 terminates the brightness anomaly detection process.

[0157] If there is a change in the average value of the luminance signal of the reference image that exceeds a predetermined threshold, the image analysis server 10 2 detects a malfunction in camera 101 and notifies the user of the result via monitoring terminal 103. (ST304)

[0158] Furthermore, the brightness anomaly detection process performed by the image analysis server 102 according to the second embodiment must be carried out in an environment where an appropriate amount of light can be obtained when the camera image 301 is captured. For this reason, the image analysis server 102 according to this embodiment may be configured to allow switching the execution of the aforementioned brightness anomaly detection process on and off during specific time periods when an appropriate amount of light cannot be obtained, using a timer or other function.

[0159] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments. [Industrial applicability]

[0160] The monitoring device and monitoring system according to the present invention are useful as a monitoring device and monitoring system that reliably detects when a camera's field of view shift occurs, allows for easy adjustment of the amount of the field of view shift, and detects and notifies when a person enters a restricted area based on camera images taken of the monitoring area. [Explanation of symbols]

[0161] 101...Camera 102...Image analysis server 103... Surveillance terminal 201…Intrusion detection area 301...Camera image (live image) 608... Fixed object area (reference area) 608' ... Fixed object area (reference area) 1001...Reference image 1201…Adjustment 1 screen 1301…Adjustment 2 screen

Claims

1. A monitoring device equipped with a processor that performs intrusion detection processing, which, based on an image captured by an imaging device, uses a machine learning model to detect an object that has entered an intrusion detection area and issues an alarm to a monitoring terminal, The aforementioned processor, In addition to the intrusion detection process mentioned above, it also performs a field of view shift detection process. As a reference image registration process, which is a preprocessing step for the angle of view shift detection process, after obtaining the reference image from the captured image, the intrusion detection area is set, and further, a reference area corresponding to the fixed object recognized in the reference image is set. Next, as part of the angle of view shift detection process, a live image of the captured image is obtained, and a reference area corresponding to the fixed object recognized in the live image is extracted. Based on the comparison result between the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area and the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area, the angle of view shift amount is calculated. Furthermore, a monitoring screen is generated that includes a field-of-view adjustment screen that guides the user through the operation to correct the aforementioned field-of-view shift. The monitoring screen is displayed on the display unit of the monitoring terminal. In the aforementioned field of view adjustment screen, an operation is performed to correct the field of view misalignment based on the display of a frame image representing the reference area and a frame image representing the reference area, and user operation, and the live image from which the field of view misalignment has been corrected is updated as the new reference image. A monitoring device characterized by the following features.

2. The aforementioned reference area is, Set based on the contour of the aforementioned fixed object, The monitoring device according to feature 1.

3. The aforementioned angle of view adjustment screen is, The system includes parameters related to the orientation of the imaging device. Based on the user's operation to adjust the aforementioned parameters, the angle of view shift is eliminated. The monitoring device according to feature 1.

4. A monitoring system that performs intrusion detection processing, which uses a machine learning model to detect an object that has entered an intrusion detection area based on an image captured by an imaging device, and issues an alarm when such object is detected. Multiple imaging devices, A server device connected to the aforementioned imaging device via a network, The system includes a monitoring terminal that receives alarms issued from the server device via a network, The server device is In addition to the intrusion detection process mentioned above, it also performs a field of view shift detection process. As a reference image registration process, which is a preprocessing step for the angle of view shift detection process, after obtaining the reference image from the captured image, the intrusion detection area is set, and further, a reference area corresponding to the fixed object recognized in the reference image is set. Next, as part of the angle of view shift detection process, a live image of the captured image is obtained, and a reference area corresponding to the fixed object recognized in the live image is extracted. Based on the comparison result between the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area and the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area, the angle of view shift amount is calculated. Furthermore, a monitoring screen is generated that includes a field-of-view adjustment screen that guides the user through the operation to correct the field-of-view misalignment, and this screen is transmitted to the monitoring terminal. In the field of view adjustment screen displayed on the monitoring terminal, an operation to correct the field of view misalignment is performed based on the display of a frame image representing the reference area and a frame image representing the reference area, and user operation, and the live image from which the field of view misalignment has been corrected is updated as the new reference image. A monitoring system characterized by the following features.

5. A monitoring method in which, based on an image captured by an imaging device, a machine learning model detects an object that has entered an intrusion detection area, and a processor executes an intrusion detection process, which is the process of issuing an alarm to a monitoring terminal, The aforementioned processor, In addition to the intrusion detection process described above, the angle of view shift detection process is performed. As a preprocessing step for the angle of view shift detection process, after obtaining the reference image from the captured image, the intrusion detection area is set, and then a reference area corresponding to the fixed object recognized in the reference image is set. Next, as part of the angle of view shift detection process, a live image of the captured image is obtained, and a reference area corresponding to the fixed object recognized in the live image is extracted. Based on the comparison result between the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area and the position of the representative coordinate representing the reference position of the reference area relative to the representative point of the intrusion detection area, the angle of view shift amount is calculated. Furthermore, a monitoring screen is generated that includes a field-of-view adjustment screen that guides the user through the operation to correct the field-of-view misalignment, and the monitoring screen is displayed on the display unit of the monitoring terminal. In the angle of view adjustment screen, an operation is performed to correct the angle of view misalignment based on the display of a frame image representing the reference area and a frame image representing the reference area, and user operation, and the live image from which the angle of view misalignment has been corrected is updated as the new reference image. A monitoring method characterized by the following features.

Images