Monitoring system, monitoring method, program, and recording medium

Abstract

This system provides a monitoring system that can more reliably identify the location of a monitored object within a target space, as captured in an image taken by a camera whose shooting range covers a portion of the target space. [Solution] In the monitoring system, the feature point identification unit 22 individually generates identified feature point-related information corresponding to each camera 1. The camera position identification unit identifies the absolute coordinate position of each camera 1 in the target space A based on each identified feature point-related information and pre-stored absolute feature point position information. The monitoring object detection unit 24 generates detected object-related information. The detected object position identification unit identifies the absolute coordinate position of the detected object in the target space A based on the absolute coordinate position of each camera 1 in the target space A and the detected object-related information.

Description

Technical Field

[0001] The present disclosure relates to a monitoring system, a monitoring method, a program, and a recording medium.

Background Art

[0002] Patent Document 1 discloses a monitoring system that performs centralized monitoring while arranging and displaying images of a plurality of monitoring cameras on one screen.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

[0007] The monitoring system relating to this disclosure is installed in a target space where three or more individually identifiable feature points are located, and includes a camera that generates an image by capturing a shooting range in the target space, a camera control unit that controls the camera's shooting, a feature point identification unit that identifies at least three feature points in the captured image as identification feature points based on the image generated by the camera, calculates the distance from the camera to each identification feature point based on feature point size information indicating the actual size of each feature point and the image size of each identification feature point in the captured image, and generates identification feature point related information that is specified by the image position of each identification feature point in the captured image, the camera's orientation, and the distance from the camera to each identification feature point, and the absolute position of each feature point in the target space The system includes: a camera position identification unit that stores in advance feature point absolute position information indicating the target location and identifies the absolute coordinate position of the camera in the target space based on the identified feature point related information and the feature point absolute position information; a monitoring object detection unit that detects the presence or absence of a target object based on the captured image generated by the camera, and if a target object is detected as a detected object, calculates the distance from the camera to the detected object based on the captured image in which the detected object is captured, and generates detected object related information that is identified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object; and a detected object position identification unit that identifies the absolute coordinate position of the detected object in the target space based on the camera's absolute coordinate position in the target space and the detected object related information. Furthermore, the monitoring method relating to this disclosure includes: a setting shooting step in which a camera installed in a target space where three or more individually identifiable feature points are located is made to capture a shooting range in the target space, thereby causing the camera to generate a captured image; a feature point identification step in which, based on the captured image generated by the camera in the setting shooting step, at least three feature points captured in the captured image are identified as identifiable feature points, and the distance from the camera to each identifiable feature point is calculated based on feature point size information indicating the actual size of each feature point and the image size of each identifiable feature point captured in the captured image, and identifiable feature point related information is generated, which is specified by the image position of each identifiable feature point in the captured image, the orientation of the camera, and the distance from the camera to each identifiable feature point; and a feature point absolute position information indicating the absolute coordinate position of each feature point in the target space is stored in advance, and after the feature point identification step, identifiable feature point related information is stored in advance. The computer performs the following steps: a camera position identification step that identifies the absolute coordinate position of the camera in the target space based on linked information and feature point absolute position information; a surveillance shooting step that, after the camera position identification step, causes the camera to capture an image of the shooting range in the target space; a surveillance object detection step that, based on the image generated by the camera in the surveillance shooting step, detects the presence or absence of a target object to be monitored, and if a target object to be monitored is detected as a detected object, generates detected object-related information based on the image of the detected object in the captured image, the camera's orientation, and the distance from the camera to the detected object; and a detected object position identification step that, after the surveillance object detection step, identifies the absolute coordinate position of the detected object in the target space based on the camera's absolute coordinate position in the target space and the detected object-related information. [Effects of the Invention]

[0008] According to this disclosure, it is possible to more reliably determine the location of a monitored object within a target space when the surveillance camera captures a portion of the target space within its shooting range. [Brief explanation of the drawing]

[0009] [Figure 1]This is a diagram showing the configuration of the monitoring system according to Embodiment 1. [Figure 2] Figure 1 is a plan view showing the target space in which the monitoring system is used. [Figure 3] Figure 2 is an explanatory diagram showing the captured image generated by the camera. [Figure 4] Figure 1 is an explanatory diagram showing the relationship between the camera's image sensor and its discriminant feature points. [Figure 5] Figure 4 is an explanatory diagram showing the relationship between the actual size of the distinguishing feature points and the size of the image captured by the camera's image sensor. [Figure 6] This is an explanatory diagram showing the process of correcting the image of the distinguishing feature points captured in the camera image shown in Figure 4. [Figure 7] This diagram illustrates the principle by which the setting information output unit in Figure 1 determines the absolute coordinate position of the camera. [Figure 8] Figure 7 is an explanatory diagram showing the situation when a vehicle and a person are present within the camera's field of view. [Figure 9] This is a plan view showing the situation in the target space of Figure 2, where a vehicle is within the shooting range of one camera and a person is within the shooting range of the other camera. [Figure 10] Figure 1 shows the monitoring method used by the monitoring system. [Figure 11] Figure 10 is a flowchart showing the process of the feature point identification step. [Figure 12] Figure 10 is a flowchart showing the process for the camera position identification step. [Figure 13] Figure 10 is a flowchart showing the process of the monitoring object detection step. [Figure 14] Figure 10 is a flowchart showing the process for the detected object location step. [Figure 15] Figure 10 is a flowchart showing the processing steps for the determination step. [Figure 16] This is a plan view showing the camera's shooting range in the target space where the monitoring system according to Embodiment 2 is used. [Figure 17]It is an explanatory diagram showing a captured image generated by the camera in FIG. 16 capturing the shooting range before turning. [Figure 18] It is an explanatory diagram showing a zoomed-in captured image of part XVIII in the captured image of FIG. 17 enlarged by the zoom function of the camera. [Figure 19] It is an explanatory diagram showing a captured image generated by the camera in FIG. 16 capturing the shooting range after turning. [Figure 20] It is an explanatory diagram showing a zoomed-in captured image of part XX in the captured image of FIG. 19 enlarged by the zoom function of the camera. [Figure 21] It is a flowchart showing the processing of the feature point identification step in the monitoring method by the monitoring system according to Embodiment 2. [Figure 22] It is an explanatory diagram showing other examples of each feature point used in the monitoring system according to Embodiments 1 to 3. [Figure 23] It is a plan view showing a state where a part of the shooting range of one camera and a part of the shooting range of the other camera of the monitoring system according to Embodiments 1 and 3 overlap in the target space. [Figure 24] It is a configuration diagram showing a first example of a processing circuit that realizes the functions of each processing unit and the management device according to Embodiments 1 to 3. [Figure 25] It is a configuration diagram showing a second example of a processing circuit that realizes the functions of each processing unit and the management device according to Embodiments 1 to 3.

Embodiments for Carrying Out the Invention

[0010] Embodiments for carrying out the subject of the present disclosure will be described with reference to the accompanying drawings. In each figure, the same or corresponding parts are denoted by the same reference numerals, and duplicate descriptions are appropriately simplified or omitted. Note that the subject of the present disclosure is not limited to the following embodiments, and within the scope not departing from the gist of the present disclosure, deformation of any component of the embodiment or omission of any component of the embodiment is possible.

[0011] Embodiment 1. Figure 1 is a configuration diagram showing a monitoring system according to Embodiment 1. Figure 2 is a plan view showing the target space in which the monitoring system of Figure 1 is used. Examples of target space A monitored by the monitoring system include highway parking areas, construction sites, and manufacturing sites. In this embodiment, as shown in Figure 2, target space A is a highway parking area.

[0012] The target space A contains three or more individually identifiable feature points 5. Each feature point 5 is displayed, for example, on a display board placed in the target space A. Each feature point 5 is located at a different position within the target space A.

[0013] Feature points 5 can include AR (Augmented Reality) markers, 2D codes, symbols, signs, road markings, streetlights, highway equipment, etc. The size and shape of each feature point 5 placed in the target space A are predetermined. The shape of each feature point 5 can be a rectangle, a sphere, etc. If the shape of each feature point 5 is rectangular, the size of each feature point 5 may be, for example, 500 mm in height and 500 mm in width. In this embodiment, AR markers are used as each feature point 5.

[0014] As shown in Figure 1, the monitoring system includes multiple cameras 1, multiple processing units 2, and a management device 3.

[0015] Multiple cameras 1 are positioned at different locations within the target space A. Each camera 1 individually captures multiple shooting ranges within the target space A. The shooting ranges of each camera 1 are different from each other. Therefore, each camera 1 has a portion of the target space A as its respective shooting range. Each camera 1 generates an image by capturing its corresponding shooting range.

[0016] In this embodiment, two cameras 1 are installed in the target space A. In this embodiment, the shooting range 11a of one camera 1a and the shooting range 11b of the other camera 1b are two separate ranges.

[0017] Each camera 1 is positioned with its orientation adjusted so that at least three feature points 5 are within its corresponding shooting range. In this embodiment, as shown in Figure 2, each camera 1a and 1b is positioned with its orientation adjusted so that three feature points 5 are within the shooting range 11a of one camera 1a, and another three feature points 5 are within the shooting range 11b of the other camera 1b. The cameras 1 and the at least three feature points 5 within the shooting range corresponding to each camera 1 are not located on the same plane or on the same straight line.

[0018] The orientation of each camera 1 is determined by its horizontal angle and elevation angle. Each camera 1 is configured with camera orientation information that identifies its orientation. Specifically, each camera 1 is configured with camera orientation information that indicates its horizontal angle and elevation angle. In this embodiment, the orientation of each camera 1 is fixed so that the shooting range of each camera 1 does not change.

[0019] Here, Figure 3 is an explanatory diagram showing the captured image generated by camera 1a in Figure 2. In the captured image generated by camera 1a, three feature points 5 located in the shooting range 11a of camera 1a are individually identifiable. In the captured image generated by camera 1b, although not shown, three feature points 5 located in the shooting range 11b of camera 1b are also individually identifiable.

[0020] As shown in Figure 1, the multiple processing units 2 correspond individually to the multiple cameras 1. In this embodiment, each processing unit 2 is individually integrated with the corresponding camera 1. Each processing unit 2 includes a camera control unit 21, a feature point identification unit 22, a setting information output unit 23, a monitored object detection unit 24, and a monitoring information output unit 25.

[0021] The camera control unit 21 controls the shooting of the corresponding camera 1. Therefore, in the surveillance system, multiple camera control units 21 individually control the shooting of each camera 1.

[0022] The feature point identification unit 22 acquires the captured image generated by the corresponding camera 1 from the camera 1. Based on the captured image generated by the camera 1, the feature point identification unit 22 identifies at least three feature points 5 in the captured image as identified feature points 5.

[0023] The feature point identification unit 22 identifies feature points 5 using a feature point learning model generated by a general-purpose machine learning computer. The feature point learning model takes image data generated by camera 1 as input and outputs feature point inference data that infers feature points 5 to be identified from the image. The feature point identification unit 22 identifies feature points 5 using AI (Artificial Intelligence) processing such as segmentation.

[0024] Furthermore, the feature point identification unit 22 has feature point size information pre-stored, which indicates the actual size of each feature point 5. Based on the feature point size information and the size of the image of each identified feature point 5 captured in the image, the feature point identification unit 22 calculates the distance from the corresponding camera 1 to each identified feature point 5.

[0025] Here, Figure 4 is an explanatory diagram showing the relationship between the image sensor of camera 1 and the discriminant feature point 5 in Figure 1. Camera 1 has an image sensor 10. The image sensor 10 is positioned at the focal point where light converges within camera 1 through the lens of camera 1. As a result, the image 50 of the discriminant feature point 5 is captured by the image sensor 10. The image sensor 10 generates a signal corresponding to the light that converges on it. Camera 1 generates the captured image from the signal generated by the image sensor 10.

[0026] Figure 5 is an explanatory diagram showing the relationship between the actual size of the identification feature point 5 in Figure 4 and the size of the image 50 captured by the image sensor 10 of camera 1. The ratio of the vertical dimension h, which represents the actual size of the identification feature point 5, to the distance L from camera 1 to the identification feature point 5 is the same as the ratio of the vertical dimension h0, which represents the size of the image 50 of the identification feature point 5, to the focal length L0 of camera 1. That is, the relationship h / L = h0 / L0 holds true.

[0027] In this embodiment, the focal length L0 of camera 1 is a value determined by the lens of camera 1. Information regarding the focal length L0 of camera 1 is acquired from camera 1 by the feature point identification unit 22. In addition, in this embodiment, the vertical dimension h of each feature point 5 is pre-stored in the feature point identification unit 22 as feature point size information indicating the actual size of each feature point 5. Therefore, the feature point identification unit 22 calculates the vertical dimension h0 of the image 50 of the identified feature point 5 and calculates the distance L from camera 1 to the identified feature point 5 by using the relationship h / L = h0 / L0.

[0028] The image 50 of the distinguishing feature point 5 captured by the image sensor 10 of camera 1 is generated as the image of the distinguishing feature point 5 captured by camera 1. Therefore, the vertical dimension h0 of the image 50 of the distinguishing feature point 5 is calculated as the size of the image 50 of the distinguishing feature point 5 from the number of pixels that make up the vertical portion of the image 50 of the distinguishing feature point 5 captured by the image sensor 10, that is, the number of pixels that make up the vertical portion of the image of the distinguishing feature point 5 captured by camera 1. As the number of pixels that make up the vertical portion of the image of the distinguishing feature point 5, for example, the number of pixels that make up the vertical portion of the bounding box of the image of the distinguishing feature point 5 may be used. The distinguishing feature point identification unit 22 may also store the horizontal dimension of each distinguishing feature point 5 in advance as distinguishing feature point size information that indicates the actual size of the distinguishing feature point 5. In this case, the feature point identification unit 22 calculates the size of the image 50 of the feature point 5 as the horizontal portion of the image 50 of the feature point 5 captured by the image sensor 10, that is, the number of pixels that make up the horizontal portion of the image of the feature point 5 captured in the image, and calculates the horizontal dimension of the image 50 of the feature point 5 based on the calculated number of pixels.

[0029] Figure 6 is an explanatory diagram showing the process of correcting the image of the identification feature point 5 captured by camera 1 in Figure 4. If the image 50 of the identification feature point 5 is captured at an angle by the image sensor 10, there is a risk that the size of the image 50 of the identification feature point 5 captured by the image sensor 10 will not be calculated accurately. Therefore, the feature point identification unit 22 corrects the image 51 of the identification feature point 5 captured by camera 1, and then calculates the vertical dimension h0 of the image 50 of the identification feature point 5 as the size of the image 50 from the size of the corrected image 51 of the identification feature point 5.

[0030] The correction of the image 51 of the identification feature point 5 is performed by orienting the image 51 so that its shape is the same as the image 51 viewed from directly in front of it. For example, the feature point identification unit 22 specifies the four corner points of the rectangular image 51 of the identification feature point 5, estimates a projection transformation matrix using the position coordinates of the four specified points, and orients the image 51 of the identification feature point 5 based on the estimated projection transformation matrix. The feature point identification unit 22 calculates the vertical dimension h0 of the image 50 of the identification feature point 5 as captured by the image sensor 10, based on the number of pixels that make up the vertical portion of the image 51 of the identification feature point 5 corrected by orienting, as the size of the image 50 of the identification feature point 5. The distance L from the camera 1 to the identification feature point 5 is calculated using the vertical dimension h0 of the image 50 calculated based on the image 51 of the identification feature point 5 corrected by orienting. In this way, the feature point identification unit 22 calculates the distance from the corresponding camera 1 to the identification feature point 5.

[0031] Furthermore, the feature point identification unit 22 individually generates feature point-related information corresponding to each camera 1, which is determined by the image position of each feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each feature point 5. Specifically, the feature point identification unit 22 individually generates feature point-related information corresponding to each camera 1, directly using the information indicating the image position of each feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each feature point 5. The orientation of camera 1 is determined by the feature point identification unit 22 acquiring camera orientation information from the corresponding camera 1.

[0032] In this way, the feature point identification unit 22 generates information related to identified feature points. In the monitoring system, multiple feature point identification units 22 individually generate information related to identified feature points corresponding to each camera 1.

[0033] As shown in Figure 1, the setting information output unit 23 acquires the identified feature point-related information generated by the corresponding feature point identification unit 22. Each setting information output unit 23 has feature point absolute position information pre-stored, which indicates the position of each feature point 5 in the target space A as the absolute coordinate position of each feature point 5. The feature point absolute position information indicates the coordinate position of each feature point 5 in the three-dimensional absolute coordinate system of the target space A as the absolute coordinate position of each feature point 5. The three-dimensional absolute coordinate system of the target space A is a three-dimensional coordinate system that defines the entire target space A with a fixed reference point in the target space A as the origin.

[0034] The feature point-related information generated by each feature point identification unit 22 includes information corresponding to camera 1, such as the image position of each feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each feature point 5. Each setting information output unit 23 identifies the absolute coordinate position of camera 1 in target space A based on the feature point-related information generated by the corresponding feature point identification unit 22 and the absolute position information of the feature point, and individually generates individual camera position information indicating the absolute coordinate position of camera 1 in target space A. Each setting information output unit 23 individually outputs the generated individual camera position information to the management device 3 as setting information. Therefore, in the monitoring system, multiple setting information output units 23 individually output individual camera position information corresponding to each camera 1 to the management device 3 as setting information.

[0035] Figure 7 is an explanatory diagram illustrating the principle by which the setting information output unit 23 of Figure 1 determines the absolute coordinate position of camera 1. The setting information output unit 23 determines the absolute coordinate position of the identification feature point 5a as (x1, y1, z1), the absolute coordinate position of the identification feature point 5b as (x2, y2, z2), and the absolute coordinate position of the identification feature point 5c as (x3, y3, z3), based on the previously stored absolute position information of the feature points. Individual camera position information is generated by each setting information output unit 23 based on the absolute position information of the feature points and the identification feature point related information. Specifically, the setting information output unit 23 determines the direction of each identification feature point 5a, 5b, and 5c as seen from camera 1, based on the image position of each identification feature point 5a, 5b, and 5c in the captured image and the orientation of camera 1. The setting information output unit 23 then determines the absolute coordinate position of camera 1 in target space A by using the absolute coordinate positions of each identification feature point 5a, 5b, and 5c in target space A as a reference, along with the direction of each identification feature point 5 as seen from camera 1 and the distances L1, L2, and L3 from camera 1 to each identification feature point 5a, 5b, and 5c. The individual camera position information indicates the coordinate position of camera 1 in the three-dimensional absolute coordinate system of target space A as the absolute coordinate position of camera 1. Methods such as triangulation based on each identification feature point 5a, 5b, and 5c, or identifying the intersection of three spheres centered on each identification feature point 5a, 5b, and 5c, are used to determine the positional relationship between each identification feature point 5a, 5b, and 5c and camera 1.

[0036] The surveillance object detection unit 24 detects the presence or absence of a target object based on the captured image generated by the corresponding camera 1. The detection of the presence or absence of a target object by the surveillance object detection unit 24 is performed using a surveillance object learning model generated by a general machine learning computer. The surveillance object learning model takes the data of the captured image generated by camera 1 as input and outputs surveillance object inference data that identifies objects in the captured image whose probability of being a target object is above a threshold as detected objects. Alternatively, the surveillance object learning model may take the data of the captured image generated by camera 1 as input and output surveillance object classification data that classifies objects in the captured image into types such as vehicles and people based on differences in size and identifies the target object as a detected object. Furthermore, the surveillance object learning model may take the data of the captured image generated by camera 1 as input and output both the above-mentioned surveillance object inference data and surveillance object classification data. In the surveillance system, multiple surveillance object detection units 24 individually detect the presence or absence of a target object for each captured image from each camera 1.

[0037] Figure 8 is an explanatory diagram showing the state when a vehicle and a person are present within the shooting range of camera 1 in Figure 7. When the monitored objects, vehicle 6a and person 6b, are present within the shooting range of camera 1, camera 1 generates an image showing vehicle 6a and person 6b. The monitored object detection unit 24, corresponding to camera 1, uses a monitored object learning model to detect vehicle 6a and person 6b as detected objects based on the image showing vehicle 6a and person 6b. For example, if multiple vehicles are present within the shooting range of camera 1, the monitored object detection unit 24 will detect vehicles facing a different direction than the parked vehicles as detected objects.

[0038] The object detection unit 24 has pre-stored object size information that indicates the actual size of the object being monitored, corresponding to the type of object being monitored. If the objects being monitored are a vehicle 6a and a person 6b, the object size information will show the average size of the vehicle and the person, respectively. The object size information may also be included in the object learning model. In this case, the type of object being monitored and the actual size of the object being monitored are determined by the object learning model.

[0039] When the monitoring object detection unit 24 detects at least one of the monitoring target objects, a vehicle 6a and a person 6b, as a detected object, it calculates the distance from the corresponding camera 1 to the detected object based on the captured image in which the detected object is visible. The distance from camera 1 to the detected object is calculated in the same way as the feature point identification unit 22, which calculates the distance from camera 1 to the identification feature point 5. Therefore, the distance from camera 1 to the detected object is calculated by the monitoring object detection unit 24 based on the monitoring object size information and the size of the image of the detected object captured in the captured image.

[0040] Furthermore, when the monitoring object detection unit 24 detects a target object as a detected object, it generates detected object-related information that is identified by the position of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object. Specifically, the monitoring object detection unit 24 generates the detected object-related information as is, based on the position of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object. In the monitoring system, multiple monitoring object detection units 24 individually generate detected object-related information corresponding to each camera 1.

[0041] The monitoring information output unit 25 acquires detected object-related information generated by the corresponding monitoring object detection unit 24. The detected object-related information includes the position of the detected object in the captured image, the orientation of camera 1, and information indicating the distance from camera 1 to the detected object, all of which are associated with camera 1. The monitoring information output unit 25 also acquires corresponding camera position-specific information from the setting information output unit 23. The camera position-specific information indicates the absolute coordinate position of camera 1 in target space A. When the monitoring information output unit 25 acquires the detected object-related information from the corresponding monitoring object detection unit 24, it generates detected object position-specific information indicating the absolute coordinate position of the detected object in target space A.

[0042] Individual detected object position information is generated by the monitoring information output unit 25 based on individual camera position information and detected object-related information. Specifically, the monitoring information output unit 25 identifies the direction of the detected object as seen from camera 1, based on the position of the image of the detected object in the captured image and the orientation of camera 1. Then, using the absolute coordinate position of camera 1 in target space A as a reference, the monitoring information output unit 25 identifies the absolute coordinate position of the detected object in target space A by using the direction of the detected object as seen from camera 1 and the distance from camera 1 to the detected object. Each monitoring information output unit 25 individually generates individual detected object position information corresponding to camera 1, showing the identified absolute coordinate position of the detected object. The individual detected object position information indicates the coordinate position of the detected object in the three-dimensional absolute coordinate system of target space A as the absolute coordinate position of the detected object.

[0043] Each monitoring information output unit 25 generates individual detected object location information when the corresponding monitoring object detection unit 24 detects a target object as a detected object, and outputs the generated individual detected object location information to the management device 3 as monitoring information. Therefore, in the monitoring system, multiple monitoring information output units 25 can individually output individual detected object location information as monitoring information to the management device 3.

[0044] As shown in Figure 1, the management device 3 includes a setting information integration unit 31, a monitoring information integration unit 32, a determination unit 33, a notification unit 34, and a display unit 35. The management device 3 is installed, for example, in an unshown control room adjacent to the target space A. In the control room, the monitored objects present in the target space A are monitored by a monitor. The management device 3 may also be installed in any of the cameras 1.

[0045] The configuration information integration unit 31 receives individual camera position information output individually as configuration information from each configuration information output unit 23. The configuration information integration unit 31 also generates integrated camera position information that collectively shows the absolute coordinate positions of each camera 1 in the target space A, based on each individual camera position information. Specifically, the configuration information integration unit 31 generates integrated camera position information by reflecting the absolute coordinate positions of the individual cameras 1 in the target space A shown in each individual camera position information into the common target space A. The integrated camera position information collectively shows the coordinate positions of each camera 1 in the three-dimensional absolute coordinate system of the target space A as the absolute coordinate positions of each camera 1.

[0046] In the monitoring system, the camera position identification unit is composed of the respective setting information output unit 23 of each processing unit 2 and the setting information integration unit 31. Based on the information related to each identified feature point generated by each feature point identification unit 22 and the absolute position information of the feature points stored in advance in each setting information output unit 23, the camera position identification unit identifies the absolute coordinate position of camera 1 in the target space A and generates individual camera position information. Then, based on the absolute coordinate position of camera 1 in the target space A shown individually in each camera position information, the camera position identification unit generates integrated camera position information that shows the absolute coordinate position of each camera 1 in the target space A collectively.

[0047] The monitoring information integration unit 32 receives individual detected object position information output as monitoring information from the monitoring information output unit 25, which corresponds to the monitoring object detection unit 24 that detected the detected object. The monitoring information integration unit 32 also acquires camera position integration information from the setting information integration unit 31. The camera position integration information shows the absolute coordinate positions of each camera 1 in the target space A all together.

[0048] When the monitoring information integration unit 32 receives multiple individual detection object position information, it generates integrated detection object position information that collectively shows the absolute coordinate positions of each detected object in the target space A based on each individual detection object position information. Specifically, the monitoring information integration unit 32 generates integrated detection object position information by reflecting the absolute coordinate positions of the individual detected objects shown in each individual detection object position information into the common target space A in the camera position integration information. The integrated detection object position information collectively shows the coordinate positions of each detected object in the three-dimensional absolute coordinate system of the target space A as the absolute coordinate positions of each detected object. If the number of received individual detection object position information is one, the monitoring information integration unit 32 generates integrated detection object position information by directly reflecting the individual detection object position information into the common target space A in the camera position integration information. Alternatively, the monitoring information integration unit 32 may generate integrated detection object position information by directly reflecting the individual detection object position information received from the monitoring information output unit 25 into the common target space A, without obtaining camera position integration information from the setting information integration unit 31.

[0049] In the monitoring system, the detected object location unit is composed of the monitoring information output unit 25 of each processing unit 2 and the monitoring information integration unit 32. The detected object location unit identifies the absolute coordinate position of the detected object in the target space A based on the detected object-related information generated by the monitoring object detection unit 24 and the absolute coordinate position of each camera 1 in the target space A identified by each setting information output unit 23, and generates individual detected object location information individually. Then, the detected object location unit generates integrated detected object location information that shows the absolute coordinate position of each detected object in the target space A collectively, based on the absolute coordinate position of the detected object in the target space A individually indicated in each individual detected object location information.

[0050] Figure 9 is a plan view showing the state in the target space A of Figure 2 where a vehicle 6a is within the shooting range 11a of one camera 1a and a person 6b is within the shooting range 11b of the other camera 1b. In Figure 9, an image of the vehicle 6a, which is the object to be monitored, is generated by one camera 1a, and an image of the person 6b, which is the object to be monitored, is generated by the other camera 1b. As a result, the object detection unit 24 corresponding to camera 1a detects the vehicle 6a as a detected object and generates detected object-related information indicating the position of the image of the vehicle 6a in the captured image, the orientation of camera 1a, and the distance from camera 1a to vehicle 6a. Similarly, the object detection unit 24 corresponding to camera 1b detects the person 6b as a detected object and generates detected object-related information indicating the position of the image of the person 6b in the captured image, the orientation of camera 1b, and the distance from camera 1b to person 6b.

[0051] As a result, the monitoring information output unit 25 corresponding to camera 1a generates individual detected object position information indicating the absolute coordinate position of vehicle 6a in target space A. In addition, the monitoring information output unit 25 corresponding to camera 1b generates individual detected object position information indicating the absolute coordinate position of person 6b in target space A.

[0052] Then, in the management device 3, based on the individual location information of each detected object, the monitoring information integration unit 32 generates integrated detected object location information that shows the absolute coordinate positions of the vehicle 6a and the person 6b in the target space A. As shown in Figure 9, in the integrated detected object location information, the absolute coordinate positions of the vehicle 6a and the person 6b are reflected in the common target space A, making it possible to understand the positional relationship between the vehicle 6a and the person 6b.

[0053] Returning to Figure 1, the determination unit 33 determines whether or not to generate an alarm based on the integrated detection object position information generated by the monitoring information integration unit 32. In this embodiment, if multiple detection objects exist in the target space A, the determination unit 33 calculates the distance between each detection object as the inter-object distance based on the absolute coordinate position of each detection object, and determines whether or not to generate an alarm by comparing the inter-object distance with a preset distance. The determination unit 33 determines to generate an alarm when the inter-object distance is shorter than the preset distance. The determination unit 33 also determines not to generate an alarm when the inter-object distance is greater than or equal to the preset distance.

[0054] The notification unit 34 issues an alarm if the determination unit 33 determines that an alarm should be issued. Examples of alarms include warnings about the risk of a collision and cautionary reminders prompting the driver to check their surroundings. The notification unit 34 maintains a state where no alarm is issued if the determination unit 33 determines that no alarm should be issued. The notification unit 34 may include a speaker that emits the alarm audibly and a display that shows the alarm.

[0055] The display unit 35 displays a monitoring image based on the integrated detection object position information generated by the monitoring information integration unit 32. The monitoring image is an image of a three-dimensional environmental diagram representing the target space A, with the positions of the detected objects shown. In the monitoring image, the positions of the detected objects are indicated by circles, illustrations, etc. A display or the like can be used as the display unit 35. The display unit 35 may be connected locally to the monitoring information integration unit 32, or it may be connected to the monitoring information integration unit 32 via a network. The monitoring image displayed on the display unit 35 is recognizable by the monitoring personnel. The display unit 35 may also function as a notification unit 34 by displaying alarms.

[0056] Next, a monitoring method for monitoring the target space A using a monitoring system will be described. Figure 10 is a flowchart showing the monitoring method using the monitoring system in Figure 1. As preparation for the monitoring system to execute the monitoring method, first, multiple cameras 1a and 1b are installed by workers at different positions in the target space A. Each camera 1a and 1b is installed at an arbitrary position in the target space A. At this time, the orientation of each camera 1a and 1b is adjusted so that their respective shooting ranges 11a and 11b are different. In this embodiment, the orientation of each camera 1a and 1b is fixed so that it does not change. The orientation of camera 1 is determined by the horizontal angle and elevation angle of camera 1. For each camera 1a and 1b, information that identifies the orientation of camera 1 at the time of installation, i.e., the horizontal angle and elevation angle of camera 1 at the time of installation, is individually set as camera orientation information.

[0057] Furthermore, as preparation for the monitoring system to execute the monitoring method, three or more feature points 5 are placed by an operator at different locations within the target space A. Each feature point 5 is placed in the target space A such that at least three feature points 5 are individually contained within the respective shooting ranges 11a and 11b of each camera 1a and 1b. At this time, each feature point 5 is placed such that camera 1 and at least three feature points 5 contained within the shooting range corresponding to camera 1 are not placed on the same plane or on the same line. Therefore, when two cameras 1a and 1b are installed in the target space A, each feature point 5 is placed such that at least three feature points 5 are contained within the shooting range 11a of one camera 1a, and at least three other feature points 5 are contained within the shooting range 11b of the other camera 1b. As a result, when two cameras 1a and 1b are installed in the target space A, the number of feature points 5 placed in the target space A is at least six. In this embodiment, two cameras 1a and 1b are installed in the target space A, and six feature points 5 are arranged in the target space A.

[0058] Furthermore, as preparation for the monitoring system to execute the monitoring method, the operator inputs absolute position information of each feature point 5 in the target space A, corresponding to the actual location where each feature point 5 is located, into the setting information integration unit 31 of the management device 3. Specifically, the coordinate position of each feature point 5 in the three-dimensional absolute coordinate system of the target space A is input to the setting information integration unit 31 of the management device 3 as absolute position information of the feature point. In addition, the operator inputs feature point size information indicating the actual size of each feature point 5 into the feature point identification unit 22 of each processing unit 2. In this embodiment, the vertical dimension h of each feature point 5 is input to each feature point identification unit 22 as feature point size information. Furthermore, the operator inputs monitoring object size information indicating the actual size of the monitored object, corresponding to the type of monitored object, into the monitored object detection unit 24 of each processing unit 2.

[0059] The monitoring method by the monitoring system involves one or more computers, which perform the following steps: a setup shooting step S1, a feature point identification step S2, a camera position identification step S3, a monitoring shooting step S4, a monitoring object detection step S5, a detected object position identification step S6, and a determination step S7.

[0060] <Setting Shooting Step S1> Once each camera 1 is installed in the target space A by a worker, the camera control unit 21 of each processing unit 2 executes the setup shooting step S1. In the setup shooting step S1, each camera control unit 21 causes each camera 1a and 1b to individually photograph multiple areas in the target space A, designated as the shooting ranges 11a and 11b for each camera 1a and 1b. As a result, each camera 1a and 1b generates images corresponding to the shooting ranges 11a and 11b.

[0061] <Feature Point Identification Step S2> After the setup shooting step S1, the feature point identification step S2 is executed by the feature point identification unit 22 of each processing unit 2. The execution of the feature point identification step S2 identifies the three feature points 5 in the captured image as identified feature points 5. Furthermore, the execution of the feature point identification step S2 generates identified feature point-related information, which is determined by the image position of each identified feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each identified feature point 5, for each camera 1a and 1b. Specifically, the execution of the feature point identification step S2 generates identified feature point-related information for each camera 1a and 1b, which includes information indicating the image position of each identified feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each identified feature point 5.

[0062] Here, Figure 11 is a flowchart showing the processing of the feature point identification step S2 in Figure 10. In the feature point identification step S2, the processing of steps 21 to S24 is executed. In step S21, each feature point identification unit 22 individually acquires the captured images generated by each camera 1a and 1b from each camera 1a and 1b. At this time, each feature point identification unit 22 also individually acquires camera pose information from each camera 1a and 1b. After this, in step S22, each feature point identification unit 22 identifies the three feature points 5 captured in the captured images as identification feature points 5 based on the captured images acquired from each camera 1a and 1b. After this, in step S23, each feature point identification unit 22 individually calculates the distance from camera 1 to identification feature point 5 corresponding to each camera 1a and 1b, based on the feature point size information and the size of the image of each identification feature point 5 captured in the captured image. Subsequently, in step S24, each feature point identification unit 22 individually generates information as feature point-related information corresponding to each camera 1a, 1b, indicating the image position of each feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each feature point 5. The orientation of camera 1 is determined by the camera orientation information acquired by the feature point identification unit 22 from camera 1.

[0063] <Camera position identification step S3> Following the feature point identification step S2, the camera position identification step S3 is performed by the camera position identification unit, which includes the setting information output unit 23 of each processing unit 2 and the setting information integration unit 31 of the management device 3. The execution of the camera position identification step S3 generates integrated camera position information that collectively shows the absolute coordinate positions of each camera 1a and 1b in the target space A.

[0064] Here, Figure 12 is a flowchart showing the processing of the camera position identification step S3 in Figure 10. In the camera position identification step S3, the processing of steps 31 and S32 is executed. When the identified feature point related information is individually generated for each camera 1a and 1b in the feature point identification step S2, each setting information output unit 23 individually generates individual camera position information for each camera 1a and 1b in step S31, based on the identified feature point related information and the previously stored absolute position information of the feature points, indicating the absolute coordinate position of camera 1 in the target space A. That is, in step S31, each setting information output unit 23 individually identifies the direction of each identified feature point 5 as seen from camera 1 for each camera 1a and 1b, based on the image position of each identified feature point 5 in the captured image and the orientation of camera 1. Then, in step S31, each setting information output unit 23 individually identifies the absolute coordinate position of camera 1 for each camera 1a and 1b by using the absolute coordinate position of each identification feature point 5 as a reference, the direction of each identification feature point 5 as seen from camera 1, and the distance from camera 1 to each identification feature point 5. In this way, in step S31, each setting information output unit 23 individually generates individual camera position information indicating the absolute coordinate position of camera 1 for each camera 1a and 1b. After this, each setting information output unit 23 individually outputs the individual camera position information as setting information to the setting information integration unit 31. After this, in step S32, the setting information integration unit 31 generates integrated camera position information by reflecting the absolute coordinate positions of each camera 1a and 1b identified by each setting information output unit 23 in a common target space A.

[0065] In this way, in the camera position identification step S3, integrated camera position information is generated based on the information related to each identified feature point and the absolute position information of the feature points. The integrated camera position information generated in the camera position identification step S3 is stored in the management device 3.

[0066] <Surveillance shooting step S4> After the camera position identification step S3, the monitoring and shooting step S4 is executed by each camera control unit 21. The monitoring and shooting step S4 is executed with the integrated camera position information stored in the management device 3. In the monitoring and shooting step S4, each camera control unit 21 causes each camera 1a and 1b to individually capture images within their respective shooting ranges 11a and 1b. As a result, each camera 1a and 1b generates images corresponding to their respective shooting ranges 11a and 11b.

[0067] <Monitoring Object Detection Step S5> After the surveillance shooting step S4, the surveillance object detection step S5 is executed by the surveillance object detection unit 24 of each processing unit 2. In the surveillance object detection step S5, each surveillance object detection unit 24 individually detects the presence or absence of a target object for each image captured by each camera 1a, 1b. In addition, in the surveillance object detection step S5, if at least one of the surveillance object detection units 24 detects a target object as a detected object, the surveillance object detection unit 24 that detected the detected object generates detected object-related information that is specified by the position of the image of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object. Specifically, in the surveillance object detection step S5, the surveillance object detection unit 24 that detected the detected object generates the information indicating the position of the image of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object as detected object-related information.

[0068] Here, Figure 13 is a flowchart showing the processing of the object detection step S5 in Figure 10. In the object detection step S5, the processing of steps S51 to S54 is executed. In step S51, each object detection unit 24 individually acquires the captured images generated by each camera 1a and 1b from each camera 1a and 1b. At this time, each object detection unit 24 also individually acquires camera pose information from each camera 1a and 1b. After this, in step S52, each object detection unit 24 individually detects the presence or absence of a target object for each captured image from each camera 1a and 1b by using a detection object learning model based on the captured images acquired from each camera 1a and 1b.

[0069] As a result, if the monitored object is not detected by any of the monitored object detection units 24, each monitored object detection unit 24 repeats the processing in steps S51 and S52 until the monitored object is detected as a detected object by at least one of the monitored object detection units 24. On the other hand, if the monitored object is detected as a detected object by at least one of the monitored object detection units 24, the monitored object detection unit 24 that detected the detected object calculates the distance from the camera 1 to the detected object in step S53 based on the captured image in which the detected object is visible. After this, the monitored object detection unit 24 that detected the detected object generates detected object-related information in step S54, which includes the position of the image of the detected object in the captured image, the orientation of the camera 1, and the distance from the camera 1 to the detected object. The orientation of the camera 1 is determined by the camera orientation information acquired by the monitored object detection unit 24 from the camera 1.

[0070] <Detected object position identification step S6> Following the object detection step S5, the object location identification step S6 is performed by the object location identification unit, which includes the monitoring information output unit 25 of each processing unit 2 and the monitoring information integration unit 32 of the management device 3. The object location identification step S6 is performed when object-related information is generated by at least one of the object detection units 24. The execution of the object location identification step S6 generates integrated object location information indicating the absolute coordinate position of the detected object in the target space A.

[0071] Here, Figure 14 is a flowchart showing the processing of the detected object position identification step S6 in Figure 10. In the detected object position identification step S6, the processing of steps S61 and S62 is executed. When detected object-related information is generated in the monitored object detection step S5, the monitoring information output unit 25 corresponding to the monitoring object detection unit 24 that generated the detected object-related information, in step S61, individually identifies the absolute coordinate position of the detected object in the target space A for each detected object and generates individual detected object position information indicating the absolute coordinate position of the detected object in the target space A. The individual detected object position information is generated based on the detected object-related information generated by the monitoring object detection unit 24 and the absolute coordinate position of the corresponding camera 1 from the absolute coordinate positions of each camera 1a, 1b identified in the camera position identification step S3. That is, in step S61, the monitoring information output unit 25 identifies the direction of the detected object as seen from camera 1 for each detected object, corresponding to camera 1, based on the position of the image of the detected object in the captured image and the orientation of camera 1. Then, in step S61, the monitoring information output unit 25 uses the absolute coordinate position of camera 1 in the target space A as a reference, and by using the direction of the detected object as seen from camera 1 and the distance from camera 1 to the detected object, it identifies the absolute coordinate position of the detected object in the target space A for each detected object in relation to camera 1. As a result, in step S61, the monitoring information output unit 25 generates individual detected object position information indicating the absolute coordinate position of the detected object for each detected object. After this, the monitoring information output unit 25 outputs the individual detected object position information as monitoring information to the monitoring information integration unit 32. After this, in step S62, the monitoring information integration unit 32 generates integrated detected object position information by reflecting the absolute coordinate positions of the individual detected objects shown in the individual detected object position information into the common target space A in the camera position integration information. In addition, the monitoring information integration unit 32 may generate the detected object position integration information in step S62 by directly reflecting the absolute coordinate positions of individual detected objects indicated in the individual detected object position information in the common target space A, rather than relying on the camera position integration information.

[0072] Thus, if object-related information is generated by at least one of the object detection units 24 in the object detection step S5, object-position identification step S6 generates integrated object position information based on the object-related information and the absolute coordinate position of the corresponding camera 1. In the integrated object position information, the absolute coordinate positions of the objects detected by at least one of the object detection units 24 are shown together in a common target space A.

[0073] <Judgment Step S7> After the detection object position identification step S6, the determination unit 33 performs the determination step S7.

[0074] Here, Figure 15 is a flowchart showing the processing of the determination step S7 in Figure 10. In the determination step S7, the processing of steps S71 and S72 is executed. When the integrated detection object position information is generated in the detection object position identification step S6, the determination unit 33 determines in step S71 whether or not to generate an alarm based on the integrated detection object position information. In step S71, if multiple detection objects exist in the target space A, the distance between each detection object is calculated as the inter-object distance based on the position of each detection object, and it is determined whether or not to generate an alarm by determining whether or not the inter-object distance is shorter than the set distance.

[0075] In step S71, if the distance between objects is shorter than the set distance, the determination unit 33 determines whether to generate an alarm, and in step S72, the notification unit 34 generates an alarm. On the other hand, in step S71, if the distance between objects is greater than or equal to the set distance, the determination unit 33 determines whether to not generate an alarm, and the notification unit 34 maintains a state where it does not generate an alarm.

[0076] <Display step S8> Following the detection object position identification step S6, the display step S8 is performed by the display unit 35 along with the execution of the determination step S7. In the display step S8, the display unit 35 displays a monitoring image based on the integrated detection object position information. The monitoring image shows the absolute coordinate position of the detected object on a three-dimensional environmental diagram representing the target space A.

[0077] In this type of surveillance system, multiple cameras 1, installed in a target space A where three or more individually identifiable feature points 5 are located, each generate an image by individually capturing multiple shooting ranges in the target space A. Each feature point identification unit 22 identifies at least three feature points 5 in the captured image as identified feature points 5 based on the image generated by the camera 1. Each feature point identification unit 22 also calculates the distance from the camera 1 to each identified feature point 5 based on feature point size information indicating the actual size of each feature point 5 and the image size of each identified feature point 5 in the captured image. Furthermore, each feature point identification unit 22 individually generates identified feature point-related information corresponding to each camera 1, which is determined by the image position of each identified feature point 5 in the captured image, the orientation of the camera 1, and the distance from the camera 1 to each identified feature point 5. The camera position identification unit has feature point absolute position information indicating the absolute coordinate position of each feature point 5 in the target space A stored in advance. The camera positioning unit determines the absolute coordinate position of each camera 1 in target space A based on the identification feature point related information and the feature point absolute position information. When each monitoring object detection unit 24 detects a monitoring target object as a detected object, it generates detected object related information based on the captured image in which the detected object is visible. The detected object positioning unit determines the absolute coordinate position of the detected object in target space A based on the absolute coordinate position of each camera 1 in target space A and the detected object related information.

[0078] Therefore, even if multiple cameras 1 are installed at arbitrary positions in the target space A, the generation of integrated camera position information allows for more accurate identification of the absolute coordinate position of each camera 1 in the target space A. As a result, when a monitored object is detected as a detected object from the images captured by each camera 1, the coordinate position of the detected object can be determined based on the absolute coordinate position of each camera 1 in the target space A, thereby allowing for more accurate identification of the absolute coordinate position of the detected object in the target space A. Consequently, the location of a monitored object captured in the image of a camera 1 whose shooting range covers a portion of the target space can be more accurately determined within the target space A.

[0079] Furthermore, in this monitoring method, in the feature point identification step S2, at least three feature points 5 captured in the image generated by each camera 1 are identified as identified feature points 5. In the feature point identification step S2, the distance from camera 1 to each identified feature point 5 is calculated based on feature point size information indicating the actual size of each feature point 5 and the image size of each identified feature point 5 captured in the image. Also in the feature point identification step S2, identified feature point related information is individually generated for each camera 1, specified by the image position of each identified feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each identified feature point 5. After this, in the camera position identification step S3, the absolute coordinate position of each camera 1 in the target space A is determined based on each identified feature point related information and the absolute feature point position information. After this, in the monitored object detection step S5, if a monitored object is detected as a detected object, detected object related information is generated based on the captured image in which the detected object is captured. Subsequently, in the detected object positioning step S6, the absolute coordinate position of the detected object in target space A is determined based on the absolute coordinate position of each camera 1 in target space A and the detected object-related information.

[0080] Therefore, similar to a surveillance system, the position of each camera 1 in target space A can be determined more accurately. As a result, when a monitored object is detected as a detected object from the image captured by each camera 1, the absolute coordinate position of the detected object in target space A can be determined more accurately. Consequently, the location of a monitored object captured in the image of camera 1, which has a shooting range that covers a portion of the target space, can be determined more accurately within target space A.

[0081] Embodiment 2. Figure 16 is a plan view showing the camera's shooting range in the target space where the monitoring system according to Embodiment 2 is used. Figure 16 shows only the shooting range of one of the two cameras 1a and 1b, camera 1a. Each camera 1 is rotatable under the control of the corresponding camera control unit 21. The shooting range of camera 1 in target space A changes according to the rotation of camera 1.

[0082] Each camera control unit 21 rotates the camera 1 until the number of distinguishable feature points 5 identified by the feature point identification unit 22 becomes three, if the number of distinguishable feature points 5 identified by the feature point identification unit 22 based on the captured image generated by the corresponding camera 1 is two or less. As a result, each camera control unit 21 sets the orientation of the corresponding camera 1 to a different orientation after rotation than the orientation before rotation, and causes the camera 1 to capture an image in a different shooting range after rotation than the shooting range of the camera 1 before rotation.

[0083] For each camera 1, the orientation of camera 1 at the time of installation is considered the reference orientation, and the orientation of camera 1 after rotation is determined according to the angle by which camera 1 rotates from the reference orientation. Therefore, the orientation of camera 1 after rotation is determined by adding the rotational horizontal angle component and rotational elevation / depression angle component of camera 1, respectively, to the horizontal angle and elevation / depression angle of camera 1 that determine the reference orientation. For each camera 1, the orientation of camera 1 determined by the horizontal angle and elevation / depression angle of camera 1 is set as camera orientation information. The camera orientation information set for camera 1 is acquired from each camera 1 by the corresponding feature point identification unit 22 and the corresponding monitored object detection unit 24, respectively.

[0084] In Figure 16, when camera 1a rotates, the shooting range of camera 1a changes from the shooting range 11a before rotation to the shooting range 12a after rotation. Two feature points 5 are contained in the shooting range 11a of camera 1a before rotation. One feature point 5 is contained in the shooting range 12a of camera 1a after rotation. Therefore, two feature points 5 are captured in the image taken by camera 1a before rotation, and one feature point 5 is captured in the image taken by camera 1a after rotation. In other words, the total number of feature points 5 captured in the image taken by camera 1a before rotation and the total number of feature points 5 captured in the image taken by camera 1a after rotation is 3. Similarly to camera 1a, the total number of feature points 5 captured in the image taken by camera 1b before rotation and the total number of feature points 5 captured in the image taken by camera 1b after rotation is 3.

[0085] Furthermore, each camera 1 has a zoom function that changes the size of the camera's shooting range. The zoom function of each camera 1 is executed by the control of the corresponding camera control unit 21. When the zoom function of camera 1 is executed, the size of the subject in the captured image produced by camera 1 changes according to the magnification of the camera 1's zoom function. The magnification of the camera 1's zoom function can be adjusted by changing the focal length of camera 1. The magnification of the camera 1's zoom function is set as camera zoom information for each camera 1 in accordance with the execution of the camera 1's zoom function. The camera zoom information is acquired from each camera 1 by the corresponding feature point identification unit 22.

[0086] Each camera control unit 21, if the size of the identified feature point 5 identified by the feature point identification unit 22 in the captured image is smaller than a preset reference value, instructs the corresponding camera 1 to execute a zoom function to enlarge the size of the identified feature point 5 in the captured image. The camera 1 that has executed the zoom function captures the shooting range of the camera 1, thereby generating a captured image that includes the image of the identified feature point 5 enlarged according to the magnification of the zoom function, and calling this the zoomed-in captured image.

[0087] Figure 17 is an explanatory diagram showing the captured image generated by the camera 1a in Figure 16 capturing the shooting range 11a before rotation. In Figure 17, of the two images of the two distinguishing feature points 5 in the captured image generated by capturing the shooting range 11a, the size of the image of the other distinguishing feature point 5, which is located further away than the other distinguishing feature point 5, is smaller than the reference value. Therefore, the camera 1a, under the control of the corresponding camera control unit 21, performs a zoom function to enlarge the XVIII portion of the captured image in Figure 17 that contains only the image of the other distinguishing feature point 5.

[0088] Figure 18 is an explanatory diagram showing a zoomed-in image of the captured image in Figure 17, where section XVIII is magnified by the zoom function of camera 1a. In the zoomed-in image of Figure 18, the size of the image of the other of the two discriminant feature points 5 in the captured image in Figure 17, which is smaller than the reference value, is magnified. As a result, the size of the image of the other discriminant feature point 5 in the zoomed-in image of Figure 18 can be calculated more accurately than the size of the image of the other discriminant feature point 5 in the captured image in Figure 17.

[0089] Figure 19 is an explanatory diagram showing an image generated when camera 1a in Figure 16 captures the shooting range 12a after rotating. In Figure 19, the size of the image of one of the distinguishing feature points 5 in the image generated by capturing the shooting range 12a is smaller than the reference value. Therefore, camera 1a performs a zoom function to enlarge the XX portion in the image of distinguishing feature point 5 in Figure 19, under the control of the corresponding camera control unit 21.

[0090] Figure 20 is an explanatory diagram showing a zoomed-in image of the captured image in Figure 19, where the XX portion is enlarged by the zoom function of camera 1a. In the zoomed-in image of Figure 20, the size of the image of the distinguishing feature point 5 in the captured image in Figure 19 is enlarged. As a result, the size of the image of the distinguishing feature point 5 in the zoomed-in image of Figure 20 can be calculated more accurately than the size of the image of the distinguishing feature point 5 in the captured image in Figure 19.

[0091] Each feature point identification unit 22 calculates the distance from the corresponding camera 1 to the identified feature point 5 based on the camera zoom information of the corresponding camera 1, the size of the image of the identified feature point 5 in the zoomed-in image, and the feature point size information. That is, each feature point identification unit 22 calculates the distance from the corresponding camera 1 to the identified feature point 5 based on the relationship between the focal length corresponding to the magnification of the zoom function of the corresponding camera 1 and the size of the identified feature point 5 in the zoomed-in image, and based on the actual size of the feature point 5 in the feature point size information. The method for calculating the distance from the corresponding camera 1 to the identified feature point 5 is the same as in Embodiment 1.

[0092] Furthermore, each feature point identification unit 22 identifies the orientation of camera 1 when it generated the captured image in which the identified feature point 5 is visible, based on camera orientation information acquired from the corresponding camera 1. Each feature point identification unit 22 individually generates identified feature point-related information corresponding to each camera 1, which is determined by the position of the identified feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to the identified feature point 5. In other words, each feature point identification unit 22 individually generates identified feature point-related information corresponding to each camera 1, which includes information indicating the position of the identified feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to the identified feature point 5.

[0093] In this embodiment, once the camera position integration information is generated by the camera position identification unit, each camera 1 individually captures the corresponding shooting range while fixing its orientation to a reference orientation, similar to the first embodiment. The other configurations are the same as in the first embodiment.

[0094] Next, a monitoring method for monitoring the target space A using a monitoring system will be described. In this embodiment, as preparation for the monitoring system to execute the monitoring method, first, multiple cameras 1a and 1b are installed by workers at different positions in the target space A. Each camera 1a and 1b is installed at an arbitrary position in the target space A. At this time, the orientation of each camera 1a and 1b is adjusted so that their respective shooting ranges 11a and 11b are different. In this embodiment, the orientation of each camera 1 can be changed by rotating the camera 1. Camera orientation information is individually set for each camera 1a and 1b. Therefore, the orientation of camera 1, as identified by the camera orientation information, changes according to the rotation of camera 1. The camera orientation information individually set for each camera 1 is acquired by the corresponding feature point identification unit 22 and also by the corresponding monitored object detection unit 24.

[0095] Furthermore, as preparation for the monitoring system to execute the monitoring method, three or more feature points 5 are placed by a worker at different locations in the target space A. Each feature point 5 is placed corresponding to each camera 1a, 1b such that at least three feature points 5 are located within the area through which the camera 1's shooting range passes when the camera 1 rotates. Each feature point 5 is placed in a common area through which both the shooting ranges of each camera 1a, 1b pass when each camera 1a, 1b rotates. As a result, when two cameras 1a, 1b are installed in the target space A, each feature point 5 can be placed in the target space A as a common feature point 5 that falls within the respective shooting ranges of each camera 1a, 1b. Therefore, in this embodiment, two cameras 1a, 1b are installed in the target space A, and three feature points 5 are placed in the target space A as common feature points 5 that fall within the respective shooting ranges of each camera 1a, 1b. In each target space A, the feature points 5 are arranged such that each of the cameras 1a and 1b, and the three feature points 5 corresponding to the respective shooting ranges of each camera 1a and 1b, are not located on the same plane or on the same straight line.

[0096] Furthermore, in the same manner as in Embodiment 1, the operator inputs feature point absolute position information, which indicates the absolute coordinate position of each feature point 5 in the target space A, to the setting information integration unit 31 of the management device 3, according to the actual location where each feature point 5 is placed. Also, in the same manner as in Embodiment 1, the operator inputs feature point size information, which indicates the actual size of each feature point 5, to the feature point identification unit 22 of each processing unit 2.

[0097] The monitoring method in this embodiment is the same as in Embodiment 1, except for the feature point identification step S2.

[0098] Figure 21 is a flowchart showing the processing of the feature point identification step S2 in the monitoring method using the monitoring system according to Embodiment 2. In this embodiment, the processing of steps S201 to S208 is executed in the feature point identification step S2. In step S201, each feature point identification unit 22 individually acquires the captured images generated by each camera 1a and 1b from each camera 1a and 1b. At this time, each feature point identification unit 22 also individually acquires camera pose information from each camera 1a and 1b.

[0099] Subsequently, in step S202, each feature point identification unit 22 determines whether or not a feature point 5 is present in the captured image using a feature point learning model.

[0100] If each feature point identification unit 22 determines in step S202 that feature points 5 are not present in the captured image, it rotates the corresponding camera 1 in step S203. When camera 1 rotates, the orientation of camera 1 changes according to the rotation. As a result, camera orientation information that identifies the orientation of camera 1 that has changed according to the rotation of camera 1 is set in the rotated camera 1.

[0101] After this, each feature point identification unit 22 repeats the processing in steps S201 to S203 until a feature point 5 is captured in the image.

[0102] If each feature point identification unit 22 determines in step S202 that a feature point 5 is present in the captured image, in step S204 it identifies the feature point 5 in the captured image as an identified feature point 5 and determines whether the size of the image of the identified feature point 5 in the captured image is greater than or equal to a standard value.

[0103] If each feature point identification unit 22 determines that the size of the image of the identified feature point 5 in the captured image is smaller than a reference value, in step S205, it causes the corresponding camera 1 to execute a zoom function to enlarge the identified feature point 5 until the size of the image of the identified feature point 5 in the captured image is equal to or greater than the reference value.

[0104] If each feature point identification unit 22 determines that the size of the image of the identified feature point 5 in the captured image is greater than or equal to a standard value, in step S206, it calculates the distance from the camera 1 to the identified feature point 5 based on the feature point size information and the size of the image of the identified feature point 5 in the captured image. In addition, in step S206, each feature point identification unit 22 obtains camera orientation information from the camera 1 to determine the orientation of the camera 1 when it generated the captured image in which the identified feature point 5 is visible.

[0105] After this, each feature point identification unit 22 determines in step S207 whether the number of identified feature points 5 identified by the feature point identification unit 22 has become 3. If the number of identified feature points 5 in step S207 is less than 3, each feature point identification unit 22 repeats the process from steps S201 to S207 until the number of identified feature points 5 becomes 3.

[0106] If the number of identified feature points 5 becomes three in step S207, each feature point identification unit 22 individually generates identified feature point-related information corresponding to each camera 1 in step S208, including the image position of the identified feature point 5 in the captured image, the orientation of the camera 1, and the distance from the camera 1 to the identified feature point 5. Other processing is the same as in Embodiment 1.

[0107] In this type of surveillance system, each camera control unit 21 rotates the camera 1, changing its orientation to a different orientation after rotation than its orientation before rotation. Therefore, even if the number of identified feature points 5 identified by the feature point identification unit 22 is two or less, the camera 1 can capture an image in a range after rotation that is different from the range of camera 1 before rotation. This allows the respective shooting ranges of each camera 1 to be moved to a position where the common feature points 5 are located, and the common feature points 5 can be captured in the images generated by each camera 1. Consequently, the common feature points 5 can be used for the respective images captured by each camera 1, and the number of feature points 5 to be placed in the target space A can be reduced compared to Embodiment 1.

[0108] Furthermore, if the size of the image of the identification feature point 5 in the captured image is smaller than a reference value, each camera control unit 21 causes the camera 1 to execute a zoom function to enlarge the image of the identification feature point 5 in the captured image. As a result, the size of each enlarged image of the identification feature point 5 in the captured image can be identified more accurately and easily by the feature point identification unit 22. This allows the distance from the camera 1 to each identification feature point 5 to be calculated more accurately and easily by the feature point identification unit 22.

[0109] In Embodiment 2, each camera control unit 21 is configured to rotate the corresponding camera 1. However, if each feature point 5 is positioned in the target space A such that at least three feature points 5 are individually contained within the shooting range of each camera 1, then each camera 1 does not need to have a rotation function.

[0110] Furthermore, in Embodiment 2, the camera 1 performs a zoom function to enlarge the image size of the distinguishing feature point 5 in the captured image. However, if the image size of the distinguishing feature point 5 in the captured image can be determined, the camera 1 does not need to have a zoom function.

[0111] Embodiment 3. In Embodiments 1 and 2, information indicating the image position of each distinguishing feature point 5 in the captured image, the orientation of the camera 1, and the distance from the camera 1 to each distinguishing feature point 5 is directly generated as distinguishing feature point-related information by each feature point identification unit 22 for each camera 1. In contrast, in this embodiment, information indicating the relative coordinate position of each distinguishing feature point 5 with respect to the camera 1 is generated as distinguishing feature point-related information by each feature point identification unit 22 for each camera 1. The relative coordinate position of each distinguishing feature point 5 with respect to the camera 1 is determined by each feature point identification unit 22 based on the image position of each distinguishing feature point 5 in the captured image, the orientation of the camera 1, and the distance from the camera 1 to each distinguishing feature point 5. Therefore, the distinguishing feature point-related information in this embodiment is determined by the image position of each distinguishing feature point 5 in the captured image, the orientation of the camera 1, and the distance from the camera 1 to each distinguishing feature point 5.

[0112] The identification feature point related information shows the coordinate position of each identification feature point 5 in a three-dimensional camera coordinate system with the corresponding camera 1 as the origin. Since the three-dimensional camera coordinate system is a coordinate system with the camera 1 as the origin, the coordinate position of each identification feature point 5 shown in the identification feature point related information is the relative coordinate position of each identification feature point 5 with respect to the corresponding camera 1.

[0113] As explained using Figure 7, the feature point identification unit 22 identifies the direction of each of the three identified feature points 5a, 5b, and 5c as seen from camera 1 in a three-dimensional camera coordinate system with camera 1 as the origin, based on the image positions of the three identified feature points 5a, 5b, and 5c in the captured image generated by camera 1, and the orientation of camera 1. Then, based on the direction of each of the identified feature points 5a, 5b, and 5c as seen from camera 1, and the distances L1, L2, and L3 from camera 1 to each of the identified feature points 5a, 5b, and 5c, the feature point identification unit 22 identifies the coordinate position of each of the identified feature points 5a, 5b, and 5c in the three-dimensional camera coordinate system as the relative coordinate position of each of the identified feature points 5a, 5b, and 5c with respect to camera 1. In this way, the feature point identification unit 22 generates identified feature point related information. In a monitoring system, multiple feature point identification units 22 individually generate identified feature point related information corresponding to each camera 1. In this embodiment, each setting information output unit 23 outputs identification feature point-related information, which indicates the relative coordinate positions of each identification feature point 5a, 5b, and 5c with respect to the camera 1, as setting information to the setting information integration unit 31 of the management device 3.

[0114] Furthermore, in this embodiment, the monitoring object detection unit 24 generates information indicating the relative coordinate position of the detected object with respect to the corresponding camera 1 as detected object-related information, based on the image position of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object. Therefore, the detected object-related information is identified by the image position of the detected object in the captured image, the orientation of camera 1, and the distance from camera 1 to the detected object. In the detected object-related information, the coordinate position of the detected object in a three-dimensional camera coordinate system with the corresponding camera 1 as the origin is shown as the relative coordinate position of the detected object with respect to the corresponding camera 1.

[0115] Specifically, the monitoring object detection unit 24 determines the direction of the detected object as seen from camera 1 in a three-dimensional camera coordinate system with camera 1 as the origin, based on the position of the image of the detected object in the captured image generated by camera 1 and the orientation of camera 1. Then, based on the direction of the detected object as seen from camera 1 and the distance from camera 1 to the detected object, the monitoring object detection unit 24 determines the coordinate position of the detected object in the three-dimensional camera coordinate system as the relative coordinate position of the detected object with respect to camera 1. In this way, the monitoring object detection unit 24 generates detected object-related information. In the monitoring system, multiple monitoring object detection units 24 individually generate detected object-related information corresponding to each camera 1. In this embodiment, each monitoring information output unit 25 outputs the detected object-related information indicating the relative coordinate position of the detected object with respect to camera 1 as monitoring information to the monitoring information integration unit 32 of the management device 3.

[0116] Furthermore, in this embodiment, the absolute position information of the feature points is stored in the setting information integration unit 31 rather than in each setting information output unit 23. The setting information integration unit 31 generates individual camera position information based on the absolute position information of the feature points and the identification feature point related information. In this embodiment, the setting information integration unit 31 identifies the absolute position of camera 1 in the target space A by adding the relative position of each identification feature point 5 with respect to camera 1 to the absolute position of each identification feature point 5 in the target space A. As explained using Figure 7, the setting information integration unit 31 identifies the absolute position of identification feature point 5a as (x1, y1, z1), the absolute position of identification feature point 5b as (x2, y2, z2), and the absolute position of identification feature point 5c as (x3, y3, z3) based on the previously stored absolute position information of the feature points. The configuration information integration unit 31 identifies the absolute coordinate position of camera 1 in target space A by adding the relative coordinate position of each identification feature point 5 with respect to camera 1 to the absolute coordinate position of each identification feature point 5a, 5b, 5c. Each camera position information shows the coordinate position of camera 1 in the three-dimensional absolute coordinate system of target space A as the absolute coordinate position of camera 1.

[0117] The monitoring information integration unit 32 receives detected object-related information output as monitoring information from the monitoring information output unit 25 corresponding to the monitoring object detection unit 24 that detected the detected object. The detected object-related information indicates the relative coordinate position of the detected object with respect to the corresponding camera 1. The monitoring information integration unit 32 also acquires camera position integration information from the setting information integration unit 31. The camera position integration information shows the absolute coordinate position of each camera 1 together in the target space A. For each piece of detected object-related information received from each monitoring information output unit 25, the monitoring information integration unit 32 individually generates individual detected object position information indicating the absolute coordinate position of the detected object in the target space A.

[0118] The monitoring information integration unit 32 generates individual detected object position information based on camera position integration information indicating the absolute coordinate position of each camera 1 and detected object related information indicating the relative coordinate position of the detected object with respect to camera 1. In this embodiment, the monitoring information integration unit 32 identifies the absolute coordinate position of the detected object in target space A by adding the relative coordinate position of the detected object with respect to camera 1 to the absolute coordinate position of camera 1 in target space A. The individual detected object position information indicates the coordinate position of the detected object in the three-dimensional absolute coordinate system of target space A as the absolute coordinate position of the detected object. The other configurations are the same as in Embodiment 1.

[0119] The monitoring method in this embodiment is similar to that in Embodiment 1, and the following steps are performed by a monitoring system, which is one or more computers: a setting shooting step S1, a feature point identification step S2, a camera position identification step S3, a monitoring shooting step S4, a monitoring object detection step S5, a detected object position identification step S6, and a determination step S7. Therefore, the flowchart illustrating the monitoring method in this embodiment is the same as the flowchart shown in Figure 10.

[0120] In the monitoring method, the setting shooting step S1 is performed in the same manner as in Embodiment 1, and captured images corresponding to the shooting ranges 11a and 11b are generated by each camera 1a and 1b.

[0121] In the feature point identification step S2, each feature point identification unit 22 individually generates feature point-related information corresponding to each camera 1a, 1b, indicating the relative coordinate position of the feature point 5 with respect to camera 1, based on the image position of each feature point 5 in the captured image, the orientation of camera 1, and the distance from camera 1 to each feature point 5. Other processing in the feature point identification step S2 is the same as in Embodiment 1.

[0122] In the camera position identification step S3, each setting information output unit 23 outputs identification feature point-related information, which indicates the relative coordinate position of the identification feature point 5 with respect to camera 1, to the setting information integration unit 31 as setting information. Then, in the camera position identification step S3, the setting information integration unit 31 calculates the absolute coordinate position of camera 1 in target space A individually for each camera 1a, 1b by adding the relative coordinate position of each identification feature point 5 with respect to camera 1 to the absolute coordinate position of each identification feature point 5 in target space A. Other processing in the camera position identification step S3 is the same as in Embodiment 1. As a result, the setting information integration unit 31 generates camera position integration information that shows the absolute coordinate positions of each camera 1a, 1b in target space A together.

[0123] After the camera position identification step S3, the monitoring shooting step S4 is performed in the same manner as in Embodiment 1, so that captured images corresponding to the shooting ranges 11a and 11b are generated by each camera 1a and 1b.

[0124] In the object detection step S5, the object detection unit 24 that has detected the object generates information indicating the relative coordinate position of the detected object with respect to the camera 1 as object-related information, based on the position of the image of the detected object in the captured image, the orientation of the camera 1, and the distance from the camera 1 to the detected object. Other processing in the object detection step S5 is the same as in Embodiment 1.

[0125] In the camera position identification step S3, each setting information output unit 23 outputs identification feature point related information, which indicates the relative coordinate position of the identification feature point 5 with respect to camera 1, to the setting information integration unit 31 as setting information. Then, in the detected object position identification step S6, the monitoring information integration unit 32 generates individual detected object position information for each detected object corresponding to camera 1, based on the camera position integration information indicating the absolute coordinate position of each camera 1 and the detected object related information indicating the relative coordinate position of the detected object with respect to camera 1. The individual detected object position information indicates the absolute coordinate position of the detected object in the target space A. In this embodiment, in the detected object position identification step S6, the monitoring information integration unit 32 calculates the absolute coordinate position of the detected object in the target space A for each detected object corresponding to camera 1 by adding the relative coordinate position of the detected object with respect to camera 1 to the absolute coordinate position of camera 1 in the target space A. Other processing in the detected object position identification step S6 is the same as in Embodiment 1. As a result, the monitoring information integration unit 32 generates integrated detection object position information, which collectively shows the absolute coordinate positions of detected objects detected by at least one of the monitoring object detection units 24 in a common target space A.

[0126] The determination step S7 and display step S8 following the detected object position identification step S6 are the same as in Embodiment 1.

[0127] In this embodiment, information indicating the relative coordinate position of each identification feature point 5 with respect to camera 1 is defined as identification feature point-related information. By adding the relative coordinate position of each identification feature point 5 with respect to camera 1 to the absolute coordinate position of each identification feature point 5 in target space A, the absolute coordinate position of camera 1 in target space A is determined. This method also allows for more accurate determination of the absolute coordinate position of each camera 1 in target space A. As a result, when a monitored object is detected as a detected object from the image captured by each camera 1, the coordinate position of the detected object can be determined based on the absolute coordinate position of each camera 1 in target space A, thereby allowing for more accurate determination of the absolute coordinate position of the detected object in target space A. Therefore, the location of a monitored object captured in the image of camera 1, which covers a portion of the target space, in target space A can be determined more accurately.

[0128] In Embodiment 3, the configuration applied to Embodiment 1 involves generating information indicating the relative coordinate position of each identification feature point 5 with respect to the camera 1 as identification feature point-related information, and generating information indicating the relative coordinate position of the detected object with respect to the camera 1 as detected object-related information. However, the configuration applied to Embodiment 2 may also involve generating information indicating the relative coordinate position of each identification feature point 5 with respect to the camera 1 as identification feature point-related information, and generating information indicating the relative coordinate position of the detected object with respect to the camera 1 as detected object-related information.

[0129] Furthermore, in Embodiment 3, the setting information integration unit 31 generates a plurality of individual camera position information, each indicating the absolute coordinate position of camera 1 in the target space A. However, the individual camera position information may be generated individually by each setting information output unit 23 instead of the setting information integration unit 31. In this case, each setting information output unit 23 stores in advance feature point absolute position information, which indicates the absolute coordinate position of each feature point 5 in the target space A. In this case, each setting information output unit 23 individually generates individual camera position information based on the identified feature point related information generated by the corresponding feature point identification unit 22 and the feature point absolute position information. In this case, the individual camera position information generated by each setting information output unit 23 is individually output from each setting information output unit 23 to the setting information integration unit 31 as setting information.

[0130] Furthermore, if individual camera position information is generated individually by each setting information output unit 23, individual detected object position information, which indicates the absolute coordinate position of a detected object in the target space A, may be generated individually by each monitoring information output unit 25 instead of the monitoring information integration unit 32. In this case, when the monitoring object detection unit 24 detects an object, each monitoring information output unit 25 generates individual detected object position information based on the detected object-related information generated by the corresponding monitoring object detection unit 24 and the individual camera position information generated by the corresponding setting information output unit 23. In this case, the individual detected object position information is output as monitoring information from the monitoring information output unit 25 that generated the individual detected object position information to the monitoring information integration unit 32.

[0131] Furthermore, each of the above embodiments is a program that causes a monitoring system, which is a computer, to execute the monitoring method according to each of the above embodiments.

[0132] Furthermore, each recording medium according to the above embodiment records a program that causes a monitoring system, which is a computer, to execute the monitoring method according to each above embodiment, and is a recording medium that can read the program from the computer.

[0133] Furthermore, in embodiments 1 and 2, each setting information output unit 23 individually generates a plurality of individual camera position information, each indicating the absolute coordinate position of camera 1 in the target space A. However, the individual camera position information may be generated by a setting information integration unit 31 instead of each setting information output unit 23. In this case, the setting information integration unit 31 pre-stores feature point absolute position information, which indicates the absolute coordinate position of each feature point 5 in the target space A. In this case, the identified feature point related information generated by each feature point identification unit 22 is individually output from each setting information output unit 23 to the setting information integration unit 31 as setting information. Furthermore, in this case, the setting information integration unit 31 individually generates individual camera position information by identifying the absolute coordinate position of each camera 1 in the target space A based on the identified feature point related information generated by each feature point identification unit 22 and the feature point absolute position information.

[0134] Furthermore, if individual camera position information is generated individually by the setting information integration unit 31, individual detected object position information may be generated individually by the monitoring information integration unit 32 instead of each monitoring information output unit 25. In this case, the detected object-related information generated by the monitoring object detection unit 24 is output as monitoring information from the corresponding monitoring information output unit 25 to the monitoring information integration unit 32. In this case, the monitoring information integration unit 32 also obtains camera position integration information from the setting information integration unit 31. Based on the detected object-related information received from the monitoring information output unit 25 and the camera position integration information obtained from the setting information integration unit 31, the monitoring information integration unit 32 individually generates individual detected object position information by identifying the absolute coordinate position of the detected object in the target space A. The monitoring information integration unit 32 generates integrated detected object position information by reflecting the absolute coordinate position of the detected object shown in the individual detected object position information into the common target space A.

[0135] Furthermore, in each of the above embodiments, in the determination step S7 of the monitoring method, it is determined whether or not to generate an alarm by comparing the distance between objects with the set distance. However, in the determination step S7, it may be determined whether or not to generate an alarm based on the motion vector of each detected object and the distance between objects. In this case, the motion vector of each detected object is calculated by the determination unit 33 based on the detected object position integration information continuously generated by the monitoring information integration unit 32. In this case, in the determination step S7, if the motion vectors of each detected object are vectors moving toward each other and the distance between objects is shorter than the set distance, the determination unit 33 determines whether to generate an alarm.

[0136] Furthermore, in each of the above embodiments, newly placed AR markers, rather than existing installations in the target space A, are used as feature points 5. However, as shown in Figure 22, existing installations in the target space A may be used as feature points 5. In Figure 22, an existing sign is used as feature point 5a, an existing manhole cover as feature point 5b, and an existing guardrail as feature point 5c. In this case, absolute position information of feature points, indicating the absolute coordinate positions of each feature point 5a, 5b, and 5c in the target space A, is pre-stored in the setting information integration unit 31. Also, in this case, feature point size information, indicating the actual size of each feature point 5a, 5b, and 5c, is pre-stored in the feature point identification unit 22. Even in this way, the position of each camera 1 in the target space A can be more accurately determined, and when a monitored object is detected as a detected object from the image captured by each camera 1, the absolute coordinate position of the detected object in the target space A can be more accurately determined.

[0137] Furthermore, in embodiments 1 and 3, the shooting ranges 11a and 11b of each camera 1a and 1b are different ranges that do not overlap with each other. However, as shown in Figure 23, the shooting ranges 11a and 11b of each camera 1a and 1b may overlap in some parts.

[0138] Furthermore, in each of the above embodiments, the number of cameras 1 and processing units 2 included in the monitoring system is multiple. However, the number of cameras 1 and processing units 2 included in the monitoring system may be one each. In this case, the shooting range of camera 1 is a part of the target space A. Also in this case, processing unit 2 is provided in correspondence with camera 1. In this case, the camera position identification unit is composed of one setting information output unit 23 in processing unit 2 and a setting information integration unit 31 in management device 3. Also in this case, the detected object position identification unit is composed of one monitoring information output unit 25 in processing unit 2 and a monitoring information integration unit 32 in management device 3. Even if the number of cameras 1 and processing units 2 included in the monitoring system is one each, the position of camera 1 in the target space A can be identified more accurately. Therefore, when a monitored object is detected as a detected object from the image captured by camera 1, the absolute coordinate position of the detected object in the target space A can be identified more accurately. This makes it possible to more reliably identify the position of the monitored object captured in the image captured by camera 1 in the target space.

[0139] Furthermore, the functions of each processing unit 2 and management device 3 according to each of the above embodiments are realized by processing circuits. Figure 24 is a configuration diagram showing a first example of a processing circuit that realizes the functions of each processing unit 2 and management device 3 according to embodiments 1 to 3. The processing circuit 100 in the first example is dedicated hardware.

[0140] Furthermore, the processing circuit 100 may include, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0141] Figure 25 is a configuration diagram showing a second example of a processing circuit that realizes the functions of each processing unit 2 and management device 3 according to Embodiments 1 to 3. The processing circuit 200 of the second example includes a processor 201 and a memory 202.

[0142] In the processing circuit 200, the functions of each processing unit 2 and management device 3 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in memory 202. The processor 201 realizes the functions of each processing unit 2 and management device 3 by reading and executing the programs stored in memory 202.

[0143] A program stored in memory 202 can be said to cause the computer to execute the procedures or methods described above. Here, memory 202 refers to non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable and Programmable Read Only Memory). Magnetic disks, flexible disks, optical disks, compact disks, minidiscs, DVDs, etc., also fall under the category of memory 202.

[0144] Furthermore, some of the functions of each processing unit 2 and management device 3 described above may be implemented using dedicated hardware, while others may be implemented using software or firmware.

[0145] Thus, the processing circuit can realize the above-described functions of each processing unit 2 and management device 3 through hardware, software, firmware, or a combination thereof.

[0146] The configurations shown in the embodiments described above are examples of the content of this disclosure. The embodiments can be combined with other known technologies. Some parts of the configurations of the embodiments can be omitted or modified without departing from the gist of this disclosure.

[0147] Examples of aspects that may be included in this disclosure are listed below as an addendum. (Note 1) A camera installed in a target space where three or more individually identifiable feature points are located, and which generates an image by capturing the shooting range within the target space, A camera control unit that controls the shooting of the aforementioned camera, A feature point identification unit identifies at least three of the feature points captured in the captured image as identification feature points based on the captured image generated by the camera, calculates the distance from the camera to each identification feature point based on feature point size information indicating the actual size of each feature point and the image size of each identification feature point captured in the captured image, and generates identification feature point related information specified by the image position of each identification feature point in the captured image, the orientation of the camera, and the distance from the camera to each identification feature point. The camera position identification unit stores in advance feature point absolute position information indicating the absolute coordinate position of each feature point in the target space, and identifies the absolute coordinate position of the camera in the target space based on the identified feature point related information and the feature point absolute position information. A monitoring object detection unit detects the presence or absence of a target object based on the captured image generated by the camera, and if the target object is detected as a detected object, it calculates the distance from the camera to the detected object based on the captured image in which the detected object is visible, and generates detected object-related information specified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object. A detection object position identification unit identifies the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of the camera in the target space and the detected object-related information. A monitoring system equipped with these features. (Note 2) Multiple cameras are installed in a target space where three or more individually identifiable feature points are located, and each camera generates an image by individually capturing multiple shooting ranges within the target space. A plurality of camera control units that individually control the shooting of each of the aforementioned cameras, A plurality of feature point identification units that, based on the captured image generated by the camera, identify at least three of the feature points captured in the captured image as identification feature points, calculate the distance from the camera to each identification feature point based on feature point size information indicating the actual size of each feature point and the image size of each identification feature point captured in the captured image, and individually generate identification feature point-related information corresponding to each camera, which is specified by the image position of each identification feature point in the captured image, the orientation of the camera, and the distance from the camera to each identification feature point. A camera position identification unit has feature point absolute position information that indicates the absolute coordinate position of each feature point in the target space, and identifies the absolute coordinate position of each camera in the target space based on each identified feature point related information and the feature point absolute position information. A plurality of monitoring object detection units individually detect the presence or absence of a target object for each of the images captured by each camera based on the captured images generated by each camera, and when a target object is detected as a detected object, calculate the distance from the camera to the detected object based on the captured image in which the detected object is visible, and generate detected object-related information that is identified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object. A detection object position identification unit identifies the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of each camera in the target space and the detected object-related information. A monitoring system equipped with these features. (Note 3) The monitoring system according to Appendix 1 or Appendix 2, wherein the camera control unit rotates the camera until the number of identified feature points identified by the feature point identification unit becomes three, if the number of identified feature points identified by the feature point identification unit is two or less based on the captured image generated by the corresponding camera, thereby changing the camera's orientation to a different orientation after rotation from the orientation before rotation, and causing the camera to capture an area different from the camera's shooting range before rotation. (Note 4) The monitoring system according to any one of the appendices 1 to 3, wherein the camera control unit causes the camera to execute a zoom function to enlarge the size of the image of the identification feature point captured in the captured image if the size of the image of the identification feature point captured in the captured image is smaller than a preset reference value. (Note 5) A setting shooting step involves causing a camera installed in a target space, where three or more individually identifiable feature points are located, to capture the shooting range within the target space, thereby causing the camera to generate a captured image. A feature point identification step in which, based on the captured image generated by the camera in the setting shooting step, at least three of the feature points captured in the captured image are identified as identification feature points, and the distance from the camera to each of the identification feature points is calculated based on feature point size information indicating the actual size of each feature point and the image size of each of the identification feature points captured in the captured image, and identification feature point related information is generated which is specified by the image position of each of the identification feature points in the captured image, the orientation of the camera, and the distance from the camera to each of the identification feature points, The absolute coordinate position information of each feature point in the target space is stored in advance, and after the feature point identification step, the absolute coordinate position of the camera in the target space is identified based on the identified feature point related information and the feature point absolute position information. After the camera position identification step, a monitoring shooting step is performed in which the camera is made to capture the shooting range in the target space, thereby generating a captured image on the camera. A monitoring object detection step in which, based on the captured image generated by the camera in the monitoring shooting step, the presence or absence of a target object to be monitored is detected, and if the target object to be monitored is detected as a detected object, the monitoring object detection step generates detected object-related information, which is identified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object, based on the captured image in which the detected object is captured. After the monitoring object detection step, a detection object position identification step is performed to identify the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of the camera in the target space and the detected object-related information. A monitoring method performed by a computer. (Note 6) A setting shooting step involves having multiple cameras, each positioned in a target space where three or more individually identifiable feature points are located, individually capture multiple shooting ranges within the target space, thereby generating captured images for each camera. A feature point identification step in which, based on the captured image generated by the camera in the setting shooting step, at least three of the feature points captured in the captured image are identified as identification feature points, the distance from the camera to each of the identification feature points is calculated based on feature point size information indicating the actual size of each feature point and the image size of each of the identification feature points captured in the captured image, and identification feature point related information is individually generated corresponding to each of the cameras, which is determined by the image position of each of the identification feature points in the captured image, the orientation of the camera, and the distance from the camera to each of the identification feature points. The absolute coordinate position information of each feature point in the target space is stored in advance, and after the feature point identification step, the absolute coordinate position of each camera in the target space is identified based on the identified feature point related information and the feature point absolute position information. After the camera position identification step, a monitoring shooting step is performed in which each camera individually captures multiple shooting ranges in the target space, thereby generating captured images for each camera. A monitoring object detection step in which, based on the captured images generated by each camera in the monitoring shooting step, the presence or absence of a target object to be monitored is individually detected for each captured image from each camera, and if a target object to be monitored is detected as a detected object, the monitoring object detection step generates detected object-related information, which is specified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object, based on the captured image in which the detected object is captured. After the monitoring object detection step, a detection object position identification step is performed to identify the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of each camera in the target space and the detected object-related information. A monitoring method performed by a computer. (Note 7) A program that causes a computer to execute the monitoring method described in Appendix 5 or Appendix 6. (Note 8) A recording medium that contains a program that causes a computer to execute the monitoring method described in Appendix 5 or Appendix 6, and the program is readable by the computer. [Explanation of Symbols]

[0148] 1,1a,1b Camera, 5,5a,5b Feature points (identification feature points), 6a Vehicle (object to be monitored), 6b Person (object to be monitored), 21 Camera control unit, 22 Feature point identification unit, 23 Setting information output unit (camera position identification unit), 24 Monitoring object detection unit, 25 Monitoring information output unit (detected object position identification unit), 31 Setting information integration unit (camera position identification unit), 32 Monitoring information integration unit (detected object position identification unit).

Claims

1. A camera installed in a target space where three or more individually identifiable feature points are located, and which generates an image by capturing the shooting range within the target space, A camera control unit that controls the shooting of the aforementioned camera, A feature point identification unit identifies at least three of the feature points captured in the captured image as identification feature points based on the captured image generated by the camera, calculates the distance from the camera to each identification feature point based on feature point size information indicating the actual size of each feature point and the image size of each identification feature point captured in the captured image, and generates identification feature point related information specified by the image position of each identification feature point in the captured image, the orientation of the camera, and the distance from the camera to each identification feature point. The camera position identification unit stores in advance feature point absolute position information indicating the absolute coordinate position of each feature point in the target space, and identifies the absolute coordinate position of the camera in the target space based on the identified feature point related information and the feature point absolute position information. A monitoring object detection unit detects the presence or absence of a target object based on the captured image generated by the camera, and if the target object is detected as a detected object, it calculates the distance from the camera to the detected object based on the captured image in which the detected object is visible, and generates detected object-related information specified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object. A detection object position identification unit identifies the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of the camera in the target space and the detected object-related information. A monitoring system equipped with these features.

2. Multiple cameras are installed in a target space where three or more individually identifiable feature points are located, and each camera generates an image by individually capturing multiple shooting ranges within the target space. A plurality of camera control units that individually control the shooting of each of the aforementioned cameras, A plurality of feature point identification units that, based on the captured image generated by the camera, identify at least three of the feature points captured in the captured image as identification feature points, calculate the distance from the camera to each identification feature point based on feature point size information indicating the actual size of each feature point and the image size of each identification feature point captured in the captured image, and individually generate identification feature point related information corresponding to each camera, which is specified by the image position of each identification feature point in the captured image, the orientation of the camera, and the distance from the camera to each identification feature point. A camera position identification unit has feature point absolute position information that indicates the absolute coordinate position of each feature point in the target space, and identifies the absolute coordinate position of each camera in the target space based on each identified feature point related information and the feature point absolute position information. A plurality of monitoring object detection units individually detect the presence or absence of a target object for each of the images captured by each camera based on the captured images generated by each camera, and when a target object is detected as a detected object, calculate the distance from the camera to the detected object based on the captured image in which the detected object is visible, and generate detected object-related information that is identified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object. A detection object position identification unit identifies the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of each camera in the target space and the detected object-related information. A monitoring system equipped with these features.

3. The monitoring system according to claim 1 or 2, wherein the camera control unit rotates the camera until the number of identified feature points identified by the feature point identification unit becomes three, if the number of identified feature points identified by the feature point identification unit is two or less based on the captured image generated by the corresponding camera, thereby changing the orientation of the camera to a different orientation after rotation from the orientation before rotation, and causing the camera to capture an area different from the shooting range of the camera before rotation.

4. The monitoring system according to claim 1 or 2, wherein the camera control unit causes the camera to perform a zoom function to enlarge the size of the image of the identification feature point captured in the captured image if the size of the image of the identification feature point captured in the captured image is smaller than a preset reference value.

5. A setting shooting step involves causing a camera installed in a target space, where three or more individually identifiable feature points are located, to capture the shooting range within the target space, thereby generating a captured image in the camera. A feature point identification step in which, based on the captured image generated by the camera in the setting shooting step, at least three of the feature points captured in the captured image are identified as identification feature points, and the distance from the camera to each of the identification feature points is calculated based on feature point size information indicating the actual size of each feature point and the image size of each of the identification feature points captured in the captured image, and identification feature point related information is generated which is specified by the image position of each of the identification feature points in the captured image, the orientation of the camera, and the distance from the camera to each of the identification feature points, The absolute coordinate position information of each feature point in the target space is stored in advance, and after the feature point identification step, the absolute coordinate position of the camera in the target space is identified based on the identified feature point related information and the feature point absolute position information. After the camera position identification step, a monitoring shooting step is performed in which the camera is made to capture the shooting range in the target space, thereby generating a captured image on the camera. A monitoring object detection step in which, based on the captured image generated by the camera in the monitoring shooting step, the presence or absence of a target object to be monitored is detected, and if the target object to be monitored is detected as a detected object, the monitoring object detection step generates detected object-related information, which is identified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object, based on the captured image in which the detected object is captured. After the monitoring object detection step, a detection object position identification step is performed to identify the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of the camera in the target space and the detected object-related information. A monitoring method performed by a computer.

6. A setting shooting step involves having multiple cameras, each positioned in a target space where three or more individually identifiable feature points are located, individually capture multiple shooting ranges within the target space, thereby generating captured images for each camera. A feature point identification step in which, based on the captured image generated by the camera in the setting shooting step, at least three of the feature points captured in the captured image are identified as identification feature points, and the distance from the camera to each of the identification feature points is calculated based on feature point size information indicating the actual size of each feature point and the image size of each of the identification feature points captured in the captured image, and identification feature point related information is individually generated corresponding to each of the cameras, which is determined by the image position of each of the identification feature points in the captured image, the orientation of the camera, and the distance from the camera to each of the identification feature points. The absolute coordinate position information of each feature point in the target space is stored in advance, and after the feature point identification step, the absolute coordinate position of each camera in the target space is identified based on the identified feature point related information and the feature point absolute position information. After the camera position identification step, a monitoring shooting step is performed in which each camera individually captures multiple shooting ranges in the target space, thereby generating captured images for each camera. A monitoring object detection step in which, based on the captured images generated by each camera in the monitoring shooting step, the presence or absence of a target object to be monitored is individually detected for each captured image from each camera, and if a target object to be monitored is detected as a detected object, the monitoring object detection step generates detected object-related information, which is specified by the position of the image of the detected object in the captured image, the orientation of the camera, and the distance from the camera to the detected object, based on the captured image in which the detected object is captured. After the monitoring object detection step, a detection object position identification step is performed to identify the absolute coordinate position of the detected object in the target space based on the absolute coordinate position of each camera in the target space and the detected object-related information. A monitoring method performed by a computer.

7. A program that causes a computer to execute the monitoring method described in claim 5 or claim 6.

8. A recording medium that stores a program that causes a computer to execute the monitoring method described in claim 5 or claim 6, and the program is readable by the computer.

Images