Moving trajectory information processing device

Abstract

This movement trajectory information processing device comprises a calculation means and a conversion means. The calculation means calculates movement trajectory information, which is a time series of information pertaining to the position of a moving object in an image, from a plurality of images in which the object is continuously captured by a camera for capturing the image within an imaging range. The conversion means calculates a function for mapping from the image to a reference image, converts the information pertaining to the position of the object in the image to information pertaining to the position of the object in the reference image by using the calculated mapping function, and converts the information pertaining to the position of the object in the reference image that was obtained through conversion to information pertaining to the position of the object in a global coordinate system using camera calibration information.

Description

Technical Field

[0001] The present invention relates to a moving trajectory information processing apparatus, a moving trajectory information processing method, and a recording medium.

Background Art

[0002] An example of a conventional apparatus for calculating moving trajectory information is described in Patent Document 1. The conventional apparatus continuously photographs an object such as a moving person using a camera with fixed pan value, tilt value, and zoom value (PTZ values). Further, at each time of photographing, the conventional apparatus extracts the position of the object from the image obtained by photographing with the camera, and converts the coordinate value of the object on the image into the coordinate value in the real space using camera calibration information (internal parameters and external parameters). The conventional apparatus uses the time series of the positions in the real space of the object obtained as described above as the moving trajectory information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a camera with a fixed PTZ value, the field of view of the camera is limited. Therefore, for the purpose of monitoring a wider range, it is important to calculate the movement trajectory information of an object from a plurality of images continuously captured by the camera while capturing the object to be moved within the imaging range by changing the PTZ value. However, when the PTZ value of the camera changes, the camera calibration information of that camera changes. Also, in order to obtain the camera calibration information corresponding to the changed PTZ value, a calibration operation by an operator is required. Therefore, in a conventional device that directly converts the coordinate value of an object on an image into the coordinate value in the real space using the camera calibration information from each of a plurality of continuously captured images, it has been difficult to calculate the movement trajectory information from a plurality of images continuously captured by the camera that captures the object to be moved within the imaging range.

[0005] The present invention aims to provide a movement trajectory information processing device that solves the above-described problems.

Means for Solving the Problems

[0006] A movement trajectory information processing device according to an aspect of the present invention includes a calculation means for calculating movement trajectory information, which is a time series of position information of an object on an image, from a plurality of images continuously captured by a camera that captures an object to be moved within an imaging range; a conversion means for calculating a mapping function from the image to a reference image, calculating the position information on the reference image corresponding to the position information of the object on the image using the calculated mapping function, and converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information; and is configured to include.

[0007] A movement trajectory information processing method according to another aspect of the present invention calculates movement trajectory information, which is a time series of position information of an object on an image, from a plurality of images continuously captured by a camera that captures an object to be moved within an imaging range, Calculate a mapping function from the image to the reference image, calculate the position information on the reference image corresponding to the position information on the image of the object using the calculated mapping function, and convert the position information on the reference image obtained by the calculation into position information in the world coordinate system using camera calibration information. It is configured as follows.

[0008] A computer-readable recording medium according to another aspect of the present invention causes a computer to calculate movement trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously captured by a camera that captures a moving object within a shooting range, calculate a mapping function from the image to the reference image, calculate the position information on the reference image corresponding to the position information on the image of the object using the calculated mapping function, and convert the position information on the reference image obtained by the calculation into position information in the world coordinate system using camera calibration information, and is configured to record a program for causing the above operations.

Advantages of the Invention

[0009] With the configuration described above, the present invention can calculate movement trajectory information, which is a time series of the position of the object in the real space, from a plurality of images continuously captured by a camera that captures a moving object within a shooting range.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Mode for Carrying Out the Invention

[0011] Next, embodiments of the present invention will be described in detail with reference to the drawings. In the description in the specification, although descriptions having similar meanings to "tracking" and "tailgating" coexist, "tracking" is mainly used for convenience to explain the configuration in the part with a strong relationship with people, and "tailgating" is mainly used for convenience to explain the configuration in the part with a strong relationship with devices. [First Embodiment] First, for easy understanding, the problems assumed in the first embodiment of the present invention will be described.

[0012] Detecting people from the images of surveillance cameras and performing tracking is useful for crime prevention, marketing, etc. In a camera with fixed PTZ values, the field of view of the camera is limited. By changing the PTZ values of the camera, the field of view of the camera can be expanded. However, even then, there is a limit to the field of view of a single camera. Therefore, consider a system in which multiple PTZ cameras are installed in a distributed manner in the surveillance area with a part of the camera fields of view in common to track people over a wider range spatially and temporally. In such a tracking system, it is preferable that the system can automatically recognize that a person being traced by one camera has entered the field of view of another camera, and that the other camera can start tracing that person. To determine the person to take over the tracing from among many people present in the camera's field of view, the movement trajectory of the person can be utilized. That is, the tracking target is determined by collating the information on the movement trajectory of the person being traced by one camera (tracking target movement trajectory information) with the information on the movement trajectory of the people within the field of view of the other camera (collated movement trajectory information). To perform such collation, it is necessary that the tracking target movement trajectory information and the collated movement trajectory information be expressed in the same coordinate system.

[0013] Generally, for the coordinate systems representing the positions of people obtained by photographing with cameras, there are an image coordinate system and a camera coordinate system that are specific to each camera, and a world coordinate system that is common to multiple cameras. The image coordinate system is a two-dimensional coordinate system on the imaging device, and the positions of points on the image are usually represented in this image coordinate system. The camera coordinate system is a coordinate system determined by the image coordinate system and the internal parameters of the camera. Therefore, the image coordinate system of the camera and its camera coordinate system can be mutually converted using the internal parameters of that camera. The world coordinate system is a coordinate system determined by the camera calibration information composed of the internal parameters and external parameters of the camera and the image coordinate system. Therefore, the image coordinate system of the camera and the world coordinate system can be mutually converted using the camera calibration information of that camera.

[0014] The conventional device extracts the position of a person from an image obtained by shooting, and uses camera calibration information to convert the coordinate values of the person's image coordinate system in the image into coordinate values in the world coordinate system. However, when the PTZ values of the camera are changed to track a person, the camera calibration information changes. Also, it is difficult to obtain the camera calibration information corresponding to the changed PTZ values in real time. Therefore, it has been difficult for the conventional device to calculate the movement trajectory information while tracking an object with a camera having a tracking function that captures a moving object within the shooting range. As a result, it has been difficult to collate the movement trajectories of a person in real time among a plurality of PTZ cameras.

[0015] This embodiment solves the above problems. Hereinafter, a tracking system 10 according to an embodiment to which the present invention is applied will be described in detail with reference to the drawings.

[0016] FIG. 1 is a schematic diagram showing a configuration example of a tracking system 10 according to an embodiment of the present invention. Referring to FIG. 1, the tracking system 10 is a system that detects and tracks a person within a monitoring area, and includes two PTZ cameras 11 and 12 and a control device 20.

[0017] PTZ cameras (hereinafter simply referred to as cameras) 11 and 12 include a solid-state imaging device such as a CMOS sensor or a CCD sensor and a pan-tilt unit. Cameras 11 and 12 generate a plurality of images by continuously photographing a photographing range at a certain period, for example, in accordance with a command from the control device 20. It is preferable that the photographing periods and photographing timings of camera 11 and camera 12 are the same or similar, but they may be different. Cameras 11 and 12 can change the photographing range by changing pan, tilt, and zoom in accordance with a command from the control device 20. Here, pan means swinging the direction of the camera left and right, tilt means swinging the direction of the camera up and down, and zoom means changing the angle of view to telephoto or wide angle, respectively. Cameras 11 and 12 are installed dispersedly in the monitoring area so that a part of their photographing ranges overlaps. In the example shown in FIG. 1, camera 11 monitors monitoring areas 13A, 13B, and 13C set on a passage extending in the left-right direction of the paper surface. Also, camera 12 monitors monitoring area 13C and monitoring area 13D set on a passage extending in the up-down direction of the paper surface. Monitoring area 13C corresponds to the corner part of the passage. That is, the area of the corner of the passage is the common monitoring area 13C for the two cameras 11 and 12. In addition, on the passage, it is assumed that a person enters from the left side direction of the paper surface, passes through the corner of the passage, and exits from the lower side direction of the paper surface. Also, the monitoring areas 13A to 13D are flat surfaces.

[0018] The control device 20 is connected to the cameras 11 and 12 either by wire or wirelessly. The control device 20 calculates the camera calibration information (internal parameters and external parameters) of each camera when the cameras 11 and 12 are installed. The calculation of this camera calibration information is performed by setting the PTZ values of the cameras 11 and 12 to the reference PTZ of each camera. For example, the reference PTZ of the camera 11 is set so as to have a camera field of view capable of photographing a person in the monitoring area 13A. Also, the reference PTZ of the camera 12 is set so as to have a camera field of view capable of photographing a person in the monitoring area 13C. At the start of the operation of the system, the control device 20 monitors and displays on the screen display unit the images continuously captured by the camera 11 fixed to the reference PTZ. When the operator designates a person to be tracked on the monitored image, the control device 20 tracks the designated person while changing the PTZ value of the camera 11. Then, the control device 20 calculates in real time the information on the movement trajectory of the tracked person (tracking target movement trajectory information) in the world coordinate system.

[0019] On the other hand, at the start of the operation of the system, the control device 20 calculates in real time in the world coordinate system the information on the movement trajectories of all the persons in the monitoring area 13C (collated movement trajectory information) based on the images continuously captured by the camera 12 fixed to the reference PTZ. Then, the control device 20 collates in real time the tracking target movement trajectory information of the person being tracked by the camera 11 and the collated movement trajectory information of all the persons shown in the camera 12. Thereby, the control device 20 determines which person among the persons shown in the camera 12 is the person being tracked by the camera 11. Next, the control device 20 continues to track with the camera 12 while capturing the determined person within the shooting range of the camera 12, and monitors and displays the image on the screen display unit.

[0020] The above is the outline of the tracking system 10. Subsequently, the control device 20 will be described in detail.

[0021] FIG. 2 is a block diagram showing an example of the control device 20. Referring to FIG. 2, the control device 20 includes a communication I / F (interface) unit 21, an operation input unit 22, a screen display unit 23, a storage unit 24, and an arithmetic processing unit 25.

[0022] The communication I / F unit 21 is composed of a data communication circuit and is configured to perform data communication with the cameras 11 and 12 and other external devices (not shown) by wire or wirelessly. The operation input unit 22 is composed of an operation input device such as a keyboard or a mouse, and is configured to detect an operator's operation and output it to the arithmetic processing unit 25. The screen display unit 23 is composed of a display device such as an LCD (Liquid Crystal Display), and is configured to display images obtained by photographing with the cameras 11 and 12.

[0023] The storage unit 24 is composed of one or more storage devices of one type or multiple types such as a hard disk or a memory, and is configured to store processing information and a program 241 necessary for various processes in the arithmetic processing unit 25. The program 241 is a program that realizes various processing units when read and executed by the arithmetic processing unit 25, and is pre-read from an external device (not shown) or a recording medium via a data input / output function such as the communication I / F unit 21 and stored in the storage unit 24. The main processing information stored in the storage unit 24 includes camera calibration information 242-1 and 242-2, reference images 243-1 and 243-2, image DB (database) 244-1 and 244-2, tracking target movement trajectory information 245, and collated target movement trajectory information 246.

[0024] The camera calibration information 242-1 is the camera calibration information of the camera 11 (internal parameters and external parameters of the camera 11). The camera calibration information 242-2 is the camera calibration information of the camera 12 (internal parameters and external parameters of the camera 12).

[0025] The reference image 243-1 is an image of the monitoring area 13A taken by the camera 11 set to the pan value, tilt value, and zoom value when obtaining the camera calibration information 242-1. The reference image 243-1 may include, in addition to the image, the pan value, tilt value, and zoom value (the reference PTZ value of the camera 11) of the camera 11 at the time of taking the reference image, and the camera position (the position of the camera 11 in the world coordinate system). Further, the reference image 243-2 is an image of the monitoring area 13C taken by the camera 12 set to the pan value, tilt value, and zoom value when obtaining the camera calibration information 242-2. The reference image 243-2 may include, in addition to the image, the pan value, tilt value, and zoom value (the reference PTZ value of the camera 12) of the camera 12 at the time of taking the reference image, and the camera position (the position of the camera 12 in the world coordinate system).

[0026] The image DB 244-1 accumulates the time series of images taken by the camera 11. The camera ID of the camera 11, the shooting time, and the PTZ value at the time of shooting are added to each image accumulated in the image DB 244-1. The image DB 244-2 accumulates the time series of images taken by the camera 12. The camera ID of the camera 12, the shooting time, and the PTZ value at the time of shooting are added to each image accumulated in the image DB 244-2.

[0027] The tracking target movement trajectory information 245 is information regarding the movement trajectory of the tracking target person calculated based on the images taken by the camera 11. FIG. 3 is a diagram showing a configuration example of the tracking target movement trajectory information 245. The tracking target movement trajectory information 245 in this example is composed of an entry 2451 composed of a tracking target ID that uniquely identifies the tracking target, the camera ID of the camera 11, and association information, and a plurality of entries 2452 that correspond one-to-one to a plurality of images continuously taken by the camera 11. The plurality of entries 2452 are connected in a row in the order of shooting time by the association information in the entry 2451 and the association information in the entry 2452.

[0028] Each entry 2452 is composed of a shooting time, a person area, position information (image coordinate system), image features, position information (world coordinate system), a movement vector, a movement speed, an acceleration, and association information. The shooting time represents the shooting time of the corresponding image. The person area represents, for example, the circumscribed rectangle of the person area in the corresponding image. The position information (image coordinate system) represents the coordinate values in the image coordinate system of a single point representing the person area in the same entry. The single point representing the person area may be, for example, the center of gravity of the person area, but is not limited thereto, and may be the center of gravity of the face, the feet, etc. The image features represent the feature amounts of the image extracted from the person area in the corresponding image. Examples of the image features include, but are not limited to, the feature amounts of the face, the feature amounts of the clothing, and the size of the person. The position information (world coordinate system) is set with the coordinate values obtained by converting the position information (image coordinate system) in the same entry from the image coordinate system to the world coordinate system. The movement vector represents the amount and direction of movement of the person to be tracked between the corresponding image and the adjacent image. The movement speed represents the movement speed of the person to be tracked. The acceleration represents the acceleration of the person to be tracked. The movement vector, the movement speed, and the acceleration are calculated, for example, based on the position information (world coordinate system) in the entry 2452 of the corresponding image and the adjacent image. The types of data included in the entry 2452 are an example and are not limited to the above.

[0029] The collated movement trajectory information 246 is information regarding the movement trajectory of a person extracted from an image captured by the camera 12. The collated movement trajectory information 246 exists for each person extracted from the image captured by the camera 12. FIG. 4 is a diagram showing a configuration example of the collated movement trajectory information 246. The collated movement trajectory information 246 in this example is composed of an entry 2461 constituted by a person ID that uniquely identifies a person, a camera ID of the camera 12, and association information, and a plurality of entries 2462 that correspond one-to-one to a plurality of images continuously captured by the camera 12. The plurality of entries 2462 are connected in a row in the order of shooting time by the association information in the entry 2461 and the association information in the entry 2462. Each individual entry 2462 is composed of the same data as the entry 2452 of the tracking target movement trajectory information 245 described with reference to FIG. 3. The types of data included in the entry 2462 are an example and are not limited to the above.

[0030] The arithmetic processing unit 25 has one or more microprocessors such as an MPU and its peripheral circuits, and is configured to realize various processing units by reading and executing the program 241 from the storage unit 24 so as to cooperate the above hardware and the program 241. The main processing units realized by the arithmetic processing unit 25 include a camera calibration unit 251 and two monitoring units 252 and 253.

[0031] The camera calibration unit 251 is configured to acquire the camera calibration information of the cameras 11 and 12 through interactive processing with an operator via the operation input unit 22 and the screen display unit 23. The camera calibration method to be used is not particularly limited.

[0032] The monitoring unit 252 is configured to detect and track a person moving within the monitoring areas 13A to 13C using the camera 11. The monitoring unit 252 includes a detection unit 2521, a tracking unit 2522, and a coordinate conversion unit 2523.

[0033] The detection unit 2521 is configured to detect a specific person as a tracking target from among all the persons present in the monitoring area 13A based on the images continuously captured by the camera 11 with the reference PTZ value set thereto.

[0034] The tracking unit 2522 is configured to track the tracking target detected by the detection unit 2521 with the camera 11 so as to capture the tracking target within the shooting range, calculate information on the movement trajectory of the tracking target in the image coordinate system of the camera 11, and store the information as tracking target movement trajectory information 245 in the storage unit 24. Since the tracking unit 2522 changes the PTZ value of the camera 11 to track the tracking target, even if the tracking target moves from the monitoring area 13A to the monitoring area 13B or the monitoring area 13C, the tracking target can be captured within the shooting range.

[0035] The coordinate conversion unit 2523 is configured to convert the tracking target movement trajectory information 245 calculated by the tracking unit 2522 from the image coordinate system of the camera 11 to the world coordinate system. That is, the coordinate conversion unit 2523 converts the position information on the image from the image coordinate system to the world coordinate system. Here, the coordinate conversion unit 2523 may use a plane (common map) where z = 0 (i.e., height 0) as the world coordinate system. Compared with the configuration of converting a point on the image coordinate system to a three-dimensional world coordinate system, the conversion to a two-dimensional world coordinate system reduces the amount of calculation and error. However, the coordinate conversion unit 2523 may be configured to convert a point in the image coordinate system to a three-dimensional world coordinate system.

[0036] FIG. 5 is a schematic diagram for explaining an example of a method of converting the position information on the image from the image coordinate system to the world coordinate system. In FIG. 5, the image I0 is an image captured by the camera 11 set to the same PTZ value as when the camera calibration information 242-1 was calculated. The images I1, I2, and I3 are a plurality of images of the camera 11 continuously captured while changing the PTZ value after the image I0 was captured.

[0037] The coordinate conversion unit 2523 converts the position information on the image I0 from the image coordinate system to the world coordinate system using the camera calibration information 242-1.

[0038] Also, in the conversion of the position information on the image I1 to the world coordinate system, the coordinate conversion unit 2523 first converts the position information on the image I1 into the corresponding position information on the image I0 using the mapping function f1. Next, the coordinate conversion unit 2523 converts the position information on the image I0 obtained by the above conversion from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Also, the coordinate conversion unit 2523 calculates the above mapping function f1 as the planar projective transformation matrix H1 from the image I1 to the image I0. That is, f1 = H1. Also, in the calculation of the planar projective transformation matrix H1, the coordinate conversion unit 2523 obtains the corresponding point group between the image I1 and the image I0 using a known feature point extraction method, and calculates the planar projective transformation matrix H1 from the obtained corresponding point group. The corresponding point group between the image I1 and the image I0 is, in other words, a pair of a plurality of corresponding feature points between the image I1 and the image I0.

[0039] Also, in the conversion of the position information on the image I2 to the world coordinate system, the coordinate conversion unit 2523 first converts the position information on the image I2 into the corresponding position information on the image I0 using the mapping function f2, and then converts the position information on the image I0 obtained by this conversion from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Also, the coordinate conversion unit 2523 obtains the above mapping function f2 from the mapping function f1 and the planar projective transformation matrix H2 from the image I2 to the image I1. That is, f2 = f1 * H2. Also, the coordinate conversion unit 2523 obtains the planar projective transformation matrix H2 by using a known feature point extraction method to obtain the corresponding point group between the image I2 and the image I1, and calculates the planar projective transformation matrix H2 from the obtained corresponding point group.

[0040] Further, when converting the position information on the image I3 into the world coordinate system, the coordinate conversion unit 2523 first converts the position information on the image I3 into the corresponding position information on the image I0 using the mapping function f3, and then converts the position information on the image I0 obtained by this conversion from the image coordinate system into the world coordinate system using the camera calibration information 242-1. Also, the coordinate conversion unit 2523 obtains the above mapping function f3 from the mapping function f2 and the planar projective transformation matrix H3 from the image I3 to the image I2. That is, f3 = f2 * H3. Further, the coordinate conversion unit 2523 obtains the above planar projective transformation matrix H3 using a known feature point extraction method for the corresponding point group between the image I3 and the image I2, and calculates the planar projective transformation matrix H3 from the obtained corresponding point group.

[0041] The method for obtaining the mapping function is not limited to the above. For example, the coordinate conversion unit 2523 uses the mapping function f3, the mapping function f1, and the planar projective transformation matrix H from the image I3 to the image I1 31 to calculate it as f3 = f1 * H 31 Alternatively, the coordinate conversion unit 2523 uses the mapping function f3 and the planar projective transformation matrix H from the image I3 to the image I0 30 to calculate it as f3 = H 30 However, although the common area between adjacent images continuously captured by the camera 11 while tracking to capture the moving person within the shooting range is relatively large, the common area between images tends to become smaller as the time interval increases. It becomes difficult to obtain the corresponding point group between the two images with a small common area. Also, even if there is a common area, it may not be possible to accurately obtain the corresponding point group when the difference between the two images is large. Therefore, it is desirable that the shooting time interval between the two images for obtaining the planar projective transformation matrix is determined in consideration of the above circumstances.

[0042] Referring to FIG. 2 again, the monitoring unit 253 is configured to detect the person being tracked by the camera 11 from among the people in the monitoring area 13C using the camera 12, and track the detected person in the monitoring areas 13C and 13D. The monitoring unit 253 includes a collation unit 2531, a tracking unit 2532, and a coordinate conversion unit 2533.

[0043] The matching unit 2531 is configured to calculate, from a plurality of images of the monitoring area 13C continuously captured by the camera 12 with the reference PTZ values set, information on the movement trajectory (the movement trajectory information 246 to be matched) for each person moving within the monitoring area 13C in the image coordinate system of the camera 12.

[0044] The coordinate conversion unit 2533 is configured to convert the movement trajectory information 246 to be matched calculated by the matching unit 2531 from the image coordinate system of the camera 12 to the world coordinate system using the camera calibration information 242-2. Here, the coordinate conversion unit 2533 uses, as the world coordinate system in the same manner as the coordinate conversion unit 2523, a plane (common map) where z = 0 (that is, height 0).

[0045] Further, the matching unit 2531 is configured to match the movement trajectory information 246 to be matched converted to the world coordinate system with the movement trajectory information 245 of the person being tracked by the monitoring unit 252. Also, the matching unit 2531 is configured to determine, based on the result of the matching, which person among the persons moving within the monitoring area 13C captured by the camera 12 is the person being tracked by the camera 11.

[0046] An example of a method for matching the movement trajectory information 245 of the person to be tracked and the movement trajectory information 246 to be matched in the matching unit 2531 will be described.

[0047] First, the matching unit 2531 targets the latest N frames of each trajectory for matching. Here, N is a predetermined value of 2 or more determined in advance. Therefore, the matching unit 2531 extracts N entries 2452 in sequence from the last entry 2452 of the tracking target movement trajectory information 245, and uses these N extracted entries 2452 as the movement trajectory information of the tracking target. If there are not N entries 2452 in the tracking target movement trajectory information 245, the matching at that time is not performed. On the other hand, the matching unit 2531 extracts N entries 2462 in sequence from the last entry 2462 of each of one or more movement trajectory information 246 to be matched, and uses these N extracted entries 2462 as the movement trajectory information to be matched. The matching unit 2531 does not perform matching at that time for the movement trajectory information 246 to be matched that does not have N entries 2462. If there is not even one movement trajectory information 246 to be matched that has N entries 2462, the matching at that time is not performed.

[0048] Next, the matching unit 2531 calculates the matching between the movement trajectory information of the tracking target and the movement trajectory information of each individual movement trajectory to be matched based on the shape matching degree and the direction matching degree.

[0049] In calculating the shape matching degree, the matching unit 2531 first normalizes each movement trajectory with the center of gravity to eliminate the influence of the absolute position. Here, the movement trajectory of the tracking target is formed by connecting the position information (world coordinate system) in the N entries 2452 that make up the movement trajectory information of the tracking target with line segments in the order of shooting time. Also, the movement trajectory to be matched is formed by connecting the position information (world coordinate system) in the N entries 2462 that make up the movement trajectory information to be matched with line segments in the order of shooting time. Next, the matching unit 2531 obtains the error between the positions at the same time of the movement trajectories of the tracking target and the movement trajectory to be matched using the least squares method or the like, and calculates a value that becomes higher as the error is smaller as the shape matching degree.

[0050] Also, in calculating the direction matching degree, the matching unit 2531 calculates a value that becomes higher as the total error between the movement vectors at the same time of the movement trajectories of the tracking target and the movement trajectory to be matched is smaller as the direction matching degree.

[0051] Next, the matching unit 2531 calculates a matching result from the shape matching degree and the orientation matching degree. For example, the matching unit 2531 may use, as the matching result, a value obtained by multiplying the shape matching degree and the orientation matching degree. Alternatively, the matching unit 2531 may use, for example, a value obtained by adding the shape matching degree and the orientation matching degree as the matching result.

[0052] The above is an example of a method for the matching unit 2531 to match the tracking target movement trajectory information 245 and the movement trajectory information 246 to be matched. However, the matching method is not limited to the above. For example, the matching unit 2531 may match the movement trajectory information of both parties in consideration of the image features, movement speed, and acceleration included in the entry 2452 of the tracking target movement trajectory information 245 and the entry 2462 of the movement trajectory information 246 to be matched.

[0053] Based on the result of matching the tracking target movement trajectory information 245 and one or more pieces of movement trajectory information 246 to be matched as described above, the matching unit 2531 is configured to determine which person among the people moving within the monitoring area 13C photographed by the camera 12 is the person being tracked by the camera 11. For example, if the highest matching result is equal to or greater than a certain threshold value, the matching unit 2531 determines that the person related to the movement trajectory information 246 with the highest matching result is the person being tracked by the camera 11. On the other hand, for example, if the highest matching result is not equal to or greater than a certain threshold value, the matching unit 2531 determines that the person being tracked by the camera 11 does not exist among the people photographed by the camera 12.

[0054] The tracking unit 2532 is configured to track with the camera 12 so as to capture the tracking target (the person being tracked by the camera 11 shown in the camera 12) within the shooting range. Since the tracking unit 2532 changes the PTZ value of the camera 12 to track the tracking target, even if the tracking target moves from the monitoring area 14C to the monitoring area 13D, the tracking target can be captured within the shooting range.

[0055] Next, the operation of the control device 20 will be described. First, the camera calibration performed before system operation will be described.

[0056] The camera calibration unit 251 of the control device 20 calibrates the cameras 11 and 12 at an arbitrary time point before system operation, such as when the cameras 11 and 12 are installed.

[0057] In the calibration of camera 11, the camera calibration unit 251 first adjusts the PTZ values of camera 11 so that the entire monitoring area 13A can be photographed. Next, the camera calibration unit 251 calculates the camera calibration information of camera 11 using a predetermined camera calibration method through interactive processing with the operator via the operation input unit 22 and the screen display unit 23. The camera calibration unit 251 stores the calculated camera calibration information in the storage unit 24 as camera calibration information 242-1. The camera calibration unit 251 also acquires the PTZ value of camera 11 at the time of calibration as the reference PTZ value. Furthermore, the camera calibration unit 251 acquires the image obtained by photographing area 13A with camera 11 set to the reference PTZ value as the reference image. The camera calibration unit 251 adds the reference PTZ value to the acquired reference image and stores it in the storage unit 24 as reference image 243-1.

[0058] In the calibration of camera 12, the camera calibration unit 251 first adjusts the PTZ values of camera 12 so that the entire monitoring area 13C (the common area with camera 11) can be photographed. Next, the camera calibration unit 251 calculates the camera calibration information of camera 12 using a predetermined camera calibration method through interactive processing with the operator via the operation input unit 22 and the screen display unit 23. The camera calibration unit 251 stores the calculated camera calibration information in the storage unit 24 as camera calibration information 242-2. The camera calibration unit 251 also acquires the PTZ value of camera 12 at the time of calibration as the reference PTZ value. Furthermore, the camera calibration unit 251 acquires the image obtained by photographing area 15 with camera 12 set to the reference PTZ value as the reference image. The camera calibration unit 251 adds the reference PTZ value to the acquired reference image and stores it in the storage unit 24 as reference image 243-2.

[0059] Next, the operation of the control device 20 during system operation will be described. FIG. 6 is a flowchart showing an example of the tracking process performed using the camera 11. Hereinafter, with reference to FIG. 6, the tracking process performed by the control device 20 using the camera 11 will be described.

[0060] First, the detection unit 2521 in the monitoring unit 252 of the control device 20 performs initialization (step S11). In this initialization, the detection unit 2521 sets the camera 11 to the reference PTZ value. In the initialization, the detection unit 2521 further clears all entries of the tracking target movement trajectory information 245. Next, the detection unit 2521 acquires an image obtained by photographing the monitoring area 13A with the camera 11 set to the reference PTZ value, stores it in the image DB 244-1, and displays it on the screen display unit 23 for monitoring (step S12). Next, the detection unit 2521 detects all persons from the image saved this time using various methods such as pattern recognition and machine learning (step S13). Next, the detection unit 2521 displays the detection result on the screen display unit 23 (step S14). For example, the detection unit 2521 generates an image in which the circumscribed rectangle for each person detected from the image is superimposed on the image and displays it on the screen display unit 23. Next, the detection unit 2521 determines whether a person to be tracked has been specified from the operation input unit 22 (step S15). The operator can specify the person to be tracked, for example, by performing an operation such as clicking on the rectangle of the person on the image of the camera 11 displayed on the screen display unit 23. If the person to be tracked is not specified, the detection unit 2521 returns to step S12 and repeats the same process as the above-described process. On the other hand, when the person to be tracked is specified, the detection unit 2521 updates the tracking target movement trajectory information 245 (step S16).

[0061] In the update of the tracking target movement trajectory information 245 in step S16, the detection unit 2521 sets the ID assigned to the tracking target person designated by the operator and the camera ID of the camera 11 in the entry 2451. Further, the detection unit 2521 secures one empty entry 2452, and associates the secured entry 2452 and the entry 2451 with each other by setting association information for the secured entry 2452 and the entry 2451. Next, the detection unit 2521 focuses on the secured entry 2452, and sets the shooting time, the person area, the position information (image coordinate system), and the image features in the entry 2452 being focused on, and sets the position information (world coordinate system), the movement vector, the movement speed, and the acceleration to NULL values.

[0062] Next, the coordinate conversion unit 2523 of the monitoring unit 252 performs coordinate conversion on the entry 2452 being focused on in the tracking target movement trajectory information 245 (step S17). In this step S17, the coordinate conversion unit 2523 first uses the camera calibration information 242-1 to convert the position information (image coordinate system) set in the entry 2452 being focused on from the image coordinate system to the world coordinate system. Next, the coordinate conversion unit 2523 sets the position information (world coordinate system) obtained by the above conversion in the entry 25452 being focused on.

[0063] Next, the tracking unit 2522 of the monitoring unit 252 performs tracking control (step S18). For example, the tracking unit 2522 generates a command for controlling the PTZ values of the camera 11 according to the position information (image coordinate system) set in the entry 2452 being focused on, and transmits it to the camera 11 through the communication I / F unit 21. At this time, for example, the tracking unit 2522 may adjust the pan and tilt so that the center of gravity of the circumscribed rectangle of the tracking target person represented by the position information (image coordinate system) of the entry 2452 being focused on is displayed at the center of the image, and adjust the zoom so that the entire circumscribed rectangle is within a predetermined viewing angle. The camera 11 changes the imaging range by changing the pan, tilt, and zoom in response to the above command.

[0064] Next, the trailing unit 2522 acquires the image captured by the camera 11 set to the changed PTZ value, stores it in the image DB 244-1, and monitors and displays it on the screen display unit 23 (step S19). Next, the trailing unit 2522 detects the person to be tracked from the image saved this time using various methods such as pattern recognition and machine learning (step S20), generates an image with the circumscribed rectangle of the detected person superimposed on the image, and displays it on the screen display unit 23 (step S21). As a result, the operator can confirm the tracking status of the specified person in real time on the image displayed on the screen display unit 23.

[0065] Next, the trailing unit 2522 updates the tracking target movement trajectory information 245 (step S22). In updating the tracking target movement trajectory information 245 in step S22, the trailing unit 2522 first secures one empty entry 2452, sets association information for the secured empty entry 2452 and the entry being focused on 2452, and then moves the focus to the secured empty entry 2452. Next, the trailing unit 2522 sets the shooting time, the person area, the position information (image coordinate system), and the image features in the newly focused entry 2452, and sets the position information (world coordinate system), the movement vector, the movement speed, and the acceleration to NULL values.

[0066] Next, the coordinate conversion unit 2523 of the monitoring unit 252 converts the position information (image coordinate system) set in the entry 2452 of interest from the image coordinate system to the world coordinate system (step S23). The image corresponding to the entry 2452 of interest is an image captured by the camera 11 set to a PTZ value different from the reference image. Therefore, as described with reference to FIG. 5, the coordinate conversion unit 2523 first calculates a mapping function for converting the position information (image coordinate system) set in the entry 2452 of interest to the position information on the reference image, and uses the calculated mapping function to convert the position information (image coordinate system) to the position information on the reference image. Next, the coordinate conversion unit 2523 converts the position information on the reference image obtained by the conversion from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Also, in step S23, the coordinate conversion unit 2523 calculates a movement vector, a movement speed, and an acceleration based on the position information (world coordinate system) of the entry 2452 of interest and the previous entry 2452, and sets them in the entry 2452 of interest.

[0067] Next, the tracking unit 2522 of the monitoring unit 252 determines whether to end the tracking (step S24). For example, when the tracking unit 2522 detects that the person to be tracked has disappeared from the monitoring areas 13A, 13B, and 13C, it may determine to end the tracking. Alternatively, when the tracking unit 2522 detects that the person to be tracked has become untrackable by the camera 11, it may end the tracking. If the tracking unit 2522 does not determine to end the tracking, it returns to step S18 and repeats the same processing as described above. As a result, the tracking of the person to be tracked by the camera 11 continues, and accordingly, the tracking target movement trajectory information 245 is further updated.

[0068] When the trailing part 2522 determines that the trailing has ended, it determines whether to end the monitoring (step S25). For example, when a monitoring end command is input from the operator through the operation input unit 22, the trailing part 2522 determines to end the monitoring. If it is not determined to end the monitoring, the monitoring unit 252 returns to step S11 and repeats the same processing as described above. If it is determined to end the monitoring, the monitoring unit 252 ends the processing shown in FIG. 6.

[0069] FIG. 7 is a flowchart showing an example of the tracking process performed using the camera 12. Hereinafter, with reference to FIG. 7, the tracking process performed by the control device 20 using the camera 12 will be described.

[0070] First, the collating unit 2531 in the monitoring unit 253 of the control device 20 performs initialization (step S31). In this initialization, the collating unit 2531 sets the camera 12 to the reference PTZ value. Further, in the initialization, the collating unit 2531 also clears all entries of the collated movement trajectory information 246. Next, the collating unit 2531 acquires an image of the monitoring area 13C taken by the camera 12 set to the reference PTZ value, saves it in the image DB 244-2, and displays it on the screen display unit 23 for monitoring (step S32). Next, the collating unit 2531 detects all persons from the image saved this time using various methods such as pattern recognition and machine learning (step S33). Next, the collating unit 2531 displays the detection result on the screen display unit 23 (step S34).

[0071] Next, the collating unit 2531 updates the collated movement trajectory information 246 based on the person detection result (step S35). Since all the persons detected from the first acquired image of the camera 12 after initialization are the persons detected for the first time, in step S35, the collating unit 2531 assigns one piece of collated movement trajectory information 246 to each detected person, and performs the following processing for each of the assigned collated movement trajectory information 246.

[0072] First, the matching unit 2531 sets the person ID assigned to the person detected in the entry 2461 of the collated movement trajectory information 246 and the camera ID of the camera 12. Next, the matching unit 2531 secures one empty entry 2462, sets information for mutually associating the secured empty entry 2462 and the entry 2461, and then focuses on the secured empty entry 2462. Next, the matching unit 2531 sets the shooting time, person area, position information (image coordinate system), and image features in the entry 2462 being focused on, and sets the position information (world coordinate system), movement vector, movement speed, and acceleration to NULL values.

[0073] Next, the coordinate conversion unit 2533 of the monitoring unit 253 performs the following processing on the entry 2462 being focused on (step S36). First, the coordinate conversion unit 2533 uses the camera calibration information 442-2 to convert the position information (image coordinate system) set in the entry 2462 being focused on from the image coordinate system to the world coordinate system. Next, the coordinate conversion unit 2533 sets the position information in the world coordinate system obtained by the above conversion as the position information (world coordinate system) of the entry 2462 being focused on.

[0074] Next, the matching unit 2531 of the monitoring unit 253 collates the tracking target movement trajectory information 245 and one or more collated movement trajectory information 246 (step S37). The matching unit 2531 determines, by this collation, which of the persons moving within the monitoring area 13C photographed by the camera 12 is the person being tracked by the camera 11. Next, the matching unit 2531 determines whether the detection of the person being tracked by the camera 11 was successful (step S38). If the matching unit 2531 fails in the above detection, it returns to step S32 and repeats the same processing as described above. On the other hand, if the matching unit 2531 succeeds in the above detection, it transmits the person ID of the person determined to be the person being tracked by the camera 11 to the tracking unit 2532.

[0075] The trailing part 2532 focuses on the last entry 2462 of the collated movement trajectory information 246 in which the person ID received by the collation part 2531 and the like is set in the entry 2461, and tracks the person to be tracked using the camera 12. For example, the trailing part 2532 generates a command for controlling the PTZ value of the camera 12 according to the position information (image coordinate system) set in the entry 2462 being noted, and transmits it to the camera 12 through the communication I / F part 21. At this time, for example, the trailing part 2532 adjusts the pan and tilt so that the center of gravity of the circumscribed rectangle of the person to be tracked represented by the position information (image coordinate system) of the entry 2462 being noted is displayed at the center of the image, and adjusts the zoom so that the entire circumscribed rectangle is within a predetermined viewing angle. The camera 12 changes the imaging range by changing the pan, tilt, and zoom in response to the above command. Also, the trailing part 2532 acquires the image taken by the camera 12 after the PTZ change in step S39, and performs processing such as detection of the person to be tracked and monitor display on the screen display part 23.

[0076] Furthermore, the trailing part 2532 determines whether to end the tracking (step S40). For example, when the trailing part 2532 detects that the person to be tracked no longer exists in the monitoring areas 13C and 13D, it may determine to end the tracking. Alternatively, when the trailing part 2532 detects that the person to be tracked has become in a state where it cannot be tracked by the camera 12, it may end the tracking. If the trailing part 2532 does not determine to end the tracking, it returns to step S38 and repeats the same processing as described above. As a result, the tracking of the person to be tracked by the camera 12 continues. If the trailing part 2532 determines to end the tracking, it determines whether to end the monitoring (step S41). For example, when the trailing part 2532 receives a monitoring end command from the operator through the operation input part 22, it determines to end the monitoring. If it is not determined to end the monitoring, the monitoring part 253 returns to step S31 and repeats the same processing as described above. If it is determined to end the monitoring, the monitoring part 253 ends the processing shown in FIG. 7.

[0077] According to this embodiment, the tracking unit 2522 acquires the position information (image coordinate system) of the person to be tracked from a plurality of images continuously captured while tracking with the camera 11 having a tracking function of capturing the moving person to be tracked within the shooting range, and calculates each entry 2452 of the tracking target movement trajectory information 245. Further, the coordinate conversion unit 2523 calculates a mapping function from the image corresponding to each entry 2452 obtained by shooting with the camera 11 to the reference image, and uses the calculated mapping function to convert the position information (image coordinate system) of each entry 2452 of the tracking target movement trajectory information 245 into position information on the reference image, and converts the position information on the reference image obtained by this conversion from the image coordinate system to the world coordinate system using the camera calibration information 242-1. Unlike the camera calibration information, the above mapping function can be calculated without manual intervention. Therefore, according to this embodiment, it is possible to calculate the movement trajectory information, which is the time series of the position of the person in the real space, while tracking the person with a PTZ camera having a tracking function of capturing the moving person within the shooting range. As a result, the movement trajectories of the person can be collated in real time among a plurality of PTZ cameras, and automatic collation of the moving object between the PTZ cameras becomes possible.

[0078] Subsequently, a modified example of this embodiment will be described.

[0079] The coordinate conversion unit 2523 may calculate a planar projective transformation matrix from the image obtained by shooting with the camera 11 to the reference image based on the PTZ value of the camera 11 and the reference PTZ value when the image was shot.

[0080] A computer may be mounted on the camera 11, and all or part of the functions of the monitoring unit 252 may be mounted on the computer. Similarly, a computer may be mounted on the camera 12, and all or part of the functions of the monitoring unit 253 may be mounted on the computer. Further, the operation input unit 22, the screen display unit 23, and the storage unit 24 may be mounted on a computer connected to the camera 11 and the camera 12 through a network.

[0081] The monitoring area may be a place other than a passageway, for example, a store, a factory, a platform of a station, a ground, a stadium, etc.

[0082] The object to be tracked may be a moving object other than a person, for example, an animal, a car, a walking robot, etc.

[0083] Cameras sharing a part of the camera field of view are not limited to two cameras, camera 11 and camera 12, and there may be three or more cameras.

[0084] [Second Embodiment] Next, a second embodiment of the present invention will be described in detail with reference to the drawings.

[0085] Referring to FIG. 8, the moving trajectory information processing apparatus 30 according to this embodiment includes a calculating means 31 and a converting means 32.

[0086] The calculating means 31 is configured to calculate moving trajectory information, which is a time series of position information of an object on an image, from a plurality of images in which the object is continuously photographed by a camera that captures the moving object within the shooting range. The calculating means 31 can be configured in the same manner as, for example, the tracking unit 2522 in FIG. 2, but is not limited thereto.

[0087] The converting means 32 calculates a mapping function from the above image to a reference image, calculates position information on the reference image corresponding to the position information of the object on the image using the calculated mapping function, and converts the obtained position information on the reference image into position information in the world coordinate system using camera calibration information. The converting means 32 can be configured in the same manner as, for example, the coordinate conversion unit 2523 in FIG. 2, but is not limited thereto.

[0088] The moving trajectory information processing apparatus 30 configured as described above operates as follows. That is, the calculating means 31 calculates moving trajectory information, which is a time series of the position information of the object on the image, from a plurality of images continuously captured by a camera that captures the moving object within the imaging range. Next, the conversion means 32 calculates a mapping function from the above image to the reference image, calculates the position information on the reference image corresponding to the position information of the object on the image using the calculated mapping function, and converts the obtained position information on the reference image into position information in the world coordinate system using the camera calibration information.

[0089] According to the moving trajectory information processing apparatus 30 configured and operating as described above, it is possible to calculate moving trajectory information, which is a time series of the position of the object in the real space, from a plurality of images continuously captured by a camera that captures the moving object within the imaging range. The reason is that the above mapping function can be calculated without manual intervention, unlike the camera calibration information.

[0090] Although the present invention has been described with reference to the above embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0091] For example, the moving trajectory of the object calculated according to the present invention may be used for purposes other than monitoring and tracking, such as for personal authentication or detection of abnormal behavior based on the moving trajectory. Further, the camera used in the present invention does not necessarily have to have a tracking function. For example, the present invention is also applicable when the imaging person manually changes the posture of the camera so as to capture the moving object within the imaging range and calculates the moving trajectory information of the object from a plurality of images continuously captured while tracking the object.

Industrial Applicability

[0092] It can be used in a system that detects and tracks an object from a plurality of images continuously captured by a camera that captures a moving object such as a PTZ camera within the imaging range.

[0093] Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto. [Appendix 1] Calculating means for calculating movement trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously captured by a camera that captures a moving object within a shooting range; Converting means for calculating a mapping function from the image to a reference image, calculating position information on the reference image corresponding to the position information on the image of the object using the calculated mapping function, and converting the obtained position information on the reference image into position information in a world coordinate system using camera calibration information; A movement trajectory information processing apparatus comprising the above. [Appendix 2] The conversion means calculates a planar projective transformation matrix from the image to the reference image, and calculates the mapping function using the calculated planar projective transformation matrix. The movement trajectory information processing apparatus according to Appendix 1. [Appendix 3] The movement trajectory information processing apparatus further comprising collating means for collating the movement trajectory information with other movement trajectory information. The movement trajectory information processing apparatus according to Appendix 1 or 2. [Appendix 4] The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the camera. The movement trajectory information processing apparatus according to Appendix 3. [Appendix 5] The camera is a PTZ camera. The movement trajectory information processing apparatus according to any one of Appendices 1 to 4. [Appendix 6] Calculating movement trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously captured by a camera that captures a moving object within a shooting range; Calculate a mapping function from the said image to a reference image, calculate the position information on the reference image corresponding to the position information on the image of the said object using the calculated mapping function, and convert the calculated position information on the reference image into position information in a world coordinate system using camera calibration information. Moving trajectory information processing method. [Appendix 7] In the said conversion, calculate a planar projective transformation matrix from the said image to the said reference image, and calculate the mapping function using the calculated planar projective transformation matrix. The moving trajectory information processing method according to Appendix 6. [Appendix 8] Furthermore, collate the said moving trajectory information with other moving trajectory information. The moving trajectory information processing method according to Appendix 6 or 7. [Appendix 9] The said other moving trajectory information is moving trajectory information calculated from a plurality of images continuously captured by another camera installed so that a part of the camera field of view overlaps with the said camera. The moving trajectory information processing method according to Appendix 8. [Appendix 10] On a computer, A process of calculating moving trajectory information, which is a time series of position information on the image of the said object, from a plurality of images continuously captured by a camera that captures a moving object within a shooting range, A process of calculating a mapping function from the said image to a reference image, calculating the position information on the reference image corresponding to the position information on the image of the said object using the calculated mapping function, and converting the calculated position information on the reference image into position information in a world coordinate system using camera calibration information, A computer-readable recording medium storing a program for causing the above processes to be performed.

Explanation of Signs

[0094] 10 Tracking system 11, 12 PTZ camera 13A~13D Monitoring area 20 Control device 21 Communication I / F unit 22 Operation input unit 23 Screen display unit 24 Memory unit 25 Arithmetic processing unit 30 Moving trajectory information processing device 31 Calculation means 32 Conversion means 241 Program 242-1, 242-2 Camera calibration information 243-1, 243-2 Reference images 244-1, 244-2 Image DB 245 Tracking target moving trajectory information 246 Collated moving trajectory information 251 Camera calibration section 252, 253 Monitoring section 2521 Detection section 2522, 2532 Tracking sections 2523, 2533 Coordinate conversion sections 2531 Collation section

Claims

1. Calculation means for calculating movement trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously captured by a PTZ camera that captures a moving object within a shooting range; Using, as a reference image, an image of the shooting range captured by the PTZ camera set to the pan value, tilt value, and zoom value when camera calibration information was obtained, calculating a mapping function from the image to the reference image, and using the calculated mapping function to calculate position information on the reference image corresponding to the position information on the image of the object, and converting the calculated position information on the reference image into position information in a world coordinate system using the camera calibration information; Conversion means; A movement trajectory information processing device comprising:

2. The conversion means calculates a planar projective transformation matrix from the image to the reference image, and calculates the mapping function using the calculated planar projective transformation matrix. The movement trajectory information processing device according to claim 1.

3. Further comprising collating means for collating the movement trajectory information with other movement trajectory information. The movement trajectory information processing device according to claim 1 or 2.

4. The other movement trajectory information is movement trajectory information calculated from a plurality of images continuously captured by another PTZ camera installed so that a part of the camera field of view overlaps with the PTZ camera. The movement trajectory information processing device according to claim 3.

5. Calculating movement trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously captured by a PTZ camera that captures a moving object within a shooting range; Using, as a reference image, an image of the shooting range captured by the PTZ camera set to the pan value, tilt value, and zoom value when camera calibration information was obtained, calculating a mapping function from the image to the reference image, and using the calculated mapping function to calculate position information on the reference image corresponding to the position information on the image of the object, and converting the calculated position information on the reference image into position information in a world coordinate system using the camera calibration information; A movement trajectory information processing method.

6. In the conversion, a planar projective transformation matrix from the image to the reference image is calculated, and the mapping function is calculated using the calculated planar projective transformation matrix. The movement trajectory information processing method according to claim 5.

7. Furthermore, the movement trajectory information is collated with other movement trajectory information. The method for processing moving trajectory information according to claim 5 or 6.

8. The other moving trajectory information is moving trajectory information calculated from a plurality of images continuously photographed by another PTZ camera installed so that a part of the camera field of view overlaps with the PTZ camera. The method for processing moving trajectory information according to claim 7.

9. On a computer, A process of calculating moving trajectory information, which is a time series of position information on the image of the object, from a plurality of images continuously photographed by a PTZ camera that captures the moving object within the shooting range; Using the image of the shooting range photographed by the PTZ camera set to the pan value, tilt value, and zoom value when obtaining the camera calibration information as a reference image, calculating a mapping function from the image to the reference image, and using the calculated mapping function Calculating the position information on the reference image corresponding to the position information on the image of the object, and converting the position information on the obtained reference image into position information in the world coordinate system using the camera calibration information; A program for causing the above to be performed.

Images