Object tracking device, object tracking method, and program

Abstract

In the present invention, an image acquisition means acquires a wide-angle photographic image photographed by a wide-angle camera. A multi-viewpoint division means performs division and distortion correction of the wide-angle photographing image to generate a plurality of individual-viewpoint images. An object detection means detects tracking target objects from the plurality of individual-viewpoint images. A same-object determination means limits the same objects included in the detected tracking target objects in duplicate to one object. A specification means converts coordinates of the tracking target objects in the coordinate system of the individual-viewpoint images to coordinates in the coordinate system of the wide-angle photographic image, and specifies the converted coordinates as positions of the tracking target objects in the wide-angle photographic image. An object tracking means tracks the tracking target objects on the basis of the wide-angle photographic image and the specified positions of the tracking target objects.

Description

Object tracking device, object tracking method, and recording medium

[0001] The present disclosure relates to techniques for tracking objects in video.

[0002] In recent years, 360-degree cameras have become available at low cost. Because 360-degree cameras can capture a wide range of images, there is a possibility that the number of cameras required can be reduced, and their use in video analysis is being considered. Patent Document 1 discloses a method for detecting objects from omnidirectional images.

[0003] JP 2013-183176 A

[0004] 360° images have significant distortion, making it difficult to apply existing object detection and tracking methods. Furthermore, to enable object detection and tracking from 360° images, training using a dedicated dataset containing 360° images is required.

[0005] One objective of the present disclosure is to enable object tracking from images with distortion, such as 360° images, without requiring training using a dedicated dataset.

[0006] In one aspect of the present disclosure, an object tracking device includes: an image acquisition means for acquiring a wide-angle captured image by a wide-angle camera; a multi-viewpoint division means for dividing and correcting the wide-angle captured image to generate a plurality of individual viewpoint images; an object detection means for detecting a tracked object from the plurality of individual viewpoint images; an identical object determination means for limiting identical objects that are included in multiple detected tracked objects to one; a designation means for converting coordinates of the tracked object in a coordinate system of the individual viewpoint image to coordinates in a coordinate system of the wide-angle captured image and designating the converted coordinates as the position of the tracked object in the wide-angle captured image; and an object tracking means for tracking the tracked object based on the wide-angle captured image and the designated position of the tracked object.

[0007] In another aspect of the present disclosure, an object tracking method executed by a computer includes acquiring a wide-angle captured image using a wide-angle camera, dividing and correcting distortion of the wide-angle captured image to generate a plurality of individual viewpoint images, detecting a tracked object from the plurality of individual viewpoint images, limiting duplicated objects included in the detected tracked objects to one object, converting coordinates of the tracked object in a coordinate system of the individual viewpoint images to coordinates in a coordinate system of the wide-angle captured image, and specifying the converted coordinates as the position of the tracked object in the wide-angle captured image, and tracking the tracked object based on the wide-angle captured image and the specified position of the tracked object.

[0008] In yet another aspect of the present disclosure, a recording medium has recorded thereon a program that causes a computer to execute a process of acquiring a wide-angle captured image by a wide-angle camera, dividing and correcting distortion of the wide-angle captured image to generate a plurality of individual viewpoint images, detecting a tracked object from the plurality of individual viewpoint images, limiting identical objects that are included multiple times among the detected tracked objects to one, converting coordinates of the tracked object in a coordinate system of the individual viewpoint images to coordinates in a coordinate system of the wide-angle captured image, and specifying the converted coordinates as the position of the tracked object in the wide-angle captured image, and tracking the tracked object based on the wide-angle captured image and the specified position of the tracked object.

[0009] 1 shows the concept of an object tracking device according to a first embodiment; FIG. 2 is a block diagram showing the hardware configuration of an object tracking device; FIG. 3 is a block diagram showing the functional configuration of an object tracking device; FIG. 4 is a diagram explaining processing on a 360° image; FIG. 5 is a diagram explaining object detection by an object detection unit; FIG. 6 is a diagram explaining object selection by an identical object determination unit; FIG. 7 is a diagram explaining coordinate transformation by a tracking object designation unit; FIG. 8 is a diagram explaining correction by a distortion correction unit; FIG. 9 is a flowchart of object tracking processing; FIG. 10 is a diagram explaining the angle of an individual viewpoint image, which is a parameter for multi-viewpoint division; FIG. 11 is a configuration when changing a multi-viewpoint division method based on an object detection result; FIG. 12 is a configuration when changing a multi-viewpoint division method based on a tracking result; FIG. 13 is an example of a composite image of individual viewpoint images; FIG. 14 is a diagram explaining an identical object determination method using coordinate transformation; FIG. 15 is a block diagram showing the configuration of an object tracking device according to a second embodiment; FIG. 16 is a flowchart showing processing of an object tracking device according to a second embodiment.

[0010] Preferred embodiments of the present disclosure will now be described with reference to the drawings. First Embodiment Overall Configuration FIG. 1 shows the concept of an object tracking device according to a first embodiment. The object tracking device 100 detects and tracks a predetermined tracked object from a 360-degree image captured by a 360-degree camera. A 360-degree image is input to the object tracking device 100. The 360-degree image is a moving image made up of a plurality of frame images. The object tracking device 100 detects and tracks the tracked object from the input 360-degree image and outputs the tracking result. The tracking result may be, for example, time-series data of the position information of the tracked object in the 360-degree image, or a moving image in which the position of the tracked object is indicated by a rectangle or the like in the 360-degree image.

[0011] 2 is a block diagram showing the hardware configuration of object tracking device 100. As shown in the figure, object tracking device 100 includes an interface (IF) 12, a processor 13, a memory 14, a recording medium 15, a database (DB) 16, a display unit 17, and an input unit 18.

[0012] IF12 acquires 360° images from the 360° camera. Note that if pre-captured 360° images are stored in an image database (hereinafter, the database will be referred to as "DB"), IF12 may acquire the 360° images from the image DB. IF12 also outputs the tracking results of object tracking device 100 to an external device as needed.

[0013] The processor 13 is a computer such as a CPU (Central Processing Unit) and controls the entire object tracking device 100 by executing a pre-prepared program. The processor 13 may be a CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof. The processor 13 executes the object tracking process described below.

[0014] The memory 14 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 14 stores various programs executed by the processor 13. The memory 14 is also used as a working memory while the processor 13 is executing various processes.

[0015] Recording medium 15 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from object tracking device 100. Recording medium 15 records various programs executed by processor 13. When object tracking device 100 executes various processes, the programs recorded on recording medium 15 are loaded into memory 14 and executed by processor 13.

[0016] The DB 16 stores the 360° images input via the IF 12. In addition, the tracking results obtained by the object tracking device 100 are saved in the DB 16 as needed.

[0017] The display unit 17 is configured with, for example, a liquid crystal display, etc. The input unit 18 includes, for example, a keyboard, a mouse, etc. The display unit 17 and the input unit 18 are used, for example, by an operator of the object tracking device 100 to perform necessary operation inputs and to view the object tracking results.

[0018] 3 is a block diagram showing the functional configuration of the object tracking device 100. Functionally, the object tracking device 100 includes an image acquisition unit 21, a multi-viewpoint division unit 22, an object detection unit 23, a same object determination unit 24, a tracked object designation unit 25, a distortion correction unit 26, a tracking unit 27, and an output unit 28.

[0019] The image acquisition unit 21 acquires a 360° image. FIG. 4 illustrates processing of the 360° image. As shown in FIG. 4, the 360° image WI is an image captured by a 360° camera, capturing a predetermined range in the vertical direction of a sphere centered on the camera position at the time of capture over 360° (all directions). If the center direction of the 360° image WI is assumed to be the 0° direction, the left edge of the 360° image WI corresponds to the -180° direction, and the right edge corresponds to the +180° direction, as shown. In other words, the left and right edge regions of the 360° image WI represent image content obtained by dividing a continuous area. The 360° image WI illustrated in FIG. 4 is a captured image of a lakeside, with a pier reflected in the foreground region of the image and the opposite shore of the lake reflected in the background region of the image. For convenience, the tracked object OB in the 360° image WI in FIG. 4 is indicated by a star (★). The image acquisition unit 21 outputs the acquired 360° image to the multi-viewpoint division unit 22 .

[0020] The multi-viewpoint division unit 22 divides the input 360-degree image into individual viewpoint images from multiple different viewpoints. In the example of FIG. 4 , the multi-viewpoint division unit 22 divides the 360-degree image WI into eight individual viewpoint images using eight division lines C1 to C8 shown in the figure. In the example of FIG. 4 , the division lines C1 to C4 divide the upper region of the 360-degree image WI into four viewpoints shifted 90 degrees in the circumferential direction. Similarly, the division lines C5 to C8 divide the lower region of the 360-degree image WI into four viewpoints shifted 90 degrees in the circumferential direction. In addition, when dividing the 360-degree image into multiple individual viewpoint images, the multi-viewpoint division unit 22 corrects distortion in each individual viewpoint image. As a result, multiple individual viewpoint images with different viewpoints and corrected distortion are generated from the 360-degree image WI.

[0021] In the example of Fig. 4, the multi-viewpoint dividing unit 22 divides the 360-degree image WI using dividing lines C1 to C8, as shown in the image WIx in the middle left corner, and corrects the distortion of each image. As a result, individual viewpoint images VI1 to VI8 are obtained, as shown in the lower left corner of Fig. 4.

[0022] When dividing the 360-degree image WI into individual viewpoint images VI1 to VI8 as described above, the multi-viewpoint division unit 22 divides the image so that adjacent divided regions formed by division lines C1 to C8 partially overlap, as shown in Fig. 4. For example, as shown in Fig. 4, the divided region formed by division line C1 and the divided region formed by division line C2 have an overlapping portion OV1 in the horizontal direction of the image. Furthermore, the divided region formed by division line C2 and the divided region formed by division line C7 have an overlapping portion OV2 in the vertical direction of the image. The multi-viewpoint division unit 22 divides the 360-degree image so that each individual viewpoint image VI has an overlapping portion with another adjacent individual viewpoint image VI.

[0023] By providing such overlapping portions, the area near the division boundary in the original 360° image is included in both of the two adjacent individual viewpoint images after division. Therefore, if a tracking target object is located near the division boundary, the object will be included in both of the two adjacent individual viewpoint images and will be subject to detection in each of the individual viewpoint images, increasing the likelihood of correct detection by the object detection unit 23 (described later). In particular, since the two adjacent individual viewpoint images are images captured from different viewpoints, the same object will appear differently in each of the two individual viewpoint images. Therefore, the object detection unit 23 will detect the same object from images with different appearances, thereby increasing the probability and accuracy of detecting the object. The multi-viewpoint division unit 22 outputs the obtained individual viewpoint images VI1 to VI8 to the object detection unit 23.

[0024] The object detection unit 23 detects a tracked object from each of the individual viewpoint images. As described above, each individual viewpoint image is an image in which distortion has been corrected, so that the object can be detected using an object detection model that detects objects from normal images. Note that the object detection unit 23 can be configured using an existing object detection model that uses a neural network.

[0025] FIG. 5 is a diagram illustrating object detection by the object detection unit 23. As shown in FIG. 5, the tracked object OB included in the 360-degree image WI in FIG. 4 is detected as object OB1 from the individual viewpoint image VI4 and as object OB2 from the individual viewpoint image VI8. Note that in the examples of FIGS. 4 and 5, only one object is detected from the 360-degree image WI, but if multiple tracked objects are included in the 360-degree image, each of them will be detected by the object detection unit 23. The object detection unit 23 outputs information about the detected tracked object to the same object determination unit 24. Note that the information about the tracked object includes, for example, position information about a rectangle surrounding the tracked object and information indicating the class of the tracked object.

[0026] The identical object determination unit 24 first determines whether or not multiple identical objects are included among the multiple objects detected by the object detection unit 23. If multiple identical objects are detected among the multiple objects detected by the object detection unit 23, the identical object determination unit 24 selects one of them. That is, if multiple identical objects are detected by the object detection unit 23, the identical object determination unit 24 limits them to one object. Note that the simplest method for the identical object determination unit 24 is to randomly select one object from multiple objects corresponding to the same tracking target. Then, the identical object determination unit 24 outputs the one or more tracking target objects after limiting the multiple identical objects detected to one to the tracking object designation unit 25.

[0027] FIG. 6 is a diagram illustrating a method for selecting an object by the identical object determination unit 24. In the example of FIG. 5, two objects OB1 and OB2 are detected from multiple individual viewpoint images VI1 to VI8. Therefore, the identical object determination unit 24 first calculates the similarity between objects OB1 and OB2. For example, the identical object determination unit 24 obtains feature vectors as images of objects OB1 and OB2 and calculates the distance between these feature vectors as the similarity. If the similarity is greater than a predetermined value, the identical object determination unit 24 determines that the two objects OB1 and OB2 are the same object. The identical object determination unit 24 then selects one of objects OB1 and OB2 and outputs information about that object to the tracked object designation unit 25. Note that if the similarity between the two objects is equal to or less than a predetermined value as a result of the similarity-based determination, the identical object determination unit 24 determines that the two objects are different objects and outputs information about the two objects to the tracked object designation unit 25. The object output by the same object determination unit 24 becomes the target of tracking by the tracking unit 27 at the subsequent stage.

[0028] The tracking object designation unit 25 performs coordinate transformation of the object output from the identical object determination unit 24. FIG. 7 is a diagram illustrating the coordinate transformation by the tracking object designation unit 25. As described above, the object detection unit 23 detects an object from the individual viewpoint image VI, and the coordinates of the object included in the object information output from the identical object determination unit 24 are coordinates in the coordinate system of the individual viewpoint image VI. In the example of FIG. 7, the coordinates of the object OB1 output from the identical object determination unit 24 are coordinates (x ｖ , y ｖ ) of the object OB1 in the coordinate system of the individual viewpoint OB14. ｖ , y ｖ ) into the coordinates (x 360 , y 360 ) to the original 360° image. This designates the tracked object in the original 360° image. Usually, the task of designating the tracked object in the original 360° image is often performed manually, but in this embodiment, by converting the coordinates of the tracked object detected from the individual viewpoint image VI as described above, it is possible to automate the designation of the tracked object in the 360° image. The tracked object designation unit 25 outputs information indicating the tracked object in the converted coordinates, i.e., coordinates in the coordinate system of the 360° image, to the distortion correction unit 26.

[0029] The distortion correction unit 26 corrects distortion in a predetermined range around the tracked object in the original 360° image, and outputs the corrected image to the tracking unit 27. FIG. 8 is a diagram illustrating a correction method performed by the distortion correction unit 26. The distortion correction unit 26 sets an area of ​​a predetermined size around the tracked object OB1 acquired from the tracked object designation unit 25 in the 360° image WI as a correction range SR, and performs distortion correction on the image within the correction range SR. This results in an image in which the distortion around the tracked object in the 360° image has been corrected. The distortion correction unit 26 outputs the distortion-corrected image to the tracking unit 27.

[0030] The tracking unit 27 receives the distortion-corrected image including the tracked object and tracks the tracked object. Since the image input to the tracking unit 27 has already been distortion-corrected by the distortion correction unit 26, the tracking unit 27 can detect and track the object using a normal object detection model and object tracking model. In a preferred example, the tracking unit 27 can use an object detection model and object tracking model using a neural network. The tracking unit 27 outputs the tracking result of the tracked object to the output unit 28.

[0031] The output unit 28 outputs the tracking result received from the tracking unit 27 to an external device, etc. The output unit 28 may store the received tracking result in the DB 16 shown in FIG. 2 or may display it on the display unit 17.

[0032] In the above configuration, the image acquisition unit 21 is an example of an image acquisition means, the multi-viewpoint division unit 22 is an example of a multi-viewpoint division means, the object detection unit 23 is an example of an object detection means, the same object determination unit 24 is an example of a same object determination means, the tracked object designation unit 25 is an example of a designation means, and the tracking unit 27 is an example of an object tracking means.

[0033] [Object Tracking Process] Next, we will explain the object tracking process performed by the object tracking device 100. Fig. 9 is a flowchart of the object tracking process. This process is realized by the processor 13 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3.

[0034] First, the image acquisition unit 21 acquires a 360° image (step S11). Next, the multi-viewpoint division unit 22 divides the 360° image into multiple individual viewpoint images and corrects distortion in each individual viewpoint image (step S12). Next, the object detection unit 23 detects objects from each individual viewpoint image (step S13). Next, if the object detection unit 23 detects multiple identical objects, the identical object determination unit 24 narrows them down to a single object (step S14). Note that if the object detection unit 23 detects no identical objects, the identical object determination unit 24 outputs each object as a tracking target object.

[0035] Next, the tracking object designation unit 25 converts the coordinates of the tracking target object in the individual viewpoint image output from the identical object determination unit 24 into coordinates in the 360° image (step S15). Next, the distortion correction unit 26 corrects the distortion of the image of the area surrounding the tracking target object in the 360° image (step S16). Next, the tracking unit 27 tracks the tracking target object using the distortion-corrected image (step S17). Then, the output unit 28 outputs the tracking result by the tracking unit 27 (step S18).

[0036] The object tracking device 100 performs the above steps S11 to S18 for each frame image of the input 360° image. Then, the object tracking device 100 determines whether or not there is a next frame image (step S19). If there is a next frame image (step S19: Yes), the process returns to step S11. On the other hand, if there is not a next frame image (step S19: No), the object tracking device ends the process.

[0037] [Modifications] Modifications of this embodiment will be described below. The following modifications can be applied in appropriate combination. (Modification 1) In the above embodiment, the multi-viewpoint division unit 22 divides the 360° image into eight individual viewpoint images, but the division method used by the multi-viewpoint division unit 22 can be changed as appropriate by user settings. Specifically, the user can arbitrarily set the following parameters as parameters that define the division method used by the multi-viewpoint division unit 22 to divide the 360° image: Number of divisions (number of individual viewpoint images) Overlap rate of adjacent individual viewpoint images Size of individual viewpoint image Angle of individual viewpoint image Color scheme of individual viewpoint image Selection of the viewpoint to actually process based on differences in appearance depending on the installation location and angle of the camera

[0038] Note that the "angle of the individual viewpoint image" includes rotating the angle of the individual viewpoint image after division by any angle, as shown in Fig. 10. By rotating the angle of the individual viewpoint image in this way, the object detection accuracy by the object detection unit 23 at the subsequent stage may be improved.

[0039] (Variation 2) The multi-viewpoint division unit 22 may change the division method based on the processing result of a later stage in the object tracking process. In one example, as shown in FIG. 11 , the multi-viewpoint division unit 22 may change the division method based on the object detection result by the object detection unit 23. In this case, the object detection unit 23 inputs information 31 such as the object detection rate to the multi-viewpoint division unit 22. For example, if the number of individual viewpoint images is too large and the object detection rate by the object detection unit 23 is low, the multi-viewpoint division unit 22 may change the division method to increase the object detection rate by reducing the number of individual viewpoint images or increasing the size of the individual viewpoint images.

[0040] 12 , the multi-viewpoint division unit 22 may change the division method based on the tracking result by the tracking unit 27. In this case, the tracking unit 27 inputs information 32 such as the tracking success rate to the multi-viewpoint division unit 22. For example, if the number of individual viewpoint images is too large and the tracking success rate by the tracking unit 27 is low, the multi-viewpoint division unit 22 may change the division method to increase the tracking success rate by reducing the number of individual viewpoint images or increasing the size of the individual viewpoint images.

[0041] (Variation 3) In the above embodiment, the object detection unit 23 detects objects from multiple individual viewpoint images VI. Alternatively, the object detection unit 23 may synthesize multiple individual viewpoint images VI to generate a single composite image and detect objects from the composite image. FIG. 13 shows an example of a composite image. As shown in the figure, the object detection unit 23 generates a composite image SI by synthesizing the individual viewpoint images VI1 to VI8 and detects objects OB1 and OB2 from the composite image SI. In this way, by detecting objects from a single composite image SI, it is possible to reduce the calculation cost and calculation time required for object detection.

[0042] (Variation 4) In the above embodiment, when the object detection unit 23 detects multiple identical objects, the identical object determination unit 24 randomly selects one of the objects. Instead, the identical object detection unit 24 may compare the rectangular sizes of the multiple detected objects and select the object with the largest rectangular size. Alternatively, the identical object detection unit 24 may compare the reliability scores of the multiple detected objects and select the object with the highest reliability score.

[0043] (Variation 5) In the above embodiment, the identical object determination unit 24 determines whether multiple objects are the same object based on the similarity of the objects detected by the object detection unit 23. This method is called a "method of determining whether an object is the same object based on similarity." Alternatively, the identical object determination unit 24 may determine whether an object is the same object by converting the coordinates of each object into the coordinate system of the 360° image. This method is called a "method of determining whether an object is the same object based on coordinate transformation."

[0044] FIG. 14 is a diagram illustrating a method for determining whether an object is the same using coordinate transformation. As shown in FIG. 14 , assume that the object detection unit 23 detects two objects OB3 and OB4. In this case, the coordinates of the objects OB3 and OB4 are indicated by coordinates in the coordinate systems of the individual viewpoint images VI10 and VI11. The identical object determination unit 24 converts the coordinates of the objects OB3 and OB4 in the individual viewpoint images VI10 and VI11 into coordinates in the coordinate system of the 360° image WI. If the objects OB3 and OB4 are the same tracking target object, the coordinates of the objects OB3 and OB4 should be close to each other after the coordinate transformation. Therefore, if the positions of the objects OB3 and OB4 after the coordinate transformation are within a predetermined distance, the identical object determination unit 24 determines that the objects OB3 and OB4 are the same tracking target object. On the other hand, if the positions of objects OB3 and OB4 after the coordinate transformation are not within a predetermined distance, the same object determination unit 24 determines that objects OB3 and OB4 are different objects to be tracked. Note that the same object determination unit 24 may use the above-mentioned method of determining the same object by coordinate transformation instead of the above-mentioned method of determining the same object by similarity, or may use it in combination with the method of determining the same object by similarity.

[0045] (Variation 6) In the above embodiment, the image input to object tracking device 100 is a 360° image, but the application of the present disclosure is not limited to this. The method of the present disclosure is applicable to object tracking from various captured images that include image distortion depending on the shape of the lens, etc. Specifically, the method of the present disclosure is applicable to images captured using a wide-angle lens in general that have image distortion, such as images captured using a fisheye lens (fisheye format image) and dual fisheye format image, in addition to 360° images (equirectangular format image or panoramic image).

[0046] 15 is a block diagram showing the configuration of an object tracking device according to the second embodiment. An object tracking device 70 according to the second embodiment includes an image acquisition unit 71, a multi-viewpoint division unit 72, an object detection unit 73, a same object determination unit 74, a designation unit 75, and an object tracking unit 76.

[0047] FIG. 16 is a flowchart of processing by the object tracking device 70 according to the second embodiment. The image acquisition unit 71 acquires a wide-angle image captured by a wide-angle camera (step S71). The multi-viewpoint division unit 72 divides the wide-angle image and corrects distortion to generate multiple individual viewpoint images (step S72). The object detection unit 73 detects a tracked object from the multiple individual viewpoint images (step S73). The identical object determination unit 74 limits the number of identical objects contained in the detected tracked objects to one (step S74). The designation unit 75 converts the coordinates of the tracked object in the coordinate system of the individual viewpoint image to coordinates in the coordinate system of the wide-angle image and designates the converted coordinates as the position of the tracked object in the wide-angle image (step S75). The object tracking unit 76 tracks the tracked object based on the wide-angle image and the position of the designated tracked object (step S76).

[0048] According to the object tracking device 70 of the second embodiment, it is possible to track an object from an image having distortion, such as an image taken by a wide-angle camera, without requiring learning using a dedicated data set.

[0049] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0050] (Supplementary Note 1) An object tracking device comprising: image acquisition means for acquiring a wide-angle photographed image by a wide-angle camera; multi-viewpoint division means for dividing and correcting distortion of the wide-angle photographed image to generate a plurality of individual viewpoint images; object detection means for detecting a tracked object from the plurality of individual viewpoint images; identical object determination means for limiting identical objects that are included in multiple detected tracked objects to one; designation means for converting coordinates of the tracked object in a coordinate system of the individual viewpoint images to coordinates in a coordinate system of the wide-angle photographed image, and designating the converted coordinates as the position of the tracked object in the wide-angle photographed image; and object tracking means for tracking the tracked object based on the wide-angle photographed image and the designated position of the tracked object.

[0051] (Supplementary Note 2) The object tracking device according to Supplementary Note 1, wherein the object tracking means comprises: a distortion correction means for performing distortion correction on a predetermined range including the position of the designated object to be tracked in the wide-angle captured image; and a tracking means for tracking the object to be tracked using an image obtained by the distortion correction and outputting a tracking result.

[0052] (Supplementary Note 3) The object tracking device according to Supplementary Note 1, wherein the multi-viewpoint dividing means divides the wide-angle captured image so that each of the individual viewpoint images has an overlapping portion with an adjacent individual viewpoint image.

[0053] (Supplementary Note 4) The object tracking device according to Supplementary Note 1, wherein the same object determination means determines that the plurality of detected objects are the same object when a similarity between the plurality of detected objects is equal to or greater than a predetermined value.

[0054] (Appendix 5) The object tracking device described in Appendix 1, wherein the same object determination means converts the coordinates of the detected multiple objects in the coordinate system of the individual viewpoint image into coordinates in the coordinate system of the wide-angle captured image, and determines that the multiple objects are the same object if the converted coordinates are closer than a predetermined distance.

[0055] (Supplementary Note 6) The object tracking device according to Supplementary Note 1, wherein the identical object determination means limits the overlapping identical objects to an object having a maximum rectangular size among the detected objects.

[0056] (Supplementary Note 7) The object tracking device according to Supplementary Note 1, wherein the identical object determination means limits the tracking target objects of the overlapping identical objects to an object having the highest reliability of the detected object.

[0057] (Supplementary Note 8) The object tracking device according to Supplementary Note 1, wherein the object detection means generates a composite image by synthesizing the plurality of individual viewpoint images, and detects the object from the composite image.

[0058] (Supplementary Note 9) An object tracking method executed by a computer, comprising: acquiring a wide-angle photographed image by a wide-angle camera; dividing and correcting distortion of the wide-angle photographed image to generate a plurality of individual viewpoint images; detecting a tracked object from the plurality of individual viewpoint images; limiting duplicated objects included in the detected tracked objects to one; converting coordinates of the tracked object in a coordinate system of the individual viewpoint images to coordinates in a coordinate system of the wide-angle photographed image; and specifying the converted coordinates as the position of the tracked object in the wide-angle photographed image; and tracking the tracked object based on the wide-angle photographed image and the specified position of the tracked object.

[0059] (Supplementary Note 10) A recording medium having recorded thereon a program that causes a computer to execute a process of acquiring a wide-angle photographed image by a wide-angle camera, dividing and correcting distortion of the wide-angle photographed image to generate a plurality of individual viewpoint images, detecting a tracked object from the plurality of individual viewpoint images, limiting identical objects that are included in the detected tracked objects to one, converting coordinates of the tracked object in a coordinate system of the individual viewpoint images to coordinates in a coordinate system of the wide-angle photographed image, and specifying the converted coordinates as the position of the tracked object in the wide-angle photographed image, and tracking the tracked object based on the wide-angle photographed image and the specified position of the tracked object.

[0060] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0061] 13 Processor 21 Image acquisition unit 22 Multi-viewpoint division unit 23 Object detection unit 24 Identical object determination unit 25 Tracked object designation unit 26 Distortion correction unit 27 Tracking unit 28 Output unit 100 Object tracking device

Claims

1. image acquisition means for acquiring a wide-angle photographed image by a wide-angle camera; A multi-viewpoint division method is performed by dividing the wide-angle photographed image and correcting distortion to generate a plurality of individual viewpoint images. and an object detection means for detecting a tracked object from the plurality of individual viewpoint images; A method for determining the same object that is included in the detected tracked objects is used to limit the number of identical objects to one. Step by step, The coordinates of the object to be tracked in the coordinate system of the individual viewpoint image are calculated by dividing the coordinates of the object to be tracked in the wide-angle photographed image by the coordinates of the object to be tracked in the coordinate system of the individual viewpoint image. The coordinates after the transformation are converted into coordinates in the reference frame, and the coordinates after the transformation are used to calculate the coordinates of the tracked object in the wide-angle photographed image. A designation means for designating the position of Based on the wide-angle photographed image and the position of the designated object to be tracked, an object tracking means for tracking an object; An object tracking device comprising:

2. The object tracking means In the wide-angle photographed image, distortion in a predetermined range including the position of the specified tracking target object is detected. distortion correction means for performing correction; The image obtained by the distortion correction is used to track the object to be tracked, and a tracking result is output. a tracking means for The object tracking device of claim 1 .

3. The multi-viewpoint division means divides the individual viewpoint images into adjacent individual viewpoint images.

2. The object tracking device according to claim 1, wherein the wide-angle captured image is divided into a plurality of portions.

4. The same object determination means determines whether the degree of similarity between the detected plurality of objects is equal to or greater than a predetermined value.

2. The object tracking device according to claim 1, wherein the plurality of objects are determined to be the same object.

5. The same object determination means determines whether the detected plurality of objects are identical in the coordinate system of the individual viewpoint images. The coordinates are converted into coordinates in the coordinate system of the wide-angle photographed image, and the converted coordinates are within a predetermined distance.

2. The object tracking device according to claim 1, wherein the plurality of objects are determined to be the same object if they are close to each other.

6. The identical object determination means determines whether the overlapping identical objects are included in a rectangular area of ​​the detected object.

2. The object tracking device according to claim 1, wherein the object is limited to the object with the largest size.

7. The identical object determination means determines whether the overlapping identical objects are included in a single image by determining the reliability of the detected objects.

2. The object tracking device according to claim 1, wherein the object is limited to an object for which the maximum value is

8. The object detection means generates a composite image by synthesizing the plurality of individual viewpoint images, and The object tracking device according to claim 1, wherein the object is detected from an image.

9. 1. A computer-implemented method for tracking an object, comprising: Obtain wide-angle images using a wide-angle camera, Dividing and correcting the distortion of the wide-angle captured image to generate a plurality of individual viewpoint images; Detecting a tracked object from the plurality of individual viewpoint images; The number of identical objects included in the detected tracking target objects is limited to one. The coordinates of the object to be tracked in the coordinate system of the individual viewpoint image are calculated by dividing the coordinates of the object to be tracked in the wide-angle photographed image by the coordinates of the object to be tracked in the coordinate system of the individual viewpoint image. The coordinates after the transformation are converted into coordinates in the reference frame, and the coordinates after the transformation are used to calculate the coordinates of the tracked object in the wide-angle photographed image. Specify it as the position of Based on the wide-angle photographed image and the position of the designated object to be tracked, An object tracking method for tracking an object.

10. Obtain wide-angle images using a wide-angle camera, Dividing and correcting the distortion of the wide-angle captured image to generate a plurality of individual viewpoint images; Detecting a tracked object from the plurality of individual viewpoint images; The number of identical objects included in the detected tracking target objects is limited to one. The coordinates of the object to be tracked in the coordinate system of the individual viewpoint image are calculated by dividing the coordinates of the object to be tracked in the wide-angle photographed image by the coordinates of the object to be tracked in the coordinate system of the individual viewpoint image. The coordinates after the transformation are converted into coordinates in the reference frame, and the coordinates after the transformation are used to calculate the coordinates of the tracked object in the wide-angle photographed image. Specify it as the position of Based on the wide-angle photographed image and the position of the designated object to be tracked, A program that causes a computer to perform the process of tracking an object.