Imaging control device, imaging device, imaging control method, and program

Abstract

To provide an imaging control device capable of continuous tracking of a subject to be tracked even when the features on the image of the subject to be tracked change.SOLUTION: The imaging control device is configured to control each detector to perform a series of processing to detect coordinate that indicates a region of a part concerning multiple parts of the subject from a captured image and control each correction unit to calculate corrected coordinates of a corrected part according to preset magnification and relative position from the coordinates indicating the area of each detected part. A tracking coordinate generator is configured to generate tracking coordinates by using corrected coordinates of multiple parts calculated by each correction unit, where the distance between the regions of the parts is within the threshold with respect to the multiple parts, and to control the imaging area using the generated tracking coordinates.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to tracking of a subject. [Background technology]

[0002] In a device that detects a subject by image analysis and tracks the subject with a camera, there is a technique for improving detection accuracy by detecting multiple parts in combination, since the detection of only one specific part may not be able to detect the part depending on the situation and tracking of the subject may fail. Patent Document 1 proposes a technique for registering image features of the face and surrounding area of ​​the subject in advance, and judging and tracking the part that matches the previously registered features as the subject. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-201387 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology disclosed in Patent Document 1, if the orientation or posture of the subject changes from the time when the image features of the face and surrounding area are registered, the image features also change, which can cause feature comparison to fail, leading to failure to detect the subject and to tracking the subject.The present invention aims to make it possible to continue tracking the subject to be tracked even if the image features of the subject to be tracked change due to a change in the orientation of the subject, etc. [Means for solving the problem]

[0005] The imaging control device of the present invention is characterized in having a plurality of detection means for detecting coordinates indicating the area of ​​a plurality of parts of a subject from an image captured by an imaging means, a correction means for calculating corrected coordinates of the parts corrected according to a preset magnification and relative position from the coordinates indicating the detected area of ​​the parts, a tracking coordinate generation means for generating tracking coordinates using the corrected coordinates of the plurality of parts calculated by the correction means, where the distance between the areas of the parts is within a threshold value, and a control means for controlling the imaging range of the imaging means using the generated tracking coordinates. Effect of the Invention

[0006] According to the present invention, it is possible to continue tracking a subject to be tracked even if the characteristics of the subject in an image to be tracked change. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of the configuration of an imaging system according to a first embodiment. [Diagram 2] 2 is a diagram illustrating an example of the hardware configuration of a tracking device according to the first embodiment. FIG. [Figure 3A] 5 is a flowchart showing an example of an imaging control process according to the first embodiment. [Figure 3B] 5 is a flowchart showing an example of an imaging control process according to the first embodiment. [Figure 4] 5A to 5C are diagrams illustrating calculation of corrected coordinates according to the first embodiment. [Diagram 5] 4A to 4C are diagrams illustrating generation of tracking coordinates according to the first embodiment. [Figure 6] FIG. 11 is a diagram illustrating an example of the configuration of an imaging system according to a second embodiment. [Figure 7A] 10 is a flowchart showing an example of an imaging control process according to the second embodiment. [Figure 7B] 10 is a flowchart showing an example of an imaging control process according to the second embodiment. [Figure 8] 13A to 13C are diagrams illustrating generation of tracking coordinates according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the configurations shown in the embodiments described below are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0009] (First embodiment) FIG. 1 is a block diagram showing an example of a functional configuration of an imaging system 100A including a tracking device 150A as an imaging control device according to the first embodiment. The imaging system 100A is a system that tracks a specific subject designated by a user and automatically captures the image. The imaging system 100A performs pan-tilt control so that the designated tracking target subject is captured at an appropriate position in the angle of view (imaging range), tracks the subject, and records the image. In addition, the imaging system 100A detects multiple parts of the subject by video analysis to identify the position of the tracking target subject, and if any one of the multiple parts is detected, tracks the subject based on the detected part, thereby improving the continuity of tracking. In the following, a case will be described as an example in which the tracking target subject is a person and three parts of the person's face, upper body, and whole body are detected, but the tracking target subject and the parts to be detected are not limited to this.

[0010] The imaging system 100A has a video input device 110, a pan / tilt drive device 130, a tracking device 150A, and a recording device 170. The tracking device 150A and the recording device 170 are connected via a video interface. The video input device 110 is an example of an imaging means that captures images and generates image data, and the tracking device 150A is an example of an imaging control device that controls the imaging means (video input device 110).

[0011] Video input device 110 is a device that captures the surroundings and generates a captured video (image), and is configured with a camera or the like. Video input device 110 can change the imaging range, and can change the imaging direction, for example, by performing at least one of a panning operation and a tilting operation by pan / tilt driving device 130. In this specification, the "pan / tilt operation" refers to an operation including at least one of a panning operation and a tilting operation. Video input device 110 outputs the captured video to tracking device 150A as video information.

[0012] Pan / tilt driving device 130 is a device that changes the imaging direction of video input device 110, and is composed of a motor, gears, etc. When pan / tilt driving device 130 receives a series of operation commands for pan / tilt operations from tracking device 150A, it controls the motor in accordance with the operation commands to perform pan / tilt operations.

[0013] Tracking device 150A analyzes the video information input from video input device 110 and generates a pan / tilt operation command (pan / tilt operation command) so that the subject to be tracked is captured at a predetermined position in the angle of view. Tracking device 150A detects the face, upper body, and entire body of the subject to be tracked, and generates a pan / tilt operation command based on the detection result. Tracking device 150A outputs the generated pan / tilt operation command to pan / tilt driving device 130, and outputs the video information to recording device 170.

[0014] The tracking device 150A includes a video input unit 151, a first detection unit 152, a first correction unit 153, a second detection unit 154, a second correction unit 155, a third detection unit 156, a third correction unit 157, and a tracking coordinate generation unit 158A. The tracking device 150A also includes an ID assignment unit 159, a target coordinate calculation unit 160A, a pan-tilt drive control unit 161, a video distribution unit 162, a tracking target input unit 163, and a communication unit 164. The first detection unit 152, the second detection unit 154, and the third detection unit 156 are examples of detection means for detecting coordinates indicating the areas of a plurality of parts of a subject. The first correction unit 153, the second correction unit 155, and the third correction unit 157 are examples of correction means for calculating the correction coordinates of the parts. Moreover, the ID assigning unit 159 is an example of a setting unit that sets one tracking coordinate as a tracking target, and the pan / tilt drive control unit 161 is an example of a control unit that controls the imaging range.

[0015] In this embodiment, the first detection unit 152 and the first correction unit 153 perform processing related to the face of the subject. The second detection unit 154 and the second correction unit 155 perform processing related to the upper body of the subject, and the third detection unit 156 and the third correction unit 157 perform processing related to the whole body of the subject.

[0016] Video input section 151 inputs video information (image) captured by video input device 110 , and outputs it to first detection section 152 , second detection section 154 , third detection section 156 , and video distribution section 162 .

[0017] The first detection unit 152 performs a process of detecting a face in an image using the video information input from the video input unit 151. The face detection method may be, for example, a template matching method or a method of detecting a face by deep learning. These detection methods are well-known technologies, and therefore detailed explanations are omitted. The first detection unit 152 outputs coordinate information indicating the area of ​​the detected face (hereinafter, also simply referred to as "face coordinate information") to the first correction unit 153. The coordinate information is output in absolute coordinates according to a spherical coordinate system.

[0018] The first correction unit 153 calculates corrected coordinates of the face from the face coordinate information input from the first detection unit 152. The first correction unit 153 performs enlargement / reduction at a preset magnification and position correction at a relative position on the face coordinate information based on a preset magnification and relative position, and calculates corrected coordinates of the face. The first correction unit 153 outputs the corrected coordinate information of the face to the tracking coordinate generation unit 158A.

[0019] Here, an example of a method for calculating the corrected coordinates of the face will be described with reference to FIG. 4(A). In FIG. 4(A), 411 is the face area indicated by the face coordinate information, and 412 is the center coordinate of the face area 411. Also, 413 is the corrected coordinate of the face area, and 414 is the corrected center coordinate of the face. In this embodiment, as an example, the center position of the face coordinate information is shifted upward by 35% according to a preset magnification and relative position, and the height and width of the area are both enlarged to 180%, which is set as the corrected center coordinate of the face. The first correction unit 153 multiplies the height HA11 of the face area 411 by 0.35 to calculate the center movement distance, and sets the point obtained by moving the center coordinate 412 of the face area 411 upward by the center movement distance as indicated by an arrow 415 as the corrected center coordinate 414 of the face. In addition, the first correction unit 153 multiplies the height HA11 of the face area 411 by 1.8 to obtain the height HA12 of the corrected coordinates 413 of the face area, and multiplies the width WA11 of the face area 411 by 1.8 to obtain the width WA12 of the corrected coordinates 413 of the face area.

[0020] The second detection unit 154 performs detection processing of the upper body in the image using the video information input from the video input unit 151. The upper body detection method may be, for example, a template matching method or a method of detecting the upper body by deep learning. These detection methods are well-known technologies, so detailed explanations are omitted. The second detection unit 154 outputs coordinate information indicating the detected upper body area (hereinafter also simply referred to as "upper body coordinate information") to the second correction unit 155.

[0021] The second correction unit 155 calculates corrected coordinates of the upper body from the coordinate information of the upper body input from the second detection unit 154. The second correction unit 155 performs enlargement / reduction at a preset magnification and position correction at a relative position on the coordinate information of the upper body based on a preset magnification and a relative position, and calculates corrected coordinates of the upper body. The second correction unit 155 outputs the corrected coordinate information of the upper body to the tracking coordinate generation unit 158A.

[0022] Here, an example of a method for calculating the corrected coordinates of the upper body will be described with reference to FIG. 4B. In FIG. 4B, 421 is the upper body area indicated by the coordinate information of the upper body, and 422 is the center coordinate of the upper body area 421. Also, 423 is the corrected coordinate of the upper body area, and 424 is the corrected center coordinate of the upper body. In this embodiment, as an example, the center position is shifted upward by 30% according to a preset magnification and relative position for the coordinate information of the upper body, the height of the area is reduced by 40%, and the width of the area is reduced by 30% to obtain the corrected coordinates of the upper body. The second corrector 155 multiplies the height HA21 of the upper body area 421 by 0.3 to calculate the center movement distance, and the point where the center coordinate 422 of the upper body area 421 is moved upward by the center movement distance as shown by the arrow 425 is obtained as the corrected center coordinate 424 of the upper body. In addition, the second correction unit 155 multiplies the height HA21 of the upper body area 421 by 0.4 to obtain the height HA22 of the corrected coordinates 423 of the upper body area, and multiplies the width WA21 of the upper body area 421 by 0.3 to obtain the width WA22 of the corrected coordinates 423 of the upper body area.

[0023] The third detection unit 156 performs a process of detecting the whole body in the image using the video information input from the video input unit 151. The whole body detection method may be, for example, a template matching method or a method of detecting the whole body by deep learning. These detection methods are well-known technologies, so detailed explanations are omitted. The third detection unit 156 outputs coordinate information indicating the detected whole body area (hereinafter, also simply referred to as "whole body coordinate information") to the third correction unit 157. The coordinate information is output in absolute coordinates according to a spherical coordinate system.

[0024] The third correction unit 157 calculates corrected coordinates of the whole body from the coordinate information of the whole body input from the third detection unit 156. The third correction unit 157 performs enlargement / reduction at a preset magnification and position correction at a relative position on the coordinate information of the whole body based on a preset magnification and relative position, and calculates corrected coordinates of the whole body. The third correction unit 157 outputs the corrected coordinate information of the whole body to the tracking coordinate generation unit 158A.

[0025] Here, an example of a method for calculating the corrected coordinates of the whole body will be described with reference to FIG. 4(C). In FIG. 4(C), 431 is the whole body area indicated by the coordinate information of the whole body, and 432 is the center coordinate of the whole body area 431. Also, 433 is the corrected coordinate of the whole body area, and 434 is the center corrected coordinate of the whole body. In this embodiment, as an example, the center position is shifted upward by 35% according to the preset magnification and relative position for the coordinate information of the whole body, the height of the area is reduced by 20%, and the width of the area is reduced by 30% to obtain the corrected coordinates of the whole body. The third corrector 157 multiplies the height HA31 of the whole body area 431 by 0.35 to calculate the center movement distance, and the point where the center coordinate 432 of the whole body area 431 is moved upward by the center movement distance as shown by the arrow 435 is obtained as the center corrected coordinate 434 of the whole body. In addition, the third correction unit 157 multiplies the height HA31 of the whole body area 431 by 0.2 to obtain the height HA32 of the corrected coordinates 433 of the whole body area, and multiplies the width WA31 of the whole body area 431 by 0.3 to obtain the width WA32 of the corrected coordinates 433 of the whole body area.

[0026] When the face corrected coordinate information from the first corrector 153, the upper body corrected coordinate information from the second corrector 155, and the whole body corrected coordinate information from the third corrector 157 are input, the tracking coordinate generator 158A generates tracking coordinates from each corrected coordinate information. The tracking coordinate generator 158A outputs the generated tracking coordinates to the ID assigner 159.

[0027] Here, the generation of tracking coordinates by the tracking coordinate generation unit 158A will be described with reference to FIG. 5. 510 shown in FIG. 5(A) is an image of an angle of view captured by an imaging system, and 511, 512, 513, and 514 are the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body of each person who is a subject. However, depending on conditions such as the direction and posture of the person and the presence or absence of an obstacle, it may be impossible to detect any of the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body. The tracking coordinate generation unit 158A calculates an average position and size from the correction coordinates of the detected parts of the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body to generate tracking coordinates. 520 shown in FIG. 5(B) is an image after the generation of tracking coordinates, and one tracking coordinate is generated for each person of the subjects, as in 521, 522, 523, and 524.

[0028] Furthermore, a method for generating tracking coordinates will be described using the subject person 515 as an example. 530 and 540 shown in Figures 5(C) and 5(D), respectively, are enlarged views of the periphery of person 515. The enlarged view shown in Figure 5(C) illustrates the face correction coordinates, upper body correction coordinates, and whole body correction coordinates for person 515, and the enlarged view shown in Figure 5(D) illustrates the tracking coordinates for person 515.

[0029] First, a method for extracting correction coordinates for one person from all the face correction coordinates, upper body correction coordinates, and whole body correction coordinates detected from an image of an angle of view captured by an imaging system will be described with reference to Fig. 5(C). In Fig. 5(C), 531 is the face correction coordinate, and 532 is the face center correction coordinate. Also, 533 is the upper body correction coordinate, 534 is the upper body center correction coordinate, 535 is the whole body correction coordinate, and 536 is the whole body center correction coordinate.

[0030] If the distance between the regions indicated by the correction coordinates of the parts, specifically, the distance between the center points of the two correction coordinates, is within the judgment threshold, the tracking coordinate generation unit 158A judges that the two correction coordinates belong to the same person, and associates the two correction coordinates as correction coordinates for one person. That is, if the distance between the center correction coordinates of the two correction coordinates is within the judgment threshold, the tracking coordinate generation unit 158A associates the two correction coordinates as correction coordinates for one person. The judgment threshold is a preset threshold that determines that the correction coordinates belong to the same person, and in the example shown in FIG. 5, the length 502 that is 5% of the height 501 of the angle of view is set as the judgment threshold. In the example shown in FIG. 5(C), since the distance D31 between the two points of the center correction coordinate 532 of the face and the center correction coordinate 534 of the upper body is within the judgment threshold, the tracking coordinate generation unit 158A associates the correction coordinate 531 of the face and the correction coordinate 533 of the upper body as correction coordinates for one person. In addition, since the distance D32 between the corrected center coordinate 532 of the face and the corrected center coordinate 536 of the whole body is within the judgment threshold, the tracking coordinate generation unit 158A associates the corrected coordinate 531 of the face with the corrected coordinate 535 of the whole body as corrected coordinates for one person. As a result, the corrected coordinate 531 of the face, the corrected coordinate 533 of the upper body, and the corrected coordinate 535 of the whole body are associated as corrected coordinates for one person 515.

[0031] Next, a method of generating tracking coordinates from each correction coordinate associated as described above will be described. The tracking coordinate generation unit 158A calculates the average of the size and position of the correction coordinates associated as the correction coordinates for one person to generate tracking coordinates. In the example shown in FIG. 5(C), each correction coordinate 531, 533, 535 is associated as the correction coordinates for one person. At this time, the tracking coordinate generation unit 158A calculates the average value of the X coordinate and Y coordinate of each center correction coordinate 532, 534, 536, and sets the point located at the calculated average X coordinate and average Y coordinate as the center coordinate 542 of the tracking coordinate 541. In addition, the tracking coordinate generation unit 158A sets the average value of the height HB11 of the face correction coordinate 531, the height HB21 of the upper body correction coordinate 533, and the height HB31 of the whole body correction coordinate 535 as the height HB41 of the tracking coordinate 541. Furthermore, the tracking coordinate generation unit 158A sets the average value of the width WB11 of the corrected coordinates 531 of the face, the width WB21 of the corrected coordinates 533 of the upper body, and the width WB31 of the corrected coordinates 535 of the whole body as the width WB41 of the tracking coordinates 541.

[0032] The ID assigning unit 159 assigns identification information (ID) to the tracking coordinates generated by the tracking coordinate generating unit 158A. The ID assigning unit 159 holds the tracking coordinates generated in the image of the previous frame (hereinafter referred to as "previous tracking coordinates") and the ID corresponding thereto, and when the tracking coordinates are input from the tracking coordinate generating unit 158A, the ID assigning unit 159 calculates the distance between each tracking coordinate and each previous tracking coordinate. The ID assigning unit 159 associates the tracking coordinates whose distance between the coordinates is within a preset threshold value with the previous tracking coordinates, and sets the ID set in the previous tracking coordinates as the current tracking coordinates. When there are tracking coordinates that are not associated with any previous tracking coordinates, the ID assigning unit 159 assigns a new ID to the tracking coordinates. When the ID assigning unit 159 receives coordinate information of the tracking target from the tracking target input unit 163, the ID assigning unit 159 sets the tracking coordinates that are closest to the coordinate information as the tracking target, and sets the tracking target ID. When the tracking target input unit 163 has not inputted coordinate information of the tracking target and the tracking target has not yet been set, the tracking process is not performed. When the tracking target ID is set but there are no tracking coordinates corresponding to the ID, the ID assigning unit 159 determines that the tracking target has been lost, deletes the tracking target ID, and does not perform the tracking process. When the tracking target ID is set, the ID assigning unit 159 outputs the tracking coordinates associated with the tracking target ID to the target coordinate calculation unit 160A.

[0033] The target coordinate calculation unit 160A calculates a target pan / tilt position related to the pan / tilt operation for capturing the subject of the tracking target within the angle of view for capturing the subject of the tracking target from the tracking coordinates of the tracking target. When the tracking coordinates of the tracking target are input from the ID assignment unit 159, the target coordinate calculation unit 160A determines a target angle of view so that the center coordinate of the tracking coordinates is a predetermined position within the angle of view, and calculates a target pan / tilt position for reaching the target angle of view. In this embodiment, as an example, the target coordinate calculation unit 160A sets the angle of view where the X coordinate of the center coordinate of the tracking coordinates is the center of the angle of view and the Y coordinate is 30% from the upper end of the angle of view as the target angle of view, and calculates a target pan / tilt position for reaching the target angle of view. In addition, the target coordinate calculation unit 160A outputs information for driving to the calculated target pan / tilt position to the pan / tilt drive control unit 161.

[0034] When information for driving to a target pan / tilt position is input from target coordinate calculation unit 160A, pan / tilt drive control unit 161 outputs a pan / tilt operation command to pan / tilt drive device 130 so as to move the imaging direction to the target position.

[0035] When the video information is input from the video input unit 151, the video distribution unit 162 outputs the input video information to the communication unit 164. When the coordinate information of the tracking target is input from the communication unit 164, the tracking target input unit 163 outputs the coordinate information of the tracking target to the ID assignment unit 159. The communication unit 164 outputs the video information input from the video distribution unit 162 to the recording device 170. In addition, the communication unit 164 inputs coordinate information on the subject of the tracking target input by the user via a user operation unit (not shown) to the tracking target input unit 163. The recording device 170 performs a process of storing the input video input from the tracking device 150A in a storage device such as an HDD (hard disk drive).

[0036] 2 is a diagram showing an example of the hardware configuration of a tracking device in this embodiment. The tracking device 200 in this embodiment has a CPU 201, a ROM 202, a RAM 203, a storage unit 204, an input unit 205, an output unit 206, and a communication unit 207. The CPU 201, the ROM 202, the RAM 203, the storage unit 204, the input unit 205, the output unit 206, and the communication unit 207 are communicatively connected via a system bus 208. Note that the tracking device 200 in this embodiment may further have other configurations.

[0037] The CPU (Central Processing Unit) 201 comprehensively controls the operation of the tracking device 200, and controls the operation of each functional unit connected via a system bus 208. The ROM (Read Only Memory) 202 is a non-volatile memory that stores control programs and the like required for the CPU 201 to execute processing. The programs may be stored in the storage unit 204 or a storage medium that is detachable from the tracking device. The RAM (Random Access Memory) 203 functions as the main memory, work area, and the like of the CPU 201. When executing processing, the CPU 201 reads out necessary programs and the like from the ROM 202, etc., into the RAM 203, and executes the programs and the like to realize various functional operations.

[0038] The storage unit 204 stores, for example, various data and various information required when the CPU 201 performs processing using a program. In addition, the storage unit 204 stores, for example, various data and various information obtained by the CPU 201 performing processing using a program. The input unit 205 accepts instructions from a user of the tracking device via an operation unit (not shown). The input unit 205 accepts, for example, input of coordinate information of a tracking target based on a user operation. The output unit 206 outputs control information from the tracking device 200 to the outside. The output unit 206 outputs, for example, a pan-tilt operation command to the pan-tilt driving device 130. The communication unit 207 is an interface unit for communicating with an external device (for example, the recording device 170).

[0039] With reference to Fig. 3A and Fig. 3B, the imaging control process in the imaging system according to this embodiment will be described. Fig. 3A and Fig. 3B are flowcharts showing an example of the imaging control process according to this embodiment. As shown in Fig. 3A, when the imaging system 100A is started by a user operation and the automatic imaging process is started, in S301, the video input unit 151 of the tracking device 150A acquires video information from the video input device 110.

[0040] In S302, first detection unit 152 of tracking device 150A performs a process of detecting a face in the video by analyzing the video (image) input from video input unit 151. Then, in S303, first correction unit 153 of tracking device 150A corrects the coordinate information of the face input from first detection unit 152 with a preset magnification and relative position to generate corrected coordinates.

[0041] In S304, second detection unit 154 of tracking device 150A performs detection processing of the upper body in the video by analyzing the video (image) input from video input unit 151. Then, in S305, second correction unit 155 of tracking device 150A corrects the coordinate information of the upper body input from second detection unit 154 with a preset magnification and relative position to generate corrected coordinates.

[0042] In S306, the third detection unit 156 of the tracking device 150A performs a detection process of the whole body in the video by analyzing the video (image) input from the video input unit 151. Then, in S307, the third correction unit 157 of the tracking device 150A corrects the coordinate information of the whole body input from the third detection unit 156 with a preset magnification and relative position to generate corrected coordinates.

[0043] The processing order for the face, upper body, and whole body of the subject in S302 to S307 may be random and may be different from the order shown in Fig. 3A, or may be executed in parallel. After the detection processing for each part is performed, the corrected coordinates for that part may be generated. For example, the detection processing for the face, upper body, and whole body of the subject may be performed, and then the corrected coordinates for the face, upper body, and whole body of the subject may be generated.

[0044] In S308, the tracking coordinate generation unit 158A of the tracking device 150A determines whether or not any of the face, upper body, and whole body of the subject has been detected within the angle of view. If the tracking coordinate generation unit 158A determines that any of the face, upper body, and whole body of the subject has been detected within the angle of view (YES in S308), the tracking coordinate generation unit 158A proceeds to S309. On the other hand, if the tracking coordinate generation unit 158A determines that none of the face, upper body, and whole body of the subject has been detected within the angle of view (NO in S308), the tracking coordinate generation unit 158A proceeds to S301.

[0045] In S309, the tracking coordinate generation unit 158A calculates the distance between the center points of the corrected coordinates calculated in S303, S305, and S307. In S310, the tracking coordinate generation unit 158A associates the corrected coordinates in which the distance between the center points is within the determination threshold as coordinate information for one person. In S311, the tracking coordinate generation unit 158A calculates the average of the size and position of the corrected coordinates from the corrected coordinates of the face, upper body, and whole body for each person, and generates tracking coordinates.

[0046] In S312, the ID assigning unit 159 of the tracking device 150A associates each tracking coordinate generated by the tracking coordinate generating unit 158A in S311 with past tracking coordinates (previous tracking coordinates), and sets the same ID as the past. Then, the process proceeds to S313 shown in FIG. 3B.

[0047] Next, in S313, the ID assigning unit 159 determines whether there are tracking coordinates that cannot be associated with past tracking coordinates (previous tracking coordinates). If the ID assigning unit 159 determines that the tracking coordinates generated in S311 cannot be associated with the previous tracking coordinates (YES in S313), the process proceeds to S314. If the ID assigning unit 159 determines that the tracking coordinates generated in S311 can be associated with the previous tracking coordinates (NO in S313), the process proceeds to S315. In S314, the ID assignment unit 159 assigns and sets a new ID to the tracking coordinates that cannot be associated with the past tracking coordinates (previous tracking coordinates), and the process proceeds to S315.

[0048] In S315, the ID assignment unit 159 acquires the coordinate information of the tracking target from the tracking target input unit 163. In S316, the ID assignment unit 159 determines whether or not the coordinate information of the tracking target has been acquired from the tracking target input unit 163. If the ID assignment unit 159 determines that the coordinate information of the tracking target has been acquired (YES in S316), the process proceeds to S317, and if the ID assignment unit 159 determines that the coordinate information of the tracking target cannot be acquired (NO in S316), the process proceeds to S318. In S317, the ID assignment unit 159 selects tracking coordinates that are closest to the coordinate information of the tracking target acquired from the tracking target input unit 163, sets the ID of the tracking coordinates as the tracking target ID, and proceeds to S321.

[0049] In S318, the ID assigning unit 159 determines whether or not a tracking target ID is set. If the ID assigning unit 159 determines that a tracking target ID is set (YES in S318), the process proceeds to S319, and if the ID assigning unit 159 determines that a tracking target ID is not set (NO in S318), the process proceeds to S301 in FIG. 3A. In S319, the ID assigning unit 159 determines whether or not tracking coordinates corresponding to the tracking target ID are present among the tracking coordinates generated in S311. If the ID assigning unit 159 determines that tracking coordinates corresponding to the tracking target ID are present (YES in S319), the process proceeds to S321, and if the ID assigning unit 159 determines that tracking coordinates corresponding to the tracking target ID are not present (NO in S319), the process proceeds to S320. In S320, the ID assignment unit 159 deletes the set tracking target ID, and the process proceeds to S301 in FIG. 3A.

[0050] In S321, the target coordinate calculation unit 160A calculates a target pan / tilt position for changing the imaging range so that the tracking coordinates of the tracking target come to a preset target position. In S322, the pan / tilt drive control unit 161 outputs a pan / tilt operation command to the pan / tilt drive device 130 so as to move to the target pan / tilt position calculated by the target coordinate calculation unit 160A, and controls the pan / tilt drive device 130. This controls the imaging direction (imaging range) so that the video input device 110 captures the subject to be tracked at an appropriate position within the angle of view.

[0051] In S323, the tracking device 150A determines whether or not the user has operated an imaging system On / Off switch (not shown) to stop the imaging system. If the tracking device 150A determines that the imaging system has not been stopped (NO in S323), it proceeds to S301 in Fig. 3A, and if the tracking device 150A determines that the imaging system has been stopped (YES in S323), it ends the operation of the imaging system.

[0052] As described above, in a system that automatically tracks and captures a target so that it is captured at an appropriate position in the angle of view, multiple parts are detected by video analysis to identify the target subject, and if even one part is detected, tracking can be performed based on that, thereby improving the continuity of tracking. In addition, by tracking the target subject using coordinate information rather than image features, tracking can be continued even if image features change due to a change in the direction of the target.

[0053] Second embodiment FIG. 6 is a block diagram showing a functional configuration example of an imaging system 100B including a tracking device 150B as an imaging control device according to the second embodiment. In FIG. 6, components having the same functions as those shown in FIG. 1 are given the same reference numerals, and duplicated explanations are omitted. The imaging system 100B is a system that tracks a specific subject designated by a user and performs automatic shooting. The imaging system 100B performs pan-tilt-zoom control so that the designated tracking target subject is captured at an appropriate position in the angle of view (imaging range), tracks the subject, and records the video. In addition, the imaging system 100B detects multiple parts of the subject by video analysis to identify the position of the tracking target, and if any one of the multiple parts is detected, tracks the subject based on the detected part, thereby improving the continuity of tracking. In the following, as in the first embodiment, a case will be described as an example in which the tracking target subject is a person and three parts of the person's face, upper body, and whole body are detected, but the tracking target subject and the parts to be detected are not limited to this. Furthermore, in this embodiment, in order to accommodate the characteristics of the upper body and whole body where the detection coordinates are likely to shift depending on the direction and posture of the person, the system is designed to determine that a single person has been detected even if the detection coordinates shift vertically. Also, when the detection of the subject's face is successful, the pan / tilt / zoom operation is controlled to target the coordinates of the face where shifting is unlikely to occur, thereby improving the accuracy of the angle of view control.

[0054] The imaging system 100B has a video input device 110, a pan / tilt driving device 130, a zoom driving device 190, a tracking device 150B, and a recording device 170. The tracking device 150B and the recording device 170 are connected via a video interface. The tracking device 150B is an example of an imaging control device that controls the imaging means (video input device 110).

[0055] The zoom driving device 190 is a device that changes the zoom ratio of the video input device 110, and is composed of a motor, gears, etc. When the zoom driving device 190 receives a series of operation commands for the zoom operation from the tracking device 150B, it controls the motor according to the operation commands to perform the zoom operation.

[0056] The tracking device 150B analyzes the video information input from the video input device 110 and generates a command (pan-tilt-zoom operation command) to perform at least one of panning, tilting, and zooming so that the tracking target is captured at a predetermined position in the angle of view. The "pan-tilt-zoom operation" refers to an operation including at least one of panning, tilting, and zooming. The tracking device 150B detects the face, upper body, and whole body of the subject to be tracked, and generates a pan-tilt-zoom operation command based on the detection result. The tracking device 150B outputs the generated pan-tilt operation command to the pan-tilt driving device 130, outputs the generated zoom operation command to the zoom driving device 190, and outputs the video information to the recording device 170.

[0057] The tracking device 150B includes a video input unit 151, a first detection unit 152, a first correction unit 153, a second detection unit 154, a second correction unit 155, a third detection unit 156, a third correction unit 157, and a tracking coordinate generation unit 158B. The tracking device 150B also includes an ID assignment unit 159, a target coordinate calculation unit 160B, a pan / tilt drive control unit 161, a zoom position calculation unit 165, a zoom drive control unit 166, a video distribution unit 162, a tracking target input unit 163, and a communication unit 164. In this embodiment, the pan / tilt drive control unit 161 and the zoom drive control unit 166 are examples of a control unit that controls the imaging range.

[0058] When the face corrected coordinate information from the first corrector 153, the upper body corrected coordinate information from the second corrector 155, and the whole body corrected coordinate information from the third corrector 157 are input, the tracking coordinate generator 158B generates tracking coordinates from each corrected coordinate information. The tracking coordinate generator 158B outputs the generated tracking coordinates to the ID assigner 159.

[0059] Here, the generation of tracking coordinates by the tracking coordinate generation unit 158B will be described with reference to FIG. 8. 810 shown in FIG. 8(A) is an image of an angle of view captured by an imaging system, and 811, 812, 813, and 814 are the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body of each person who is a subject. However, depending on conditions such as the direction and posture of the person and the presence or absence of an obstacle, it may be impossible to detect any of the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body. In this embodiment, the reliability of the coordinates is set for each part of the face, the upper body, and the whole body of the person. The tracking coordinate generation unit 158B uses the correction coordinates with higher reliability (highest reliability) among the correction coordinates of the parts that can be detected among the correction coordinates of the face, the correction coordinates of the upper body, and the correction coordinates of the whole body as the tracking coordinates. In this embodiment, the reliability is set in the order of the face, the upper body, and the whole body. Reference numeral 820 shown in FIG. 8B is an image after tracking coordinate generation, and tracking coordinates such as 821, 822, 823, and 824 are generated for each person in the subjects.

[0060] Furthermore, a method for generating tracking coordinates will be described using the subject person 815 as an example. 830 and 840 shown in Fig. 8(C) and Fig. 8(D), respectively, are enlarged views of the periphery of the person 815. The enlarged view shown in Fig. 8(C) illustrates the face correction coordinates, upper body correction coordinates, and whole body correction coordinates for the person 815, and the enlarged view shown in Fig. 8(D) illustrates the tracking coordinates for the person 815.

[0061] First, a method for extracting correction coordinates for one person from all the face correction coordinates, upper body correction coordinates, and whole body correction coordinates detected from an image of an angle of view captured by an imaging system will be described with reference to Fig. 8(C). In Fig. 8(C), 831 is the face correction coordinate, and 832 is the face center correction coordinate. Also, 833 is the upper body correction coordinate, 834 is the upper body center correction coordinate, 835 is the whole body correction coordinate, and 836 is the whole body center correction coordinate. The figure shows an example of a case where a deviation occurs in the detection of the subject's whole body coordinates, and the whole body correction coordinate 835 and the whole body center correction coordinate 836 are located away from the other coordinates.

[0062] In FIG. 8C, the distance (horizontal distance) in the X coordinate between the two points of the corrected center coordinate 832 of the face and the corrected center coordinate 834 of the upper body is D31, and the distance (vertical distance) in the Y coordinate is D32. The distance (horizontal distance) in the X coordinate between the two points of the corrected center coordinate 832 of the face and the corrected center coordinate 836 of the whole body is D33, and the distance (vertical distance) in the Y coordinate is D34. In this embodiment, if the distances in the X coordinate and the Y coordinate between the areas (center points) indicated by the two corrected coordinates are both within a judgment threshold, the tracking coordinate generation unit 158B judges that they are of the same person, and associates the two corrected coordinates as corrected coordinates for one person. In this embodiment, the judgment threshold is a length 803 that is 5% of the height 801 of the angle of view for the X coordinate (horizontal direction), and a length 802 that is larger than the judgment threshold in the horizontal direction and is 20% of the height 801 of the angle of view for the Y coordinate (vertical direction). In the example shown in FIG. 8C, the distance D31 in the X coordinate between the corrected center coordinate 832 of the face and the corrected center coordinate 834 of the upper body is within the judgment threshold, and the distance D32 in the Y coordinate is within the judgment threshold. Therefore, the tracking coordinate generation unit 158B associates the corrected coordinate 831 of the face and the corrected coordinate 833 of the upper body as corrected coordinates for one person. Similarly, the distance D33 in the X coordinate between the corrected center coordinate 832 of the face and the corrected center coordinate 836 of the whole body is within the judgment threshold, and the distance D34 in the Y coordinate is within the judgment threshold. Therefore, the tracking coordinate generation unit 158B associates the corrected coordinate 831 of the face and the corrected coordinate 835 of the whole body as corrected coordinates for one person.

[0063] Next, a method of generating tracking coordinates from each correction coordinate will be described. The tracking coordinate generation unit 158B sets tracking coordinates according to the reliability (priority) of each coordinate set for a plurality of parts of the subject. In this embodiment, when face correction coordinates exist, the face correction coordinates are set as tracking coordinates. When face correction coordinates do not exist but upper body correction coordinates exist, the upper body correction coordinates are set as tracking coordinates, and when neither face nor upper body correction coordinates exist, the whole body correction coordinates are set as tracking coordinates.

[0064] The target coordinate calculation unit 160B calculates a target position related to the pan / tilt / zoom operation for capturing the subject of the tracking target within the angle of view for capturing an image of the tracking target from the tracking coordinates of the tracking target. When the tracking coordinates of the tracking target are input from the ID assignment unit 159, the target coordinate calculation unit 160B sets the angle of view where the X coordinate of the central coordinate of the tracking coordinate is the center of the angle of view and the Y coordinate is 30% from the upper end of the angle of view as the target angle of view, and calculates a target pan / tilt position for setting the target angle of view. The target coordinate calculation unit 160B outputs information for driving to the calculated target pan / tilt position to the pan / tilt drive control unit 161. The target coordinate calculation unit 160B also outputs the tracking coordinates of the tracking target to the zoom position calculation unit 165.

[0065] When the tracking coordinates of the tracking target are input from the target coordinate calculation unit 160B, the zoom position calculation unit 165 calculates a target zoom position so that the height of the tracking coordinates is 10% of the height of the angle of view. The zoom position calculation unit 165 outputs information for driving to the calculated target zoom position to the zoom drive control unit 166. When the zoom drive control section 166 receives information for driving to a target zoom position from the zoom position calculation section 165, the zoom drive control section 166 outputs a zoom operation command to the zoom drive device 190 to move to the target position. Other configurations are the same as those of the imaging system in the first embodiment. In addition, the hardware configuration of the tracking device is also the same as that of the first embodiment, so a description thereof will be omitted.

[0066] With reference to Fig. 7A and Fig. 7B, the imaging control process in the imaging system according to this embodiment will be described. Fig. 7A and Fig. 7B are flowcharts showing an example of the imaging control process according to this embodiment. As shown in Fig. 7A, when the imaging system 100B is started by a user operation and the automatic shooting process is started, in S701, the video input unit 151 of the tracking device 150B acquires video information from the video input device 110.

[0067] In S702, first detection unit 152 of tracking device 150B performs a process of detecting a face in the video by analyzing the video (image) input from video input unit 151. Then, in S703, first correction unit 153 of tracking device 150B corrects the coordinate information of the face input from first detection unit 152 with a preset magnification and relative position to generate corrected coordinates.

[0068] In S704, second detection unit 154 of tracking device 150B analyzes the video (image) input from video input unit 151 to perform detection processing of the upper body in the video. Then, in S705, second correction unit 155 of tracking device 150B corrects the coordinate information of the upper body input from second detection unit 154 with a preset magnification and relative position to generate corrected coordinates.

[0069] In S706, the third detection unit 156 of the tracking device 150B analyzes the video (image) input from the video input unit 151 to perform a detection process of the whole body in the video. Then, in S707, the third correction unit 157 of the tracking device 150B corrects the coordinate information of the whole body input from the third detection unit 156 with a preset magnification and relative position to generate corrected coordinates.

[0070] The processing order for the face, upper body, and whole body of the subject in S702 to S707 may be random and may be different from the order shown in Fig. 7A, or may be executed in parallel. After the detection processing for each part is performed, the correction coordinates for that part may be generated. For example, the correction coordinates for the face, upper body, and whole body of the subject may be generated after the detection processing for the face, upper body, and whole body of the subject.

[0071] In S708, the tracking coordinate generation unit 158B of the tracking device 150B determines whether or not any of the face, upper body, and whole body of the subject has been detected within the angle of view. If the tracking coordinate generation unit 158B determines that any of the face, upper body, and whole body of the subject has been detected within the angle of view (YES in S708), the process proceeds to S709. On the other hand, if the tracking coordinate generation unit 158B determines that none of the face, upper body, and whole body of the subject has been detected within the angle of view (NO in S708), the process proceeds to S701.

[0072] In S709, the tracking coordinate generation unit 158B calculates the distance between the center points of the corrected coordinates calculated in S703, S705, and S707. In S710, the tracking coordinate generation unit 158B associates corrected coordinates in which the distance in the X coordinate between the center points is within a judgment threshold (horizontal judgment threshold) and the distance in the Y coordinate is within a judgment threshold (vertical judgment threshold) as coordinate information for one person.

[0073] In S711, the tracking coordinate generation unit 158B judges whether or not face coordinate information is detected for each person. If the tracking coordinate generation unit 158B judges that face coordinate information is detected (YES in S711), the process proceeds to S712, and if the tracking coordinate generation unit 158B judges that face coordinate information is not detected (NO in S711), the process proceeds to S713. In S712, the tracking coordinate generation unit 158B sets the corrected coordinates of the face as tracking coordinates for that person, and the process proceeds to S716.

[0074] In S713, the tracking coordinate generation unit 158B determines whether or not the coordinate information of the upper body is detected for each person. If the tracking coordinate generation unit 158B determines that the coordinate information of the upper body is detected (YES in S713), the process proceeds to S714, and if the tracking coordinate generation unit 158B determines that the coordinate information of the upper body is not detected (NO in S713), the process proceeds to S715. In S714, the tracking coordinate generation unit 158B sets the corrected coordinates of the upper body as tracking coordinates for that person, and the process proceeds to S716. In S715, the tracking coordinate generation unit 158B sets the corrected coordinates of the whole body as tracking coordinates for that person, and the process proceeds to S716.

[0075] In S716, ID assigning unit 159 of tracking device 150B associates each tracking coordinate with past tracking coordinates (previous tracking coordinates) and sets the same ID as the previous one. Then, the process proceeds to S717 shown in FIG. 7B.

[0076] Next, in S717, the ID assigning unit 159 determines whether there are tracking coordinates that cannot be associated with past tracking coordinates (previous tracking coordinates). If the ID assigning unit 159 determines that the currently generated tracking coordinates cannot be associated with the previous tracking coordinates (YES in S717), the process proceeds to S718. If the ID assigning unit 159 determines that the currently generated tracking coordinates can be associated with the previous tracking coordinates (NO in S717), the process proceeds to S719. In S718, the ID assignment unit 159 assigns and sets a new ID to the tracking coordinates that cannot be associated with the past tracking coordinates (previous tracking coordinates), and the process proceeds to S719.

[0077] In S719 , the ID assignment unit 159 acquires the coordinate information of the tracking target from the tracking target input unit 163 . In S720, the ID assignment unit 159 determines whether or not the coordinate information of the tracking target has been acquired from the tracking target input unit 163. If the ID assignment unit 159 determines that the coordinate information of the tracking target has been acquired (YES in S720), the process proceeds to S721, and if the ID assignment unit 159 determines that the coordinate information of the tracking target cannot be acquired (NO in S720), the process proceeds to S722. In S721, the ID assignment unit 159 selects tracking coordinates that are closest to the coordinate information of the tracking target acquired from the tracking target input unit 163, sets the ID of the tracking coordinates as the tracking target ID, and proceeds to S725.

[0078] In S722, the ID assigning unit 159 determines whether or not a tracking target ID is set. If the ID assigning unit 159 determines that a tracking target ID is set (YES in S722), the process proceeds to S723, and if the ID assigning unit 159 determines that a tracking target ID is not set (NO in S722), the process proceeds to S701 in FIG. 7A. In S723, the ID assigning unit 159 determines whether or not tracking coordinates corresponding to the tracking target ID exist. If the ID assigning unit 159 determines that tracking coordinates corresponding to the tracking target ID exist (YES in S723), the process proceeds to S725, and if the ID assigning unit 159 determines that tracking coordinates corresponding to the tracking target ID do not exist (NO in S723), the process proceeds to S724. In S724, the ID assignment unit 159 deletes the set tracking target ID, and the process proceeds to S701 in FIG. 7A.

[0079] In S725, the target coordinate calculation unit 160B calculates a target pan / tilt position for changing the imaging range so that the tracking coordinates of the tracking target come to a preset target position. In S726, the pan / tilt drive control unit 161 outputs a pan / tilt operation command to the pan / tilt drive device 130 so as to move to the target pan / tilt position calculated by the target coordinate calculation unit 160B, and controls the pan / tilt drive device 130. This controls the imaging direction (imaging range) so that the video input device 110 captures the subject to be tracked at an appropriate position within the angle of view.

[0080] In S727, the zoom position calculation unit 165 calculates the target zoom position so that the height of the tracking coordinates of the tracking target becomes the height of a preset angle of view. In S728, the zoom drive control unit 166 outputs a zoom operation command to the zoom drive device 190 to move to the target zoom position calculated by the zoom position calculation unit 165B, and controls the zoom drive device 190. This controls the zoom magnification (imaging range) so that the video input device 110 captures the subject to be tracked at an appropriate size within the angle of view.

[0081] In S729, the tracking device 150B determines whether or not the user has operated an imaging system On / Off switch (not shown) to stop the imaging system. If the tracking device 150B determines that the imaging system has not been stopped (NO in S729), it proceeds to S701 in Fig. 7A, and if the tracking device 150B determines that the imaging system has been stopped (YES in S729), it ends the operation of the imaging system.

[0082] As described above, this system automatically captures images by tracking the target so that the target is captured at an appropriate position in the field of view. In order to identify the target subject, multiple parts are detected by video analysis, and if even one part is detected, tracking can be performed based on that, thereby improving the continuity of tracking. In addition, by tracking the target subject using coordinate information rather than image features, tracking can be continued even if the image features change due to changes in the direction of the target. Furthermore, in order to deal with the characteristics of the upper body and whole body, which are prone to shifts in detection coordinates depending on the direction and posture of a person, the system determines that a person is being detected even if the detection coordinates are shifted vertically. In addition, if a face is successfully detected, the pan, tilt, and zoom operations are controlled to target the coordinates of the face, which are less likely to shift, thereby improving the accuracy of the field of view control.

[0083] (Another embodiment of the present invention) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0084] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0085] 100A, 100B: Imaging system 110: Video input device 130: Pan / tilt drive device 150A, 150B: Tracking device 151: Video input unit 152: First detection unit 153: First correction unit 154: Second detection unit 155: Second correction unit 156: Third detection unit 157: Third correction unit 158A, 158B: Tracking coordinate generation unit 159: ID assignment unit 160A, 160B: Target coordinate calculation unit 161: Pan / tilt drive control unit 162: Video distribution unit 163: Tracking target input unit 164: Communication unit 165: Zoom position calculation unit 166: Zoom drive control unit 170: Recording device 190: Zoom drive device

Claims

1. a plurality of detection means for detecting coordinates indicating areas of a plurality of parts of a subject from an image captured by the imaging means; a correction means for calculating corrected coordinates of the detected part based on coordinates indicating the area of ​​the part, the corrected coordinates being corrected according to a preset magnification and relative position; a tracking coordinate generating means for generating tracking coordinates using the corrected coordinates of the plurality of parts calculated by the correcting means, the distance between the regions of the parts being within a threshold; and a control unit that controls an imaging range of the imaging unit using the generated tracking coordinates.

2. The plurality of detection means include a first detection means for detecting coordinates indicating a face area of ​​the subject; a second detection means for detecting coordinates indicating an upper body region of the subject; 2. The imaging control device according to claim 1, further comprising a third detection means for detecting coordinates indicating a whole body area of ​​the subject.

3. The imaging control device according to claim 1 or 2, characterized in that the tracking coordinate generation means associates, with one subject, corrected coordinates of a plurality of parts calculated by the correction means, where the distance between the center points of the areas of the parts is within a threshold value, and sets the average value of the associated corrected coordinates as the tracking coordinates of the one subject.

4. 4. The imaging control device according to claim 3, further comprising: a horizontal threshold value and a vertical threshold value greater than the horizontal threshold value; and the tracking coordinate generation means associates with one subject corrected coordinates in which the horizontal distance between center points of the area of ​​the part is within the horizontal threshold value and the vertical distance between center points is within the vertical threshold value.

5. The imaging control device according to claim 1 or 2, characterized in that the tracking coordinate generation means sets the corrected coordinates of the part having the highest set reliability among the parts whose coordinates indicating the area of ​​the part have been detected as the tracking coordinates.

6. The imaging control device according to any one of claims 1 to 5, characterized in that the correction means performs position correction of the center position of the detected area of ​​the part at a predetermined relative position, and calculates corrected coordinates of the part by enlarging or reducing the height and width of the detected area of ​​the part at a predetermined magnification.

7. a setting means for setting one of the one or more tracking coordinates generated by the tracking coordinate generating means as a tracking target; 7. The imaging control device according to claim 1, wherein the control means controls an imaging range of the imaging means by using the tracking coordinates of the tracking target set by the setting means.

8. The imaging control device according to any one of claims 1 to 7, characterized in that the control means instructs at least one of a pan operation, a tilt operation, and a zoom operation on the imaging means so that the tracking coordinates of the tracking target are located at a predetermined position within the imaging range of the imaging means.

9. An imaging means for capturing an image and generating an image; a plurality of detection means for detecting coordinates indicating areas of a plurality of parts of a subject from the image captured by the imaging means; a correction means for calculating corrected coordinates of the detected part based on coordinates indicating the area of ​​the part, the corrected coordinates being corrected according to a preset magnification and relative position; a tracking coordinate generating means for generating tracking coordinates using the corrected coordinates of the plurality of parts calculated by the correcting means, the distance between the regions of the parts being within a threshold; and a control unit that controls an imaging range of the imaging unit using the generated tracking coordinates.

10. a plurality of detection steps for detecting coordinates indicating areas of a plurality of parts of a subject from an image captured by an imaging means; a correction step of calculating corrected coordinates of the detected part based on coordinates indicating the area of ​​the part, the corrected coordinates being corrected according to a preset magnification and relative position; a tracking coordinate generating step of generating tracking coordinates using the corrected coordinates of the plurality of parts calculated in the correcting step, the distance between the regions of the parts being within a threshold; and a control step of controlling an imaging range of the imaging means by using the generated tracking coordinates.

11. a detection step of detecting coordinates indicating areas of a plurality of parts of the subject from an image captured by the imaging means; a correction step of calculating corrected coordinates of the detected part based on coordinates indicating the area of ​​the part, the corrected coordinates being corrected according to a preset magnification and relative position; a tracking coordinate generating step of generating tracking coordinates using the corrected coordinates of the plurality of parts calculated in the correcting step, the distance between the regions of the parts being within a threshold; and a control step of controlling an imaging range of the imaging means by using the generated tracking coordinates.

Images