Imaging control device, imaging control method, and program

Abstract

To perform imaging in accordance with a gesture of a person whose image is to be picked up.SOLUTION: An imaging control device analyzes video information to detect a human body that performs a gesture and motion information on the gesture. The imaging control device determines if the detected gesture is a gesture that requires a change in angle of view, and when the gesture is the gesture that requires a change in angle of view, further determines if the time elapsed from the time point at which the gesture that requires a change in angle of view is previously performed is within a predetermined time. When determining that the gesture is performed within the predetermined time, the imaging control device detects attention areas in the video information corresponding to the gesture that requires a change in angle of view that is previously performed and the gesture that requires a change in angle of view that is performed this time. The imaging control device generates a pan operation instruction for changing the angle of view based on all the detected attention areas.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an imaging control device, an imaging control method, and a program. [Background technology]

[0002] It has been known in the past that various operations of a camera are controlled by a subject making a predetermined gesture (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-223244 A Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to capture an image with an appropriate angle of view in accordance with the gesture of a person to be imaged. [Means for solving the problem]

[0005] An image capture control device according to one aspect of the present invention includes: an image input unit that inputs image information from an image capture device that captures an image of a person to be captured; Movement information A first determination means for determining whether a gesture is a predetermined gesture; means By What? The gesture determined to be the predetermined gesture First motion information and a second motion information The predetermined gesture is determined. Be Ta Second movement information and The elapsed time is specified of A second determination means for determining whether the predetermined gesture is within a certain time period, and Depending on The image capturing apparatus includes: an area detection means for detecting an area of ​​interest in the image information; and an angle-of-view control means for controlling the image capturing apparatus so as to change the angle of view based on all of the areas of interest detected by the area detection means. and when the first determination means determines that the first movement information is the predetermined gesture within the predetermined time and then determines that the second movement information is the predetermined gesture, the area detection means detects an attention area in the video information according to the first movement information and the second movement information. It is characterized by: [Brief description of the drawings]

[0006] [Figure 1a] 1 is a diagram illustrating an example of an overall configuration of an imaging system according to a first embodiment. [Figure 1b] FIG. 2 is a diagram illustrating an example of a configuration of an imaging control device. [Diagram 2] 4 is a flowchart showing a view angle control process according to the first embodiment. FIG. [Diagram 3] 4A to 4C are diagrams illustrating a method of calculating an angle of view according to the first embodiment. [Figure 4] FIG. 11 is a diagram illustrating an example of the overall configuration of an imaging system according to a second embodiment. [Diagram 5] FIG. 11 is a flowchart showing a view angle control process according to the second embodiment. [Figure 6] FIG. 11 is a diagram showing feature information according to the second embodiment. [Figure 7] 13 is a diagram illustrating a method for calculating an angle of view according to the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, a first embodiment of the present invention will be described with reference to the accompanying drawings.

[0008] <First embodiment> (Overall configuration of imaging system) 1a is a diagram showing an example of an overall configuration of an imaging system 100 according to a first embodiment. The imaging system 100 includes an imaging control device 130 according to the first embodiment. The imaging system 100 is a system for capturing images by changing the angle of view according to a gesture made by a person to be imaged.

[0009] 1a, the imaging system 100 includes a video input device 110, a pan drive device 120, an imaging control device 130, and a recording device 140. The imaging control device 130 and the recording device 140 are connected via a video interface. Each component will be described below.

[0010] The video input device 110 is a device that captures an image of the surroundings and generates video information, and is configured with a camera, etc. The video input device 110 outputs the video information to the imaging control device 130. The video input device 110 is an example of an imaging device. The pan driving device 120 is a device that changes the imaging direction (pan direction) of the video input device 110, and is composed of a motor, gears, etc. When the pan driving device 120 receives a series of operation commands for a pan operation from the imaging control device 130, it drives the motor to perform a pan operation according to the received operation commands.

[0011] The imaging control device 130 is a device that analyzes video information captured by the video input device 110 and controls the video input device 110 to change the angle of view. Specifically, the imaging control device 130 generates an operation command for a pan operation based on a gesture detected from the video information. The imaging control device 130 outputs the generated operation command to the pan driving device 120. The imaging control device 130 also outputs the video information input from the video input device 110 to the recording device 140. The recording device 140 performs a process of storing the video information input from the imaging control device 130 in a storage device such as a HDD (hard disk drive).

[0012] (Hardware configuration of the imaging control device) 1b, a description will be given of the hardware configuration of the imaging control device 130. The imaging control device 130 has, as hardware, a CPU 151, a storage device 152, an input / output interface 153, and a bus 154 connecting these. The CPU 151 controls the entire imaging control device 130. The CPU 151 executes processing based on a program stored in the storage device 152 or the like to realize various functions of the imaging control device 130 and the processing of the flowchart shown in FIG. The storage device 152 is a RAM, a ROM, a HDD, or the like, and stores programs and data used when the CPU 151 executes processes based on the programs. The input / output interface 153 controls input / output between the imaging control device 130 and external devices such as the video input device 110. The input / output interface 153 includes a video interface that connects the imaging control device 130 and the recording device 140.

[0013] (Functional configuration of the imaging control device) 1a, the imaging control device 130 functions as a video input unit 131, a human body detection unit 132, a gesture detection unit 133, a view angle change determination unit 134, a consecutive operation determination unit 135, an attention area detection unit 136, a view angle setting unit 137, a video distribution unit 138, and a communication unit 139 by the CPU 151 executing a program stored in the storage device 152 or the like. Each component of the imaging control device 130 will be described in detail below.

[0014] Video input unit 131 inputs video information captured by video input device 110, and outputs it to human body detection unit 132 and video distribution unit 138. Video input unit 131 also records the video information in storage device 152. Video input unit 131 records, for example, the past 60 frames of video information in storage device 152. Video input unit 131 corresponds to a video input means.

[0015] The human body detection unit 132 detects a human body by analyzing the video information input from the video input unit 131. Specifically, the human body detection unit 132 performs a human body detection process for detecting a human body in the video information. For example, a template matching method or a method of detecting a human body by deep learning is used as a method of detecting a human body. When the human body detection unit 132 detects a human body, it acquires coordinate information (area information) of the human body. In addition, the human body detection unit 132 assigns a unique ID to all detected human bodies. In this case, the human body detection unit 132 refers to past human body detection results recorded in the storage device 152, and if the detected human body is the same as a human body detected in the past, it assigns the same ID as the human body detected in the past. The human body detection unit 132 outputs the video information, the coordinate information of the human body, and the ID information to the gesture detection unit 133. The coordinate information is output in absolute coordinates according to a spherical coordinate system. The human body detection unit 132 corresponds to a person detection means.

[0016] The gesture detection unit 133 detects a gesture by analyzing the video information and the like input from the human body detection unit 132. Specifically, the gesture detection unit 133 detects a gesture made by a human body in a video by using video information of the past 60 frames read from the storage device 152. Furthermore, the storage device 152 stores past coordinate information and video information of a human body for each ID of the human body, and the gesture detection unit 133 detects a gesture for each ID of the human body. As a gesture detection method, for example, a block matching method or a method of detecting from the movement of a human body by deep learning is used. When the gesture detection unit 133 detects a gesture, it selects the coordinate information and ID information of the human body that detected the gesture from the coordinate information and ID information of the human body input from the human body detection unit 132. Then, it outputs the coordinate information and ID information of the selected human body to the view angle change determination unit 134 together with the movement information and video information of the detected gesture. The gesture detection unit 133 corresponds to a gesture detection means.

[0017] The angle-of-view change determination unit 134 determines whether the gesture detected by the gesture detection unit 133 is a gesture that requires a change in the angle of view. The determination process executed by the angle-of-view change determination unit 134 corresponds to a first determination process. Specifically, the angle-of-view change determination unit 134 reads out movement information of a gesture that requires a change in the angle of view (hereinafter, referred to as an angle-of-view change gesture; a predetermined gesture) that is stored in advance in the storage device 152. Then, the angle-of-view change determination unit 134 matches the movement information of the read angle-of-view change gesture with the movement information of the gesture input from the gesture detection unit 133. If the matching is successful, the angle-of-view change determination unit 134 determines that the detected gesture is a gesture that requires a change in the angle of view, and records the time in the storage device 152 as the time when the angle-of-view change gesture was performed (corresponding to the time when the first determination process was performed). Further, the angle-of-view change determination unit 134 records the information input from the gesture detection unit 133 in the storage device 152 in association with the time when the angle-of-view change gesture was made. That is, the storage device 152 stores the time when the angle-of-view change gesture was made and the movement information of the gesture (corresponding to the gesture to be determined in the first determination process) in chronological order. Then, the angle-of-view change determination unit 134 outputs the human body coordinate information, ID information, gesture movement information, and video information input from the gesture detection unit 133 to the consecutive operation determination unit 135. Note that, if the above matching fails, the angle-of-view change determination unit 134 determines that the detected gesture is not a gesture that requires a change in the angle of view, and does not output anything. The angle-of-view change determination unit 134 corresponds to a first determination means.

[0018] The consecutive operation determination unit 135 determines whether the angle-of-view change gestures are performed consecutively. Specifically, when the consecutive operation determination unit 135 receives an input from the angle-of-view change determination unit 134, it calculates the time that has elapsed since the previous angle-of-view change gesture was performed to the present. If the elapsed time is within a predefined time (predetermined time), it determines that the previous angle-of-view change gesture and the current angle-of-view change gesture are consecutive gestures. Then, it outputs to the attention area detection unit 136 the information input from the current angle-of-view change determination unit 134 and the information input from the previous angle-of-view change determination unit 134 together. For example, the consecutive operation determination unit 135 reads out information stored in association with the time points at which the current and previous angle-of-view change gestures were performed from the storage device 152, and outputs to the attention area detection unit 136. If the elapsed time exceeds a predefined time (predetermined time), the currently performed view angle change gesture is determined to be a single gesture, and only the information input from the currently performed view angle change determination unit 134 is output to the attention area detection unit 136. The consecutive operation determination unit 135 corresponds to a second determination means.

[0019] The attention area detection unit 136 detects the attention area pointed by the gesture. For example, when a human body makes a gesture of pointing, the angle of the pointing is calculated from the information of the estimated joints of the human body detected by using deep learning, thereby estimating the pointed area, and the estimated area is detected as the attention area. When the movement information of a plurality of gestures is input from the continuous operation determination unit 135, the attention area is detected for the movement information of all the input gestures. The attention area detection unit 136 outputs the coordinate information of the human body input from the continuous operation determination unit 135 and the coordinate information of all the attention areas detected by the attention area detection unit 136 to the angle of view setting unit 137. The coordinate information is output in absolute coordinates according to a spherical coordinate system. The attention area detection unit 136 corresponds to an area detection means.

[0020] The angle of view setting unit 137 calculates an angle of view so that the human body and all of the attention areas are included, based on the coordinate information of the human body input from the attention area detection unit 136 and the coordinate information of all of the attention areas. The method of calculating the angle of view will be described in detail later with reference to FIG. 3. Then, the angle of view setting unit 137 generates a pan operation command so as to obtain the calculated angle of view. The angle of view setting unit 137 outputs the generated pan operation command to the pan driving device 120. The angle of view setting unit 137 corresponds to an angle of view control means. When video information is input from video input unit 131 , video distribution unit 138 outputs the video information to communication unit 139 . The communication unit 139 outputs the video information input from the video distribution unit 138 to the recording device 140 .

[0021] (How to calculate the angle of view) Here, an example of a method for calculating the angle of view in the first embodiment will be described with reference to Fig. 3. Image 300 is an image captured by controlling the angle of view in imaging system 100. Image 300 includes areas 301, 302, and 303. Area 301 is an area corresponding to coordinate information of a human body that has made a gesture. Areas 302 and 303 are areas corresponding to coordinate information of attention areas, respectively.

[0022] The angle of view setting unit 137 calculates a left edge coordinate 304 and a right edge coordinate 306. The left edge coordinate 304 is the horizontal coordinate (hereinafter, the horizontal coordinate is referred to as the x coordinate) located at the leftmost end among the coordinates of the human body and all the attention areas. The left edge margin 305 is the distance in x coordinate from the left edge of the angle of view of the video 300 to the left edge coordinate 304. Similarly, the right edge coordinate 306 is the x coordinate located at the right edge of the angle of view of the human body and all the attention areas. The right edge margin 307 is the distance in x coordinate from the right edge of the angle of view of the video 300 to the right edge coordinate 306. The angle of view setting unit 137 calculates the angle of view in the horizontal direction (pan direction) so that the distances between the left edge margin 305 and the right edge margin 307 are the same.

[0023] (Angle of view control processing) Next, a view angle control process executed by the imaging system 100 of the first embodiment will be described with reference to the flowchart of Fig. 2. The view angle control process is started when the imaging system 100 is started by a user operation. In S101, the CPU 151 acquires video information from the video input device 110, and records the acquired video information in the storage device 152. Then, the process proceeds to S102. In S102, the CPU 151 performs a human body detection process on the acquired image information. Then, the process proceeds to S103. In S103, the CPU 151 determines whether or not a human body has been detected as a result of the human body detection process. If the CPU 151 determines that a human body has been detected, the process proceeds to S104. If the CPU 151 determines that a human body has not been detected, the process returns to S101. In S104, the CPU 151 determines whether or not a new human body that has not been detected in the past has been detected. If the CPU 151 determines that a new human body has been detected, the process proceeds to S105. If the CPU 151 determines that the human body is the same as the human body detected in the past, the same ID as the human body detected in the past is assigned to the new human body, and the process proceeds to S106.

[0024] In S105, the CPU 151 assigns a new ID to the newly detected human body, and the process proceeds to S106. In S106, the CPU 151 detects a gesture made by a human body in the image, using the coordinate information of the human body obtained in S102, the ID information, and past coordinate information of the human body and image information stored in the storage device 152. Then, the process proceeds to S107. In S107, the CPU 151 determines whether or not a gesture has been detected. If the CPU 151 determines that a gesture has been detected, the process proceeds to S108. If the CPU 151 determines that a gesture has not been detected, the process returns to S101. In S108, the CPU 151 performs matching between the movement information of the angle-of-view changing gesture stored in the storage device 152 and the movement information of the detected gesture. Then, the process proceeds to S109.

[0025] In S109, the CPU 151 determines whether the motion information of the detected gesture matches the motion information of the view angle changing gesture. If the CPU 151 determines that the motion information of the detected gesture matches, the motion information of the gesture determined to match is recorded in the storage device 152 in association with the time when the motion information of the detected gesture matches, and the process proceeds to S110. If the CPU 151 determines that the motion information of the detected gesture does not match, the process returns to S101. In S110, the CPU 151 calculates the time that has elapsed since it was determined in the previous S109 that a match occurred. Then, the process proceeds to S111. Note that, if it is determined in S109 that a match occurred for the first time, the process proceeds to S113. In S111, CPU 151 determines whether the calculated elapsed time has exceeded a predefined time. If CPU 151 determines that the calculated elapsed time is within the predefined time (within the specified time), the process proceeds to S112. If CPU 151 determines that the calculated elapsed time has exceeded the predefined time (outside the specified time), the process proceeds to S113.

[0026] In S112, the CPU 151 focuses on both the gesture movement information determined to be a match in the previous S109 and the gesture movement information determined to be a match in the current S109. Then, the process proceeds to S114. In S113, the CPU 151 focuses only on the gesture movement information determined to be a match in S109, and the process proceeds to S114. In S114, the CPU 151 detects the coordinate information of the attention area designated by the gesture using the motion information of the gesture that is the attention object. When both the motion information of the previous gesture and the motion information of the current gesture are the attention objects, the CPU 151 detects the coordinate information of the attention area for each of the gesture motion information. Then, the process proceeds to S115. In S115, the CPU 151 calculates the angle of view based on the coordinate information of all of the attention areas detected in S114 and the coordinate information of the human body obtained in S102. Then, the process proceeds to S116.

[0027] In S116, the CPU 151 generates a pan operation command to obtain the calculated angle of view, and outputs the generated operation command to the pan driving device 120. Then, the process proceeds to S117. In S117, the CPU 151 determines whether or not a gesture detection function ON / OFF switch (not shown) of the imaging system 100 has been turned OFF. If the CPU 151 determines that the gesture detection function ON / OFF switch has not been turned OFF, the process returns to S101. If the CPU 151 determines that the gesture detection function ON / OFF switch has been turned OFF, the process ends.

[0028] As described above, in the imaging system 100 of the first embodiment, when the person to be imaged makes consecutive gestures, it is possible to perform imaging so that the optimal angle of view is taken into consideration of the consecutive gestures. For example, when distributing a video of a lecture, if the lecturer points alternately at the slide and the writing on the board in a short time when detailing the materials projected as slides with writing on the board, both the slide and the writing on the board will be included within the angle of view. Also, when the person to be imaged makes a single gesture, it is possible to perform imaging so that the angle of view is taken into consideration only of the single gesture. This makes it possible to capture a suitable image captured at the angle of view intended by the person to be imaged.

[0029] As a modified example, in the present embodiment, CPU 151 records the gesture information determined to be a match in association with the time point at which it was determined to be a match in S109 of Fig. 2, but may record the coordinate information of the attention area detected in S114 of Fig. 2. In this case, when the movement information of the previous gesture is the attention target, in S114, CPU 151 may read out the coordinate information of the attention area recorded in association with the time point at which it was determined to be a match in the previous S109.

[0030] <Second embodiment> A second embodiment of the present invention will now be described. (Overall configuration of imaging system) Next, an example of the overall configuration of an imaging system 200 according to the second embodiment will be described with reference to FIG. 4. As shown in FIG. 4, the imaging system 200 according to the second embodiment has a video input device 110, a pan-tilt-zoom driving device 220, an imaging control device 230, and a recording device 140. In the second embodiment, the pan-tilt-zoom driving device 220 is used instead of the pan driving device 120 of the first embodiment. The imaging system 200 also includes an imaging control device 230 according to the second embodiment. The same components as those in the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted. Below, the differences from the first embodiment will be mainly described.

[0031] The pan-tilt-zoom driving device 220 is a device that changes the imaging direction (pan / tilt direction) and zoom ratio of the video input device 110, and is composed of a motor, gears, etc. When the pan-tilt-zoom driving device 220 receives a series of operation commands for pan-tilt-zoom operations from the imaging control device 230, it controls the motor to perform pan-tilt-zoom operations in accordance with the received operation commands. As in the first embodiment, the imaging control device 230 is a device that analyzes image information captured by the image input device 110 and controls the image input device 110 to change the angle of view. Specifically, the imaging control device 230 generates an operation command for pan-tilt-zoom operation based on a gesture detected from the image information. The imaging control device 230 outputs the generated operation command to the pan-tilt-zoom driving device 220.

[0032] (Hardware configuration of the imaging control device) As in FIG. 1b, the imaging control device 230 has, as hardware, a CPU 151, a storage device 152, an input / output interface 153, and a bus 154 connecting these. The CPU 151 controls the entire imaging control device 230. The CPU 151 executes processing based on a program stored in the storage device 152 or the like to realize various functions of the imaging control device 230 and the processing of the flowchart shown in FIG. The storage device 152 is a RAM, a ROM, a HDD, or the like, and stores programs and data used when the CPU 151 executes processes based on the programs. The input / output interface 153 controls input / output between the imaging control device 230 and external devices such as the video input device 110. The input / output interface 153 includes a video interface that connects the imaging control device 230 and the recording device 140.

[0033] (Functional configuration of the imaging control device) 4, the imaging control device 230 functions as the video input unit 131, the human body detection unit 132, the gesture detection unit 133, the gesture feature determination unit 231, the angle of view change determination unit 134, the consecutive operation determination unit 135, the attention area detection unit 136, the angle of view setting unit 137, the video distribution unit 138, and the communication unit 139. The imaging control device 230 further includes a gesture feature determination unit 231 in addition to the components of the imaging control device 130 of the first embodiment. The gesture feature determination unit 231 and components having functions different from those of the first embodiment will be described in detail below.

[0034] The gesture feature determination unit 231 determines feature information of a gesture detected by the gesture detection unit 133. Specifically, a database in which feature information related to gestures is registered (referred to as a feature information database) is stored in advance in the storage device 152, and the gesture feature determination unit 231 reads out the feature information database upon receiving an input from the gesture detection unit 133. Then, the gesture feature determination unit 231 searches the feature information database using the gesture movement information detected by the gesture detection unit 133 as a key, and acquires feature information corresponding to the detected gesture. The gesture feature determination unit 231 outputs the coordinate information of the human body, the ID information, the movement information of the detected gesture, the gesture feature information, and the video information input from the gesture detection unit 133 to the view angle change determination unit 134.

[0035] FIG. 6 is an example of the feature information database. Feature information on gestures is set in association with gesture movement information 601. Examples of the gesture movement information 601 include "pointing at an area," "drawing a circle around an area with a finger," "making an X with both hands," and "lowering both hands that have been raised up." The gesture feature determination unit 231 matches the gesture movement information 601 with the gesture movement information detected by the gesture detection unit 133, and acquires feature information that is set in association with the matching gesture movement information 601. The acquired feature information is used for processing executed by the view angle change determination unit 134 and the continuous operation determination unit 135. The feature information is composed of a view angle change necessity 602, a gesture continuous acceptance possibility 603, a gesture continuous acceptance time correction value 604, a gesture inhibition processing flag 605, and a gesture inhibition time 606.

[0036] In the field of view angle change necessity 602, whether or not a change of the field of view is necessary is set. In the field of gesture continuous acceptance possibility 603, whether or not the judgment by the continuous operation judgment unit 135 is valid or invalid is set. In the field of gesture continuous acceptance time correction value 604 (corresponding to correction time), the field of view angle change determination unit 135 calculates the continuous acceptance time. In the field of gesture inhibition processing flag 605, whether to inhibit the change of the field of view or to release the inhibition is set. In the field of gesture inhibition processing flag 605, the field of view angle change inhibition release time is set. In the field of gesture inhibition processing flag 605, whether to inhibit the change of the field of view is set ...

[0037] In the second embodiment, when the gesture feature information of the gesture is input from the gesture feature determination unit 231, the view angle change determination unit 134 sets the internal state to the view angle change inhibited state if the gesture inhibit processing flag 605 of the feature information is "inhibited". Furthermore, the view angle change determination unit 134 calculates the view angle change inhibit release time using the time set in the gesture inhibit time 606 of the feature information, and stores it in the storage device 152. The gesture inhibit time 606 is the duration of the view angle change inhibited state. For example, if the gesture inhibit time 606 is "30 seconds", the time 30 seconds after the current time is stored as the view angle change inhibit release time. Furthermore, the view angle change determination unit 134 releases the setting of the view angle change inhibited state if the gesture inhibit processing flag 605 of the feature information is "inhibited" or if the current time is later than the view angle change inhibit release time stored in the storage device 152.

[0038] In the second embodiment, the angle-of-view change determination unit 134 determines whether the internal state is not set to the angle-of-view change inhibited state and whether the angle-of-view change necessity 602 of the gesture feature information input from the gesture feature determination unit 231 is "necessary". Then, when these conditions are met, the angle-of-view change determination unit 134 determines that the angle of view change is necessary. In the second embodiment, when the angle-of-view change determination unit 134 determines that the angle of view change is necessary, it stores the time in the storage device 152 as the time when the angle-of-view change gesture was performed (corresponding to the time when the first determination process was executed). In addition, the angle-of-view change determination unit 134 records the information input from the gesture feature determination unit 231 in the storage device 152 in association with the time when the angle-of-view change gesture was performed. Then, the angle-of-view change determination unit 134 outputs the human body coordinate information, ID information, movement information of the detected gesture, feature information, and video information input from the gesture feature determination unit 231 to the continuous operation determination unit 135.

[0039] In the second embodiment, the consecutive operation determination unit 135 calculates a consecutive reception time used to determine whether the view angle change gesture has been performed consecutively. First, the consecutive operation determination unit 135, when receiving an input from the view angle change determination unit 134, calculates the elapsed time from the time when the previous view angle change gesture was performed to the present, as in the first embodiment. Next, the consecutive operation determination unit 135 compares the elapsed time with the consecutive reception time. The consecutive reception time Y is calculated by the following formula (1) with the current number of consecutively received gestures as X and the gesture consecutive reception time correction value 604 of the feature information previously input to the consecutive operation determination unit 135 as Z. Note that the method of calculating the consecutive reception time is not limited to the method using the following formula (1), and may be calculated by other methods as long as it changes according to the number of consecutively received gestures. The number of consecutively received gestures corresponds to the number of times that the consecutive operation determination unit 135 has determined that the gestures are consecutive. Y=(1.0+(X-1)×0.5))×Z···(1)

[0040] If the elapsed time is shorter than the continuous reception time as a result of the comparison, the consecutive operation determination unit 135 determines that the previous view angle change gesture and the current view angle change gesture are consecutive gestures. Then, the consecutive operation determination unit 135 adds 1 to the number of consecutively received gestures stored in the storage device 152. In this way, the consecutive operation determination unit 135 counts the number of consecutively determined gestures as consecutive gestures. Furthermore, the consecutive operation determination unit 135 outputs to the attention area detection unit 136 the information input this time from the view angle change determination unit 134 and all the information input from the view angle change determination unit 134 up to the previous time. For example, the consecutive operation determination unit 135 reads out from the storage device 152 a number of pieces of information corresponding to the number of consecutively received gestures in order of newness from among the pieces of information stored in association with the time point when the view angle change gesture was performed, and outputs the pieces of information to the attention area detection unit 136. Furthermore, if the comparison result indicates that the elapsed time is equal to or longer than the continuous reception time, the consecutive operation determination unit 135 determines that the currently performed angle-of-view change gesture is a single gesture. The consecutive operation determination unit 135 sets the number of consecutively received gestures stored in the storage device 152 to 1, and outputs only the information input from the currently performed angle-of-view change determination unit 134 to the attention area detection unit 136.

[0041] However, in the second embodiment, when the continuous gesture acceptance possibility 603 of the feature information input from the angle of view change determination unit 134 is not “possible”, the continuous operation determination unit 135 determines that the currently performed angle of view change gesture is a single gesture, regardless of the result of the comparison. In this case, the continuous operation determination unit 135 sets the number of continuously accepted gestures stored in the storage device 152 to 1, and outputs only the information currently input from the angle of view change determination unit 134 to the attention area detection unit 136. In this way, when the continuous gesture acceptance possibility 603 of the feature information is “possible”, the determination process by the continuous operation determination unit 135 is enabled, and when the continuous gesture acceptance possibility 603 of the feature information is not “possible”, the determination process by the continuous operation determination unit 135 is disabled.

[0042] In the second embodiment, the angle-of-view setting unit 137 calculates an angle of view so as to include the human body and all of the attention areas, based on the coordinate information of the human body and the coordinate information of all of the attention areas input from the attention area detection unit 136. The method of calculating the angle of view will be described in detail later with reference to Fig. 7. Then, the angle-of-view setting unit 137 generates a pan operation command, a tilt operation command, and a zoom operation command so as to obtain the calculated angle of view.

[0043] (How to calculate the angle of view) Here, an example of a method for calculating the angle of view in the second embodiment will be described with reference to Fig. 7. The following description will be divided into a pan direction, a tilt direction, and a zoom ratio. An image 700 is an image captured by controlling the angle of view in the imaging system 200. The image 700 includes an area 701, an area 702, an area 703, an area 704, and an area 705. The area 701 is an area corresponding to the coordinate information of the human body that performed the gesture. The areas 702, 703, 704, and 705 are areas corresponding to the coordinate information of the attention area, respectively.

[0044] First, a method of calculating the angle of view in the pan direction will be described. The angle of view setting unit 137 calculates the x coordinate of the leftmost position among all the coordinates of all the bodies and all the attention areas, such as the left edge coordinate 706. The left edge margin 708 is the distance in x coordinate from the left edge of the angle of view of the image 700 to the left edge coordinate 706. Similarly, the right edge coordinate 707 is the x coordinate of the right edge of all the bodies and the attention area, and the right edge margin 709 is the distance in x coordinate from the right edge of the angle of view of the image 700 to the right edge coordinate 707. The angle of view setting unit 137 calculates the angle of view in the horizontal direction (pan direction) so that the distance between the left edge margin 708 and the right edge margin 709 is the same.

[0045] Next, a method of calculating the angle of view in the tilt direction will be described. The angle of view setting unit 137 calculates the vertical coordinate (hereinafter, referred to as y coordinate) located at the top end among the coordinates of all the bodies and the attention area, such as the top end coordinate 710. The top end margin 712 is the distance in y coordinate from the top end of the angle of view of the image 700 to the top end coordinate 710. Similarly, the bottom end coordinate 711 is the y coordinate of the bottom end of all the bodies and the attention area, and the bottom end margin 713 is the distance in y coordinate from the bottom end of the angle of view of the image 700 to the bottom end coordinate 711. The angle of view setting unit 137 calculates the angle of view in the vertical direction (tilt direction) so that the distance between the top end margin 712 and the bottom end margin 713 is the same.

[0046] Finally, a method for calculating the zoom ratio will be described. The zoom magnification of the angle of view is calculated so as to satisfy the following two conditions. The first condition is a condition for zooming in the x coordinate, in which, assuming that the width 715 is the horizontal length of the angle of view of the image 700, the distance between the left margin 708 and the right margin 709 is 10% or more of the length of the width 715. The second condition is a condition for zooming in the y coordinate, in which, assuming that the height 714 is the vertical length of the angle of view of the image 700, the distance between the top margin 712 and the bottom margin 713 is 10% or more of the length of the height 714. The angle of view setting unit 137 calculates the maximum zoom ratio while satisfying these two conditions.

[0047] (Angle of view control processing) Next, a view angle control process executed by the imaging system 200 of the second embodiment will be described with reference to the flowchart of Fig. 5. The view angle control process is started when the imaging system 200 is started by a user operation.

[0048] First, in steps S201 to S205, the CPU 151 executes the same processes as those in steps S101 to S105 of FIG. In S206, the CPU 151 detects gestures made by a human body in the video. In the second embodiment, the CPU 151 detects gestures that match any of the gesture movement information 601 in the feature information database stored in the storage device 152 from among the gestures made by the human body in the video. In S207, CPU 151 determines whether or not a gesture has been detected. If CPU 151 determines that a gesture has been detected, the process proceeds to S208. If CPU 151 determines that a gesture has not been detected, the process returns to S201. In S208, the CPU 151 acquires feature information of the detected gesture from the feature information database, and the process proceeds to S209.

[0049] In S209, the CPU 151 determines whether or not the gesture suppression processing flag 605 of the feature information acquired in S208 is “suppressed.” If the CPU 151 determines that the gesture suppression processing flag 605 is “suppressed,” the process proceeds to S210. If the CPU 151 determines that the gesture suppression processing flag 605 is not “suppressed,” the process proceeds to S212. In S210, the CPU 151 sets the internal state to a view angle change inhibit state, and the process proceeds to S211. In S211, the CPU 151 calculates the view angle change inhibition release time from the time set in the gesture inhibition time 606 of the feature information acquired in S208, and stores the time in the storage device 152. Then, the process proceeds to S212.

[0050] In S212, the CPU 151 determines whether or not the gesture suppression processing flag 605 of the feature information acquired in S208 is “Release suppression”. If the CPU 151 determines that the gesture suppression processing flag 605 is “Release suppression”, the process proceeds to S214. If the CPU 151 determines that the gesture suppression processing flag 605 is not “Release suppression”, the process proceeds to S213. In S213, the CPU 151 determines whether or not the current time is later than the angle of view change inhibition release time stored in the storage device 152. If the CPU 151 determines that the current time is later than the angle of view change inhibition release time, the process proceeds to S214. If the CPU 151 determines that the current time is not later than the angle of view change inhibition release time, the process proceeds to S215.

[0051] In S214, the CPU 151 releases the internal state of the view angle change inhibit state, and the process proceeds to S215. In S215, the CPU 151 determines whether the internal state is the angle of view change inhibited state. If the CPU 151 determines that the internal state is the angle of view change inhibited state, the process returns to S201. If the CPU 151 determines that the internal state is not the angle of view change inhibited state, the process proceeds to S216.

[0052] In S216, the CPU 151 determines whether the angle of view change necessity 602 of the characteristic information acquired in S208 is "necessary". If the CPU 151 determines that the angle of view change necessity 602 is "necessary", the process proceeds to S217. If the CPU 151 determines that the angle of view change necessity 602 is "unnecessary", the process returns to S201. In S217, the CPU 151 calculates the elapsed time from the time when the necessity for changing the angle of view 602 was determined to be "necessary" in the previous S216 to the present. Then, the process proceeds to S217. Note that, if it is the first time that the necessity for changing the angle of view 602 is determined to be "necessary" in S216, the process proceeds to S225.

[0053] In S218, the CPU 151 acquires the number of consecutive accepted gestures stored in the storage device 152. Then, the process proceeds to S219. In S219, the CPU 151 acquires the gesture continuous acceptance time correction value 604 of the feature information acquired in S208. Note that the CPU 151 may read out the feature information previously acquired in S208 from the storage device 152 and acquire the gesture continuous acceptance time correction value 604 of the read out feature information. Then, the process proceeds to S220.

[0054] In S220, the CPU 151 calculates the continuous acceptance time by using the number of continuous accepted gestures and the gesture continuous acceptance time correction value 604. Then, the process proceeds to S221. In S221, the CPU 151 determines whether the elapsed time calculated in S217 is equal to or greater than the continuous reception time calculated in S220. If the CPU 151 determines that the elapsed time is equal to or greater than the continuous reception time, the process proceeds to S225. If the CPU 151 determines that the elapsed time is less than the continuous reception time, the process proceeds to S222. In S222, the CPU 151 determines whether the consecutive gesture acceptance 603 of the feature information acquired in S208 is "possible". If the CPU 151 determines that the consecutive gesture acceptance 603 is "possible", the movement information of the gesture determined to be "possible" is recorded in the storage device 152 in association with the time when it was determined to be "possible", and the process proceeds to S223. If the CPU 151 determines that the consecutive gesture acceptance 603 is not "possible", the process proceeds to S225. In S223, the CPU 151 adds 1 to the number of consecutive accepted gestures stored in the storage device 152. Then, the process proceeds to S224. In S224, the CPU 151 regards the movement information of all consecutively performed gestures as a target of attention, and the process proceeds to S227.

[0055] In S225, the CPU 151 sets the number of consecutive accepted gestures stored in the storage device 152 to 1. Then, the process proceeds to S226. In S226, the CPU 151 focuses on only the gesture movement information for which the view angle change necessity 602 has been determined to be “necessary” in the current S216. Then, the process proceeds to S227. In S227, the CPU 151 detects coordinate information of the attention area designated by the gesture using the movement information of all the gestures that have been targeted for attention. Then, the process proceeds to S228.

[0056] In S228, the CPU 151 calculates the angle of view based on the coordinate information of all of the attention areas detected in S227 and the coordinate information of the human body obtained in S102. Then, the process proceeds to S229. In S229, the CPU 151 generates a pan-tilt-zoom operation command so as to obtain the calculated angle of view, and outputs the generated operation command to the pan-tilt-zoom drive device 220. Then, the process proceeds to S230. In S230, the CPU 151 determines whether or not a gesture detection function ON / OFF switch (not shown) of the imaging system 100 has been turned OFF. If the CPU 151 determines that the gesture detection function ON / OFF switch has not been turned OFF, the process returns to S201. If the CPU 151 determines that the gesture detection function ON / OFF switch has been turned OFF, the process ends.

[0057] As described above, in the imaging system 200 of the second embodiment, when a person to be imaged performs consecutive gestures, it is possible to perform imaging so that the optimal angle of view is obtained by considering all the consecutive gestures. For example, when three or more gestures are performed consecutively in a short time, imaging can be performed so that the optimal angle of view is obtained by considering the three or more consecutive gestures. In addition, depending on the gesture performed, it is possible to realize control that considers only the last gesture even if consecutive gestures are performed, or control that suppresses the change of the angle of view. This makes it possible to capture a suitable image captured at the angle of view intended by the person to be imaged.

[0058] Although the present invention has been described above with reference to the embodiments, the above embodiments are merely illustrative of the specific examples of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be embodied in various forms without departing from its technical concept or main features.

[0059] (Other embodiments) The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a recording medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0060] 100, 200: imaging system, 110: video input device, 120: pan drive device, 130, 230: imaging control device, 140: recording device, 220: pan / tilt / zoom drive device

Claims

1. a video input unit for inputting video information from an imaging device that images a person as an imaging target; a first determination means for analyzing the video information and determining whether the person's movement information is a predetermined gesture; a second determination means for determining whether an elapsed time between a first piece of movement information determined by the first determination means to be the predetermined gesture and a second piece of movement information determined to be the predetermined gesture subsequent to the first piece of movement information is within a predetermined time; an area detection means for detecting an attention area in the video information in response to the movement information of the predetermined gesture; a field-of-view control means for controlling the imaging device so as to change the field-of-view angle based on all of the attention areas detected by the area detection means; having When the first determination means determines that the first movement information is the predetermined gesture and then the second movement information is determined to be the predetermined gesture within the predetermined time, the area detection means detects an attention area in the video information according to the first movement information and the second movement information. An imaging control device comprising:

2. The imaging control device according to claim 1, characterized in that the area detection means detects the area of ​​interest corresponding to the second motion information when the second determination means does not determine that it is within the specified time, and does not detect the area of ​​interest for the first motion information.

3. 3. The imaging control device according to claim 1, further comprising a storage means for storing, when the first determination means determines that the person's movement information is the specified gesture, the movement information of the gesture that was the subject of the determination in association with the time when the determination process was performed by the first determination means.

4. The image processing device further includes a person detection unit that detects area information of the person by analyzing the image information, 4. The imaging control device according to claim 1, wherein the angle-of-view control means controls the imaging device so as to change the angle of view based on area information of a person detected by the person detection means.

5. The second determination means counts the number of times that the predetermined gesture is consecutively determined by the first determination means within the predetermined time period, the area detection means detects the attention area corresponding to the movement information of the predetermined gesture determined by the first determination means within the predetermined time based on the number of times counted by the second determination means; 5. The imaging control device according to claim 1, wherein the second determination means changes the predetermined time based on the counted number of times.

6. a determination process by the second determination means is set to be valid or invalid for the predetermined gesture; the area detection means detects an attention area in the video information corresponding to each of the first movement information and the second movement information when the setting of the predetermined gesture is valid and when the second determination means determines that the setting is within the predetermined time, An imaging control device as described in any one of claims 1 to 5, characterized in that when the setting of the specified gesture is invalid, even if the second judgment means judges that it is within the specified time, the attention area corresponding to the movement information of the specified gesture is detected, and the attention area is not detected for the first movement information.

7. the imaging control device has a gesture detection means for detecting a gesture of the person, the first determination means determines whether the gesture detected by the gesture detection means is a gesture for inhibiting a change in an angle of view, and when it is determined that the gesture is a gesture for inhibiting a change in the angle of view, sets a state in which an angle of view change is inhibited; The imaging control device according to any one of claims 1 to 6, characterized in that when the angle of view control means is in the angle of view change suppression state, the angle of view is controlled not to be changed even if the first determination means determines that the gesture is the specified gesture.

8. the first determination means determines whether the gesture detected by the gesture detection means is a gesture for canceling the view angle change inhibiting state, and when it is determined that the gesture detected by the gesture detection means is a gesture for canceling the view angle change inhibiting state, cancels the view angle change inhibiting state; 8. The imaging control device according to claim 7, wherein the angle of view control means controls to change the angle of view when the first determination means determines that the gesture is the predetermined gesture after the angle of view change suppression state is released.

9. a duration of the angle-of-view change inhibiting state is set for the gesture for inhibiting the change of the angle of view, the first determination means cancels the angle-of-view change inhibiting state when the duration time has elapsed since the time point when the gesture is determined to be a gesture for inhibiting the change of the angle of view; 9. The imaging control device according to claim 7, wherein the angle of view control means controls to change the angle of view when the first determination means determines that the gesture is the specified gesture after the angle of view change inhibition state is released.

10. 10. The imaging control device according to claim 1, wherein the angle-of-view control means performs control so that all of the attention areas detected by the area detection means are included in the angle of view of the imaging device.

11. 11. The imaging control device according to claim 1, wherein the angle-of-view control means controls an operation of the imaging device in a panning direction.

12. 12. The imaging control device according to claim 1, wherein the angle-of-view control means controls an operation of the imaging device in a tilt direction.

13. 13. The imaging control device according to claim 1, wherein the angle-of-view control means controls a zoom ratio of the imaging device.

14. an image input step of inputting image information from an imaging device that images a person as an imaging target; a first determination step of analyzing the video information to determine whether the person's movement information is a predetermined gesture; a second determination step of determining whether an elapsed time between first movement information determined to be the predetermined gesture in the first determination step and second movement information determined to be the predetermined gesture subsequent to the first movement information is within a predetermined time; a region detection step of detecting a region of interest in the video information according to movement information of the predetermined gesture; a field of view control step of controlling the imaging device so as to change the field of view based on all of the regions of interest detected by the region detection step; Including, When the first motion information is determined to be the predetermined gesture by the first determination step within the predetermined time and then the second motion information is determined to be the predetermined gesture, the area detection step detects an attention area in the video information according to the first motion information and the second motion information.

4. An imaging control method comprising:

15. A program for causing a computer to function as each of the means of the imaging control device according to any one of claims 1 to 13.

Images