Photographing control device, photographing control method, and program

Abstract

To acquire an angle of view including a target subject inside the angle of view and a target subject outside the angle of view.SOLUTION: A photographing control device controls the angle of view of a photographing device that has driving means capable of pan-tilt driving, and comprises: setting means that sets a plurality of preset angles of view; control means that controls the driving means to the present angles of view; coordinate conversion means that, when at least one of a plurality of target subjects is detected in a photographed image, converts the coordinates in the angle of view of the detected target subject into a pan-tilt coordinate value; storage means that stores the pan-tilt coordinate value; and angle of view acquisition means that, when a pan-tilt coordinate value of another target subject different from the detected target subject is stored, acquires an angle of view including the detected target subject and the other target subject, on the basis of the target subject and the pan-tilt coordinate value of the other target subject. When the target subject is not detected in the photographed image, the photographing control device controls the driving means to a preset angle of view that is not used for photographing, of the plurality of preset angles of view.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a control technique for an image capturing device. [Background technology]

[0002] In recent years, there is an increasing need for a camera that automatically captures a lecture scene. When a lecture scene is automatically captured and displayed, a close-up image of the lecturer, who is the target subject, is normally displayed, and when the lecturer points to a blackboard or the like, an image including both the lecturer and the target subject of the area pointed to is displayed. Patent Document 1 discloses a technique for capturing an image of a podium or the like at a predetermined wide-angle reference angle of view, determining the center of gravity coordinates of the lecturer and the center of gravity coordinates of the pointing area from the coordinates within the reference angle of view, and calculating the angle of view including both the lecturer and the target subject of the pointing area based on the center of gravity coordinates. Patent Document 1 also discloses a technique for cutting out an image of an angle of view including both the target subjects from an image captured at a reference angle of view by digital PTZ processing. Patent Document 1 also discloses a technique for preparing a camera for a reference angle of view and a camera for tracking, and controlling the angle of view of the tracking camera by PTZ according to the angle of view including both the target subjects. Note that PTZ is an abbreviation of pan (P), tilt (T), and zoom (Z). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-158680 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 can only acquire images within a predetermined reference angle of view. For example, when a lecturer or the like points out a whiteboard that is outside the reference angle of view, it is not possible to calculate the angle of view that includes the lecturer and the whiteboard, since there is no coordinate within the reference angle of view that corresponds to the whiteboard. In other words, the technology described in Patent Document 1 cannot calculate the angle of view that includes a target subject within a predetermined angle of view and another target subject that is outside the angle of view at that time, and therefore cannot acquire an image that includes both of these target subjects.

[0005] Therefore, an object of the present invention is to make it possible to obtain an angle of view that includes target subjects within the angle of view and target subjects that were outside the angle of view at that time, and to obtain an image that includes these target subjects. [Means for solving the problem]

[0006] The present invention is an imaging control device that controls an angle of view of an imaging device having a driving means capable of pan / tilt drive, comprising: a setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to the preset angles of view set by the setting means; a coordinate conversion means for converting coordinates within the angle of view of the detected target subject into pan / tilt coordinate values ​​when at least one of a plurality of target subjects is detected in an image captured by the imaging device; a storage means for storing the pan / tilt coordinate values ​​of the target subject converted by the coordinate conversion means; and an angle of view acquisition means for acquiring an angle of view that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject when pan / tilt coordinate values ​​of a target subject other than the detected target subject are stored in the storage means, and is characterized in that when a target subject is not detected in the captured image, the driving means is controlled to a preset angle of view that is not captured among the plurality of preset angles of view. Effect of the Invention

[0007] According to the present invention, it is possible to obtain an angle of view that includes a target subject within the angle of view and a target subject that was outside the angle of view at that time, and to obtain an image that includes these target subjects. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating the configuration of an automatic photography system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing a target subject to be captured within the angle of view. [Diagram 3] 11 is an explanatory diagram of a process for converting coordinates within the angle of view into PT coordinate values. [Figure 4] FIG. 2 is a diagram illustrating a current angle of view. [Diagram 5] FIG. 2 is an explanatory diagram of the angle of view control according to the first embodiment. [Figure 6] 4 is a flowchart of a view angle adjustment process according to the first embodiment. [Figure 7] FIG. 11 is a block diagram showing the configuration of an automatic photography system according to a second embodiment. [Figure 8] 11 is an explanatory diagram of a process for extracting a board region within an object. FIG. [Figure 9] FIG. 11 is an explanatory diagram of the angle of view control according to the second embodiment. [Figure 10] 10 is a flowchart of a view angle adjustment process according to the second embodiment. [Figure 11] FIG. 11 is a block diagram showing the configuration of an automatic photography system according to a third embodiment. [Figure 12] 13 is an explanatory diagram of the angle of view control according to the third embodiment. FIG. [Figure 13] 13 is a flowchart of a view angle adjustment process according to the third embodiment. [Figure 14] FIG. 13 is a block diagram showing the configuration of an automatic photography system according to a fourth embodiment. [Figure 15] FIG. 1 is a diagram showing the movement history of a human body within a certain period of time. [Figure 16] 1 is a diagram showing the PT coordinate values ​​of an object and the PT coordinate values ​​of a human body. [Figure 17] 13 is an explanatory diagram of the angle of view control according to the fourth embodiment. FIG. [Figure 18] FIG. 13 is a diagram showing another example of the movement history of a human body within a certain period of time. [Figure 19] 11A and 11B are explanatory diagrams of angle of view control based on the movement history of a human body. [Figure 20] 13 is a flowchart of a view angle adjustment process according to the fourth embodiment. [Figure 21] FIG. 13 is a block diagram showing the configuration of an automatic photography system according to a fifth embodiment. [Figure 22] 11 is an explanatory diagram of an example of calculating the angle of view based on the P coordinates of a human body and an object. FIG. [Figure 23] 11 is an explanatory diagram of an example of calculating the angle of view based on the T coordinates of a human body and an object. FIG. [Figure 24] 13 is a flowchart of a view angle adjustment process according to the fifth embodiment. [Diagram 25] FIG. 2 is a diagram illustrating an example of a hardware configuration of a view angle adjustment device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings. In the following embodiments, the same configurations and processes are described with the same reference numerals.

[0010] First Embodiment An example of the configuration of an imaging control device according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the functional configuration of an automatic imaging system 100 including a field-of-view adjustment device 102, which is an imaging control device according to this embodiment. The automatic photography system 100 of this embodiment is configured to include an image capture device 101, a field of view adjustment device 102, and a monitor device 114. The field of view adjustment device 102 and the monitor device 114 are connected via a video interface.

[0011] The image acquisition device 101 is configured to have an imaging device such as a camera that captures an object and its surroundings to generate a captured image. The imaging device has a zoom optical system and is configured to be able to adjust the zoom magnification by controlling the zoom value. The imaging device also includes a camera platform, and the pan direction can be adjusted by controlling the pan value for the camera platform, and similarly, the tilt direction can be adjusted by controlling the tilt value. Therefore, the imaging device is configured to be able to adjust the angle of view by controlling the pan value, tilt value, and zoom value. The image acquisition device 101 outputs image information captured by the imaging device to the angle of view adjustment device 102.

[0012] The angle-of-view adjustment device 102 has a function of an imaging control device that adjusts the angle of view of the imaging device of the image acquisition device 101. In this embodiment, the angle-of-view adjustment device 102 detects, for example, a human body as a first target subject and an object as a second target subject from an image captured by the imaging device (hereinafter referred to as a camera) of the image acquisition device 101. For example, in the case of automatically capturing an image of a scene in which a lecturer is giving a lecture, the human body as the first target subject can be, for example, the lecturer giving the lecture, and the object as the second target subject can be a blackboard or whiteboard used in the lecture. The angle-of-view adjustment device 102 converts the coordinate value of the body in the angle of view when the person is included in the angle of view and the coordinate value of the object in the angle of view when the object is included in the angle of view into PT coordinate values ​​(pan coordinate value / tilt coordinate value) and stores them. Thereafter, even if the human body is within the angle of view and the object at that time is outside the angle of view, the angle of view adjustment device 102 calculates the angle of view using the stored PT coordinate values, obtains an angle of view that includes the human body and the object, and adjusts the angle of view of the camera to obtain that angle of view. Then, the captured image after the angle of view adjustment by the angle of view adjustment device 102 is output to the monitor device 114 and displayed.

[0013] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 102 determines whether an object to be included in the angle of view as a target subject has been detected. When the angle-of-view adjustment device 102 has not detected the target subject object, it adjusts the angle of view of the camera using a preset function, and performs object detection using an image of the target subject object. When the angle-of-view adjustment device 102 detects the target subject object, it converts the coordinates of the object in the angle of view at that time into PT coordinate values ​​using the PTZ (pan-tilt-zoom) value of the camera and stores the converted coordinates. In addition, the angle-of-view adjustment device 102 detects a human body as the target subject from within the image captured by the camera, and converts the coordinates of the human body in the angle of view into PT coordinate values ​​using the PTZ value of the camera, as in the case of the target subject object described above, and stores the converted coordinates. Based on the PT coordinate values ​​of the human body and object obtained and stored in this way, the angle-of-view adjustment device 102 calculates the angle of view that includes the human body in the angle of view when the camera is capturing the image and the object that is outside the angle of view at that time, and adjusts the angle of view according to the calculated angle of view. Thereafter, the angle-of-view adjustment device 102 outputs image information obtained after adjusting the angle of view to the monitor device 114 .

[0014] The angle of view adjustment device 102 has an image input unit 103, a detection and judgment unit 104, an angle of view storage unit 105, an object detection unit 106, a human body detection unit 107, a coordinate conversion unit 108, a position storage unit 109, a coordinate storage unit 110, an angle of view calculation unit 111, an angle of view operation unit 112, and an image output unit 113. The image input unit 103 receives image information from the image acquisition device 101 and outputs the received image information to the detection and determination unit 104 .

[0015] The detection and determination unit 104 determines whether or not an object that is a target subject to be included in the current angle of view of the camera has been detected. If the object of the target subject has not been detected in the current angle of view of the camera, the detection and determination unit 104 outputs the image information input from the image input unit 103 to the object detection unit 106. On the other hand, if the object of the target subject has been detected in the current angle of view of the camera, the detection and determination unit 104 inquires of the angle of view storage unit 105 whether or not there is a preset angle of view that has not yet been used.

[0016] The angle of view storage unit 105 stores information on multiple preset angles of view input by user operation, etc., and information on whether each preset angle of view has been used, and outputs the information in response to an acquisition request from the detection and judgment unit 104.

[0017] The preset angles of view will be explained with reference to Fig. 2. Fig. 2 is a diagram showing a human body P1 as a target subject, and objects P2 and P3 as other target subjects to be captured in the angle of view. The range shown by the dashed line in Fig. 2 represents the preset angles of view, with angle of view D2 representing the preset angle of view that captures object P2 as a target subject, and angle of view D3 representing the preset angle of view that captures object P3 as a target subject. In this way, there are a plurality of preset angles of view, and these preset angles of view are made up of a plurality of different preset positions and the shooting range of the camera at each of these positions.

[0018] If there is no unused preset angle of view among the information stored in angle of view storage unit 105, detection and determination unit 104 outputs the image information input from image input unit 103 to human body detection unit 107. On the other hand, if there is an unused preset angle of view among the information stored in angle of view storage unit 105, detection and determination unit 104 outputs information about the preset angle of view to angle of view operation unit 112, which will be described later. In other words, by outputting information about the preset angle of view to angle of view operation unit 112, detection and determination unit 104 sets the angle of view of the camera to the preset angle of view.

[0019] The object detection unit 106 performs an object detection process to detect an object in an image from the image information input from the detection and determination unit 104. For the object detection process, a highly accurate and high-speed technology based on Deep Learning can be used. Examples of object detection processes based on Deep Learning include Faster R-CNN, YOLO, SSD, and Masked R-CNN. The object detection unit 106 uses any of these object detection processing technologies, but is not limited to these. Faster R-CNN, YOLO, SSD, and Masked R-CNN are well-known technologies, so detailed descriptions thereof will be omitted. Then, when the object detection unit 106 detects an object in the image, it outputs the coordinate value within the angle of view of the detected object and image information to the coordinate conversion unit 108.

[0020] The human body detection unit 107 performs a human body detection process to detect a human body in an image from the image information input from the detection and determination unit 104. The human body detection process may use any method capable of detecting a human body, such as a template matching method or a semantic region division method. The template matching method and the semantic region division method are well-known techniques, and therefore detailed descriptions thereof will be omitted. When the human body detection unit 107 detects a human body in an image, it outputs the coordinate values ​​within the angle of view of the detected human body and image information to the coordinate conversion unit 108. In this embodiment, the human body detection unit 107 outputs the coordinate values ​​of the four vertices of a rectangle (detection rectangle) that contains the human body detected in the image as the coordinate values ​​of the detected person.

[0021] Coordinate values ​​and image information of an object are input to coordinate conversion section 108 from object detection section 106 , and coordinate values ​​and image information of a human body are input to coordinate conversion section 108 from human body detection section 107 . Furthermore, the position storage unit 109 stores the current PTZ value of the camera, and outputs information about the PTZ value to the coordinate conversion unit 108 . The coordinate conversion unit 108 converts the coordinates within the angle of view of the object and the human body into PT coordinate values ​​based on the coordinate values ​​and image information of the object, the coordinate values ​​and image information of the human body, and the current PTZ value information of the camera. The coordinate conversion unit 108 then outputs the PT coordinate values ​​and image information of the object and the human body to the coordinate storage unit 110.

[0022] The PT coordinate conversion process in which the coordinate conversion unit 108 converts the coordinates of the object and the human body within the angle of view into PT coordinate values ​​will be described with reference to FIG. 3. In FIG. 3, an image D21 is an image in which the coordinates of the human body detected by the human body detection unit 107 are expressed in coordinates within the angle of view. Also, an image D22 is an image in which the coordinates within the angle of view representing the human body in the image D21 are expressed in PT coordinate values. That is, the human body P21 in the image D21 and the human body P22 in the image D22 are the same person at the same time. The rectangle indicated by the dashed line surrounding the human body P21 in the image D21 represents the detection rectangle indicating the human body detected by the human body detection unit 107. In the image D21, the coordinates within the angle of view of the detection rectangle of the human body P21 are expressed by four coordinates, (x1, y1), (x2, y2), (x3, y3), and (x4, y4), in the order of the upper left vertex, the upper right vertex, the lower left vertex, and the lower right vertex. In addition, if the pan value of the camera when the image in which the human body is detected is represented by θ, the tilt value by Φ, and the zoom value by ω, the PTZ value is represented by (θ, Φ, ω). Coordinate conversion unit 108 converts the coordinates of the human body and object within the angle of view into PT coordinate values ​​using the coordinate values ​​within the angle of view and the PTZ value of the camera.

[0023] The conversion process of converting the coordinates in the angle of view into the PT coordinate values ​​can be realized by using a technique such as conversion from planar coordinates to spherical coordinates, calculation of coordinates projected onto an arbitrary plane perpendicular to the optical axis of the camera, and conversion of the coordinates of the projection plane into the rotation angles of the pan angle and tilt angle. These techniques are well known, and detailed explanations thereof will be omitted. The conversion process using these techniques is merely an example, and any method that can convert the coordinates in the angle of view into the PT coordinate values ​​may be used. Taking the coordinates in the angle of view of person P21 as an example, the PT coordinate values ​​after conversion are expressed by four coordinates, (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4), in the order of the top left, top right, bottom left, and bottom right vertices of the rectangle of person P22 in image D22. The coordinates (θ, Φ) in image D22 represent the PT coordinate values ​​of the center P23 of the angle of view.

[0024] The coordinate storage unit 110 stores the PT coordinate values ​​of the human body and object and the image information input from the coordinate conversion unit 108 , and outputs them to the angle of view calculation unit 111 . The angle-of-view calculation unit 111 calculates the angle of view of the camera that allows the human body within the current angle of view of the camera and the object outside the angle of view to be included, based on the PT coordinate values ​​and image information of the human body and object read from the coordinate storage unit 110. In the present embodiment, the angle-of-view calculation unit 111 calculates a PTZ value for determining the angle of view of the camera that allows the human body within the current angle of view and the object outside the angle of view to be included.

[0025] The manner in which the angle-of-view calculation unit 111 calculates the PTZ value for determining the angle of view of the camera that accommodates a human body within the current angle of view of the camera and an object outside the angle of view will be described with reference to FIGS. In FIG. 4, the angle of view D31 represents the current angle of view of the camera. Assume that the human body P31 corresponds to the human body P1 illustrated in FIG. 2, the object P32 corresponds to the object P2 in FIG. 2, and the object P33 corresponds to the object P3 in FIG. 2. Assume that the rectangular PT coordinate values ​​of the human body P31 are represented by four coordinate values ​​of the top left, top right, bottom left, and bottom right vertices, respectively: (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4). Assume that the rectangular PT coordinate values ​​of the rectangular object P33 are represented by four coordinate values ​​of the top left, top right, bottom left, and bottom right vertices, respectively: (θ5, Φ5), (θ6, Φ6), (θ7, Φ7), and (θ8, Φ8). Assume that the rectangular PT coordinate values ​​of the rectangular object P32 are represented by four coordinate values ​​of the top left, top right, bottom left, and bottom right vertices, respectively: (θ9, Φ9), (θ 10 ,Φ 10 ), (θ 11 ,Φ 11 ), (θ 12 ,Φ 12 4 is an angle of view that includes a human body P31 and an object P33, and the object P32 is outside the current angle of view of the camera. However, since the PT coordinate value of the object P32 is acquired in the PT coordinate conversion process when the preset angle of view is used and is stored in the coordinate storage unit 110, the angle of view calculation unit 111 can acquire the PT coordinate value of the object P32 from the coordinate storage unit 110.

[0026] FIG. 5 is a diagram for explaining how the angle of view calculation unit 111 calculates the angle of view that contains the human body P31 and the object P32 based on these PT coordinate values. First, a method of determining the angle of view in the angle of view calculation unit 111 will be described. The angle of view is uniquely determined once the center point, horizontal angle of view value, and vertical angle of view value are determined. The horizontal angle of view value and vertical angle of view value vary depending on the focal length of the camera, and the focal length of a camera having a zoom optical system as in this embodiment is determined by the zoom magnification. Generally, the image size is determined according to the aspect ratio of the image, so once one of the horizontal angle of view value and the vertical angle of view value is determined, the other is also determined. For these reasons, in this embodiment, the angle of view calculation unit 111 determines the angle of view by calculating the center point and the horizontal angle of view value corresponding to the zoom magnification.

[0027] In Fig. 5, the human body P41 corresponds to the human body P31 in Fig. 4, and similarly, the object P42 corresponds to the object P32 in Fig. 4, and the object P43 corresponds to the object P33 in Fig. 4. Therefore, the PT coordinate values ​​of the human body P41, the object P42, and the object P43 in Fig. 5 are the same as the PT coordinate values ​​of the human body P31, the object P32, and the object P33 in Fig. 4. In addition, the center of the angle of view that contains the human body P41 and the object P42 is P44, its PT coordinate value is (θ', Φ'), and the horizontal angle of view value is P45, whose value is ω'. The PT coordinate value (θ', Φ') and the horizontal angle of view value ω' of the center P44 can be calculated as follows.

[0028] The horizontal angle of view ω' can be calculated as the difference between the P coordinate values ​​(pan coordinate values) of the human body P41 and the object P42, as in equation (1).

[0029] ω'=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 ) Formula (1)

[0030] In the formula (1), max() represents obtaining the maximum value in the parentheses, and min() represents obtaining the minimum value in the parentheses.

[0031] Moreover, the P coordinate value θ′ of the center P44 of the angle of view can be calculated as the average value of the P coordinate values ​​of the human body P41 and the object P42, as in equation (2).

[0032] θ'=(max(θ1,θ2,θ9,θ 10 )+min(θ1,θ2,θ9,θ 10 )) / 2 Equation (2)

[0033] Moreover, the T coordinate value (tilt coordinate value) of the center P44 of the angle of view can be calculated as the average value of the T coordinate values ​​of the human body P41 and the object P42, as in equation (3).

[0034] Φ'=(max(Φ1,Φ2,Φ9,Φ 10)+min(Φ1,Φ2,Φ9,Φ 10 )) / twenty three)

[0035] The angle-of-view calculation unit 111 calculates the angle of view D41 as described above, and outputs the angle of view and image information to the angle-of-view operation unit 112. In this embodiment, in addition to the angle-of-view acquisition process by calculation as described above, the angle-of-view calculation unit 111 may also perform an angle-of-view acquisition process in which, for example, an angle of view is selected from a table in which a plurality of angles of view are prepared in advance.

[0036] The angle of view operation unit 112 operates the angle of view of the camera, that is, performs adjustment control of the angle of view of the camera, using the angle of view and image information input from the angle of view calculation unit 111, or the preset angle of view and image information input from the detection and determination unit 104. After that, the angle of view operation unit 112 outputs the image information after the angle of view adjustment to the image output unit 113. The image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. In response to this, the monitor device 114 displays an image based on the image information input from the image output unit 113 on the monitor.

[0037] 6 is a flowchart showing the flow of the view angle adjustment process in the view angle adjustment device 102 among the processes related to the automatic photography system 100 of the first embodiment. In the following explanation of each flowchart, the alphabet S attached to the reference numerals indicates a processing step. The automatic photography system 100 starts the process for automatic photography when a start instruction is input by a user operation. First, in S1, the image input unit 103 of the view angle adjustment device 102 acquires image information from the image acquisition device 101. Then, the processing of the view angle adjustment device 102 proceeds to S2.

[0038] In S2, the detection and determination unit 104 uses the image information input from the image input unit 103 to determine whether or not a target object to be included in the angle of view of the camera has been detected, that is, whether or not a target object is present in the image. The target object determined by the detection and determination unit 104 in S2 is a target subject of a human body or an object. If the determination result of S2 is true (YES in S2), that is, if a target object to be included in the angle of view has been detected, the detection and determination unit 104 advances the process to S4. On the other hand, if the determination result of S2 is false (NO in S2), that is, if a target object to be included in the angle of view has not been detected, the detection and determination unit 104 outputs the image information to the object detection unit 106, and the process of the angle of view adjustment device 102 advances to S3.

[0039] In S3, the object detection unit 106 performs a detection process for an object, for example, a second target subject, using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 102 proceeds to S7.

[0040] When proceeding to S4, the detection and determination unit 104 determines whether there is another object of a second target subject to be detected, using information on whether the preset angle of view obtained from the angle of view storage unit 105 was used. If the determination result of S4 is true (YES in S4), that is, if there is no other object to be detected, the detection and determination unit 104 outputs image information to the human body detection unit 107. Then, the processing of the angle of view adjustment device 102 proceeds to S6. On the other hand, if the determination result of S4 is false (NO in S4), that is, if there is another object to be detected, the detection and determination unit 104 proceeds to S5.

[0041] When the process proceeds from S4 to S5, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105. Then, the process of the angle of view adjustment device 102 proceeds to S10. When the process proceeds from S5 to S10, the view angle operation unit 112 performs view angle operation in S10 based on the information of the preset view angle so that the object is contained within the preset view angle. Then, the process of the view angle adjustment device 102 proceeds to S11.

[0042] In S11, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 102 proceeds to S12.

[0043] In S12, the angle-of-view adjustment device 102 determines whether or not a user has operated an automatic photography system on / off switch (not shown) to stop the automatic photography process. If the result of the determination in S12 is false (NO in S12), that is, if the angle-of-view adjustment device 102 has determined that the switch has not been turned off, it returns to S1. On the other hand, if the result of the determination in S12 is true (YES in S12), that is, if the angle-of-view adjustment device 102 has determined that the switch has been turned off, it ends the process of the flowchart in FIG. 6.

[0044] As described above, when the process proceeds from S4 to S5, and then returns to S1 via the processes of S10, S11, and S12, and then proceeds from S2 to S3, the object detection unit 106 performs object detection processing using image information acquired at the preset angle of view. The object detection unit 106 then outputs the coordinates and image information of the object detected within the preset angle of view to the coordinate conversion unit 108.

[0045] Also, when the process proceeds from S4 to S6, the human body detection unit 107 performs a human body detection process using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 102 proceeds to S7.

[0046] When the process proceeds from S3 to S7, in S7 the coordinate conversion unit 108 converts the coordinates of the object within the angle of view into PT coordinate values ​​based on the coordinates and image information of the object obtained from the object detection unit 106 and the current PTZ value of the camera obtained from the position storage unit 109. Also, when the process proceeds from S6 to S7, in S7 the coordinate conversion unit 108 converts the coordinates of the human body within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body obtained from the human body detection unit 107 and the current PTZ value of the camera obtained from the position storage unit 109. Then, the coordinate conversion unit 108 outputs the converted PT coordinate values ​​to the coordinate storage unit 110. Then, the process of the angle of view adjustment device 102 proceeds to S8.

[0047] In S8, the coordinate storage unit 110 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108. That is, the PT coordinate values ​​of the human body and the object are stored in the coordinate storage unit 110. Then, the processing of the angle of view adjustment device 102 proceeds to S9. In S9, the angle-of-view calculation unit 111 calculates the angle of view in which the human body within the angle of view and the object that is outside the angle of view at that time are accommodated, from the PT coordinate values ​​of the human body and the object acquired from the coordinate storage unit 110. That is, the angle-of-view calculation unit 111 calculates the angle of view in which the human body within the angle of view and the object that was outside the angle of view before the angle of view operation was performed by the subsequent angle-of-view operation unit 112 are accommodated. Then, the angle-of-view calculation unit 111 outputs information on the calculated angle of view to the angle-of-view operation unit 112. After that, the processing of the angle-of-view adjustment device 102 proceeds to S10.

[0048] In S10, the angle-of-view operation unit 112 performs an angle-of-view operation based on the information on the angle of view acquired from the angle-of-view calculation unit 111. That is, the angle-of-view operation unit 112 performs an angle-of-view operation so that a human body and an object are included. Then, the process of the angle-of-view adjustment device 102 proceeds to S11.

[0049] In S11, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114, and an image based on the image information is displayed on the monitor device 114. In S12, the angle of view adjustment device 102 determines whether a stop operation has been performed by operating the automatic photography system On / Off switch. If the determination result in S12 is false (NO in S12), the angle of view adjustment device 102 returns to S1, and if the determination result in S12 is true (YES in S12), the process of the flowchart in FIG. 6 is terminated.

[0050] The angle-of-view adjustment device 102 of the first embodiment converts coordinates within the angle of view when an object is detected in advance using a preset angle of view or the like into PT coordinate values ​​and stores them, and calculates an angle of view that includes a human body within the angle of view and an object that was outside the angle of view at that time using the PT coordinate values. That is, in this embodiment, when an object that is to be included in the current angle of view of the camera is outside the angle of view, it is possible to obtain an image including the human body and the object by calculating an angle of view that includes the human body within the angle of view and the object outside the angle of view and adjusting the angle of view. As a result, according to this embodiment, in an automatic photography system that automatically photographs a scene in which a lecturer is giving a lecture, for example, it is possible to continue photographing the lecturer and the object.

[0051] <Second embodiment> Hereinafter, an example of the configuration of the imaging control device according to the second embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of an automatic imaging system 200 including a field-of-view adjustment device 202, which is an imaging control device according to this embodiment. The automatic imaging system 200 of the second embodiment has an image acquisition device 101, a field-of-view adjustment device 202, and a monitor device 114. The field-of-view adjustment device 202 and the monitor device 114 are connected via a video interface. Note that in the second embodiment as well, an application in which a scene in which a lecturer is giving a lecture is automatically captured will be taken as an example.

[0052] The angle-of-view adjustment device 202 of the second embodiment detects a human body, an object, and an attention area on the object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and uses the PT coordinate values ​​to calculate the angle of view so that the attention area is at the edge of the angle of view. That is, when obtaining an angle of view that includes the human body and the object, the angle of view adjustment device 202 of the second embodiment calculates an angle of view such that the human body is positioned at a position with some margin from the edge of the angle of view. In the second embodiment, an example is given in which a blackboard area is used as the attention area. The blackboard area is, for example, an area on a blackboard or whiteboard used in a lecture where the lecture contents are written.

[0053] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 202 of the second embodiment determines whether an object to be included in the angle of view as a target subject has been detected, and if not, detects the object from an image captured by adjusting the angle of view using a preset function. The angle-of-view adjustment device 202 also converts the coordinates of the object in the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores the converted coordinates. The angle-of-view adjustment device 202 then detects a human body or a blackboard area in the image, and converts the coordinates in the angle of view into PT coordinate values ​​in the same way as for an object. The angle-of-view adjustment device 202 then adjusts the angle of view to a value with a margin for the human body to the end of the angle of view, with the blackboard area at the end of the angle of view, based on the PT coordinate values ​​of the human body, the object, and the blackboard area, thereby fitting the human body and the object into the angle of view. The angle-of-view adjustment device 202 then outputs the image information with the adjusted angle of view to the monitor device 114, and displays an image based on the image information.

[0054] In the view angle adjustment device 202, the image input unit 103, the detection and determination unit 104, the view angle storage unit 105, the object detection unit 106, the human body detection unit 107, the coordinate conversion unit 108, the position storage unit 109, the view angle operation unit 112, and the image output unit 113 are the same as those in the first embodiment. The view angle adjustment device 202 of the second embodiment differs from the first embodiment in that it has processes in the coordinate storage unit 210 and the view angle calculation unit 211, and has an area extraction unit 215. The processes and configurations that differ from those of the first embodiment will be described below.

[0055] The area extraction unit 215 extracts the blackboard area using the coordinate values ​​and image information of the human body input from the human body detection unit 107, or the coordinate values ​​and image information of the object input from the object detection unit 106. The blackboard area can be extracted using a known method such as a method of extracting the blackboard area based on the difference in luminance information of frame images at different times on the time axis of the video, or a method of extracting the blackboard area based on the results of character recognition or graphic recognition, and a detailed description thereof will be omitted. The area extraction unit 215 then outputs the coordinate values ​​of the blackboard area, the coordinate values ​​of the human body, and the coordinate values ​​of the object, which are expressed by the coordinates within the angle of view, to the coordinate conversion unit 108 together with the image information.

[0056] A coordinate conversion unit 108 converts the coordinates within the field of view of the board writing area, the human body, and the object into PT coordinate values ​​by the same processing as that described in the first embodiment. The coordinate storage unit 210 stores the PT coordinate values ​​of the human body, the PT coordinate values ​​of the object, the PT coordinate values ​​of the blackboard area, and the image information input from the coordinate conversion unit 108. Then, the coordinate storage unit 210 outputs the PT coordinate values ​​and the image information to the angle of view calculation unit 211.

[0057] Based on the PT coordinate values ​​of the human body, the PT coordinate values ​​of the object, the PT coordinate values ​​of the blackboard writing area, and the image information input from the coordinate storage unit 210, the angle of view calculation unit 211 calculates an angle of view in which the blackboard writing area is the edge of the angle of view and the human body is spaced apart from the edge of the angle of view while including the human body and the object.

[0058] Fig. 8 is a diagram showing PT coordinate values ​​input from the coordinate storage unit 210. Assume that a human body P51 corresponds to the human body P1 shown in Fig. 2, and objects P52 and P53 correspond to objects P2 and P3 in Fig. 2. Assume that a rectangular area surrounded by a dotted line in object P52 in Fig. 8 is a board writing area P54 extracted for object P52, and a rectangular area surrounded by a dotted line in object P53 is a board writing area P55 extracted for object P53. Assume that the PT coordinate values ​​of the rectangle representing the human body P51 in Fig. 8 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the top left, top right, bottom left, and bottom right vertices, in that order, as with the PT coordinates in Figs. 4 and 5 described above. Similarly, the PT coordinates of object P53 are (θ5, Φ5), (θ6, Φ6), (θ7, Φ7), and (θ8, Φ8) for the top left, top right, bottom left, and bottom right vertices, respectively, and the PT coordinates of object P52 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 The PT coordinate values ​​of the blackboard writing area P54 are the top left, top right, bottom left, and bottom right vertices, in that order (θ 13 ,Φ 13 ),(θ 14 ,Φ 14 ),(θ 15 ,Φ 15 ),(θ 16 ,Φ 16 The PT coordinate values ​​of the blackboard writing area P55 are the top left, top right, bottom left, and bottom right vertices, in that order (θ 17 ,Φ 17 ),(θ 18 ,Φ 18 ),(θ 19 ,Φ 19 ),(θ 20 ,Φ 20 )

[0059] FIG. 9 is a diagram showing how the angle of view calculation unit 211 of the second embodiment calculates the angle of view that includes the human body P51 and the blackboard writing area P54 based on these PT coordinate values. In FIG. 9, the human body P61 corresponds to the human body P51 shown in FIG. 8, and the object P62 and the blackboard writing area P63 correspond to the object P52 and the blackboard writing area P54 in FIG. 8. The center P64 corresponds to the center P44 of the angle of view calculated in the same manner as in the first embodiment, and its PT coordinate values ​​are (θ', Φ'). Similarly, the horizontal angle of view value P66 corresponds to the horizontal angle of view value P45 calculated in the same manner as in the first embodiment, and its horizontal angle of view value is ω'. The angle of view D61 corresponds to the angle of view D41 shown in FIG. 5 described above.

[0060] Here, in the case of angle of view D61, since human body P61 is located at the edge of the angle of view, there is a high possibility that human body P61 will move out of the angle of view due to movement or the like and be lost sight of. Therefore, in the second embodiment, the angle-of-view calculation unit 211 calculates an angle of view that is unlikely to lose sight of the human body P61 even if it moves to some extent, and that includes the blackboard writing area P63 of the object P62. In the case of the second embodiment, in order to accommodate the blackboard writing area P63 and to provide a margin for the human body P61 at the edge of the angle of view, the angle of view D62 is calculated by sliding the angle of view D61 obtained by the calculation described in the first embodiment so that the blackboard writing area P63 becomes the edge of the angle of view. The angle of view D62 shown in FIG. 9 is the angle of view after sliding. If the PT coordinate value of the center P65 of the angle of view D62 is (θ'',Φ'') and the horizontal angle of view value P67 is ω'', then the angle of view D62 can be expressed by equation (4). The angle of view calculation unit 211 then outputs the calculated angle of view D62 and image information to the angle of view operation unit 112.

[0061] θ''=θ'+(θ 13 -θ9), Φ''=Φ', Equation (4) ω''=ω'

[0062] 10 is a flowchart showing the flow of processing in the automatic photography system 200 according to the second embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 202. When the automatic photography system 200 is started by a user operation, it starts processing for automatic photography. First, in S101 , the image input unit 103 of the angle of view adjustment device 202 acquires image information from the image acquisition device 101 . Next, in S102, the detection and determination unit 104 determines whether or not a target object to be contained in the angle of view is detected using the image information input from the image input unit 103. If the determination result of S102 is true (YES in S102), the detection and determination unit 104 advances the process to S104. On the other hand, if the determination result of S102 is false (NO in S102), the detection and determination unit 104 outputs the image information to the object detection unit 106, and the process of the angle of view adjustment device 202 advances to S103.

[0063] In S103, the object detection unit 106 performs an object detection process using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the area extraction unit 215. Then, the process of the angle of view adjustment device 202 proceeds to S107.

[0064] When the process proceeds to S104, the detection and determination unit 104 determines whether or not there is another object to be detected, using information on whether or not a preset angle of view was used, acquired from the angle of view storage unit 105. When the determination result of S104 is true (YES in S104), the detection and determination unit 104 outputs image information to the human body detection unit 107. Then, the process of the angle of view adjustment device 202 proceeds to S106. On the other hand, when the determination result of S104 is false (NO in S104), the detection and determination unit 104 advances the process to S105.

[0065] In S105, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105. Then, the process of the angle of view adjustment device 202 proceeds to S111. The process in the case where the process proceeds from S105 to S111 or later is the same as the process in the case where the process proceeds from S5 to S10 or later in FIG. 6 described above.

[0066] Also, if the process proceeds to S106, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the area extraction unit 215. Then, the process of the angle of view adjustment device 202 proceeds to S107.

[0067] In S107, the area extraction unit 215 extracts a board writing area using the coordinates and image information of the object input from the object detection unit 106. Then, the area extraction unit 215 outputs the coordinates and image information of the extracted board writing area, the coordinates and image information of the human body input from the human body detection unit 107, and the coordinates and image information of the object input from the object detection unit 106 to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 202 proceeds to S108.

[0068] In S108, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the blackboard area, the coordinates and image information of the human body, the coordinates and image information of the object obtained from the area extraction unit 215, and the current PTZ value of the camera obtained from the position storage unit 109. Then, the coordinate conversion unit 108 outputs the converted PT coordinate values ​​to the coordinate storage unit 210. Then, the processing of the angle of view adjustment device 202 proceeds to S109.

[0069] In S109, the coordinate storage unit 210 stores the PT coordinate value acquired from the coordinate conversion unit 108. Then, the processing of the view angle adjustment device 202 proceeds to S110. In S110, the angle-of-view calculation unit 211 calculates an angle of view that contains the human body and the object, and that allows the human body to be positioned at the edge of the angle of view and the blackboard writing area to be positioned at a certain distance from the edge of the angle of view, from the PT coordinate values ​​of the blackboard writing area, the human body, and the object acquired from the coordinate storage unit 210. The angle-of-view calculation unit 211 then outputs information about the calculated angle of view to the angle-of-view operation unit 112. Then, the process of the angle-of-view adjustment device 202 proceeds to S111.

[0070] In S111, the angle-of-view operation unit 112 adjusts the angle of view so that the human body and the object are included, the blackboard writing area is at the end of the angle of view, and the human body has a margin from the end of the angle of view, based on the information on the angle of view acquired from the angle-of-view calculation unit 211. Then, the processing of the angle-of-view adjustment device 202 proceeds to S112. In S112, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 202 proceeds to S113.

[0071] In S113, the view angle adjustment device 202 determines whether the user has operated the automatic photography system On / Off switch to stop the automatic photography process. If the determination result in S113 is false (NO in S113), the view angle adjustment device 202 returns to S101. On the other hand, if the determination result in S113 is true (YES in S113), the view angle adjustment device 202 ends the process of the flowchart in FIG.

[0072] The angle-of-view adjustment device 202 of the second embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and stores them, calculates the angle of view based on the PT coordinate values, and controls the angle of view so that the area of ​​interest is at the edge of the angle of view and the human body has a margin from the edge of the angle of view. In other words, according to this embodiment, if an object that is to be included in the current angle of view of the camera is outside the angle of view, it is possible to continue shooting with the blackboard area at the edge of the image and the human body having a margin from the edge of the angle of view while including the human body within the angle of view and the object outside the angle of view within the angle of view.

[0073] <Third embodiment> Hereinafter, an example of the configuration of an imaging control device according to the third embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the functional configuration of an automatic imaging system 300 including an angle-of-view adjustment device 302, which is an imaging control device according to this embodiment. The automatic imaging system 300 of the third embodiment has an image acquisition device 101, an angle-of-view adjustment device 302, and a monitor device 114. The angle-of-view adjustment device 302 and the monitor device 114 are connected via a video interface. Note that in the third embodiment as well, an application in which a scene in which a lecturer is giving a lesson is automatically captured will be given as an example.

[0074] The angle-of-view adjustment device 302 of the third embodiment detects a human body and an object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and when calculating the angle of view using the PT coordinate values, adjusts the angle of view so that the human body has a margin of the width of the human body from the edge of the angle of view. In other words, the angle-of-view adjustment device 302 of the third embodiment adjusts the angle of view so that the human body has a margin from the edge of the angle of view when fitting the human body and the object into the angle of view.

[0075] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 302 of the third embodiment determines whether an object to be included in the angle of view as a target subject has been detected, and if not, detects the object from an image captured by adjusting the angle of view using a preset function. The angle-of-view adjustment device 302 converts the coordinates in the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores them. Then, the angle-of-view adjustment device 302 detects a human body in the image and converts the coordinates in the angle of view into PT coordinate values ​​in the same way as in the case of an object. Then, the angle-of-view adjustment device 302 calculates an angle of view that allows a margin of the human body's width from the edge of the angle of view based on the PT coordinate values ​​of the human body and the object, and adjusts the angle of view to include the human body and the object in the angle of view. Then, the angle-of-view adjustment device 302 outputs the image information after the angle of view adjustment to the monitor device 114 to display an image corresponding to the image information.

[0076] In the view angle adjustment device 302, the image input unit 103, the detection and determination unit 104, the view angle storage unit 105, the object detection unit 106, the human body detection unit 107, the coordinate conversion unit 108, the position storage unit 109, the coordinate storage unit 110, the view angle operation unit 112, and the image output unit 113 are the same as those described above. In the case of the view angle adjustment device 302 of the third embodiment, the process in the view angle calculation unit 311 differs from that of the first embodiment. Below, the process and configuration that differ from the first embodiment will be described.

[0077] The angle of view calculation unit 311 uses the PT coordinate values ​​and image information of the human body and object acquired from the coordinate storage unit 110 to calculate an angle of view that includes the human body within the angle of view and objects outside the angle of view, while allowing a margin of the horizontal width of the human body from the edge of the angle of view.

[0078] Fig. 12 is a diagram showing how the angle of view calculation unit 311 of the third embodiment calculates the angle of view based on these PT coordinate values. In Fig. 12, a human body P71 corresponds to, for example, the human body P31 shown in Fig. 5, and an object P72 corresponds to the object P42 in Fig. 5. In Fig. 12, the PT coordinate values ​​of a rectangle representing the human body P71 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the top left, top right, bottom left, and bottom right vertices, respectively, as described above. In addition, the PT coordinates of the object P72 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 ) The angle of view D71 represents an angle of view that includes a human body P71 and an object P72, while allowing the human body P71 to have a margin of the horizontal width (distance P75) from the edge of the angle of view. The PT coordinate value of the center P73 in the angle of view D71 is (θ''',Φ'''), and the horizontal angle of view value P74 is ω'''. The distance P75 between the detected rectangle representing the human body P71 and the edge of the angle of view is θ d Then, it can be expressed by equation (5).

[0079] θ d =θ2-θ1 Equation (5)

[0080] Furthermore, the P coordinate value θ''' of the center P73 of the angle of view can be calculated using equation (6) as the average value of the P coordinate value of the human body P71 plus the distance P75 corresponding to the width of the human body and the P coordinate value of the object P72.

[0081] θ'''=(max(θ1,θ2,θ9,θ 10 ) + θ d +min(θ1,θ2,θ9,θ 10 )) / 2 Equation (6)

[0082] Moreover, the T coordinate value Φ′″ of the center P73 of the angle of view can be calculated as the average value of the T coordinate value of the human body P71 and the T coordinate value of the object P72 by equation (7).

[0083] Φ'''=(max(Φ1,Φ2,Φ9,Φ10 )+min(Φ1,Φ2,Φ9,Φ 10 )) / 2 Equation (7)

[0084] Furthermore, the horizontal angle of view ω′″ can be calculated by adding the difference between the P coordinate values ​​of the human body P71 and the object P72 and the distance P75 corresponding to the width of the human body, using equation (8).

[0085] ω'''=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 ) + θ d Formula (8)

[0086] Then, the angle-of-view calculation unit 311 outputs the calculated angle of view and image information to the angle-of-view operation unit 112 . 13 is a flowchart showing the flow of processing in the automatic photography system 300 according to the third embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 302. When the automatic photography system 300 is started by a user operation, it starts processing for automatic photography.

[0087] First, in S 201 , the image input unit 103 of the angle of view adjustment device 302 acquires image information from the image acquisition device 101 . Next, in S202, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S202 is true (YES in S202), the detection determination unit 104 advances the process to S204, whereas if the determination result of S202 is false (NO in S202), the detection determination unit 104 outputs the image information to the object detection unit 106. Then, the process of the angle of view adjustment device 302 advances to S203.

[0088] In S203, the object detection unit 106 performs an object detection process using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 302 proceeds to S207.

[0089] When the process proceeds to S204, the detection and determination unit 104 determines whether or not there is another object to be detected, using information on whether or not a preset angle of view was used, acquired from the angle of view storage unit 105. When the determination result of S204 is true (YES in S204), the detection and determination unit 104 outputs image information to the human body detection unit 107, and then the process of the angle of view adjustment device 302 proceeds to S206. On the other hand, when the determination result of S204 is false (NO in S204), the detection and determination unit 104 advances the process to S205.

[0090] In S205, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105, and then the process of the angle of view adjustment device 302 proceeds to S210. The process when proceeding from S205 to S210 or later is similar to the process when proceeding from S5 to S10 or later in FIG. 6 described above. Also, in the case where the process proceeds to S206, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. After that, the process of the angle of view adjustment device 302 proceeds to S207.

[0091] In S207, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body or the coordinates and image information of the object, and the current PTZ value of the camera. The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 110. After that, the process of the angle of view adjustment device 302 proceeds to S208. In S208, the coordinate storage unit 110 stores the PT coordinate value acquired from the coordinate conversion unit 108, and then the process of the view angle adjustment device 302 proceeds to S209.

[0092] In S209, the angle-of-view calculation unit 311 calculates an angle of view in which the object is at the end of the angle of view and the human body has a margin of the horizontal width of the human body from the end of the angle of view, based on the PT coordinate values ​​of the human body and the object acquired from the coordinate storage unit 110. The angle-of-view calculation unit 311 further outputs information about the calculated angle of view to the angle-of-view operation unit 112. After that, the process of the angle-of-view adjustment device 302 proceeds to S210. In S210, the angle-of-view operation unit 112 performs angle-of-view operation according to an angle of view that includes the human body and the object, has the object at the end of the angle of view, and has a margin of the width of the human body from the end of the angle of view, based on the information on the angle of view acquired from the angle-of-view calculation unit 311. After that, the process of the angle-of-view adjustment device 302 proceeds to S211.

[0093] In S211, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, the monitor device 114 displays an image based on the image information. Next, in S212, the angle-of-view adjustment device 302 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the result of the determination in S212 is false (NO in S212), the process returns to S201. On the other hand, if the result of the determination in S212 is true (YES in S212), the angle-of-view adjustment device 302 ends the process of the flowchart in FIG.

[0094] The angle-of-view adjustment device 302 of the third embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and stores them, and calculates the angle of view using the PT coordinate values ​​of the human body and object as well as the width of the human body. As a result, according to this embodiment, it is possible to keep the human body and object within the angle of view while continuing to capture the image of the human body at an angle of view with a margin from the edge of the angle of view.

[0095] <Fourth embodiment> Hereinafter, an example of the configuration of an imaging control device according to the fourth embodiment will be described with reference to Fig. 14. Fig. 14 is a block diagram showing the functional configuration of an automatic imaging system 400 including a field-of-view adjustment device 402, which is an imaging control device according to this embodiment. The automatic imaging system 400 of the fourth embodiment has an image acquisition device 101, a field-of-view adjustment device 402, and a monitor device 114. The field-of-view adjustment device 402 and the monitor device 114 are connected via a video interface. Note that, in the fourth embodiment as well, an application in which a scene in which a lecturer is giving a lesson is automatically captured will be taken as an example.

[0096] The angle-of-view adjustment device 402 of the fourth embodiment detects a human body and an object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and calculates the angle of view using the movement history of the human body within a certain period of time and the PT coordinate value of the object. That is, the angle-of-view adjustment device 402 of this embodiment calculates the angle of view taking into account the movement history of the human body when acquiring the angle of view that includes the human body within the angle of view and the object that was outside the angle of view.

[0097] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 402 of the fourth embodiment determines whether an object to be included in the angle of view as a target subject has been detected, and if not, detects the object from an image captured by adjusting the angle of view using a preset function. The angle-of-view adjustment device 402 converts the coordinates in the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores the converted coordinates. Then, the angle-of-view adjustment device 402 detects a human body in the image and converts the coordinates in the angle of view into PT coordinate values ​​in the same way as the object. At this time, the angle-of-view adjustment device 402 stores the PT coordinate values ​​of the human body obtained at regular intervals within a certain period as movement history information. Then, the angle-of-view adjustment device 402 calculates the angle of view that includes the human body and the object from the movement history of the human body obtained at regular intervals within a certain period and the PT coordinate value of the object, and performs image adjustment. Then, the angle-of-view adjustment device 402 outputs the image information after the angle of view adjustment to the monitor device 114 to display an image based on the image information.

[0098] The image input unit 103, detection and determination unit 104, view angle storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, view angle operation unit 112, and image output unit 113 of the view angle adjustment device 402 are the same as those described above. The view angle adjustment device 402 of the fourth embodiment differs from the first embodiment in that it has processes in the coordinate storage unit 410 and view angle calculation unit 411, and an operation determination unit 416. The processes and configurations that differ from those of the first embodiment will be described below.

[0099] The coordinate storage unit 410 stores the PT coordinate values ​​of the human body and the object input from the coordinate conversion unit 108. The coordinate storage unit 410 of the fourth embodiment stores the PT coordinate values ​​of the human body as history information, and outputs the history information of the human body and the PT coordinate values ​​of the object for a period designated by the operation determination unit 416 described later.

[0100] The operation determination unit 416 determines whether a certain period of time has passed by timer processing. If the certain period of time has passed, the operation determination unit 416 obtains the PT coordinate value of the object and image information and historical information of the human body within the certain period from the coordinate storage unit 410, and outputs them to the angle of view calculation unit 411. If the certain period of time has not passed, the operation determination unit 416 outputs the image information obtained from the coordinate storage unit 410 to the image output unit 113.

[0101] The angle of view calculation unit 411 uses the history information of the human body and the PT coordinate values ​​and image information of the object input from the coordinate storage unit 410 to calculate the angle of view that contains the human body and the object, taking into account the movement history of the human body over a certain period of time.

[0102] 15 to 19 are diagrams for explaining how the angle of view calculation unit 411 of the fourth embodiment calculates the angle of view based on the history information of the human body and the PT coordinate values ​​and image information of the object. n However, any value may be used.

[0103] In Fig. 15, range D81 represents a range that includes the human body and each object as the target subject. n That is, the human body is moving, and at time t1 it is located at the position of human body P81, at time t2 it is located at the position of human body P82, and at time t n Assume that the object P84 is located at the position of a human body P83 at time t85. The objects P84 and P85 correspond to the objects P2 and P3 shown in FIG.

[0104] FIG. 16 is a diagram showing the PT coordinate values ​​of the human body (a rectangle corresponding to the human body) and the PT coordinate values ​​of the objects (obj1 and obj2 in FIG. 16) at each time shown in FIG. 15. In FIG. 16, the PT coordinate values ​​of the human body and the objects are the upper left coordinates (θ, Φ) of the rectangle and the width (θ d ) and height (Φ d )

[0105] Figure 17 shows the time from t1 to t n 11 is an explanatory diagram of a calculation of the angle of view based on the movement history of the human body within a period from A method for calculating the angle of view in angle of view calculation section 411 will be described below. Time t1 to t n Since the positions to which the human body has moved within the period up to this point are positions to which the human body may move in the future, the angle of view calculation unit 411 calculates an angle of view that includes all positions in the human body's movement history within that period and the object (object P94 in this example).

[0106] In FIG. 17, the PT coordinate value of the center P96 of the angle of view is (θ 4 ',Φ 4 '), and the horizontal angle of view P97 is ω 4 '. The P coordinate value θ of the center of the field of view P96 4 ' can be calculated from the movement history of the human body and the P coordinate value of the object P94 using equation (9).

[0107] θ 4 '=(max(θ1+θ d1 ,θ2+θ d2 ,…,θ n +θ dn ,θ o1 +θ do1 )+min(θ1,θ2,…,θ n ,θ o1 )) / 2 Equation (9)

[0108] Also, the T coordinate value of the center of the field of view P96 is Φ 4 ' can be calculated from the movement history of the human body and the T coordinate value of the object P94 using equation (10).

[0109] Φ4 '=(min(Φ1,Φ2,…,Φ n ,Φ o1 )+max(Φ1+Φ d1 ,Φ2+Φ d2 ,…,Φ n +Φ dn ,Φ o1 +Φ do1 )) / 2 Equation (10)

[0110] Horizontal angle of view ω 4 P' can be calculated as the difference between the movement history of the human body and the P coordinate value of the object P94 using equation (11).

[0111] ω 4 '=max(θ1+θ d1 ,θ2+θ d2 ,…,θ n +θ dn ,θ o1 +θ do1 )-min(θ1,θ2,…,θ n ,θ o1 ) Formula (11)

[0112] By performing the above-mentioned calculation, the angle of view calculation unit 411 obtains the angle of view D91 in FIG. 17 that can include each position to which the person may move and the object P94.

[0113] Figures 18 and 19 show the following period (time t n+1 From 2n ) is a diagram showing how the angle of view is calculated. In FIG. 18, the angle of view D101 is the same as the angle of view D91 shown in FIG. 17. The objects P103 and P104 are the same as the objects P84 and P85 shown in FIG. 15. The human bodies P101 and P102 are respectively n+1 ,t 2n It represents the human body at .

[0114] Fig. 19 is a diagram showing a state in which the angle of view is calculated based on information on the movement history of the human body illustrated in Fig. 18. The people P111 and P112 and the objects P113 and P114 in Fig. 19 are the same as the people P101 and P102 and the objects P103 and P104 shown in Fig. 18.

[0115] In FIG. 19, the PT coordinate value of the center P115 of the angle of view is (θ 5 ',Φ 5 '), and the horizontal angle of view value P116 is ω 5 '. P coordinate value θ of center P115 5 ' can be calculated from the movement history of the human body and the P coordinate value of the object P113 using equation (12).

[0116] θ 5 '=(max(θ n+1 +θ d(n+1) ,θ n+2 +θ d(n+2) ,…,θ 2n +θ d2n ,θ o1 +θ do1 )+min(θ n+1 ,θ n+2 ,…,θ 2n ,θ o1 )) / 2 Equation (12)

[0117] T coordinate value Φ of the center of the field of view P115 5 ' can be calculated from the movement history of the human body and the T coordinate value of the object P113 using equation (13).

[0118] Φ 5 '=(min(Φ n+1 ,Φ n+2 ,…,Φ 2n ,Φ o1 )+max(Φ n+1 +Φ d(n+1) ,Φ n+2 +Φ d(n+2) ,…,Φ 2n +Φ d2n ,Φ o1 +Φ do1 )) / 2 Equation (13)

[0119] Horizontal angle of view ω 5 ' can be calculated by equation (14) as the difference between the movement history of the human body and the P coordinate value of the object P113.

[0120] ω 5 '=max(θ n+1+θ d(n+1) ,θ n+2 +θ d(n+2) ,…,θ 2n +θ d2n ,θ o1 +θ do1 )-min(θ n+1 ,θ n+2 ,…,θ 2n ,θ o1 ) Formula (14)

[0121] By performing the above-mentioned calculation, the angle of view calculation unit 411 obtains the angle of view D111 in FIG. 19 that can include each position to which the person may move and the object P113. Then, the angle of view calculation unit 411 calculates the angle of view that matches the movement history of the human body at regular intervals in this manner, and outputs information on the calculated angle of view and image information to the angle of view operation unit 112 .

[0122] 20 is a flowchart showing the flow of processing in the automatic photography system 400 according to the fourth embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 402. When the automatic photography system 400 is started by a user operation, it starts processing for automatic photography.

[0123] First, in S 301 , the image input unit 103 of the angle of view adjustment device 402 acquires image information from the image acquisition device 101 . Next, in S302, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S302 is true (YES in S302), the detection determination unit 104 advances the process to S304, whereas if the determination result of S302 is false (NO in S302), the detection determination unit 104 outputs the image information to the object detection unit 106. Thereafter, the process of the angle of view adjustment device 402 advances to S303.

[0124] In S303, the object detection unit 106 performs an object detection process using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 402 proceeds to S307.

[0125] When the process proceeds to S304, the detection and determination unit 104 determines whether or not there is another object to be detected, using information on whether or not a preset angle of view was used, acquired from the angle of view storage unit 105. When the determination result of S304 is true (YES in S304), the detection and determination unit 104 outputs image information to the human body detection unit 107, and then the process of the angle of view adjustment device 402 proceeds to S306. On the other hand, when the determination result of S304 is false (NO in S304), the detection and determination unit 104 advances the process to S305.

[0126] In S305, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105, and then the process of the angle of view adjustment device 402 proceeds to S311. The process when proceeding from S305 to S311 or later is similar to the process when proceeding from S5 to S10 or later in FIG. 6 described above.

[0127] Also, in the case where the process proceeds to S306, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. After that, the process of the angle of view adjustment device 402 proceeds to S307.

[0128] In S307, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body, or the coordinates and image information of the object, and the current PTZ value of the camera. The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 410. After that, the process of the angle of view adjustment device 402 proceeds to S308.

[0129] In S308, the coordinate storage unit 410 stores the PT coordinate value acquired from the coordinate conversion unit 108. Then, the processing of the view angle adjustment device 402 proceeds to S309.

[0130] In S309, the operation determination unit 416 determines whether a certain period of time has elapsed by timer processing. If the determination result of S309 is true (YES in S309), the operation determination unit 416 acquires information on the movement history of the human body within the certain period of time, the PT coordinate value of the object, and image information from the coordinate storage unit 410, and outputs them to the view angle calculation unit 411. Then, the processing of the view angle adjustment device 402 proceeds to S310. On the other hand, if the determination result of S309 is false (NO in S309), the operation determination unit 416 acquires image information from the coordinate storage unit 410, and outputs it to the image output unit 113. Then, the processing of the view angle adjustment device 402 proceeds to S312.

[0131] When the process proceeds to S310, the angle-of-view calculation unit 411 calculates the angle of view based on the PT coordinate value of the object acquired from the coordinate storage unit 410, the image information, and the movement history information of the human body within a certain period of time. That is, the angle-of-view calculation unit 411 calculates the angle of view that includes the human body and the object, taking into account the movement history of the human body within the certain period of time. Furthermore, the angle-of-view calculation unit 411 outputs the information of the calculated angle of view to the angle-of-view operation unit 112. Then, the process of the angle-of-view adjustment device 402 proceeds to S311.

[0132] In S311, the view angle operation unit 112 performs view angle operation so that the human body and the object are included, taking into consideration the movement history of the human body within a certain period of time, based on the view angle information acquired from the view angle calculation unit 411. Then, the process of the view angle adjustment device 402 proceeds to S312.

[0133] In S312, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 402 proceeds to S313. In S313, the angle-of-view adjustment device 402 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the result of the determination in S313 is false (NO in S313), the process returns to S301. On the other hand, if the result of the determination in S313 is true (YES in S313), the angle-of-view adjustment device 402 ends the process of the flowchart in FIG.

[0134] The angle-of-view adjustment device 402 of the fourth embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and stores them, and further stores the PT coordinate values ​​of the human body as movement history information.The angle-of-view adjustment device 402 then adjusts the angle of view to include the human body that may move and the object, based on the movement history information of the human body for each fixed period and the PT coordinate values ​​of the object.Therefore, according to this embodiment, it is possible to keep the area where the most recent human body has moved within the angle of view.

[0135] <Fifth embodiment> Hereinafter, an example of the configuration of the imaging control device according to the fifth embodiment will be described with reference to Fig. 21. Fig. 21 is a block diagram showing the functional configuration of an automatic imaging system 500 including a field-of-view adjustment device 502, which is an imaging control device according to this embodiment. The automatic imaging system 500 of the fifth embodiment has an image acquisition device 101, a field-of-view adjustment device 502, and a monitor device 114. The field-of-view adjustment device 502 and the monitor device 114 are connected via a video interface. Note that in the fifth embodiment as well, an application in which a scene in which a lecturer is giving a lesson is automatically captured will be given as an example.

[0136] The angle-of-view adjusting device 502 of the fifth embodiment detects a human body and an object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and when calculating the angle of view using the PT coordinate values, obtains an angle of view that takes into account the aspect ratio of the image. That is, when calculating the angle of view that includes a human body within the angle of view and an object that was outside the angle of view, the angle of view adjusting device 502 of this embodiment calculates an angle of view that does not include the human body and the object, taking into account the aspect ratio of the image.

[0137] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 502 of the fifth embodiment determines whether an object to be included in the angle of view has been detected, and if not, adjusts the angle of view using a preset function to detect the object from the captured image. The angle-of-view adjustment device 502 also converts the coordinates within the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores the converted coordinates. The angle-of-view adjustment device 502 then detects a human body in the image and converts the coordinates within the angle of view into PT coordinate values ​​in the same way as the object. When calculating the angle of view, the angle-of-view adjustment device 502 of the fifth embodiment determines whether the human body or object will protrude from the angle of view based on the horizontal angle of view value calculated from the P coordinate value of the human body and the object and the vertical angle of view value calculated from the aspect ratio. If the human body or object will protrude from the angle of view, the angle-of-view adjustment device 502 calculates the vertical angle of view value from the T coordinate value of the human body and the object, and calculates the horizontal angle of view value from the vertical angle of view value and the aspect ratio. The view angle adjustment device 502 then calculates and adjusts the view angle to the calculated view angle, thereby calculating the view angle that contains the human body and the object. After that, the view angle adjustment device 502 outputs the image information after the view angle adjustment to the monitor device 114.

[0138] The image input unit 103, detection and determination unit 104, view angle storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, coordinate storage unit 110, view angle operation unit 112, and image output unit 113 of the view angle adjustment device 502 are the same as those described above. The view angle adjustment device 502 of the fifth embodiment differs from the first embodiment in the processing in the view angle calculation unit 511 and in the inclusion of an aspect ratio storage unit 517. The processing and configuration that differ from the first embodiment will be described below.

[0139] The angle of view calculation unit 511 uses the PT coordinate values ​​and image information of the human body and object input from the coordinate storage unit 110 and the image aspect ratio input from the aspect ratio storage unit 517 to calculate the angle of view so that the human body and object do not protrude. The aspect ratio storage unit 517 stores the image aspect ratio, and outputs the aspect ratio information to the angle of view calculation unit 511 .

[0140] 22 and 23 are diagrams for explaining how the angle of view calculation unit 511 of the fifth embodiment calculates the angle of view based on the PT coordinate values ​​of a human body and an object, image information, and an image aspect ratio. Fig. 22 shows a human body P121, an object P122, and a current angle of view D121. In the example of Fig. 22, the PT coordinate values ​​of the rectangle representing the human body P121 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the top left, top right, bottom left, and bottom right vertices, respectively. The PT coordinates of the object P122 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 22, the PT coordinate value of the center P123 in the field of view D121 where the edge of the human body P121 and the edge of the object P122 are the edges of the field of view is (θ 5 ',Φ 5 '), and the horizontal angle of view P124 is ω h 5 Here, the P coordinate value θ of the center P123 of the angle of view 5 ' can be calculated as the average value of the P coordinate values ​​of the human body P121 and the object P122 using equation (15).

[0141] θ 5 '=(max(θ1,θ2,θ9,θ 10 )+min(θ1,θ2,θ9,θ 10 )) / 2 Equation (15)

[0142] Also, the T coordinate value of the center of the field of view P123 is Φ 5 ' can be calculated by the formula (16) as the average value of the T coordinate value of the human body P121 and the T coordinate value of the object P122.

[0143] Φ 5 '=(max(Φ1,Φ2,Φ9,Φ 10 )+min(Φ1,Φ2,Φ9,Φ 10 )) / 2 Equation (16)

[0144] Also, the horizontal angle of view ω h5 ' can be calculated by equation (17) as the difference value between the P coordinate values ​​of the human body P121 and the object P122.

[0145] ω h 5 '=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 ) Formula (17)

[0146] Also, the vertical angle of view P125 is ω v 5 ', the vertical angle of view ω v 5 ' is the horizontal angle of view ω h 5 It can be calculated from the aspect ratio of the image by equation (18). Here, the image aspect ratio is 4:3 (vertical width of the angle of view: horizontal width of the angle of view), but it is not limited to this ratio and may be any ratio such as 16:9.

[0147] ω v 5 '=2tan -1 ((3 / 4)tan((1 / 2)ω h 5 ')) Formula (18)

[0148] The PT coordinate value (θ 5 ',Φ 5 ') and horizontal angle of view ω h 5 ' and the vertical angle of view ω v 5 The angle of view determined by the above formula is angle of view D121 in FIG. In the fifth embodiment, the view angle calculation unit 511 calculates a view angle whose vertical width is ω v 5 ', whether the human body or object is outside the angle of view is judged by the following equation (19).

[0149] (Φ 5 '-(1 / 2)ω v 5 ')>min(Φ1,Φ2,Φ9,Φ10 ) Or, (Φ 5 '+(1 / 2)ω v 5 ') <max(Φ1,Φ2,Φ9,Φ 10 ) Formula (19)

[0150] When this condition is met, a human body or object is outside the angle of view, so the angle-of-view calculation unit 511 calculates the horizontal angle of view from the image aspect ratio using the vertical angle of view as a reference. FIG. 23 is a diagram for explaining how the angle-of-view calculation unit 511 calculates the horizontal angle of view value from the image aspect ratio with the vertical angle of view value as a reference.

[0151] 23, a human body P131 is the same as the human body P121 in FIG. 22, and an object P132 is the same as the object P122 in FIG. 22. Also, a center P133 is the same as the center P123 in FIG. 22. That is, the PT coordinate value of the center P133 is expressed as (θ 6 ',Φ 6 Then, as shown in equation (20), the PT coordinate value (θ 5 ',Φ 5 ').

[0152] θ 6 '=θ 5 ' Φ 6 '=Φ 5 ' Formula (20)

[0153] Also, the vertical angle of view P135 is ω v 6 ', the vertical angle of view ω v 6 ' can be calculated by equation (21) as the difference value between the T coordinate values ​​of the human body P131 and the object P132.

[0154] ω v 6 '=max(Φ1,Φ2,Φ9,Φ 10 )-min(Φ1,Φ2,Φ9,Φ 10 ) Formula (21)

[0155] In addition, the horizontal angle of view P134 is set to ω h 6 ', the horizontal angle of view ω h 6 ' is the vertical angle of view ω v 6 ' and the image aspect ratio using equation (22).

[0156] ω h 6 '=2tan -1 ((3 / 4)tan((1 / 2)ω v 6 ')) Formula (22)

[0157] As described above, the angle of view calculation unit 511 calculates the angle of view at which the human body and objects do not protrude as shown in FIG.

[0158] 24 is a flowchart showing the flow of processing in the automatic photography system 500 according to the fifth embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 502. When the automatic photography system 500 is started by a user operation, it starts processing for automatic photography.

[0159] First, in S 401 , the image input unit 103 of the angle of view adjustment device 502 acquires image information from the image acquisition device 101 . Next, in S402, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S402 is true (YES in S402), the detection determination unit 104 advances the process to S404, whereas if the determination result of S402 is false (NO in S402), the detection determination unit 104 outputs the image information to the object detection unit 106. Thereafter, the process of the angle of view adjustment device 502 advances to S403.

[0160] In S403, the object detection unit 106 performs an object detection process using the image information input from the detection determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 502 proceeds to S407.

[0161] In S404, the detection and determination unit 104 determines whether or not there is another object to be detected, using information on whether or not a preset angle of view was used, acquired from the angle of view storage unit 105. If the determination result of S404 is true (YES in S404), the detection and determination unit 104 outputs image information to the human body detection unit 107, and then the processing of the angle of view adjustment device 502 proceeds to S406. On the other hand, if the determination result of S404 is false (NO in S404), the detection and determination unit 104 advances the processing to S405.

[0162] In S405, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105, and then the processing of the angle of view adjustment device 502 proceeds to S414. When the processing proceeds from S405 to S414, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114.

[0163] Also, if the process proceeds to S406, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 502 proceeds to S407.

[0164] In S407, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body, or the coordinates and image information of the object, and the current PTZ value of the camera. The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 410. After that, the process of the angle of view adjustment device 502 proceeds to S408. In S408, the coordinate storage unit 110 stores the PT coordinate value acquired from the coordinate conversion unit 108. Then, the processing of the view angle adjustment device 502 proceeds to S409.

[0165] In S409, the aspect ratio storage unit 517 acquires the aspect ratio of the image from the camera and stores it. Then, the process of the view angle adjustment device 502 proceeds to S410. When proceeding to S410, the angle of view calculation unit 511 calculates the difference between the P coordinate value and the T coordinate value and the center point from the maximum and minimum PT coordinate values ​​of the human body and object obtained from the coordinate storage unit 110, and sets the center point as the center of the angle of view.

[0166] Next, in S411, the angle-of-view calculation unit 511 sets the difference between the P coordinate values ​​as the horizontal angle of view, and acquires information on the image aspect ratio from the aspect ratio storage unit 517. Then, the angle-of-view calculation unit 511 determines whether a human body or an object is outside the angle of view when the vertical angle of view is calculated from the horizontal angle of view and the image aspect ratio. If the determination result of S411 is true (YES in S411), the angle-of-view calculation unit 511 proceeds to S413, whereas if the determination result is false (NO in S411), the angle-of-view calculation unit 511 proceeds to S412.

[0167] In S412, the view angle calculation unit 511 sets the difference value of the T coordinate value as the vertical view angle value, and calculates the horizontal view angle value from the vertical view angle value and the image aspect ratio. Then, the process of the view angle adjustment device 502 proceeds to S413.

[0168] In S413, the view angle operation unit 112 performs view angle operation so that the human body and the object are included in the image, taking into consideration the image aspect ratio, using the view angle acquired from the view angle calculation unit 511. Then, the process of the view angle adjustment device 502 proceeds to S414.

[0169] In S414, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 502 proceeds to S415. In S415, the angle-of-view adjustment device 502 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the result of the determination in S415 is false (NO in S415), the process returns to S401. On the other hand, if the result of the determination in S415 is true (YES in S415), the angle-of-view adjustment device 502 ends the process of the flowchart in FIG.

[0170] When calculating the angle of view from the PT coordinate value, the angle-of-view adjustment device 502 of the fifth embodiment calculates the horizontal angle of view from the P coordinate value of the human body and the object, calculates the vertical angle of view from the horizontal angle of view value and the image aspect ratio, and determines whether the human body and the object are beyond the angle of view. If the human body and the object are beyond the angle of view, the angle-of-view adjustment device 502 calculates the vertical angle of view from the T coordinate value of the human body and the object, and adjusts the horizontal angle of view to the angle of view calculated from the vertical angle of view value and the image aspect ratio. This makes it possible to keep the human body and the object within the angle of view according to this embodiment.

[0171] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, in the first embodiment, the angle-of-view calculation unit 111 may calculate a vertical angle-of-view value from the T coordinate values ​​of the human body and the object when calculating the angle of view, and may calculate a horizontal angle-of-view value from the vertical angle-of-view value and the image aspect ratio. Also, the image acquisition device and the angle-of-view adjustment device may be configured to be both included in one device, or may be configured to be included in separate devices. Furthermore, the angle-of-view adjustment device may be realized by application software.

[0172] Fig. 25 is a diagram showing an example of a hardware configuration of an information processing device 600 in the case where the view angle adjustment device described in the first to fifth embodiments is realized by application software. Note that Fig. 25 also shows an image acquisition device 101 and a monitor device 114 connected to the information processing device 600 (image adjustment device).

[0173] The information processing device 600 includes a CPU 601, a ROM 602, a RAM 603, a storage 604, and a communication I / F 605. Each of these components is connected to an internal bus 606 and is capable of communicating with each other. It is also assumed that the image acquisition device 101 and the monitor device 114 are connected to the communication I / F 605 as external devices.

[0174] The CPU 601 is a central processing unit that performs overall control of operations in the information processing device 600. The ROM 602 is a non-volatile memory that stores programs and various initial setting data required for the CPU 601 to execute processing. The RAM 603 is a volatile memory that functions as the main memory, work area, etc. of the CPU 601, and temporarily stores programs, image data, etc. When executing processing, the CPU 601 loads the necessary programs, etc. from the ROM 602 into the RAM 603 and executes the programs, etc. to realize various functional operations.

[0175] The storage 604 is a recording device having a larger capacity than the RAM 603, and is, for example, an HDD (hard disk drive) or an SSD (solid state drive). The storage 604 stores an OS (operating system) executed by the CPU 601, an information processing program (application program) related to the processing of the angle of view adjustment device described in each of the above-mentioned embodiments, and various other data. The storage 604 can also record image data acquired via a network.

[0176] At the time of startup, such as when the power is turned on, the CPU 601 executes a startup program stored in the ROM 602. This startup program is for reading out the OS stored in the storage 604 and loading it in the RAM 603. After the OS is started, when an execution start instruction is input by the user, the CPU 601 reads out an information processing program according to any one of the above-mentioned embodiments from the storage 604 and loads it in the RAM 603. This enables the CPU 601 to execute the information processing according to each of the above-mentioned embodiments, that is, the angle of view adjustment processing. The CPU 601 also stores various data used in the operation of the information processing program according to each embodiment in the RAM 603 and reads and writes it.

[0177] The communication I / F 605 is, for example, an interface for a LAN (local area network), and communicates with a network camera, other computers, etc. via the network. Note that images handled in the processing according to this embodiment, an information processing program for executing the processing, and various data used in the processing may be acquired via the network. The information processing device 600 can be configured as one device, such as a personal computer (PC) or a tablet PC. However, the configuration of the embodiment may be configured as separate hardware or a combination of several hardware. In other words, the information processing device 600 may be configured as multiple devices.

[0178] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. The above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0179] 100: automatic photography system, 101: image acquisition device, 102: angle of view adjustment device, 103: image input unit, 104: detection and judgment unit, 105: angle of view storage unit, 106: object detection unit, 107: human body detection unit, 108: coordinate conversion unit, 109: position storage unit, 110: coordinate storage unit, 111: angle of view calculation unit, 112: angle of view operation unit, 113: image output unit, 114: monitor device

Claims

1. A photographing control device that controls an angle of view of a photographing device having a driving means capable of panning and tilting, A setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to set the preset angle of view to the preset angle of view set by the setting means; a coordinate conversion means for converting, when at least one of a plurality of target subjects is detected in an image captured by the image capture device, a coordinate within an angle of view of the detected target subject into a pan / tilt coordinate value; a storage means for storing the pan / tilt coordinate values ​​of the target object converted by the coordinate conversion means; a view angle acquisition means for acquiring a view angle that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject when the pan / tilt coordinate values ​​of the other target subject are stored in the storage means; having 2. An image capturing control device comprising: a camera that controls the driving means to select a preset angle of view that is not used for image capturing, among the plurality of preset angles of view, when a target subject is not detected in the captured image.

2. 2 . The photographing control device according to claim 1 , wherein the control means further controls the drive means based on an angle of view that contains the detected target subject and the other target subject and that is acquired by the angle of view acquisition means.

3. 2. The photographing control device according to claim 1, wherein the angle of view acquisition means calculates pan coordinate values ​​and tilt coordinate values ​​that are the center of an angle of view that contains the detected target subject and the other target subjects, and a zoom magnification, to acquire an angle of view that contains the detected target subject and the other target subjects.

4. the coordinate conversion means converts a focus area of ​​an object into pan / tilt coordinate values ​​when an object is detected from among the plurality of target subjects in an image captured by the image capture device; 2 . The photographing control device according to claim 1 , wherein the angle of view acquisition means acquires an angle of view that includes the detected target subject and the other target subject such that the attention area is at an end of an angle of view.

5. 2 . The photographing control device according to claim 1 , wherein the angle of view acquisition means acquires the angle of view having a margin from the detected target subject to an end of the angle of view equal to a width of the detected target subject.

6. 2 . The photographing control device according to claim 1 , wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subjects based on a movement history of the detected target subject within a certain period of time.

7. the movement history of the detected target subject is composed of pan / tilt coordinate values ​​of the detected target subject obtained at regular time intervals within a certain period of time; 7. The photographing control device according to claim 6, wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subjects based on a movement history of the detected target subject and pan / tilt coordinate values ​​of the other target subjects.

8. a determination means for determining whether or not to adjust the angle of view depending on whether the certain period of time has elapsed; The photographing control device according to claim 6, characterized in that, when the determination means determines that the certain period has elapsed, the angle of view acquisition means performs a process of acquiring the angle of view using a movement history of the detected target subject within the certain period.

9. 2 . The photographing control device according to claim 1 , wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subject by also using an aspect ratio.

10. 10. The photographing control device according to claim 9, wherein the angle of view acquisition means determines whether the detected target subject and the other target subject are outside the current angle of view based on the aspect ratio.

11. The photographing control device according to claim 10, characterized in that the angle of view acquisition means determines whether the detected target subject and the other target subjects extend beyond the angle of view based on a zoom magnification calculated as a horizontal angle of view value based on a difference value between the pan coordinate values ​​of the detected target subject and the other target subjects.

12. The photographing control device according to claim 11, characterized in that when it is determined that the subject protrudes, the angle of view acquisition means acquires the angle of view corresponding to a zoom magnification calculated by using a difference value of tilt coordinate values ​​between the detected target subject and the other target subject as a vertical angle of view value.

13. A photographing control method executed by a photographing control device that controls an angle of view of a photographing device having a driving means capable of pan / tilt driving, comprising: A setting process for setting a plurality of preset angles of view; a control step of controlling the driving means to set the preset angle of view to the preset angle of view set in the setting step; a coordinate conversion step of converting coordinates within an angle of view of the detected target subject into pan / tilt coordinate values ​​when at least one of a plurality of target subjects is detected in an image captured by the image capture device; a storage step of storing the pan / tilt coordinate values ​​of the target object converted by the coordinate conversion step; a view angle acquisition step of acquiring a view angle that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject when the pan / tilt coordinate values ​​of the other target subject are stored in the storage step; having a control method for controlling the driving means to a preset angle of view that has not been photographed among the plurality of preset angles of view when a target subject is not detected in the photographed image;

14. Computer, A photographing control device that controls an angle of view of a photographing device having a driving means capable of panning and tilting, A setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to set the preset angle of view to the preset angle of view set by the setting means; a coordinate conversion means for converting, when at least one of a plurality of target subjects is detected in an image captured by the image capture device, a coordinate within an angle of view of the detected target subject into a pan / tilt coordinate value; a storage means for storing the pan / tilt coordinate values ​​of the target object converted by the coordinate conversion means; a view angle acquisition means for acquiring a view angle that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject when the pan / tilt coordinate values ​​of the other target subject are stored in the storage means; having a program that functions as an imaging control device that controls the drive means to a preset angle of view that is not used for imaging among the plurality of preset angles of view when a target subject is not detected in the captured image;

Images