Photographing control device, photographing control method, and program

Abstract

To obtain an angle of view that includes a target subject within the angle of view and a target subject outside the angle of view. [Solution] An imaging control device that controls the angle of view of an imaging device having a driving means capable of pan / tilt drive has a setting means for setting a plurality of preset angles of view, a control means for controlling the driving means to those preset angles of view, a coordinate conversion means for converting the coordinates within the angle of view of the detected target subject into pan / tilt coordinate values ​​when at least one of a plurality of target subjects is detected in a captured image, a storage means for saving those pan / tilt coordinate values, and an angle of view acquisition means for acquiring an angle of view that includes the detected target subject and the other target subjects based on the pan / tilt coordinate values ​​of the target subject and the other target subjects when pan / tilt coordinate values ​​of other target subjects different from the detected target subject are saved, and if the target subject is not detected in the captured image, the drive means controls the driving means to a preset angle of view of the plurality of preset angles of view that is not captured.

Description

[Technical Field]

[0001] The present invention relates to a control technique for an image capturing device. [Background technology]

[0002] In recent years, there has been a growing need for imaging devices that can automatically capture scenes of lecturers teaching. When automatically capturing and displaying a lecture scene, a close-up image of the lecturer, who is the target subject, is normally displayed. When the lecturer points at something like a blackboard, an image including both the lecturer and the area pointed at is displayed. Patent Document 1 discloses a technique for capturing an image of a podium or other object at a predetermined wide-angle reference angle of view, determining the coordinates of the center of gravity of the lecturer and the center of gravity of the area pointed at from the coordinates within the reference angle of view, and then calculating the angle of view including both the lecturer and the area pointed at based on these coordinates. Patent Document 1 also discloses a technique for extracting an image with an angle of view including both the lecturer and the area pointed at using digital PTZ processing from an image captured at the reference angle of view. Patent Document 1 also discloses a technique for preparing a camera for a reference angle of view and a tracking camera, and PTZ-controlling the angle of view of the tracking camera according to the angle of view including both the target subjects. PTZ is an abbreviation of pan (P), tilt (T), and zoom (Z). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-158680 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 can only acquire images within a predetermined reference angle of view. For example, if a lecturer or other person points to a whiteboard outside the reference angle of view, it is not possible to calculate the angle of view that includes the lecturer and the whiteboard because there are no coordinates within the reference angle of view that correspond to the whiteboard. In other words, the technology described in Patent Document 1 cannot calculate the angle of view that includes a target subject within a predetermined angle of view and another target subject that is outside the angle of view at that time, and therefore cannot acquire an image that includes both target subjects.

[0005] Therefore, an object of the present invention is to make it possible to acquire an angle of view that includes target subjects within the angle of view and target subjects that were outside the angle of view at that time, and to obtain an image that includes these target subjects. [Means for solving the problem]

[0006] The present invention is an imaging control device that controls the angle of view of an imaging device having a driving means capable of pan / tilt drive, comprising: a setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to the preset angles of view set by the setting means; a coordinate conversion means for converting coordinates within the angle of view of the detected target subject into pan / tilt coordinate values ​​when at least one of a plurality of target subjects is detected in an image captured by the imaging device; a storage means for saving the pan / tilt coordinate values ​​of the target subject converted by the coordinate conversion means; and an angle of view acquisition means for acquiring an angle of view that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject when the storage means has saved pan / tilt coordinate values ​​of a target subject different from the detected target subject, and controlling the driving means to a preset angle of view that is not captured among the plurality of preset angles of view when no target subject is detected in the captured image. [Effects of the Invention]

[0007] According to the present invention, it is possible to obtain an angle of view that includes target subjects within the angle of view and target subjects that were outside the angle of view at that time, and to obtain an image that includes these target subjects. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating the configuration of an automatic photography system according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing a target subject to be captured within the angle of view. [Figure 3] FIG. 10 is an explanatory diagram of a process for converting coordinates within the angle of view into PT coordinate values. [Figure 4] FIG. 10 is a diagram illustrating the current angle of view. [Figure 5] FIG. 2 is an explanatory diagram of angle of view control according to the first embodiment. [Figure 6] 10 is a flowchart of a view angle adjustment process according to the first embodiment. [Figure 7] FIG. 10 is a block diagram illustrating the configuration of an automatic photography system according to a second embodiment. [Figure 8] FIG. 10 is an explanatory diagram of a process for extracting a blackboard area within an object. [Figure 9] FIG. 10 is an explanatory diagram of angle of view control according to the second embodiment. [Figure 10] 10 is a flowchart of a view angle adjustment process according to the second embodiment. [Figure 11] FIG. 10 is a block diagram illustrating the configuration of an automatic photography system according to a third embodiment. [Figure 12] FIG. 11 is an explanatory diagram of angle of view control according to the third embodiment. [Figure 13] 11 is a flowchart of a view angle adjustment process according to the third embodiment. [Figure 14] FIG. 10 is a block diagram illustrating the configuration of an automatic photography system according to a fourth embodiment. [Figure 15] FIG. 1 is a diagram showing the movement history of a human body within a certain period of time. [Figure 16] 1 is a diagram showing the PT coordinate values ​​of an object and the PT coordinate values ​​of a human body. [Figure 17] FIG. 10 is an explanatory diagram of angle of view control according to the fourth embodiment. [Figure 18] FIG. 10 is a diagram showing another example of a movement history of a human body within a certain period of time. [Figure 19] 10A and 10B are explanatory diagrams illustrating angle of view control based on the movement history of a human body. [Figure 20] 10 is a flowchart of a view angle adjustment process according to the fourth embodiment. [Figure 21] FIG. 10 is a block diagram illustrating the configuration of an automatic photography system according to a fifth embodiment. [Figure 22] 10A and 10B are explanatory diagrams illustrating an example of calculating the angle of view based on the P coordinates of a human body and an object. [Figure 23] 10A and 10B are explanatory diagrams illustrating an example of calculating the angle of view based on the T coordinates of a human body and an object. [Figure 24] 13 is a flowchart of a view angle adjustment process according to the fifth embodiment. [Figure 25] FIG. 2 is a diagram illustrating an example of a hardware configuration of a view angle adjustment device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings. In the following embodiments, the same configurations and processes will be described with the same reference numerals.

[0010] First Embodiment An example of the configuration of an imaging control device according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the functional configuration of an automatic imaging system 100 including a field-angle adjustment device 102, which is an imaging control device according to this embodiment. The automatic photography system 100 of this embodiment is configured to include an image capture device 101, a field of view adjustment device 102, and a monitor device 114. The field of view adjustment device 102 and the monitor device 114 are connected via a video interface.

[0011] The image acquisition device 101 is configured to include an imaging device such as a camera that captures an image of a subject and its surroundings to generate a captured image. The imaging device has a zoom optical system and is configured to be able to adjust the zoom magnification by controlling the zoom value. The imaging device also includes a camera platform, and is configured to be able to adjust the pan direction by controlling the pan value of the camera platform, and is configured to be able to adjust the tilt direction by controlling the tilt value. Therefore, the imaging device is configured to be able to adjust the angle of view by controlling the pan value, tilt value, and zoom value. The image acquisition device 101 outputs image information captured by the imaging device to the angle of view adjustment device 102.

[0012] The angle-of-view adjustment device 102 has the function of an imaging control device that adjusts the angle of view of the imaging device of the image acquisition device 101. In this embodiment, the angle-of-view adjustment device 102 detects a first target subject, such as a human body, and a second target subject, such as an object, from an image captured by the imaging device (hereinafter referred to as a camera) of the image acquisition device 101. For example, in the case of automatically capturing a scene in which a lecturer is teaching, the first target subject, such as a human body, may be the lecturer giving the lecture, and the second target subject, such as an object, may be a blackboard or whiteboard used for the lecture. The angle-of-view adjustment device 102 converts the coordinate values ​​of the person within the angle of view when the person is captured within the angle of view, and the coordinate values ​​of the object within the angle of view when the object is captured within the angle of view, into PT coordinate values ​​(pan coordinate values / tilt coordinate values), and stores them. Thereafter, even if the human body is within the angle of view and the object at that time is outside the angle of view, the angle of view adjustment device 102 calculates the angle of view using the saved PT coordinate values, obtains the angle of view that will fit the human body and the object, and adjusts the angle of view of the camera to that angle of view.The captured image after the angle of view adjustment by the angle of view adjustment device 102 is then output to and displayed on the monitor device 114.

[0013] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 102 determines whether an object to be included in the angle of view as a target subject has been detected. If the angle-of-view adjustment device 102 has not detected the target subject, it adjusts the camera's angle of view using a preset function and performs object detection using an image of the target subject. If the angle-of-view adjustment device 102 detects the target subject, it converts the object's coordinates within the angle of view at that time into PT coordinate values ​​using the camera's PTZ (pan-tilt-zoom) value and stores them. The angle-of-view adjustment device 102 also detects a human body as a target subject from the image captured by the camera, and, as in the case of the target subject object described above, converts the human body's coordinates within the angle of view into PT coordinate values ​​using the camera's PTZ value and stores them. Based on the PT coordinate values ​​of the human body and object obtained and stored in this way, the angle-of-view adjustment device 102 calculates the angle of view that will accommodate the human body within the angle of view of the camera and the object that is outside the angle of view at that time, and adjusts the angle of view according to the calculated angle of view. Thereafter, the angle-of-view adjusting device 102 outputs the image information obtained after adjusting the angle of view to the monitor device 114 .

[0014] The angle of view adjustment device 102 has an image input unit 103, a detection and judgment unit 104, an angle of view storage unit 105, an object detection unit 106, a human body detection unit 107, a coordinate conversion unit 108, a position storage unit 109, a coordinate storage unit 110, an angle of view calculation unit 111, an angle of view operation unit 112, and an image output unit 113. The image input unit 103 receives image information from the image acquisition device 101 and outputs the received image information to the detection and determination unit 104 .

[0015] The detection and determination unit 104 determines whether an object that is a target subject to be included in the current angle of view of the camera has been detected. If the object that is a target subject has not been detected in the current angle of view of the camera, the detection and determination unit 104 outputs the image information input from the image input unit 103 to the object detection unit 106. On the other hand, if the object that is a target subject has been detected in the current angle of view of the camera, the detection and determination unit 104 inquires of the angle of view storage unit 105 whether there is a preset angle of view that has not yet been used.

[0016] The angle of view storage unit 105 stores information on multiple preset angles of view input by user operation, etc., and information on whether each preset angle of view has been used, and outputs this information in response to an acquisition request from the detection and judgment unit 104.

[0017] The preset angles of view will be explained using Fig. 2. Fig. 2 is a diagram showing a human body P1 as a target subject and objects P2 and P3 as other target subjects to be captured within the angle of view. The range indicated by the dashed dotted line in Fig. 2 represents the preset angles of view, with angle of view D2 representing the preset angle of view capturing object P2 as a target subject, and angle of view D3 representing the preset angle of view capturing object P3 as a target subject. As such, there are multiple preset angles of view, and these multiple preset angles of view are made up of multiple different preset positions and the camera's shooting range at each of these positions.

[0018] If there is no unused preset angle of view among the information stored in angle of view storage unit 105, detection and determination unit 104 outputs the image information input from image input unit 103 to human body detection unit 107. On the other hand, if there is an unused preset angle of view among the information stored in angle of view storage unit 105, detection and determination unit 104 outputs information about the preset angle of view to angle of view operation unit 112, which will be described later. In other words, by outputting information about the preset angle of view to angle of view operation unit 112, detection and determination unit 104 sets the angle of view of the camera to the preset angle of view.

[0019] The object detection unit 106 performs object detection processing to detect objects in the image from the image information input from the detection and determination unit 104. For the object detection processing, a high-precision, high-speed technology based on deep learning can be used. Examples of deep learning-based object detection processing include Faster R-CNN, YOLO, SSD, and Masked R-CNN. The object detection unit 106 uses one of these object detection processing technologies, but is not limited to these. Since Faster R-CNN, YOLO, SSD, and Masked R-CNN are well-known technologies, detailed description thereof will be omitted. If the object detection unit 106 detects an object in the image, it outputs the coordinate values ​​within the angle of view of the detected object and image information to the coordinate conversion unit 108.

[0020] The human body detection unit 107 performs human body detection processing to detect a human body in the image from the image information input from the detection and determination unit 104. The human body detection processing may use any method capable of detecting a human body, such as a template matching method or a semantic region segmentation method. Since the template matching method and the semantic region segmentation method are well-known technologies, detailed description thereof will be omitted. If the human body detection unit 107 detects a human body in the image, it outputs the coordinate values ​​within the angle of view of the detected human body and image information to the coordinate conversion unit 108. In this embodiment, the human body detection unit 107 outputs the coordinate values ​​of the four vertices of a rectangle (referred to as a detection rectangle) that contains the human body detected in the image as the coordinate values ​​of the detected person.

[0021] Coordinate values ​​and image information of an object are input to coordinate conversion section 108 from object detection section 106 , and coordinate values ​​and image information of a human body are input to coordinate conversion section 108 from human body detection section 107 . Furthermore, the position storage unit 109 stores the current PTZ value of the camera, and outputs the information on the PTZ value to the coordinate conversion unit 108. The coordinate conversion unit 108 converts the coordinates within the angle of view of the object and the human body into PT coordinate values ​​based on the coordinate values ​​and image information of the object, the coordinate values ​​and image information of the human body, and the current PTZ value information of the camera. The coordinate conversion unit 108 then outputs the PT coordinate values ​​and image information of the object and the human body to the coordinate storage unit 110.

[0022] The PT coordinate conversion process in which the coordinate conversion unit 108 converts the coordinates of an object and a human body within the angle of view into PT coordinate values ​​will be described with reference to FIG. 3. In FIG. 3, image D21 is an image in which the coordinates of a human body detected by the human body detection unit 107 are expressed in coordinates within the angle of view. Image D22 is an image in which the coordinates within the angle of view representing the human body in image D21 are expressed in PT coordinate values. In other words, human body P21 in image D21 and human body P22 in image D22 are the same person at the same time. The rectangle indicated by the dashed dotted line surrounding human body P21 in image D21 represents the detection rectangle indicating the human body detected by the human body detection unit 107. In image D21, the coordinates within the angle of view of the detection rectangle of human body P21 are expressed by four coordinates: (x1, y1), (x2, y2), (x3, y3), and (x4, y4) for the upper left vertex, upper right vertex, lower left vertex, and lower right vertex, in that order. Furthermore, if the pan value of the camera when capturing the image in which the human body is detected is represented by θ, the tilt value by Φ, and the zoom value by ω, then the PTZ value is represented by (θ, Φ, ω). The coordinate conversion unit 108 converts the coordinates within the angle of view of the human body and object into PT coordinate values ​​using the coordinate values ​​within the angle of view and the PTZ value of the camera.

[0023] The conversion process for converting coordinates within the angle of view into PT coordinate values ​​can be achieved using techniques such as converting planar coordinates into spherical coordinates, calculating coordinates projected onto an arbitrary plane perpendicular to the camera's optical axis, and converting coordinates on the projection plane into rotation angles for the pan and tilt angles. These techniques are well known, and detailed descriptions thereof are omitted. Furthermore, the conversion process using these techniques is merely an example, and any method capable of converting coordinates within the angle of view into PT coordinate values ​​may be used. Taking the coordinates within the angle of view of person P21 as an example, the converted PT coordinate values ​​are represented by four coordinates: (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4), which correspond to the top left, top right, bottom left, and bottom right vertices of the rectangle of person P22 in image D22, in that order. The coordinates (θ, Φ) in image D22 represent the PT coordinate values ​​of the center P23 of the angle of view.

[0024] The coordinate storage unit 110 stores the PT coordinate values ​​of the human body and object and the image information input from the coordinate conversion unit 108 , and outputs them to the angle of view calculation unit 111 . The angle-of-view calculation unit 111 calculates the angle of view of the camera that allows the human body within the current angle of view of the camera and the object outside the angle of view to be included, based on the PT coordinate values ​​and image information of the human body and object read from the coordinate storage unit 110. In this embodiment, the angle-of-view calculation unit 111 calculates a PTZ value for determining the angle of view of the camera that allows the human body within the current angle of view and the object outside the angle of view to be included.

[0025] The manner in which the angle-of-view calculation unit 111 calculates the PTZ value for determining the angle of view of the camera that accommodates a human body within the current angle of view of the camera and an object outside the angle of view will be described with reference to FIGS. In FIG. 4, the angle of view D31 represents the current angle of view of the camera. Assume that human body P31 corresponds to human body P1 illustrated in FIG. 2, object P32 corresponds to object P2 in FIG. 2, and object P33 corresponds to object P3 in FIG. 2. Assume that the rectangular PT coordinate values ​​of human body P31 are represented by four coordinate values, (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4), for the top left, top right, bottom left, and bottom right vertices, in that order. Assume that the PT coordinate values ​​of rectangular object P33 are represented by four coordinate values, (θ5, Φ5), (θ6, Φ6), (θ7, Φ7), and (θ8, Φ8), for the top left, top right, bottom left, and bottom right vertices, in that order. Assume that the PT coordinate values ​​of rectangular object P32 are represented by four coordinate values, (θ9, Φ9), (θ 10 ,Φ 10 ), (θ 11 ,Φ 11 ), (θ 12 ,Φ 12 4 is an angle of view that includes a human body P31 and an object P33, and the object P32 is outside the current angle of view of the camera. However, since the PT coordinate value of the object P32 is acquired in the PT coordinate conversion process when the preset angle of view is used and is stored in the coordinate storage unit 110, the angle of view calculation unit 111 can acquire the PT coordinate value of the object P32 from the coordinate storage unit 110.

[0026] FIG. 5 is a diagram for explaining how the angle of view calculation unit 111 calculates the angle of view that includes the human body P31 and the object P32 based on these PT coordinate values. First, we will explain how the angle of view is determined in the angle-of-view calculation unit 111. The angle of view is uniquely determined once the center point and the horizontal and vertical angle of view values ​​are determined. The horizontal and vertical angle of view values ​​vary depending on the focal length of the camera, and the focal length of a camera with a zoom optical system, as in this embodiment, is determined by the zoom magnification. Generally, the image size is determined according to the aspect ratio of the image, so once either the horizontal or vertical angle of view value is determined, the other is also determined. For these reasons, in this embodiment, the angle-of-view calculation unit 111 determines the angle of view by calculating the center point and the horizontal angle of view value corresponding to the zoom magnification.

[0027] In Fig. 5, human body P41 corresponds to human body P31 in Fig. 4, and similarly, object P42 corresponds to object P32 in Fig. 4, and object P43 corresponds to object P33 in Fig. 4. Therefore, the PT coordinate values ​​of human body P41, object P42, and object P43 in Fig. 5 are assumed to be the same as the PT coordinate values ​​of human body P31, object P32, and object P33 in Fig. 4. Furthermore, the center of the angle of view containing human body P41 and object P42 is defined as P44, its PT coordinate value is (θ', Φ'), and the horizontal angle of view value is defined as P45, whose value is ω'. The PT coordinate value (θ', Φ') and horizontal angle of view value ω' of center P44 can be calculated as follows.

[0028] The horizontal angle of view ω′ can be calculated as the difference between the P coordinate values ​​(pan coordinate values) of the human body P41 and the object P42, as in equation (1).

[0029] ω'=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 ) Formula (1)

[0030] In the formula (1), max() indicates that the maximum value in the parentheses is obtained, and min() indicates that the minimum value in the parentheses is obtained.

[0031] Furthermore, the P coordinate value θ′ of the center P44 of the angle of view can be calculated as the average value of the P coordinate values ​​of the human body P41 and the object P42, as in equation (2).

[0032] θ'=(max(θ1,θ2,θ9,θ 10 )+min(θ1,θ2,θ9,θ 10 )) / 2 Equation (2)

[0033] Furthermore, the T coordinate value (tilt coordinate value) of the center P44 of the angle of view can be calculated as the average value of the T coordinate values ​​of the human body P41 and the object P42, as in equation (3).

[0034] Φ'=(max(Φ1,Φ2,Φ9,Φ 10)+min(Φ1,Φ2,Φ9,Φ 10 )) / 2 Equation (3)

[0035] The angle-of-view calculation unit 111 calculates the angle of view D41 as described above, and outputs the angle of view and image information to the angle-of-view operation unit 112. In this embodiment, in addition to the angle-of-view acquisition process using the calculation as described above, the angle-of-view calculation unit 111 may also perform an angle-of-view acquisition process in which, for example, an angle of view is selected from a table in which a plurality of angles of view are prepared in advance.

[0036] The angle-of-view operation unit 112 operates the angle of view of the camera, that is, controls the adjustment of the angle of view of the camera, using the angle of view and image information input from the angle-of-view calculation unit 111 or the preset angle of view and image information input from the detection and determination unit 104. After that, the angle-of-view operation unit 112 outputs the image information after the angle of view adjustment to the image output unit 113. The image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, the monitor device 114 displays an image based on the image information input from the image output unit 113 on the monitor.

[0037] 6 is a flowchart showing the flow of the processing related to the automatic photography system 100 of the first embodiment, particularly the flow of the field angle adjustment processing in the field angle adjustment device 102. In the following explanation of each flowchart, the letter S attached to the reference numeral denotes a processing step. The automatic photography system 100 starts processing for automatic photography when a start instruction is input by a user operation. First, in S1, the image input unit 103 of the view angle adjusting device 102 acquires image information from the image acquisition device 101. Then, the processing of the view angle adjusting device 102 proceeds to S2.

[0038] In S2, the detection and determination unit 104 uses the image information input from the image input unit 103 to determine whether a target object to be included in the angle of view of the camera has been detected, that is, whether a target object is present in the image. The target object determined by the detection and determination unit 104 in S2 is a human body or a target subject of an object. If the determination result of S2 is true (YES in S2), that is, if a target object to be included in the angle of view has been detected, the detection and determination unit 104 proceeds to processing in S4. On the other hand, if the determination result of S2 is false (NO in S2), that is, if a target object to be included in the angle of view has not been detected, the detection and determination unit 104 outputs the image information to the object detection unit 106, and the processing of the angle of view adjustment device 102 proceeds to S3.

[0039] In S3, the object detection unit 106 performs a detection process for an object, for example, a second target subject, using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 102 proceeds to S7.

[0040] If the process proceeds to S4, the detection and determination unit 104 determines whether there is another object of a second target subject to be detected, using information on whether the preset angle of view obtained from the angle of view storage unit 105 was used. If the determination result of S4 is true (YES in S4), that is, if there is no other object to be detected, the detection and determination unit 104 outputs image information to the human body detection unit 107. Then, the process of the angle of view adjustment device 102 proceeds to S6. On the other hand, if the determination result of S4 is false (NO in S4), that is, if there is another object to be detected, the detection and determination unit 104 proceeds to S5.

[0041] When the process proceeds from S4 to S5, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105. Then, the process of the angle of view adjustment device 102 proceeds to S10. If the process proceeds from S5 to S10, the view angle operation unit 112 performs view angle operation in S10 based on the information about the preset view angle so that the object fits within the preset view angle. Then, the process of the view angle adjustment device 102 proceeds to S11.

[0042] In S11, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 102 proceeds to S12.

[0043] In S12, the view angle adjustment device 102 determines whether a user has operated an automatic photography system on / off switch (not shown) to stop the automatic photography process. If the determination result in S12 is false (NO in S12), that is, if the view angle adjustment device 102 determines that the off switch has not been operated, it returns to S1. On the other hand, if the determination result in S12 is true (YES in S12), that is, if it determines that the off switch has been operated, the view angle adjustment device 102 ends the processing of the flowchart in FIG.

[0044] As described above, if the process proceeds from S4 to S5, then returns to S1 via the processes of S10, S11, and S12, and then proceeds from S2 to S3, object detection unit 106 performs object detection processing using image information acquired at the preset angle of view. Object detection unit 106 then outputs the coordinates and image information of the object detected within the preset angle of view to coordinate conversion unit 108.

[0045] Also, when the process proceeds from S4 to S6, the human body detection unit 107 performs a human body detection process using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 102 proceeds to S7.

[0046] If the process proceeds from S3 to S7, in S7 the coordinate conversion unit 108 converts the coordinates of the object within the angle of view into PT coordinate values ​​based on the coordinates and image information of the object obtained from the object detection unit 106 and the current PTZ value of the camera obtained from the position storage unit 109. Also, if the process proceeds from S6 to S7, in S7 the coordinate conversion unit 108 converts the coordinates of the human body within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body obtained from the human body detection unit 107 and the current PTZ value of the camera obtained from the position storage unit 109. Then, the coordinate conversion unit 108 outputs these converted PT coordinate values ​​to the coordinate storage unit 110. Then, the process of the angle of view adjustment device 102 proceeds to S8.

[0047] In S8, the coordinate storage unit 110 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108. That is, the PT coordinate values ​​of the human body and the object are stored in the coordinate storage unit 110. Then, the processing of the angle of view adjustment device 102 proceeds to S9. In S9, the angle-of-view calculation unit 111 calculates the angle of view at which the human body within the angle of view and the object currently outside the angle of view fit, based on the PT coordinate values ​​of the human body and the object acquired from the coordinate storage unit 110. That is, the angle-of-view calculation unit 111 calculates the angle of view at which the human body within the angle of view and the object that was outside the angle of view before the angle of view manipulation by the subsequent angle-of-view manipulation unit 112 fit. The angle-of-view calculation unit 111 then outputs information about the calculated angle of view to the angle-of-view manipulation unit 112. Thereafter, the processing of the angle-of-view adjustment device 102 proceeds to S10.

[0048] In S10, the angle-of-view operation unit 112 performs an angle-of-view operation based on the information about the angle of view acquired from the angle-of-view calculation unit 111. That is, the angle-of-view operation unit 112 performs an angle-of-view operation so that the human body and objects are included. Then, the processing of the angle-of-view adjustment device 102 proceeds to S11.

[0049] In S11, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114, and an image based on the image information is thereby displayed on the monitor device 114. Thereafter, in S12, the angle of view adjustment device 102 determines whether a stop operation has been performed by operating the automatic photography system On / Off switch. If the determination result in S12 is false (NO in S12), the angle of view adjustment device 102 returns to S1, and if the determination result in S12 is true (YES in S12), the angle of view adjustment device 102 ends the processing of the flowchart in FIG. 6.

[0050] The angle-of-view adjustment device 102 of the first embodiment converts coordinates within the angle of view when an object is detected in advance using a preset angle of view, etc., into PT coordinate values ​​and stores them, and then uses the PT coordinate values ​​to calculate the angle of view that will accommodate a human body within the angle of view and an object that was outside the angle of view at that time. That is, in this embodiment, if an object that you want to include in the current angle of view of the camera is outside that angle of view, it is possible to acquire an image that includes the human body within the angle of view and the object outside the angle of view by calculating the angle of view that will accommodate the human body within the angle of view and the object outside the angle of view and adjusting the angle of view. As a result, according to this embodiment, it is possible to continue capturing images that include the lecturer and the object in an automatic imaging system that automatically captures, for example, a scene in which a lecturer is giving a lecture.

[0051] Second Embodiment An example of the configuration of an imaging control device according to the second embodiment will be described below with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of an automatic imaging system 200 including a field-of-view adjustment device 202, which is an imaging control device according to this embodiment. The automatic imaging system 200 of the second embodiment has an image acquisition device 101, a field-of-view adjustment device 202, and a monitor device 114. The field-of-view adjustment device 202 and the monitor device 114 are connected via a video interface. In the second embodiment, an example will be taken in which an instructor is automatically capturing images of a lecture.

[0052] The angle-of-view adjustment device 202 of the second embodiment detects a human body, an object, and a region of interest on the object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and uses the PT coordinate values ​​to calculate the angle of view so that the region of interest is at the edge of the angle of view. That is, when obtaining the angle of view that includes the human body and the object, the angle of view adjustment device 202 of the second embodiment calculates the angle of view so that the human body is positioned with some space from the edge of the angle of view. In the second embodiment, an example is given in which a blackboard area is used as the region of interest. The blackboard area is assumed to be the area where the lecture content is written on a blackboard or whiteboard used for a lecture, for example.

[0053] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 202 of the second embodiment determines whether an object to be included in the angle of view as a target subject has been detected. If not, the angle of view adjustment device 202 adjusts the angle of view using a preset function and performs object detection from the captured image. The angle-of-view adjustment device 202 also converts the coordinates of the object within the angle of view at that time into PT coordinate values ​​using the camera's PTZ value and stores the converted coordinates. The angle-of-view adjustment device 202 then detects a human body or a blackboard area in the image and converts the coordinates within the angle of view into PT coordinate values, as in the case of an object. Based on the PT coordinate values ​​of the human body, the object, and the blackboard area, the angle of view adjustment device 202 adjusts the angle of view so that the blackboard area is at the edge of the angle of view and has enough room for the human body to extend to the edge of the angle of view, thereby fitting the human body and the object within the angle of view. The angle-of-view adjustment device 202 then outputs the image information with the adjusted angle of view to the monitor device 114, which displays an image based on the image information.

[0054] In the angle-of-view adjustment device 202, the image input unit 103, detection and determination unit 104, angle-of-view storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, angle-of-view operation unit 112, and image output unit 113 are the same as in the first embodiment. The angle-of-view adjustment device 202 of the second embodiment differs from the first embodiment in the processing in the coordinate storage unit 210 and angle-of-view calculation unit 211, and in the presence of an area extraction unit 215. Below, the processing and configuration that differ from the first embodiment will be described.

[0055] The area extraction unit 215 extracts the blackboard area using the coordinate values ​​and image information of the human body input from the human body detection unit 107, or the coordinate values ​​and image information of the object input from the object detection unit 106. The blackboard area can be extracted using any known method, such as extracting the blackboard area based on the difference in brightness information between frame images at different times on the video time axis, or extracting the blackboard area based on character recognition results or graphic recognition results, and detailed descriptions thereof will be omitted. The area extraction unit 215 then outputs the coordinate values ​​of the blackboard area, the coordinate values ​​of the human body, and the coordinate values ​​of the object, expressed in coordinates within the angle of view, to the coordinate conversion unit 108, along with the image information.

[0056] The coordinate conversion unit 108 converts the coordinates within the field of view of the blackboard area, the human body, and the object into PT coordinate values ​​by the same processing as that described in the first embodiment. The coordinate storage unit 210 stores the PT coordinate values ​​of the human body, the PT coordinate values ​​of the object, the PT coordinate values ​​of the blackboard area, and the image information input from the coordinate conversion unit 108. The coordinate storage unit 210 then outputs the PT coordinate values ​​and the image information to the angle of view calculation unit 211.

[0057] The angle of view calculation unit 211 calculates an angle of view that includes the human body and the object, the blackboard writing area as the edge of the angle of view, and the human body has a margin from the edge of the angle of view, based on the PT coordinate values ​​of the human body, the PT coordinate values ​​of the object, and the PT coordinate values ​​of the blackboard writing area input from the coordinate storage unit 210, and the image information.

[0058] 8 is a diagram showing PT coordinate values ​​input from the coordinate storage unit 210. Assume that human body P51 corresponds to human body P1 shown in FIG. 2, and objects P52 and P53 correspond to objects P2 and P3 in FIG. 2. Also, assume that the rectangular area surrounded by a dotted line for object P52 in FIG. 8 is a blackboard area P54 extracted for object P52, and the rectangular area surrounded by a dotted line for object P53 is a blackboard area P55 extracted for object P53. Also, assume that the PT coordinate values ​​of the rectangle representing human body P51 in FIG. 8 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the upper left, upper right, lower left, and lower right vertices, respectively, similar to the PT coordinates of FIGS. 4 and 5 described above. Similarly, the PT coordinates of object P53 are (θ5, Φ5), (θ6, Φ6), (θ7, Φ7), and (θ8, Φ8) for the top left, top right, bottom left, and bottom right vertices, respectively, and the PT coordinates of object P52 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 The PT coordinate values ​​of the blackboard area P54 are (θ 13 ,Φ 13 ),(θ 14 ,Φ 14 ),(θ 15 ,Φ 15 ),(θ 16 ,Φ 16 ) and the PT coordinate values ​​of the blackboard writing area P55 are (θ 17 ,Φ 17 ),(θ 18 ,Φ 18 ),(θ 19 ,Φ 19 ),(θ 20 ,Φ 20 )

[0059] FIG. 9 is a diagram showing how the angle-of-view calculation unit 211 of the second embodiment calculates the angle of view that includes a human body P51 and a board writing area P54 based on these PT coordinate values. In FIG. 9, human body P61 corresponds to human body P51 shown in FIG. 8, and object P62 and board writing area P63 correspond to object P52 and board writing area P54 shown in FIG. 8. Center P64 corresponds to center P44 of the angle of view calculated using the same method as described in the first embodiment, and its PT coordinate values ​​are (θ', Φ'). Similarly, horizontal angle of view value P66 corresponds to horizontal angle of view value P45 calculated using the same method as described in the first embodiment, and its horizontal angle of view value is ω'. Furthermore, angle of view D61 corresponds to angle of view D41 shown in FIG. 5.

[0060] Here, in the case of angle of view D61, human body P61 is located at the edge of the angle of view, so there is a high possibility that human body P61 will move out of the angle of view due to movement or the like and be lost. Therefore, in the second embodiment, the angle of view calculation unit 211 calculates an angle of view that makes it difficult to lose sight of the human body P61 even if it moves to some extent, and that also fits the blackboard writing area P63 of the object P62. In the second embodiment, in order to fit the blackboard writing area P63 and provide a margin at the edge of the angle of view for the human body P61, the angle of view D62 is calculated by sliding the angle of view D61 obtained by the calculation described in the first embodiment so that the blackboard writing area P63 becomes the edge of the angle of view. The angle of view D62 shown in FIG. 9 is the angle of view after sliding. If the PT coordinate value of the center P65 of the angle of view D62 is (θ'',Φ'') and the horizontal angle of view value P67 is ω'', then the angle of view D62 can be expressed by equation (4). The angle of view calculation unit 211 then outputs the calculated angle of view D62 and image information to the angle of view operation unit 112.

[0061] θ''=θ'+(θ 13 -θ9), Φ''=Φ', Equation (4) ω''=ω'

[0062] 10 is a flowchart showing the processing in the automatic photography system 200 according to the second embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 202. When the automatic photography system 200 is started by a user operation, it starts processing for automatic photography. First, in S101 , the image input unit 103 of the angle of view adjustment device 202 acquires image information from the image acquisition device 101 . Next, in S102, the detection determination unit 104 determines whether or not a target object to be contained in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S102 is true (YES in S102), the detection determination unit 104 proceeds to S104. On the other hand, if the determination result of S102 is false (NO in S102), the detection determination unit 104 outputs the image information to the object detection unit 106, and the processing of the angle of view adjustment device 202 proceeds to S103.

[0063] In S103, the object detection unit 106 performs object detection processing using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the area extraction unit 215. Then, the processing of the angle of view adjustment device 202 proceeds to S107.

[0064] If the process proceeds to S104, the detection and determination unit 104 determines whether or not there are any other objects to be detected, using information on whether or not a preset angle of view was used, acquired from the angle of view storage unit 105. If the determination result of S104 is true (YES in S104), the detection and determination unit 104 outputs image information to the human body detection unit 107. Then, the processing of the angle of view adjustment device 202 proceeds to S106. On the other hand, if the determination result of S104 is false (NO in S104), the detection and determination unit 104 proceeds to S105.

[0065] In S105, the detection and determination unit 104 acquires information about the preset angle of view from the angle of view storage unit 105. Then, the processing of the angle of view adjustment device 202 proceeds to S111. The processing when proceeding from S105 to S111 and subsequent steps is the same as the processing when proceeding from S5 to S10 and subsequent steps in FIG. 6 described above.

[0066] Also, if the process proceeds to S106, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the area extraction unit 215. Then, the process of the angle of view adjustment device 202 proceeds to S107.

[0067] In S107, the area extraction unit 215 extracts a blackboard area using the coordinates and image information of the object input from the object detection unit 106. Then, the area extraction unit 215 outputs the coordinates and image information of the extracted blackboard area, the coordinates and image information of the human body input from the human body detection unit 107, and the coordinates and image information of the object input from the object detection unit 106 to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 202 proceeds to S108.

[0068] In S108, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the blackboard area, the coordinates and image information of the human body, the coordinates and image information of the object obtained from the area extraction unit 215, and the current PTZ value of the camera obtained from the position storage unit 109. Then, the coordinate conversion unit 108 outputs the converted PT coordinate values ​​to the coordinate storage unit 210. Then, the processing of the angle of view adjustment device 202 proceeds to S109.

[0069] In S109, the coordinate storage unit 210 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 202 proceeds to S110. In S110, the angle-of-view calculation unit 211 calculates an angle of view that fits the human body and the object, with the blackboard writing area at the edge of the angle of view and the human body at a sufficient distance from the edge of the angle of view, from the PT coordinate values ​​of the blackboard writing area, the human body, and the object acquired from the coordinate storage unit 210. The angle-of-view calculation unit 211 then outputs information about the calculated angle of view to the angle-of-view operation unit 112. Then, the processing of the angle-of-view adjustment device 202 proceeds to S111.

[0070] In S111, the angle-of-view operation unit 112 adjusts the angle of view so that the human body and the object are included, the blackboard writing area is at the end of the angle of view, and the human body has some margin from the end of the angle of view, based on the information about the angle of view acquired from the angle-of-view calculation unit 211. Then, the processing of the angle-of-view adjustment device 202 proceeds to S112. In S112, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 202 proceeds to S113.

[0071] In S113, the view angle adjustment device 202 determines whether the user has operated the automatic photography system On / Off switch to stop the automatic photography process. If the determination result in S113 is false (NO in S113), the view angle adjustment device 202 returns to S101. On the other hand, if the determination result in S113 is true (YES in S113), the view angle adjustment device 202 ends the processing of the flowchart in FIG. 10.

[0072] The angle-of-view adjustment device 202 of the second embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and saves them, calculates the angle of view based on the PT coordinate values, and controls the angle of view so that the area of ​​interest is at the edge of the angle of view and the human body has a margin from the edge of the angle of view. In other words, according to this embodiment, if an object that you want to fit into the current angle of view of the camera is outside that angle of view, it is possible to continue shooting with the blackboard area at the edge of the image and the human body having a margin from the edge of the angle of view while fitting the human body within the angle of view and the object outside the angle of view within the angle of view.

[0073] Third Embodiment An example of the configuration of an imaging control device according to the third embodiment will be described below with reference to Fig. 11. Fig. 11 is a block diagram showing the functional configuration of an automatic imaging system 300 including a field-of-view adjustment device 302, which is an imaging control device according to this embodiment. The automatic imaging system 300 of the third embodiment has an image acquisition device 101, a field-of-view adjustment device 302, and a monitor device 114. The field-of-view adjustment device 302 and the monitor device 114 are connected via a video interface. Note that the third embodiment also takes as an example an application in which a lecturer is automatically capturing a lecture scene.

[0074] The angle-of-view adjusting device 302 of the third embodiment detects a human body and an object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and when calculating the angle of view using the PT coordinate values, adjusts the angle of view so that the human body has a margin of the width of the human body from the edge of the angle of view. In other words, the angle-of-view adjusting device 302 of the third embodiment adjusts the angle of view so that when fitting a human body and an object into the angle of view, the human body has a margin from the edge of the angle of view.

[0075] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 302 of the third embodiment determines whether an object to be included in the angle of view as a target subject has been detected. If not, the angle of view adjustment device 302 adjusts the angle of view using a preset function and performs object detection from the captured image. The angle-of-view adjustment device 302 also converts the coordinates within the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores the converted coordinates. The angle-of-view adjustment device 302 then detects a human body in the image and converts the coordinates within the angle of view into PT coordinate values, as in the case of an object. The angle-of-view adjustment device 302 then calculates an angle of view based on the PT coordinate values ​​of the human body and the object, allowing a margin of the human body's width from the edge of the angle of view, and adjusts the angle of view to fit the human body and the object within the angle of view. The angle-of-view adjustment device 302 then outputs the image information after the angle of view adjustment to the monitor device 114, displaying an image corresponding to the image information.

[0076] In the view angle adjustment device 302, the image input unit 103, detection and determination unit 104, view angle storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, coordinate storage unit 110, view angle operation unit 112, and image output unit 113 are the same as those described above. In the case of the view angle adjustment device 302 of the third embodiment, the processing in the view angle calculation unit 311 differs from that of the first embodiment. Below, the processing and configuration that differ from those of the first embodiment will be described.

[0077] The angle of view calculation unit 311 uses the PT coordinate values ​​and image information of the human body and object acquired from the coordinate storage unit 110 to calculate an angle of view that includes the human body within the angle of view and the object outside the angle of view, while leaving a margin of the width of the human body from the edge of the angle of view.

[0078] 12 is a diagram showing how the angle of view calculation unit 311 of the third embodiment calculates the angle of view based on these PT coordinate values. In FIG. 12, human body P71 corresponds to, for example, human body P31 shown in FIG. 5, and object P72 corresponds to object P42 in FIG. 5. In FIG. 12, the PT coordinate values ​​of the rectangle representing human body P71 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the upper left, upper right, lower left, and lower right vertices, respectively, as described above. In addition, the PT coordinates of object P72 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 ) The angle of view D71 represents an angle of view that includes a human body P71 and an object P72, while allowing the human body P71 to have a margin of the width of the human body (distance P75) from the edge of the angle of view. The PT coordinate value of the center P73 in the angle of view D71 is (θ''',Φ'''), and the horizontal angle of view value P74 is ω'''. The distance P75 from the detected rectangle representing the human body P71 to the edge of the angle of view is θ d Then, it can be expressed by equation (5).

[0079] θ d =θ2-θ1 Equation (5)

[0080] Furthermore, the P coordinate value θ''' of the center P73 of the angle of view can be calculated using equation (6) as the average value of the P coordinate value of the human body P71 plus the distance P75 equal to the width of the human body and the P coordinate value of the object P72.

[0081] θ'''=(max(θ1,θ2,θ9,θ 10 )+θ d +min(θ1,θ2,θ9,θ 10 )) / 2 Equation (6)

[0082] The T coordinate value Φ′″ of the center P73 of the angle of view can be calculated by equation (7) as the average value of the T coordinate value of the human body P71 and the T coordinate value of the object P72.

[0083] Φ'''=(max(Φ1,Φ2,Φ9,Φ10 )+min(Φ1,Φ2,Φ9,Φ 10 )) / 2 Equation (7)

[0084] Furthermore, the horizontal angle of view ω''' can be calculated by equation (8) as the sum of the difference between the P coordinate values ​​of the human body P71 and the object P72 and the distance P75 corresponding to the width of the human body.

[0085] ω'''=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 )+θ d Formula (8)

[0086] Then, the angle-of-view calculation unit 311 outputs the calculated angle of view and image information to the angle-of-view operation unit 112 . 13 is a flowchart showing the processing in the automatic photography system 300 according to the third embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 302. When the automatic photography system 300 is started by a user operation, it starts processing for automatic photography.

[0087] First, in S201 , the image input unit 103 of the angle of view adjustment device 302 acquires image information from the image acquisition device 101 . Next, in S202, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S202 is true (YES in S202), the detection determination unit 104 proceeds to S204, whereas if the determination result of S202 is false (NO in S202), the detection determination unit 104 outputs the image information to the object detection unit 106. Then, the processing of the angle of view adjustment device 302 proceeds to S203.

[0088] In S203, the object detection unit 106 performs object detection processing using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 302 proceeds to S207.

[0089] If the process proceeds to S204, the detection determination unit 104 determines whether or not there are any other objects to be detected, using information on whether or not a preset angle of view was used, obtained from the angle of view storage unit 105. If the determination result of S204 is true (YES in S204), the detection determination unit 104 outputs image information to the human body detection unit 107, and then the process of the angle of view adjustment device 302 proceeds to S206. On the other hand, if the determination result of S204 is false (NO in S204), the detection determination unit 104 proceeds to S205.

[0090] In S205, the detection and determination unit 104 acquires information about the preset angle of view from the angle of view storage unit 105, and then the processing of the angle of view adjustment device 302 proceeds to S210. The processing when proceeding from S205 to S210 and subsequent steps is the same as the processing when proceeding from S5 to S10 and subsequent steps in FIG. 6 described above. Also, if the process proceeds to S206, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Thereafter, the process of the angle of view adjustment device 302 proceeds to S207.

[0091] In S207, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body, or the coordinates and image information of the object, and the current PTZ value of the camera. The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 110. After that, the processing of the angle of view adjustment device 302 proceeds to S208. In S208, the coordinate storage unit 110 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108, and then the processing of the angle of view adjustment device 302 proceeds to S209.

[0092] In S209, the angle-of-view calculation unit 311 calculates an angle of view in which the object is at the end of the angle of view and the human body has a margin of the horizontal width of the human body from the end of the angle of view, based on the PT coordinate values ​​of the human body and object acquired from the coordinate storage unit 110. The angle-of-view calculation unit 311 then outputs information about the calculated angle of view to the angle-of-view operation unit 112. Thereafter, the processing of the angle-of-view adjustment device 302 proceeds to S210. In S210, the angle-of-view manipulation unit 112 manipulates the angle of view according to the angle of view that fits the human body and the object, and that has a margin of the horizontal width of the human body from the end of the angle of view, with the object at the end of the angle of view, based on the angle of view information acquired from the angle of view calculation unit 311. After that, the processing of the angle-of-view adjustment device 302 proceeds to S211.

[0093] In S211, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, the monitor device 114 displays an image based on the image information. Next, in S212, the view angle adjustment device 302 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the determination result in S212 is false (NO in S212), the process returns to S201. On the other hand, if the determination result in S212 is true (YES in S212), the view angle adjustment device 302 ends the processing of the flowchart in FIG.

[0094] The angle-of-view adjustment device 302 of the third embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and stores them, and calculates the angle of view using the PT coordinate values ​​of the human body and object as well as the width of the human body. As a result, according to this embodiment, it is possible to keep the human body and object within the angle of view while continuing to photograph the human body at an angle of view with a margin from the edge of the angle of view.

[0095] <Fourth embodiment> An example of the configuration of an imaging control device according to the fourth embodiment will be described below with reference to Fig. 14. Fig. 14 is a block diagram showing the functional configuration of an automatic imaging system 400 including a field-of-view adjustment device 402, which is an imaging control device according to this embodiment. The automatic imaging system 400 of the fourth embodiment has an image acquisition device 101, a field-of-view adjustment device 402, and a monitor device 114. The field-of-view adjustment device 402 and the monitor device 114 are connected via a video interface. Note that the fourth embodiment also takes as an example an application in which a lecturer is automatically capturing a lecture scene.

[0096] The angle-of-view adjustment device 402 of the fourth embodiment detects human bodies and objects from captured images, saves their coordinate values ​​within the angle of view as PT coordinate values, and calculates the angle of view using the movement history of the human bodies within a certain period and the PT coordinate values ​​of the objects. That is, the angle-of-view adjustment device 402 of this embodiment calculates the angle of view taking into account the movement history of the human bodies when obtaining the angle of view that includes the human bodies within the angle of view and the objects that were outside the angle of view.

[0097] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 402 of the fourth embodiment determines whether an object to be included in the angle of view as a target subject has been detected. If not, the angle of view adjustment device 402 adjusts the angle of view using a preset function and performs object detection from the captured image. The angle-of-view adjustment device 402 then converts the coordinates within the angle of view at that time into PT coordinate values ​​using the PTZ value of the camera and stores the converted coordinates. The angle-of-view adjustment device 402 then detects a human body in the image and converts the coordinates within the angle of view into PT coordinate values, just as it does for the object. The angle-of-view adjustment device 402 also stores the PT coordinate values ​​of the human body obtained at regular intervals within a certain period as movement history information. The angle-of-view adjustment device 402 then calculates the angle of view that includes the human body and the object based on the movement history of the human body obtained at regular intervals within the certain period and the PT coordinate values ​​of the object, and performs image adjustment. The angle-of-view adjustment device 402 then outputs the image information after the angle of view adjustment to the monitor device 114, which displays a video based on the image information.

[0098] The image input unit 103, detection and determination unit 104, view angle storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, view angle operation unit 112, and image output unit 113 of the view angle adjustment device 402 are the same as those described above. The view angle adjustment device 402 of the fourth embodiment differs from the first embodiment in that it includes processing in the coordinate storage unit 410 and view angle calculation unit 411, and an operation determination unit 416. The following describes the processing and configuration that differ from the first embodiment.

[0099] The coordinate storage unit 410 stores the PT coordinate values ​​of the human body and the object input from the coordinate conversion unit 108. The coordinate storage unit 410 of the fourth embodiment stores the PT coordinate values ​​of the human body as history information, and outputs the history information of the human body and the PT coordinate values ​​of the object for a period designated by the operation determination unit 416, which will be described later.

[0100] The operation determination unit 416 determines whether a certain period of time has passed by timer processing. If the certain period of time has passed, the operation determination unit 416 acquires the PT coordinate value of the object, image information, and historical information of the human body within the certain period of time from the coordinate storage unit 410, and outputs them to the angle of view calculation unit 411. If the certain period of time has not passed, the operation determination unit 416 outputs the image information acquired from the coordinate storage unit 410 to the image output unit 113.

[0101] The angle of view calculation unit 411 uses the history information of the human body and the PT coordinate values ​​of the object and image information input from the coordinate storage unit 410 to calculate the angle of view that will contain the human body and the object, taking into account the movement history of the human body over a certain period of time.

[0102] 15 to 19 are diagrams for explaining how the angle of view calculation unit 411 of the fourth embodiment calculates the angle of view based on the history information of the human body and the PT coordinate values ​​and image information of the object. n However, any value may be used.

[0103] 15, range D81 represents the range that includes the human body and other objects that are the target subjects. n That is, the human body is moving, and is located at the position of human body P81 at time t1, and at the position of human body P82 at time t2. n Assume that the object P84 is located at the position of the human body P83 at time t8. The objects P84 and P85 correspond to the objects P2 and P3 shown in FIG.

[0104] FIG. 16 is a diagram showing the PT coordinate values ​​of the human body (rectangle corresponding to the human body) and the PT coordinate values ​​of the objects (obj1 and obj2 in FIG. 16) at each time shown in FIG. 15. In FIG. 16, the PT coordinate values ​​of the human body and the objects are the coordinates (θ, Φ) of the top left corner of the rectangle and the width (θ d ) and height (Φ d ) is shown.

[0105] Figure 17 shows the time from t1 to t n 10 is an explanatory diagram of a calculation of the angle of view based on the movement history of the human body within the period from 1 to 1000. A method for calculating the angle of view in the angle of view calculation unit 411 will be described below. Time t1 to t n Since the positions to which the human body has moved within the period up to this point are positions to which the human body may move in the future, the angle of view calculation unit 411 calculates an angle of view that includes all positions in the human body's movement history within that period and the object (object P94 in this example).

[0106] In FIG. 17, the PT coordinate value of the center P96 of the angle of view is (θ 4 ',Φ 4 '), and the horizontal angle of view value P97 is ω 4 '. P coordinate value θ of the center P96 of the angle of view 4 ' can be calculated from the movement history of the human body and the P coordinate value of the object P94 using equation (9).

[0107] θ 4 '=(max(θ1+θ d1 ,θ2+θ d2 ,…,θ n +θ dn ,θ o1 +θ do1 )+min(θ1,θ2,…,θ n ,θ o1 )) / 2 Equation (9)

[0108] Also, the T coordinate value Φ of the center of the angle of view P96 4 ' can be calculated from the movement history of the human body and the T coordinate value of the object P94 using equation (10).

[0109] Φ4 '=(min(Φ1,Φ2,…,Φ n ,Φ o1 )+max(Φ1+Φ d1 ,Φ2+Φ d2 ,…,Φ n +Φ dn ,Φ o1 +Φ do1 )) / 2 Equation (10)

[0110] Horizontal angle of view ω 4 ' can be calculated by equation (11) as the difference value between the movement history of the human body and the P coordinate value of the object P94.

[0111] ω 4 '=max(θ1+θ d1 ,θ2+θ d2 ,…,θ n +θ dn ,θ o1 +θ do1 )-min(θ1,θ2,…,θ n ,θ o1 ) Formula (11)

[0112] By performing the calculations described above, the angle of view calculation unit 411 obtains the angle of view D91 in FIG. 17 that can accommodate each position to which the person may move and the object P94.

[0113] Figures 18 and 19 show the following period (time t n+1 From t 2n ) is a diagram showing how the angle of view is calculated. In FIG. 18, the angle of view D101 is the same as the angle of view D91 shown in FIG. 17. The objects P103 and P104 are the same as the objects P84 and P85 shown in FIG. 15. The human bodies P101 and P102 are respectively n+1 ,t 2n It represents the human body at this time.

[0114] Fig. 19 is a diagram showing how the angle of view is calculated based on information about the movement history of the human body shown in Fig. 18. The people P111 and P112 and the objects P113 and P114 in Fig. 19 are the same as the people P101 and P102 and the objects P103 and P104 shown in Fig. 18.

[0115] In FIG. 19, the PT coordinate value of the center P115 of the angle of view is (θ 5 ',Φ 5 '), and the horizontal angle of view value P116 is ω 5 '. P coordinate value θ of center P115 5 ' can be calculated from the movement history of the human body and the P coordinate value of the object P113 using equation (12).

[0116] θ 5 '=(max(θ n+1 +θ d(n+1) ,θ n+2 +θ d(n+2) ,…,θ 2n +θ d2n ,θ o1 +θ do1 )+min(θ n+1 ,θ n+2 ,…,θ 2n ,θ o1 )) / 2 Equation (12)

[0117] T coordinate value Φ of the center of the angle of view P115 5 ' can be calculated from the movement history of the human body and the T coordinate value of the object P113 using equation (13).

[0118] Φ 5 '=(min(Φ n+1 ,Φ n+2 ,…,Φ 2n ,Φ o1 )+max(Φ n+1 +Φ d(n+1) ,Φ n+2 +Φ d(n+2) ,…,Φ 2n +Φ d2n ,Φ o1 +Φ do1 )) / 2 Equation (13)

[0119] Horizontal angle of view ω 5 ' can be calculated by equation (14) as the difference between the movement history of the human body and the P coordinate value of the object P113.

[0120] ω 5 '=max(θ n+1+θ d(n+1) ,θ n+2 +θ d(n+2) ,…,θ 2n +θ d2n ,θ o1 +θ do1 )-min(θ n+1 ,θ n+2 ,…,θ 2n ,θ o1 ) Formula (14)

[0121] By performing the calculations described above, the angle of view calculation unit 411 obtains the angle of view D111 in FIG. 19 that can accommodate each position to which the person may move and the object P113. In this way, the angle of view calculation unit 411 calculates the angle of view that matches the movement history of the human body at regular intervals, and outputs information about the calculated angle of view and image information to the angle of view operation unit 112 .

[0122] 20 is a flowchart showing the processing in the automatic photography system 400 according to the fourth embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 402. When the automatic photography system 400 is started by a user operation, it starts processing for automatic photography.

[0123] First, in S301 , the image input unit 103 of the angle of view adjustment device 402 acquires image information from the image acquisition device 101 . Next, in S302, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S302 is true (YES in S302), the detection determination unit 104 proceeds to S304, whereas if the determination result of S302 is false (NO in S302), the detection determination unit 104 outputs the image information to the object detection unit 106. Thereafter, the processing of the angle of view adjustment device 402 proceeds to S303.

[0124] In S303, the object detection unit 106 performs an object detection process using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 402 proceeds to S307.

[0125] If the process proceeds to S304, the detection determination unit 104 determines whether there are any other objects to be detected, using information on whether a preset angle of view was used, obtained from the angle of view storage unit 105. If the determination result of S304 is true (YES in S304), the detection determination unit 104 outputs image information to the human body detection unit 107, and then the process of the angle of view adjustment device 402 proceeds to S306. On the other hand, if the determination result of S304 is false (NO in S304), the detection determination unit 104 proceeds to S305.

[0126] In S305, the detection and determination unit 104 acquires information on the preset angle of view from the angle of view storage unit 105, and then the processing of the angle of view adjustment device 402 proceeds to S311. The processing when proceeding from S305 to S311 or later is the same as when proceeding from S5 to S10 or later in FIG. 6 described above.

[0127] Also, if the process proceeds to S306, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Thereafter, the process of the angle of view adjustment device 402 proceeds to S307.

[0128] In S307, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body, or the coordinates and image information of the object, and the current PTZ value of the camera. The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 410. Thereafter, the processing of the angle of view adjustment device 402 proceeds to S308.

[0129] In S308, the coordinate storage unit 410 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 402 proceeds to S309.

[0130] In S309, the operation determination unit 416 determines by timer processing whether a certain period of time has elapsed. If the determination result of S309 is true (YES in S309), the operation determination unit 416 acquires information on the movement history of the human body within the certain period of time, the PT coordinate values ​​of the object, and image information from the coordinate storage unit 410, and outputs them to the field of view calculation unit 411. Then, the processing of the field of view adjustment device 402 proceeds to S310. On the other hand, if the determination result of S309 is false (NO in S309), the operation determination unit 416 acquires image information from the coordinate storage unit 410 and outputs it to the image output unit 113. Then, the processing of the field of view adjustment device 402 proceeds to S312.

[0131] In S310, the angle-of-view calculation unit 411 calculates the angle of view based on the PT coordinate values ​​of the object acquired from the coordinate storage unit 410, the image information, and the movement history information of the human body within a certain period of time. That is, the angle-of-view calculation unit 411 calculates the angle of view that fits the human body and the object, taking into account the movement history of the human body within the certain period of time. The angle-of-view calculation unit 411 then outputs information about the calculated angle of view to the angle-of-view operation unit 112. Then, the processing of the angle-of-view adjustment device 402 proceeds to S311.

[0132] In S311, the angle of view operation unit 112 performs an angle of view operation to fit the human body and the object in consideration of the movement history of the human body within a certain period of time, based on the information on the angle of view acquired from the angle of view calculation unit 411. Then, the processing of the angle of view adjustment device 402 proceeds to S312.

[0133] In S312, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 402 proceeds to S313. In S313, the view angle adjustment device 402 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the determination result in S313 is false (NO in S313), the process returns to S301. On the other hand, if the determination result in S313 is true (YES in S313), the view angle adjustment device 402 ends the processing of the flowchart in FIG.

[0134] The angle-of-view adjustment device 402 of the fourth embodiment converts coordinates within the angle of view when an object is detected in advance into PT coordinate values ​​and stores them, and also stores the PT coordinate values ​​of the human body as movement history information.The angle-of-view adjustment device 402 then adjusts the angle of view to include the human body and the object, which may be moving, based on the human body movement history information for each fixed period and the PT coordinate values ​​of the object.As a result, according to this embodiment, it is possible to continue to include the area through which the most recent human body has moved within the angle of view.

[0135] Fifth Embodiment An example of the configuration of an imaging control device according to the fifth embodiment will be described below with reference to Fig. 21. Fig. 21 is a block diagram showing the functional configuration of an automatic imaging system 500 including a field-of-view adjustment device 502, which is an imaging control device according to this embodiment. The automatic imaging system 500 of the fifth embodiment has an image acquisition device 101, a field-of-view adjustment device 502, and a monitor device 114. The field-of-view adjustment device 502 and the monitor device 114 are connected via a video interface. Note that the fifth embodiment also takes as an example an application in which a scene in which a lecturer is giving a lesson is automatically captured.

[0136] The angle-of-view adjusting device 502 of the fifth embodiment detects a human body and an object from a captured image, saves their coordinate values ​​within the angle of view as PT coordinate values, and when calculating the angle of view using the PT coordinate values, obtains an angle of view that takes into account the aspect ratio of the image. That is, when calculating the angle of view that includes a human body within the angle of view and an object that was outside the angle of view, the angle of view adjusting device 502 of this embodiment calculates an angle of view that does not include the human body or object protruding from the image, taking into account the aspect ratio of the image.

[0137] When image information is input from the image acquisition device 101, the angle-of-view adjustment device 502 of the fifth embodiment determines whether an object to be included in the angle of view has been detected. If not, the angle of view adjustment device 502 adjusts the angle of view using a preset function and performs object detection from the captured image. The angle-of-view adjustment device 502 also converts the coordinates within the angle of view at that time into PT coordinate values ​​using the camera's PTZ values ​​and saves them. The angle-of-view adjustment device 502 then detects a human body in the image and converts the coordinates within the angle of view into PT coordinate values, just like the object. When calculating the angle of view, the angle-of-view adjustment device 502 of the fifth embodiment determines whether the human body or object exceeds the angle of view based on the horizontal angle of view calculated from the P coordinate values ​​of the human body and object and the vertical angle of view calculated from the aspect ratio. If the human body or object exceeds the angle of view, the angle-of-view adjustment device 502 calculates the vertical angle of view from the T coordinate values ​​of the human body and object, and then calculates the horizontal angle of view from this and the aspect ratio. The angle-of-view adjusting device 502 then calculates and adjusts the angle of view to the calculated angle of view that accommodates the human body and the object. After that, the angle-of-view adjusting device 502 outputs the image information after the angle of view adjustment to the monitor device 114.

[0138] The image input unit 103, detection and determination unit 104, view angle storage unit 105, object detection unit 106, human body detection unit 107, coordinate conversion unit 108, position storage unit 109, coordinate storage unit 110, view angle operation unit 112, and image output unit 113 of the view angle adjustment device 502 are the same as those described above. The view angle adjustment device 502 of the fifth embodiment differs from the first embodiment in the processing in the view angle calculation unit 511 and in the inclusion of an aspect ratio storage unit 517. Below, the processing and configuration that differ from the first embodiment will be described.

[0139] The angle of view calculation unit 511 uses the PT coordinate values ​​and image information of the human body and object input from the coordinate storage unit 110 and the image aspect ratio input from the aspect ratio storage unit 517 to calculate an angle of view that will prevent the human body and object from protruding. The aspect ratio storage unit 517 stores the image aspect ratio, and outputs the aspect ratio information to the angle of view calculation unit 511 .

[0140] 22 and 23 are diagrams for explaining how the angle of view calculation unit 511 of the fifth embodiment calculates the angle of view based on the PT coordinate values ​​of the human body and object, image information, and the image aspect ratio. Fig. 22 shows a human body P121, an object P122, and a current angle of view D121. In the example of Fig. 22, the PT coordinate values ​​of the rectangle representing the human body P121 are (θ1, Φ1), (θ2, Φ2), (θ3, Φ3), and (θ4, Φ4) ​​for the top left, top right, bottom left, and bottom right vertices, respectively. The PT coordinates of the object P122 are (θ9, Φ9), (θ 10 ,Φ 10 ),(θ 11 ,Φ 11 ),(θ 12 ,Φ 12 22, the PT coordinate value of the center P123 in the field of view D121 where the edge of the human body P121 and the edge of the object P122 are at the edge of the field of view is (θ 5 ',Φ 5 '), and the horizontal angle of view P124 is ω h 5 Here, the P coordinate value θ of the center P123 of the angle of view 5 ' can be calculated by equation (15) as the average value of the P coordinate values ​​of the human body P121 and the object P122.

[0141] θ 5 '=(max(θ1,θ2,θ9,θ 10 )+min(θ1,θ2,θ9,θ 10 )) / 2 Equation (15)

[0142] Also, the T coordinate value Φ of the center of the angle of view P123 5 ' can be calculated by equation (16) as the average value of the T coordinate value of the human body P121 and the T coordinate value of the object P122.

[0143] Φ 5 '=(max(Φ1,Φ2,Φ9,Φ 10 )+min(Φ1,Φ2,Φ9,Φ 10 )) / 2 Equation (16)

[0144] The horizontal angle of view value ω h5 ' can be calculated by equation (17) as the difference value between the P coordinate values ​​of the human body P121 and the object P122.

[0145] ω h 5 '=max(θ1,θ2,θ9,θ 10 )-min(θ1,θ2,θ9,θ 10 ) Formula (17)

[0146] Also, the value of the vertical angle of view P125 is ω v 5 ', the vertical angle of view value ω v 5 ' is the horizontal angle of view value ω h 5 It can be calculated from the image aspect ratio using equation (18). Here, the image aspect ratio is set to 4:3 (vertical width of field of view: horizontal width of field of view), but it is not limited to this ratio and can be any ratio, such as 16:9.

[0147] ω v 5 '=2tan -1 ((3 / 4)tan((1 / 2)ω h 5 ')) Formula (18)

[0148] The PT coordinate value (θ 5 ',Φ 5 ') and horizontal angle of view ω h 5 ' and vertical angle of view value ω v 5 The angle of view determined by the above formula is the angle of view D121 in FIG. In the fifth embodiment, the angle of view calculation unit 511 calculates the angle of view such that the vertical width of the angle of view is ω v 5 When ', it is determined whether the human body or object is outside the angle of view by the following equation (19).

[0149] (Φ 5 '-(1 / 2)ω v 5 ')>min(Φ1,Φ2,Φ9,Φ10 ) Or, (Φ 5 '+(1 / 2)ω v 5 ') <max(Φ1,Φ2,Φ9,Φ 10 ) Formula (19)

[0150] If this condition is met, the human body or object is outside the angle of view, so the angle of view calculation unit 511 calculates the horizontal angle of view from the image aspect ratio using the vertical angle of view as a reference. FIG. 23 is a diagram for explaining how the angle-of-view calculation unit 511 calculates the horizontal angle of view value from the image aspect ratio using the vertical angle of view value as a reference.

[0151] In FIG. 23, a human body P131 is the same as the human body P121 in FIG. 22, and an object P132 is the same as the object P122 in FIG. 22. Also, a center P133 is the same as the center P123 in FIG. 22. That is, the PT coordinate value of the center P133 is expressed as (θ 6 ',Φ 6 '), then the PT coordinate value (θ 5 ',Φ 5 ') is the same value.

[0152] θ 6 '=θ 5 ' Φ 6 '=Φ 5 ' Formula (20)

[0153] Also, the value of the vertical angle of view P135 is ω v 6 ', the vertical angle of view value ω v 6 ' can be calculated by equation (21) as the difference value between the T coordinate values ​​of the human body P131 and the object P132.

[0154] ω v 6 '=max(Φ1,Φ2,Φ9,Φ 10 )-min(Φ1,Φ2,Φ9,Φ 10 ) Formula (21)

[0155] Also, the horizontal angle of view value P134 is set to ω h 6 ', the horizontal angle of view value ω h 6 ' is the vertical angle of view value ω v 6 ' and the image aspect ratio, it can be calculated using equation (22).

[0156] ω h 6 '=2tan -1 ((3 / 4)tan((1 / 2)ω v 6 ')) Formula (22)

[0157] As described above, the angle of view calculation unit 511 calculates the angle of view at which the human body and objects do not protrude as shown in FIG. 23, and outputs information about the angle of view to the angle of view operation unit 112.

[0158] 24 is a flowchart showing the processing in the automatic photography system 500 according to the fifth embodiment, particularly the flow of the angle-of-view adjustment processing in the angle-of-view adjustment device 502. When the automatic photography system 500 is started by a user operation, it starts processing for automatic photography.

[0159] First, in S401 , the image input unit 103 of the angle of view adjustment device 502 acquires image information from the image acquisition device 101 . Next, in S402, the detection determination unit 104 determines whether an object of a target subject to be included in the angle of view has been detected using the image information input from the image input unit 103. If the determination result of S402 is true (YES in S402), the detection determination unit 104 proceeds to S404, whereas if the determination result of S402 is false (NO in S402), the detection determination unit 104 outputs the image information to the object detection unit 106. Thereafter, the processing of the angle of view adjustment device 502 proceeds to S403.

[0160] In S403, the object detection unit 106 performs object detection processing using the image information input from the detection and determination unit 104, and outputs the coordinates and image information of the detected object to the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 502 proceeds to S407.

[0161] In S404, the detection determination unit 104 determines whether there are any other objects to be detected, using information on whether a preset angle of view was used, obtained from the angle of view storage unit 105. If the determination result of S404 is true (YES in S404), the detection determination unit 104 outputs image information to the human body detection unit 107, and then the processing of the angle of view adjustment device 502 proceeds to S406. On the other hand, if the determination result of S404 is false (NO in S404), the detection determination unit 104 proceeds to S405.

[0162] In S405, the detection and determination unit 104 acquires information about the preset angle of view from the angle of view storage unit 105, and then the processing of the angle of view adjustment device 502 proceeds to S414. When the processing proceeds from S405 to S414, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114.

[0163] Also, if the process proceeds to S406, the human body detection unit 107 detects a human body using the image information acquired from the detection and determination unit 104, and outputs the coordinates of the detected human body and the image information to the coordinate conversion unit 108. Then, the process of the angle of view adjustment device 502 proceeds to S407.

[0164] In S407, the coordinate conversion unit 108 converts the coordinates within the angle of view into PT coordinate values ​​based on the coordinates and image information of the human body, or the coordinates and image information of the object, and the current PTZ value of the camera.The coordinate conversion unit 108 then outputs the converted PT coordinate values ​​to the coordinate storage unit 410.Then, the processing of the angle of view adjustment device 502 proceeds to S408. In S408, the coordinate storage unit 110 stores the PT coordinate values ​​acquired from the coordinate conversion unit 108. Then, the processing of the angle of view adjustment device 502 proceeds to S409.

[0165] In S409, the aspect ratio storage unit 517 acquires and stores the aspect ratio of the image from the camera. Then, the processing of the angle of view adjustment device 502 proceeds to S410. When proceeding to S410, the angle of view calculation unit 511 calculates the difference between the P coordinate value and the T coordinate value and the center point from the maximum and minimum PT coordinate values ​​of the human body and object obtained from the coordinate storage unit 110, and sets the center point as the center of the angle of view.

[0166] Next, in S411, the angle-of-view calculation unit 511 sets the difference between the P coordinate values ​​as the horizontal angle of view, and acquires information on the image aspect ratio from the aspect ratio storage unit 517. The angle-of-view calculation unit 511 then determines whether a human body or an object protrudes outside the angle of view when calculating the vertical angle of view from the horizontal angle of view and the image aspect ratio. If the determination result in S411 is true (YES in S411), the angle-of-view calculation unit 511 proceeds to S413, and if the determination result is false (NO in S411), the angle-of-view calculation unit 511 proceeds to S412.

[0167] In S412, the view angle calculation unit 511 sets the difference value of the T coordinate value as the vertical view angle value, and calculates the horizontal view angle value from the vertical view angle value and the image aspect ratio. Then, the process of the view angle adjustment device 502 proceeds to S413.

[0168] In S413, the angle of view operation unit 112 performs angle of view operation so that the human body and the object are included in the image, taking into consideration the image aspect ratio, using the angle of view acquired from the angle of view calculation unit 511. Then, the processing of the angle of view adjustment device 502 proceeds to S414.

[0169] In S414, the image output unit 113 outputs the image information input from the angle of view operation unit 112 to the monitor device 114. As a result, an image based on the image information is displayed on the monitor device 114. Then, the processing of the angle of view adjustment device 502 proceeds to S415. In S415, the view angle adjustment device 502 determines whether the automatic photography system On / Off switch has been operated to stop the automatic photography process, and if the determination result in S415 is false (NO in S415), the process returns to S401. On the other hand, if the determination result in S415 is true (YES in S415), the view angle adjustment device 502 ends the processing of the flowchart in FIG.

[0170] When calculating the angle of view from the PT coordinate values, the angle-of-view adjustment device 502 of the fifth embodiment calculates the horizontal angle of view from the P coordinate values ​​of the human body and object, calculates the vertical angle of view from the horizontal angle of view value and the image aspect ratio, and determines whether the human body and object are outside the angle of view. If the human body and object are outside the angle of view, the angle-of-view adjustment device 502 calculates the vertical angle of view from the T coordinate values ​​of the human body and object, and adjusts the horizontal angle of view to the angle of view calculated from the vertical angle of view value and the image aspect ratio. This makes it possible to keep the human body and object within the angle of view according to this embodiment.

[0171] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, in the first embodiment, the angle-of-view calculation unit 111 may calculate a vertical angle-of-view value from the T coordinate values ​​of the human body and the object when calculating the angle of view, and may calculate a horizontal angle-of-view value from the vertical angle-of-view value and the image aspect ratio. The image acquisition device and the angle-of-view adjustment device may be configured either to be included in a single device or to be included in separate devices. Furthermore, the angle-of-view adjustment device may be realized by application software.

[0172] Fig. 25 is a diagram showing an example of the hardware configuration of an information processing device 600 when the view angle adjustment device described in the first to fifth embodiments is realized by application software. Fig. 25 also shows an image acquisition device 101 and a monitor device 114 connected to the information processing device 600 (image adjustment device).

[0173] The information processing device 600 includes a CPU 601, a ROM 602, a RAM 603, a storage 604, and a communication I / F 605. These components are connected to an internal bus 606 and are capable of communicating with each other. The communication I / F 605 is also connected to the image acquisition device 101 and the monitor device 114 as external devices.

[0174] The CPU 601 is a central processing unit that performs overall control of the operations of the information processing device 600. The ROM 602 is a non-volatile memory that stores programs and various initial setting data required for the CPU 601 to execute processing. The RAM 603 is a volatile memory that functions as the main memory, work area, etc. of the CPU 601, and temporarily stores programs, image data, etc. When executing processing, the CPU 601 loads the necessary programs, etc. from the ROM 602 into the RAM 603 and executes the programs, etc. to realize various functional operations.

[0175] The storage 604 is a storage device with a larger capacity than the RAM 603, such as an HDD (hard disk drive) or SSD (solid state drive). The storage 604 stores an OS (operating system) executed by the CPU 601, an information processing program (application program) related to the processing of the angle of view adjustment device described in each of the above-mentioned embodiments, and various other data. The storage 604 can also store image data acquired via a network.

[0176] Upon startup, such as when the power is turned on, the CPU 601 executes a startup program stored in the ROM 602. This startup program is used to read out the OS stored in the storage 604 and load it into the RAM 603. After the OS has started, when a user inputs an instruction to start execution, the CPU 601 reads out an information processing program according to any one of the above-described embodiments from the storage 604 and loads it into the RAM 603. This enables the CPU 601 to execute the information processing according to each of the above-described embodiments, i.e., the angle of view adjustment processing. The CPU 601 also stores various data used in the operation of the information processing program according to each embodiment in the RAM 603 and reads and writes it.

[0177] The communication I / F 605 is, for example, an interface for a LAN (local area network), and communicates with a network camera, other computers, etc. via the network. Note that the images handled in the processing according to this embodiment, the information processing program for executing the processing, and various data used in the processing may be acquired via the network. Furthermore, the information processing device 600 can be configured as a single device, such as a personal computer (PC) or a tablet PC. However, the configuration of the embodiment may be configured by separate hardware or a combination of several hardware components. In other words, the information processing device 600 may be configured by multiple devices.

[0178] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0179] 100: Automatic photography system, 101: Image acquisition device, 102: View angle adjustment device, 103: Image input unit, 104: Detection and judgment unit, 105: View angle storage unit, 106: Object detection unit, 107: Human body detection unit, 108: Coordinate conversion unit, 109: Position storage unit, 110: Coordinate storage unit, 111: View angle calculation unit, 112: View angle operation unit, 113: Image output unit, 114: Monitor device

Claims

1. An imaging control device that controls the angle of view of an imaging device having a driving means capable of pan / tilt driving, a setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to set the preset angle of view to the preset angle of view set by the setting means; a coordinate conversion means for converting, when at least one of a plurality of target subjects is detected in an image captured by the image capture device, coordinates within the angle of view of the detected target subject into pan / tilt coordinate values; a storage means for storing the pan / tilt coordinate values ​​of the target subject converted by the coordinate conversion means; a field of view acquisition means for acquiring, when the storage means stores pan / tilt coordinate values ​​of another target subject different from the detected target subject, a field of view that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject; and 2. An imaging control device comprising: a camera that controls the driving means to select a preset angle of view that has not been used for imaging among the plurality of preset angles of view when a target subject is not detected in the captured image;

2. 2. The photographing control device according to claim 1, wherein the control means further controls the drive means based on the angle of view that includes the detected target subject and the other target subject, the angle of view being acquired by the angle of view acquisition means.

3. 2. The photographing control device according to claim 1, wherein the angle of view acquisition means calculates a pan coordinate value and a tilt coordinate value that are the center of an angle of view that includes the detected target subject and the other target subject, and a zoom magnification, to acquire an angle of view that includes the detected target subject and the other target subject.

4. the coordinate conversion means converts a focus area of ​​an object into pan / tilt coordinate values ​​when the object is detected among the plurality of target subjects in the image captured by the image capture device; 2. The photographing control device according to claim 1, wherein the angle of view acquisition means acquires an angle of view that includes the detected target subject and the other target subject so that the region of interest is at an edge of the angle of view.

5. 2. The photographing control device according to claim 1, wherein the angle of view acquisition means acquires the angle of view having a margin from the detected target subject to an end of the angle of view equal to the width of the detected target subject.

6. 2. The photographing control device according to claim 1, wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subjects based on a movement history of the detected target subject within a certain period of time.

7. the movement history of the detected target subject is made up of pan / tilt coordinate values ​​of the detected target subject obtained at regular intervals within a certain period of time; 7. The photographing control device according to claim 6, wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subject based on the movement history of the detected target subject and the pan / tilt coordinate values ​​of the other target subject.

8. a determination means for determining whether to adjust the angle of view depending on whether the predetermined period has elapsed; 7. The photographing control device according to claim 6, wherein the angle of view acquisition means, when the determination means determines that the certain period has elapsed, performs processing to acquire the angle of view using a movement history of the detected target subject within the certain period.

9. 2. The photographing control device according to claim 1, wherein the angle of view acquisition means acquires the angle of view that includes the detected target subject and the other target subject, also using an aspect ratio.

10. 10. The photographing control device according to claim 9, wherein the angle of view acquisition means determines whether the detected target subject and the other target subject extend beyond the current angle of view based on the aspect ratio.

11. The photographing control device according to claim 10, characterized in that the angle of view acquisition means determines whether the detected target subject and the other target subject extend beyond the angle of view based on a zoom magnification calculated as a horizontal angle of view value based on a difference value between the pan coordinate values ​​of the detected target subject and the other target subject.

12. 12. The photographing control device according to claim 11, wherein the angle of view acquisition means, when it is determined that the target subject protrudes, acquires the angle of view according to a zoom magnification calculated by using a difference value of tilt coordinate values ​​between the detected target subject and the other target subject as a vertical angle of view value.

13. A photographing control method executed by a photographing control device that controls the angle of view of a photographing device having a driving means capable of pan / tilt driving, comprising: a setting step of setting a plurality of preset angles of view; a control step of controlling the driving means to the preset angle of view set in the setting step; a coordinate conversion step of converting coordinates within the angle of view of the detected target subject into pan / tilt coordinate values ​​when at least one of a plurality of target subjects is detected in the image captured by the image capture device; a storing step of storing the pan / tilt coordinate values ​​of the target subject converted by the coordinate conversion step; a view angle acquisition step of acquiring a view angle that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject, when the pan / tilt coordinate values ​​of the other target subject are stored in the storage step; and a control method for controlling the driving means to a preset angle of view that has not been used for photographing among the plurality of preset angles of view when a target subject is not detected in the photographed image;

14. Computer, An imaging control device that controls the angle of view of an imaging device having a driving means capable of pan / tilt driving, a setting means for setting a plurality of preset angles of view; a control means for controlling the driving means to set the preset angle of view to the preset angle of view set by the setting means; a coordinate conversion means for converting, when at least one of a plurality of target subjects is detected in an image captured by the image capture device, coordinates within the angle of view of the detected target subject into pan / tilt coordinate values; a storage means for storing the pan / tilt coordinate values ​​of the target subject converted by the coordinate conversion means; a field of view acquisition means for acquiring, when the storage means stores pan / tilt coordinate values ​​of another target subject different from the detected target subject, a field of view that includes the detected target subject and the other target subject based on the pan / tilt coordinate values ​​of the target subject and the other target subject; and a program that functions as a photography control device that controls the drive means to a preset angle of view that has not been photographed among the plurality of preset angles of view when a target subject is not detected in the photographed image;

Images