Information processing device, information processing method, and program

Abstract

To facilitate grasping of a change in a position of a subject by the attitude control of a camera.SOLUTION: Based on the difference between a first position and a second position designated according to the user operation on the captured image imaged by an imaging device, the control information for controlling an attitude of the imaging device is obtained. Based on control information, the attitude control of the imaging device is performed.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Pan-tilt cameras that can remotely drive the imaging unit of the camera in a pan or tilt direction are used in video production sites. Patent Document 1 discloses a technology that performs control by determining whether telephoto control or wide-angle control is to be performed based on the positional relationship between two coordinates specified in a captured image. Patent Document 2 discloses a technology that performs camera rotation control so that the camera faces a specified point based on the current camera direction and a point specified on an image in an absolute coordinate system with the camera as the origin. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-082556 A [Patent Document 2] JP 2007-235696 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Documents 1 and 2, the position of the subject that the photographer is focusing on is not directly specified before and after the change in composition, which makes it difficult to grasp from which position to which the subject will move as a result of the change in composition.

[0005] An object of the present invention is to make it easier to grasp changes in the position of a subject caused by controlling the attitude of a camera. [Means for solving the problem]

[0006] In order to achieve the object of the present invention, for example, an information processing device according to one embodiment includes the following configuration: a first calculation means for calculating control information for controlling an attitude of the imaging device based on a difference between a first position designated in response to a user operation on an image captured by the imaging device and a second position designated after the designation of the first position, and a control means for controlling the attitude of the imaging device based on the control information. When the attitude of the imaging device changes in a state in which the first position is specified and the second position is not specified, the first calculation means discards information on the specified first position. do. Effect of the Invention

[0007] This makes it easier to grasp changes in the subject's position due to camera attitude control. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a system including an information processing device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the internal configuration of the imaging device according to the first embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of a display screen for a captured image according to the first embodiment. [Figure 4] 5A to 5C are views for explaining the attitude control process of the imaging apparatus according to the first embodiment. [Diagram 5] 4 is a flowchart of a processing example of a control method according to the first embodiment. [Figure 6] 5A to 5C are diagrams for explaining a method of calculating an attitude control amount of the image capturing apparatus according to the first embodiment. [Figure 7] FIG. 11 is a diagram for explaining an example of a display screen for a captured image according to the second embodiment. [Figure 8] 11 is a flowchart of a processing example of a control method according to the second embodiment. [Figure 9] 10A to 10C are diagrams for explaining a method of calculating an attitude control amount of an image capturing apparatus according to a second embodiment. [Figure 10] 13A to 13C are diagrams for explaining attitude control and zoom control processing according to the third embodiment. [Figure 11] 11 is a flowchart of a processing example of a control method according to the third embodiment. [Figure 12]13A to 13C are diagrams for explaining a method of calculating an attitude and zoom control amount according to the third embodiment. [Figure 13] FIG. 2 is a diagram showing an example of the internal configuration of the information processing device according to the first embodiment. [Figure 14] FIG. 13 is a diagram showing an example of a hardware configuration of an information processing device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] [Embodiment 1] The information processing device according to the present embodiment acquires an image captured by an imaging device, calculates control information for controlling the attitude of the imaging device based on a difference between a first position and a second position on the acquired captured image that are specified in response to a user operation, and controls the attitude of the imaging device based on the calculated control information.

[0011] FIG. 1 is a diagram showing an example of the configuration of an imaging device control system including an information processing device according to this embodiment. 100 is an imaging device (camera) that captures an image and transmits it to an information processing device 110. Here, the imaging device 100 can capture time-sequential captured images (videos) and transmit them to the information processing device 110, but in the following description, it is assumed that the imaging device 100 simply transmits captured images. The imaging device 100 is connected to the information processing device 110 via a network 140 in a state in which it can communicate with each other. The imaging device 100 has a pan-tilt-zoom mechanism, and its attitude and zoom magnification are controlled according to instructions from the information processing device 110. In the following description, the imaging device 100 is described as being one camera, but the number of imaging devices included in the imaging device 100 is not particularly limited. The information processing device 110 may issue an instruction for attitude control to one imaging device, or may issue an instruction for attitude control to multiple imaging devices at the same time. In addition, the imaging device 100 and the information processing device 110 may be separate devices, or may be included in the same device. Hereinafter, when the term "attitude control" or "zoom control" is used simply, this refers to the attitude control (of the pan angle or tilt angle) or the zoom magnification control of the image capture device 100.

[0012] The display unit 120 is, for example, a monitor such as a display, and displays an image output by the information processing device 110. The display unit 120 can acquire and display, for example, an image captured by the imaging device 100 via a video output unit 111 included in the information processing device 110. The input unit 130 is, for example, a mouse or a keyboard, and acquires an input from a user to the information processing device 110. The user can specify, for example, a position for controlling the attitude of the imaging device 100 via the input unit 130. The input unit 130 is not particularly limited in type as long as it can acquire a user's input, and may be, for example, a type in which an input is made to a touch panel that also serves as the display unit 120. In this embodiment, the input unit 130 transmits the user's input to the information processing device 110 via an input acceptance unit 112 included in the information processing device 110.

[0013] The network 140 connects the imaging device 100 and the information processing device 110, but the type of the network 140 is not particularly limited. The network 140 may be, for example, the Internet, a LAN, a WAN, or communication using a serial communication standard such as RS-232C. The network 140 may be wired communication or wireless communication. The imaging device 100 can transmit an image to the information processing device 110 using, for example, a LAN, a WAN, SDI, or HDMI (registered trademark) as the network 140.

[0014] 2 is a block diagram showing an example of a functional configuration of the imaging device 100 according to this embodiment. The imaging device 100 includes an imaging unit 201, an image processing unit 202, a system control unit 203, a command processing unit 204, a storage unit 210, and a communication unit 211. The imaging device 100 also includes a pan driving unit 205, a tilt driving unit 206, a pan / tilt unit 207, a lens driving unit 208, and a lens control unit 209.

[0015] The imaging unit 201 is composed of an imaging lens and an imaging element, and captures an image of a subject and converts the captured image into an electrical signal. The image processing unit 202 generates image data from the image captured by the imaging unit 201 and transmits the image data to the system control unit 203. The image processing unit 202 may generate image data by performing image processing such as noise removal or gamma correction on the captured image.

[0016] The system control unit 203 has one or more processors such as a CPU or an MPU, and controls the entire imaging device 100 by executing a program stored in the storage unit 210. The system control unit 203 also transmits image data from the image processing unit 202 to the information processing device 110 via the communication unit 211. The system control unit 203 may instruct the image processing unit 202 on imaging parameters such as image quality adjustment settings. In this case, the imaging unit 201 adjusts the imaging according to the parameters instructed by the system control unit 203. Furthermore, the system control unit 203 can store such imaging parameters or setting values ​​such as video output settings, or settings related to pan-tilt control, in the storage unit 210.

[0017] The system control unit 203 also includes a command processing unit 204. The command processing unit 204 interprets a command received from the information processing device 110 via the communication unit 211. The command, which will be described in detail later, may be a command related to attitude control (pan / tilt control) of the imaging device 100, a command related to a zoom operation, or a command related to both. Depending on the content of these commands, the command processing unit 204 transmits an instruction to the pan / tilt unit 207 or the lens control unit 209.

[0018] The pan driving unit 205 has a mechanical driving system that performs a panning operation of the camera, a motor as a driving source, and an angle sensor that detects a driving angle (none of which are shown). The pan driving unit 205 can drive the attitude of the imaging device in the pan direction by driving the motor. The tilt driving unit 206 has a mechanical driving system that performs a tilting operation of the camera, a motor as a driving source, and an angle sensor that detects a driving angle (none of which are shown). The tilt driving unit 206 can drive the attitude of the imaging device in the tilt direction by driving the motor. The pan / tilt unit 207 transmits the control direction and control amount of the attitude to the pan driving unit 205 or the tilt driving unit 206 in response to an instruction received from the system control unit 203.

[0019] The lens driving unit 208 includes a mechanical driving system including a focus lens and a zoom lens, a motor as a driving source, and a sensor (not shown) for detecting the position of the driving unit. The lens driving unit 208 can perform zoom control of the image capturing by the image capturing device 100. The lens control unit 209 transmits a zoom control amount to the lens driving unit 208 in response to an instruction received from the system control unit 203.

[0020] The storage unit 210 stores various pieces of information, such as programs used by the system control unit 203 or setting values. For example, the system control unit 203 reads and writes data from and to the storage unit 210. The communication unit 211 communicates with the information processing device 110 via the network 140. The communication unit 211 transmits received information to the system control unit 203 and performs communication processing to transmit communication data transmitted from the system control unit 203 to the information processing device 110.

[0021] FIG. 13 is a block diagram showing an example of a functional configuration of the information processing device 110 according to this embodiment. The information processing device 110 includes an acquisition unit 1301, a calculation unit 1302, and a control unit 1303. The acquisition unit 1301 acquires an image captured by the imaging device 100, and acquires a first position and a second position on the captured image. In this embodiment, the acquisition unit 1301 acquires a position designated in response to a user operation via the input unit 130, but the method is not particularly limited as long as the user-designated position can be acquired. For example, the acquisition unit 1301 can acquire the first acquired position as the first position and the second acquired position as the second position. In addition, the acquisition unit 1301 may detect a slide input by the user on the touch panel and acquire the start point and end point of the slide input as the first position and the second position, respectively.

[0022] The calculation unit 1302 obtains control information for controlling the attitude of the imaging device 100 based on the difference between the acquired first position and second position. In this embodiment, the calculation unit 1302 calculates an attitude control amount based on the acquired first position and second position and the angle of view of the acquired captured image so that the subject at the first position is positioned at the second position in the captured image captured after the attitude control. A calculation method by the calculation unit 1302 will be described later with reference to FIG. 6. The control unit 1303 transmits the control information (or the attitude control amount) calculated by the calculation unit 1302 to the imaging device 100, and issues an instruction to control the attitude of the imaging device 100 based on the control information.

[0023] In this embodiment, the information processing device 110 acquires information indicating the angle of view of the captured image from the imaging device 100 in addition to the captured image, calculates a control amount for the attitude of the imaging device based on the information as control information, and transmits the control information to the imaging device 100. However, the process is not particularly limited to this. For example, the information processing device may transmit a difference between a first position and a second position on the captured image to the imaging device 100 as control information, and the imaging device may calculate a control amount for the attitude based on the difference and the angle of view. With this configuration, even when the imaging device 100 transmits captured images to the information processing device 110 frequently, it is possible to omit the process of transmitting information indicating the angle of view each time.

[0024] 3 is an example of a reception screen for the information processing device 110 to obtain input of a first position and a second position for an image captured by the imaging device 100 according to this embodiment. This reception screen is displayed when the information processing device 110 transmits a video acquisition request to the imaging device 100 and outputs image data based on the captured image received from the imaging device 100 in response to the request to the display unit 120.

[0025] The reception screen 300 displays a status 301, a video display section 302, and a pointer 303. In FIG. 3, "Normal" is displayed as the status 301, which indicates that the attitude of the imaging device 100 is not controlled. For example, when a pan / tilt operation is performed, a sentence "Pan / tilt movement completed" is displayed as the status 301, as shown in FIG. 4 described later. What the status 301 displays is not limited to this, and any status related to imaging, such as the presence or absence of a zoom operation, may be displayed. The video display section 302 displays an image currently captured by the imaging device 100.

[0026] In the example of FIG. 3, the user operates the input unit 130 to move the pointer 303 and specify a position. In the following description, the acquisition unit 1301 acquires the positions specified here in the order of the first position and the second position. That is, here, the position where a person who is a subject is captured is specified as the first position, and the second position is specified as the position where the person is to be captured after posture control. The acquisition unit 1301 may acquire the first specified position as the second position, and the second specified position as the first position. According to such processing, after specifying the position where the person is to be captured after posture control, it is possible to specify the subject to be moved to that position. Also, after specifying two points on a captured image, one of them may be selected as the first position and the other as the second position.

[0027] FIG. 4 is a diagram for explaining an example in which attitude control of the imaging device 100 is performed from the reception screen of FIG. 3. In FIG. 4, a first position 402 and a second position 403 are specified on an image display section 401 before attitude control on a reception screen 400 similar to the reception screen 300 of FIG. 3. The image display section 411 displays a captured image immediately after attitude control of the imaging device 100 is performed in accordance with the specification of the first position 402 and the second position 403. In FIG. 4, a subject person shown at the first position on the image display section 302 is now located at the second position on the image display section 411 as a result of attitude control of the imaging device 100. In addition, a sentence "Pan / tilt movement completed" is displayed as the status 301 at the top of the image display section 411.

[0028] Fig. 5 is a flowchart showing an example of the attitude control process performed by the information processing device according to this embodiment. The process shown in Fig. 5 is assumed to start in a state of waiting for the user to specify a position on the reception screen as shown in Fig. 3. In step S501, the acquisition unit 1301 determines whether or not a first position has been specified. If it has been specified, the process proceeds to step S502, and if not, the process repeats step S501. In step S502, the acquisition unit 1301 determines whether or not a second position has been specified. If it has been specified, the process proceeds to step S503, and if not, the process proceeds to step S505.

[0029] In step S503, the calculation unit 1302 calculates an attitude control amount of the imaging device 100 based on the first position accepted in step S501, the second position accepted in step S502, and the current angle of view of the imaging device 100. The calculation method will be described later with reference to FIG. 6. Next, the process proceeds to step S504. In step S504, the control unit 1303 transmits an attitude control instruction to the communication unit 211 of the imaging device 100 based on the control amount calculated in step S503, and the attitude control of the imaging device 100 is performed. Then, the process ends.

[0030] In step S505, the acquisition unit 1301 determines whether or not attitude control has been performed after the first position has been received in step S501. If the attitude of the imaging device 100 has changed due to an instruction from a device other than the information processing device 110, such as a remote controller or an external application, the process proceeds to step S506. If not, the process proceeds to step S507.

[0031] In step S507, the acquisition unit 1301 determines whether a predetermined time has elapsed since the reception of the first position in step S501. If the predetermined time has elapsed, the process proceeds to step S506, and if the predetermined time has not elapsed, the process returns to step S502. The predetermined time used as a threshold in step S507 is not particularly limited, and may be set to a desired time depending on the application, conditions, etc., such as 5 seconds, 10 seconds, 30 seconds, or 1 minute.

[0032] In step S506, the acquisition unit 1301 discards the information on the first position received in step S501, and ends the process. After this process ends, the acquisition unit 1301 may return to the top of the flowchart in FIG. 5 again and start waiting for the reception of a position designation, or may end the process completely. According to the process of returning to the top of the flowchart again, when attitude control is performed by a device other than the information processing device 110, it becomes possible to restart the reception process of the position designation from the beginning. Also, when the second position cannot be acquired even after a certain time has elapsed since the first position was acquired, it becomes possible to restart the reception process of the position designation from the beginning.

[0033] Next, with reference to FIG. 6, an example of a process in which the calculation unit 1302 according to this embodiment calculates the amount of posture control will be described. FIG. 6(a) is a diagram showing a coordinate system defined on the image of the video display unit 302 in FIG. 3. Here, coordinates in the coordinate system thus defined on the video display unit 302 are specified and acquired as the first position and the second position. Coordinate 601 is the coordinate of the upper left corner of the image 600, and is the origin (0,0) of the coordinate system shown in FIG. 6. Coordinates 602, 603, and 604 are the coordinates of the upper right corner (Xmax ,0), bottom left edge (0, Y max ), and the bottom right corner (X max ,Y max ) coordinates. Coordinates 605 are first coordinates (X1, Y1) that have been received and are acquired by the acquisition unit 1301, and coordinates 606 are second coordinates (X2, Y2) that have also been received. In this example, when control in the pan direction is performed by attitude control, the image moves parallel to the X axis, and when control in the tilt direction is performed, the image moves parallel to the Y axis.

[0034] 6B is a plan view showing the angle of view in the panning direction of the image capture device 100. Angle 611 is the horizontal angle of view θ p In addition, the angle 612 is a horizontal control amount θ of the captured image in the posture control, which is calculated by the calculation unit 1302 in this embodiment. pm θ pm is calculated based on the horizontal component of the vector from the first position to the second position. For example, θ pm is the above coordinate X max , X1, and X2, the horizontal length of the captured image is calculated based on the following formula (1) in accordance with the ratio of the horizontal length to the width of the captured image. θ pm =-θ p (X2-X1) / X max (1)

[0035] Equation (1) expresses the horizontal control amount θ pm is a formula for calculating the vertical control amount θ of the captured image. tm Similarly, θ is calculated based on the vertical component of the vector from the first position to the second position. That is, for example, θ is calculated according to the following formula (2) according to the ratio of the vertical length to the height of the captured image. tm Here, θ t is the vertical angle of view of the imaging device 100 on the same plan view. θ tm =-θ t (Y2-Y1) / Y max (2)

[0036] In this embodiment, the attitude of the imaging device 100 is controlled according to the values ​​calculated by formula (1) and formula (2). At this time, if the value of formula (1) or formula (2) is a positive value, the attitude is controlled in the positive direction of the coordinate system shown in Fig. 6(a) by the angle of that value, and if the value is negative, the attitude is controlled in the negative direction of the coordinate system shown in Fig. 6(a) by the angle of that value. That is, the control direction of the pan angle is the opposite direction to the direction of the horizontal component of the vector from the first position to the second position, and the control direction of the tilt angle is the opposite direction to the direction of the vertical component of the vector from the first position to the second position.

[0037] In this embodiment, the amount of attitude control is calculated using a plan view such as that shown in Fig. 6(b), but the method of calculating the amount of attitude control is not limited to this. For example, a spherical coordinate system centered on the image capture device 100 may be considered based on the angle of view of the image capture device and the first and second coordinates, and the amount of attitude control may be calculated based on the coordinates.

[0038] According to this process, it is possible to receive input of the first position and the second position, calculate control information for controlling the attitude of the imaging device based on the difference between those positions, and control the attitude of the imaging device. Therefore, it is possible to instruct the attitude control of the imaging device in a format that makes it easy to understand the movement of the position of the subject of interest before and after the attitude control.

[0039] [Embodiment 2] In the second embodiment, there is an area set as an area that can be captured, and the attitude of the imaging device is controlled so as not to capture an area outside that area (restricted area). Attitude control is performed in the same manner as in the first embodiment, so a duplicated description will be omitted. For the above control, when the acquisition unit 1301 acquires a first position, it acquires a second position so that a captured image after attitude control in accordance with the designation of the second position does not show the restricted area. Furthermore, the information processing device according to this embodiment can perform a visual display so that an area that can be designated as the second position for the above control and an area that cannot be designated (undesignatable area) can be distinguished.

[0040] FIG. 7 is a diagram showing an example of a captured image in which an undesignated area for the above-mentioned restricted area is visualized. A captured image is displayed on the video display unit 701, and basic position designation and attitude control processing are similar to those of the video display unit 302 shown in the third embodiment. In FIG. 7, a restricted area 702 exists, and an undesignated area 704 calculated according to a designated first position 703 is displayed on the video display unit 701. In this example, a second position cannot be designated on the undesignated area, and the acquisition unit 1301 acquires the designation of the second position from other areas. In this example, the undesignated area 704 is displayed superimposed on the captured image of the video display unit 701, but the display method of the undesignated area 704 is not limited to this. For example, the undesignated area 704 itself may not be displayed, and the area that cannot be designated may be shown to the user through a change in the display of the pointer depending on whether the pointer is on the undesignated area 704 or not. In addition, the area set as an imageable area corresponding to the restricted area is set in advance, but may be set by the user, for example.

[0041] Fig. 8 is a flowchart showing an example of the attitude control process performed by the information processing device according to this embodiment. The process shown in Fig. 8 starts in a state of waiting for the user to specify a position on the reception screen as shown in Fig. 7. In step S801, the acquisition unit 1301 determines whether a first position has been specified. If it has been specified, the process proceeds to step S802, and if not, the process repeats step S801.

[0042] In step S802, the calculation unit 1302 calculates the range of the undesignated area based on the first position accepted in step S801 and a preset restricted area. The calculation process here will be described later with reference to FIG. 9. In step S803, the acquisition unit 1301 sets so as not to acquire the designation of the second position from the undesignated area calculated in step S1302. In step S804, the control unit 1303 displays the calculated undesignated area superimposed on the video display unit 701. After step S804, the acquisition unit 1301 advances the process to step S502 and determines whether or not the second position has been designated. The processes in steps S502 to S507 are performed in the same manner as those in FIG. 5 of the first embodiment, and therefore repeated explanations will be omitted.

[0043] Next, the calculation process of the undesignable area performed by the calculation unit 1302 according to this embodiment will be described with reference to Fig. 9. In Fig. 9, the coordinates of the four corners are defined in the same way as in Fig. 6 of the first embodiment. Fig. 9(a) is a diagram showing a coordinate system on an image in which an undesignable area is superimposed and displayed on the video display unit of Fig. 7. Coordinates 901 are the first coordinates (X1, Y1) that are specified, and coordinates 903 are the coordinates (X1, Y1) of the lower left corner of the area that can be specified as the second coordinates. 2min ,Y 2max ) The area 902 is an undesignated area.

[0044] 9B is a plan view showing the angle of view in the panning direction of the image capturing apparatus 100, similar to FIG. 6B. Angle 911 is the horizontal angle of view θ p Angle 912 is the angle θ from the origin in the optical axis direction of the image capture device 100 at the current time. pnow Angle 913 is an angle θ on a plan view from the origin in the direction of the optical axis of the image capturing apparatus 100 after attitude control when the coordinate 903 in FIG. pmax The above X 2min For example, the above coordinate X max , X1, θ pnow , and θ pmax Based on this, it is calculated based on the following formula (3). X 2min =X1-(θpmax -θ pnow ) / θ p X max (3)

[0045] Equation (3) is the horizontal coordinate X of the bottom left corner that can be specified as the second coordinate. 2min This is the formula for calculating the vertical coordinate Y 2max can be calculated in a similar manner. That is, Y 2max Here, θ t is the vertical angle of view of the image capture device 100 on the same plan view, and θ tmax is the angle from the origin in the optical axis direction of the image capturing apparatus 100 after attitude control on a vertical plan view when the coordinate 903 in FIG. 9(a) is specified as the second coordinate. Y 2max =Y1-(θ tmax -θ tnow ) / θ t Y max (4)

[0046] In this embodiment, the calculation of each coordinate is performed using a plan view such as that shown in Fig. 9(b), but the positions used are not particularly limited to this. For example, a spherical coordinate system centered on the image capture device 100 may be considered based on the angle of view of the image capture device, the first coordinate, and the restricted area, and the undesignated area may be calculated based on the coordinates.

[0047] According to such a process, it is possible to specify a position on the captured image and control the attitude of the imaging device so that the imaging range falls within the area set as the imageable area.

[0048] [Embodiment 3] In the third embodiment, a third position and a fourth position are acquired to complement the first position and the second position, respectively. Next, based on a rectangle formed from the first position and the third position, a rectangle formed from the second position and the fourth position, and the angle of view of the imaging device 100, control information and magnification information for controlling the zoom magnification of the imaging device 100 are calculated. Based on the control information and magnification information calculated here, the attitude and zoom magnification of the imaging device 100 are controlled. In the following, a rectangle formed from the positions of two points refers to a rectangle having the positions of the two points as diagonals.

[0049] In this embodiment, in addition to the first position and the second position, a third position and a fourth position are specified. The third position and the fourth position are specified in the same manner as the first position and the second position. For example, the acquisition unit 1301 may acquire the first position specified by using the pointer 303 as the first position, the second position as the third position, the third position as the second position, and the fourth position as the fourth position. For example, the acquisition unit 1301 may detect a slide input by the user on the touch panel, and acquire the start point and the end point of the first slide input as the first position and the third position, respectively, and the start point and the end point of the second slide input as the second position and the fourth position, respectively.

[0050] In this embodiment, the rectangle formed from the first position and the third position and the rectangle formed from the second position and the fourth position are specified so that their aspect ratios are equal. That is, for example, when the first to third positions are specified, the fourth position is specified only from a position where the rectangle formed from the second position has the same aspect ratio as the rectangle formed from the first position and the third position. For this reason, when the first to third positions are specified, the acquisition unit 1301 limits the range in which the fourth position can be specified so that the aspect ratios are equal as described above.

[0051] The calculation unit 1302 according to this embodiment calculates the control information and the magnification information based on the position and size of a rectangle formed from the first position and the third position, the position and size of a rectangle formed from the second position and the fourth position, and the angle of view of the imaging device 100. The calculation process here will be described later with reference to FIG.

[0052] 10 is a reception screen for acquiring input of each position, showing an example of attitude control and zoom according to the above-mentioned first to fourth positions. A reception screen 1000 displays a status, an image display section, and a pointer similar to those in FIG. 3 of the first embodiment. An image display section 1001 displays an image currently captured by the imaging device 100. Positions 1002 to 1005 are a first position, a third position, a second position, and a fourth position, respectively.

[0053] The image display unit 1011 displays the captured image immediately after the attitude control and zoom control of the imaging device are performed. In Fig. 10, a subject within a rectangle formed by positions 1002 and 1003 on the image display unit 1001 is displayed within a rectangle formed by positions 1004 and 1005 on the image display unit 1011 as a result of the attitude control and zoom control. In this example, each rectangle is displayed with a dashed line, but the display format is not particularly limited, and may be a dotted line, a diagonal line may be displayed within the rectangle, or no rectangle may be displayed.

[0054] FIG. 11 is a flowchart showing an example of processing performed by the information processing device according to this embodiment. The processing shown in FIG. 11 is assumed to start in a state of waiting for the reception of a position designation by a user on a reception screen as shown in FIG. 3. In step S1101, the acquisition unit 1301 judges whether or not a first position has been designated. If it has been designated, the process proceeds to step S1102, and if not, the process repeats step S1101. In step S1102, the acquisition unit 1301 judges whether or not a third position has been designated. If it has been designated, the process proceeds to step S1103, and if not, the process repeats step S1102. In step S1103, the acquisition unit 1301 judges whether or not a second position has been designated. If it has been designated, the process proceeds to step S1104, and if not, the process repeats step S1103.

[0055] In step S1104, the acquisition unit 1301 limits the range in which the fourth position can be specified so that the aspect ratio of the rectangle formed by the second position and the fourth position is equal to the aspect ratio of the rectangle formed by the first position and the third position. In step S1105, the acquisition unit 1301 determines whether the fourth position has been specified within the range limited in step S1104. If it has been specified, the process proceeds to step S1106, and if not, the process repeats step S1105.

[0056] In step S1106, the calculation unit 1302 calculates an attitude control amount and a zoom control amount of the imaging device based on the first to fourth positions and the current angle of view of the imaging device 100. This calculation process will be described later with reference to Fig. 12. In step S1107, the control unit 1303 transmits an attitude control and zoom control instruction to the communication unit of the imaging device 100 based on the control amount calculated in step S1106, and the attitude control of the imaging device 100 is performed. After that, the process ends.

[0057] In this embodiment, the positions are specified so that the aspect ratios of the two rectangles are equal, but the present invention is not limited to this. For example, when the positions are specified so that the aspect ratios of the two rectangles are different, the subject in the rectangle formed by the first position and the third position may be resized to be the same size as the rectangle formed by the second position and the fourth position without maintaining the aspect ratio. That is, the zoom control amount in the pan direction and the tilt direction may be calculated separately.

[0058] As in the processes of steps S505 to S507 in the first embodiment, the acquisition unit 1301 may discard information on the positions specified up to that point if attitude control or zoom control is performed before acquiring the designation of all four positions, or if a certain period of time has elapsed. Similarly, the process thereafter may also return to step S1301 to wait for acceptance of position designation, or may be completely terminated.

[0059] Next, an example of a process in which the calculation unit 1302 according to this embodiment calculates the amount of attitude control and zoom control will be described with reference to Fig. 12. In Fig. 12, the coordinates of the four corners are defined in the same manner as in Fig. 6 of the first embodiment. Fig. 12(a) is a diagram showing a coordinate system on an image in which first to fourth coordinates are specified on the video display unit of Fig. 7. Coordinates 1201 to 1204 are the specified first coordinates (X1, Y1), second coordinates (X2, Y2), third coordinates (X3, Y3), and fourth coordinates (X4, Y4), respectively.

[0060] 12B is a plan view showing the angle of view in the panning direction of the image capture device 100, similar to FIG. 6B. Angle 1211 is the horizontal angle of view θ p The angle 1212 is a horizontal control amount θ in the attitude control, which is calculated by the calculation unit 1302 in this embodiment. pm The position 1213 is the horizontal center position of the rectangle formed by the coordinates 1201 and 1203. The position 1214 is the horizontal center position of the rectangle formed by the coordinates 1202 and 1204. θ pmFor example, the above coordinate X max , X1, X2, X3, and X4, according to the following formula (5). θ pm =-θ p ((X2+X4) / 2-(X1+X3) / 2) / X max (5)

[0061] Equation (5) expresses the horizontal control amount θ pm The vertical control amount θ tm can be calculated in a similar manner. That is, θ tm Here, θ t is the vertical angle of view of the imaging device 100 on the same plan view. θ tm =-θ t ((Y2+Y4) / 2-(Y1+Y3) / 2) / Y max (6)

[0062] The attitude control amount of the image capturing apparatus 100 is calculated based on the above-mentioned formulas (5) and (6). Next, the zoom control amount of the image capturing apparatus 100 is calculated based on the following formula (7). Here, Z is the zoom magnification of the image capturing apparatus 100 at the current time point, and Z after is the zoom magnification of the image capture device 100 after zoom control. Z after =Z·((X4-X2) / (X3-X1)-1) (7)

[0063] In this embodiment, the amount of attitude control is calculated using a plan view such as that shown in Fig. 12(b), but the method of calculating the amount of attitude control is not limited to this. For example, a spherical coordinate system centered on the image capture device 100 may be considered based on the angle of view of the image capture device and the first to fourth coordinates, and the amount of attitude control and the amount of zoom control may be calculated based on the coordinates.

[0064] According to this process, the input of the first to fourth positions is acquired, and the amount of attitude control and zoom control can be calculated and controlled based on these positions and the angle of view and zoom magnification of the imaging device. Therefore, it is possible to instruct the control of the imaging device in a format that makes it easy to understand the changes in position and size of the subject area of ​​interest before and after the attitude and zoom control.

[0065] [Embodiment 4] In the above-described embodiments, each processing unit shown in, for example, FIG. 13 is realized by dedicated hardware. However, some or all of the processing units of the information processing device may be realized by a computer. In this embodiment, at least a part of the processing according to each of the above-described embodiments is executed by a computer.

[0066] FIG. 14 is a diagram showing the basic configuration of a computer. In the diagram, a processor 1410 is, for example, a CPU, and controls the operation of the entire computer. A memory 1420 is, for example, a RAM, and temporarily stores programs, data, etc. A computer-readable storage medium 1430 is, for example, a hard disk or a CD-ROM, and stores programs, data, etc. for a long period of time. In this embodiment, a program that realizes the functions of each unit stored in the storage medium 1430 is read into the memory 1420. Then, the processor 1410 operates according to the program on the memory 1420, thereby realizing the functions of each unit.

[0067] 14, an input interface 1440 is an interface for acquiring information from an external device. An output interface 1450 is an interface for outputting information to an external device. A bus 1460 connects the above-mentioned units and enables data exchange.

[0068] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0069] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0070] 1301: Acquisition unit, 1302: Calculation unit, 1303: Control unit

Claims

1. a first calculation means for calculating control information for controlling an attitude of the imaging device based on a difference between a first position designated in response to a user operation on an image captured by the imaging device and a second position designated after the designation of the first position; a control means for controlling the attitude of the imaging device based on the control information; Equipped with The information processing device according to claim 1, wherein the first calculation means discards information of the specified first position when the attitude of the imaging device changes while the first position is specified and the second position is not specified.

2. The information processing apparatus according to claim 1 , further comprising an input unit configured to receive a user operation for specifying the first position and the second position on the captured image.

3. 3. The information processing device according to claim 1, wherein the first calculation means discards information about the specified first position when a predetermined time has elapsed while the first position is specified and the second position is not specified.

4. 4. The information processing device according to claim 1, wherein the first calculation means calculates control information for controlling a pan angle of the imaging device based on a horizontal component of a vector from the first position to the second position, and calculates control information for controlling a tilt angle of the imaging device based on a vertical component of the vector.

5. 5. The information processing device according to claim 4, wherein the first calculation means calculates a field angle relative to a horizontal field angle of the captured image according to a ratio of a horizontal length to a width of the captured image as a control amount for the pan angle, and calculates a direction opposite to a direction in a horizontal component of the vector as a control direction for the pan angle.

6. 6. The information processing device according to claim 4, wherein the first calculation means calculates a field angle relative to a vertical field angle of the captured image according to a ratio of a vertical length of the captured image to a height of the captured image as a control amount of the tilt angle, and calculates a direction opposite to a direction of a vertical component of the vector as a control direction of the tilt angle.

7. A setting unit is further provided for setting an area that can be captured as an image captured by the imaging device, 7. The information processing device according to claim 1, wherein the second position is specified based on the first position and the imageable area when the first position is specified, such that an image captured after controlling the attitude of the imaging device based on a difference between the first position and the second position falls within the imageable area.

8. a second calculation means for calculating, when the first position is designated, based on the first position and the imageable area, a range of the second position within which an image captured after control of the attitude of the imaging device based on a difference between the first position and the second position does not fall within the imageable area; a display unit that displays the range of the second position that does not fall within the imageable area on the captured image by superimposing the range on the captured image; The information processing device according to claim 7 , further comprising:

9. a third position and a fourth position are further specified on the captured image in response to a user operation; the first calculation means further calculates magnification information for controlling a zoom magnification of the imaging device based on a rectangle having the first position and the third position as diagonal corners, a rectangle having the second position and the fourth position as diagonal corners, and a zoom magnification of the imaging device at a time when the captured image was captured; The information processing apparatus according to claim 1 , wherein the control means further performs zoom control of the imaging device based on the calculated magnification information.

10. 10. The information processing device of claim 9, wherein the first calculation means calculates the magnification information based on a size of a rectangle having the first position and the third position as diagonal corners and a size of a rectangle having the second position and the fourth position as diagonal corners so that a subject in a rectangle having the first position and the third position as diagonal corners in the captured image before zoom magnification control is captured with the size of a rectangle having the second position and the fourth position as diagonal corners in an captured image captured after zoom control based on the magnification information.

11. 11. The information processing device according to claim 9, wherein the first position, the second position, the third position, and the fourth position are specified so that an aspect ratio of a rectangle having the first position and the third position as diagonal corners is equal to an aspect ratio of a rectangle having the second position and the fourth position as diagonal corners.

12. 12. The information processing device according to claim 11, characterized in that, when the first position, the second position, and the third position are specified, the fourth position is specified so that an aspect ratio of a rectangle having the first position and the third position as diagonal corners is equal to an aspect ratio of a rectangle having the second position and the fourth position as diagonal corners.

13. The information processing device according to claim 9 , wherein the first position, the third position, the second position, and the fourth position are specified in this order.

14. determining control information for controlling an attitude of the imaging device based on a difference between a first position designated in response to a user operation on an image captured by the imaging device and a second position designated after the designation of the first position; controlling the attitude of the imaging device based on the control information; Equipped with An information processing method, characterized in that, in the step of obtaining the control information, if the attitude of the imaging device changes when the first position is specified and the second position is not specified, information of the specified first position is discarded.

15. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 13.

Images