Information processing device, information processing method, and program

Abstract

To easily perform the setting of imaging parameters and the rotational display of a captured image.SOLUTION: An information processing device is configured to acquire information specifying an imaging range on an image captured by an imaging unit that can rotate around an optical axis, determine whether to rotate the imaging unit around the optical axis on the basis of the acquired information, and control the imaging unit on the basis of the determination result.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Imaging devices having a pan, tilt, rotation mechanism, etc., and capable of displaying captured images on an operation terminal such as a PC in real time are widespread. As such an imaging device, there is also one provided with a camera head portion that can rotate with the optical axis center as the orientation of the imaging sensor, and it is possible to change the imaging angle of view according to the user's preference. There are also those that can change the orientation of the displayed image according to the orientation of the imaging sensor.

[0003] For example, Patent Document 1 discloses an imaging device capable of changing the orientation of an imaging sensor and the orientation of a displayed image by a user performing a predetermined operation on the displayed image of an operation terminal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technique disclosed in Patent Document 1, there are many setting items by the user, and a lot of labor is required to change the orientation of the imaging sensor and the orientation of the displayed image.

[0006] An object of the present invention is to easily set imaging parameters for imaging an imaging range specified by a user in an apparatus capable of changing the orientation of an imaging sensor.

Means for Solving the Problem

[0007] In order to achieve the object of the present invention, for example, an information processing apparatus according to an embodiment includes the following configuration. That is, a first acquisition means for acquiring information specifying an imaging range on an imaging image captured by an imaging unit that can rotate around an optical axis, a determination means for determining whether to rotate the imaging unit around the optical axis based on the acquired information, and a control means for controlling the imaging unit based on the determination result of the determination means.

Effect of the Invention

[0008] The setting of imaging parameters and the rotation display of the imaging image are easily performed.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] [Embodiment 1] An example of the configuration of an imaging system including the information processing apparatus according to this embodiment is shown in FIG. 1. In the imaging system according to this embodiment, an information processing apparatus 100 (imaging apparatus) and an operation terminal 110 for operating the information processing apparatus 100 are connected via a network 109. The information processing apparatus 100 distributes the captured image captured by the information processing apparatus 100 to the operation terminal 110. The operation terminal 110 gives an imaging instruction to the information processing apparatus 100 regarding the imaging direction or imaging angle of view of the information processing apparatus 100. Further, the operation terminal 110 receives the captured image from the information processing apparatus 100 and causes a display unit 117 described later to display the received captured image.

[0012] The configuration of the information processing apparatus 100 (imaging apparatus) in the example of FIG. 1 will be described. The information processing apparatus 100 according to this embodiment will be described as being provided with an imaging unit 101 and capable of acquiring a captured image. However, the information processing apparatus 100 does not necessarily have to have an imaging function, and it may be configured to acquire a captured image to be processed from an external imaging apparatus (not shown).

[0013] Hereinafter, the configuration of the information processing apparatus 100 according to this embodiment will be described. The information processing apparatus 100 includes an imaging unit 101, an image processing unit 102, a communication control unit 103, a pan drive unit 104, a tilt drive unit 105, a rotation drive unit (R drive unit) 106, a zoom drive unit 107, and a drive control unit 108. Further, the information processing apparatus 100 is connected via the network 109 so as to be able to transmit and receive information to and from the operation terminal 110.

[0014] The imaging unit 101 includes an imaging sensor and captures an image of a subject. The image data captured by the imaging unit 101 is sent to the image processing unit 102 described later. The image processing unit 102 processes the video signal captured by the imaging unit 101.

[0015] The image processing performed in the image processing unit 102 includes known general image processing performed on the captured image data. For example, the image processing unit 102 can perform digital video signal conversion of the image signal captured by the imaging unit 101, or encoding of the digital video signal. As the format of the digital video signal, for example, YUV or the like may be used. As the encoding method of the digital video signal, for example, a method based on a standard such as MPEG4, H.265, H.264, MJPEG or JPEG may be used. The video signal on which image processing has been performed by the image processing unit 102 is output from the network 109 to the operation terminal 110 described later via the communication control unit 103.

[0016] The image processing unit 102 acquires information specifying the imaging range on the captured image. The image processing unit 102 according to the present embodiment can acquire information regarding two points designated by the user on the captured image as information specifying the imaging range. More specifically, the image processing unit 102 can acquire defining information, which is information for defining a straight line having two points designated by the user on the captured image at both ends. Hereinafter, when simply expressed as a "straight line", it refers to a line segment defined by such defining information. Further, hereinafter, such defining information is used as information specifying the imaging range, but different information may be used as long as it is information specifying the imaging range. Here, the image processing unit 102 can acquire the defining information based on a user operation input via the operation terminal 110. For example, the image processing unit 102 may detect a straight line input by the user (for example, by a drag operation using a mouse or the like) in the display image displayed on a display unit 117 of the operation terminal 110 described later, and acquire the straight line as the defining information. In the present embodiment, for example, when two points are designated on the display image, a line segment connecting the two points may be regarded as a straight line, or a line input by the user may be regarded as a straight line. Hereinafter, based on the straight line input by the user in this way, the start point and the end point of the straight line are acquired as two points, but a mode in which two points are directly input may also be possible, and it is not particularly limited in this way as long as the two designated points are acquired.

[0017] Also, for example, it is conceivable that the line input by the user is distorted and does not form a straight line. From such a viewpoint, the image processing unit 102 may acquire a plurality of coordinates on the line drawn on the display image, and generate a straight line by performing straight line approximation based on the plurality of coordinates. Also, there may be a configuration in which there are a plurality of methods for generating a straight line and the user can select one of them. Also, the "two points on the display" may both be located on the captured image, or may be located outside the captured image of the display including the captured image.

[0018] Also, the image processing unit 102 controls the rotation of the imaging unit that captures the captured image based on the acquired information for specifying the imaging range on the captured image (here, the information indicating two points). In the present embodiment, the image processing unit 102 sets a rectangular area on the captured image based on the two points, and determines the imaging range of the imaging unit 101 by controlling the imaging parameters based on the rectangular area (for example, determines the imaging range so as to capture the rectangular area). In the present embodiment, the control of the imaging range is performed by controlling the pan, tilt, zoom, and rotation drives so as to capture a predetermined imaging range. The control of the imaging posture of the imaging device performed to capture a specified area in the captured image can be performed by known general imaging techniques, and the description thereof is omitted here.

[0019] The pan drive unit 104, the tilt drive unit 105, and the R drive unit 106 respectively perform the pan drive, tilt drive, and rotation drive of the imaging unit 101. The zoom drive unit 107 changes the zoom magnification of the imaging unit 101. The rotation drive according to the present embodiment is to rotate the imaging element of the imaging unit 101 around the optical axis. The drive control unit 108 controls the pan drive unit 104, the tilt drive unit 105, the R drive unit 106, and the zoom drive unit 107 to control the imaging posture and imaging angle of view of the imaging unit 101. The control of the imaging posture according to the present embodiment includes the control of the pan drive, tilt drive, and rotation drive of the imaging unit 101.

[0020] As described above, the image processing unit 102 controls the rotation of the imaging unit 101. For example, the image processing unit 102 may control the rotation of the imaging unit based on the positional relationship between the start point and the end point of the straight line included in the specified information. For example, in a coordinate system with the upper left end of the captured image as the origin, the image processing unit 102 may determine whether to rotate the imaging unit by a predetermined angle (here, 180°) based on the vertical relationship (greater than or less than relationship) between the y coordinate of the start point and the y coordinate of the end point of the straight line. In this case, for example, when the y coordinate of the start point of the straight line is less than the y coordinate of the end point in the coordinate system with the upper left end of the captured image as the origin, the image processing unit 102 may not rotate the imaging unit, and when the y coordinate of the start point of the straight line is greater than or equal to the y coordinate of the end point, the imaging unit may be rotated by 180°. Note that in this example, only the y coordinate is compared to determine whether to rotate the imaging unit by 180°, but for example, the x coordinate may also be referred to, and the imaging unit may be rotated by 180° only when both the x coordinate and the y coordinate of the start point are greater than or equal to those of the end point. Such conditions and rotation angles are just examples and may be arbitrarily set according to the conditions desired by the user. Also, hereinafter, the coordinates in the captured image or display shall be the coordinates in a two-dimensional coordinate system with the upper left end of the captured image as the origin, the vertical downward direction of the image as the positive y axis, and the horizontal right direction of the image as the positive x axis.

[0021] The communication control unit 103 controls the transmission and reception of data with the outside of the information processing apparatus 100. For example, the communication control unit 103 can perform packetization processing on the encoded digital video signal. Further, the communication control unit 103 can also perform multiplexing processing of packets by arranging the digital video signal to be transmitted to the outside according to a predetermined format and outputting the signal to the network 109 according to the predetermined format. As the predetermined format here, for example, formats such as HTTP (Hypertext Transfer Protocol) or RTP (Real-time Transport Protocol) can be used. The communication control unit 103 transmits the digital video signal captured by the imaging unit 101 to the network 109. Also, the communication control unit 103 receives an instruction from the operation terminal 110 described later and outputs it to the drive control unit 108.

[0022] Next, the configuration of the operation terminal 110 will be described. The operation terminal 110 is a terminal that acquires operations by the user. The operation terminal 110 may be, for example, a personal computer (PC) or a mobile terminal such as a smartphone, and is not particularly limited as long as it can acquire the user's operations.

[0023] The communication control unit 111 receives the digital video signal output from the communication control unit 103 and transmits it to the display control unit 113 described later. Further, the communication control unit 111 transmits the instruction of the drive instruction unit 112 described later to the information processing apparatus 100 via the network 109.

[0024] The drive instruction unit 112 gives an instruction on the control content when controlling the posture of the imaging unit 101. For example, the drive instruction unit 112 can give an instruction on the rotation type, rotation direction, or rotation angle for rotating the imaging unit 101. Here, the instruction on the rotation type refers to an instruction on which of the pan rotation drive, tilt rotation drive, rotation drive, and zoom drive the instruction from the operation terminal 110 indicates. The instruction on the rotation direction refers to an instruction on the rotation direction for each rotation type. The instruction on the rotation angle refers to an instruction on the rotation angle for each rotation type.

[0025] The display control unit 113 performs control to display the captured image transmitted from the information processing apparatus 100 on the display unit 117. Further, the display control unit 113 receives the specified information acquired by the interface unit 118 described later and performs rotation display of the captured image based on the specified information.

[0026] The interface unit 118 acquires the specified information input by the input device 120 described later and outputs it to the display control unit 113 and the drive instruction unit 112.

[0027] The input device 120 is composed of a pointing device such as a mouse or a digitizer, a keyboard, or a joystick, etc., and receives user input. For example, the input device 120 may be used to receive input such as a rotation type, a rotation direction, or a rotation angle with respect to the drive instruction unit 112. Also, the input device 120 may be used for the user to input specified information.

[0028] For example, the input device 120 is a pointing device or the like, and based on a user input that designates two points in the image displayed on the display unit 117, a line segment between the two points can be acquired as specified information. Also, for example, the input device 120 is a mouse or the like, and specified information can be acquired based on a user input that directly draws a line in the image displayed on the display unit 117.

[0029] Details will be described later, but in this embodiment, there may be a case where the imaging parameters are controlled so that a rectangular area having a straight line defined by the specified information as a diagonal line becomes the imaging range. From such a viewpoint, the straight line of the user input for designating the specified information may be defined as the diagonal line of a rectangular area (the vertical side and the horizontal side are each parallel to the captured image) that maintains the aspect ratio of the imaging range by the imaging sensor with the starting point of the straight line being the vertex at the upper left end of the rectangular area. Here, the vertical side of the rectangle refers to the side with a smaller angle with the y-axis in the coordinate system with the upper left end of the captured image as the origin.

[0030] Also, for example, the specified information may be the straight line input by the user as it is, or the input straight line may be corrected as the specified information. For example, among the rectangular areas that maintain the aspect ratio of the imaging range by the imaging sensor with the starting point of the straight line input by the user being the vertex at the upper left end of the rectangular area, the diagonal line having the vertex at the lower right end closest to the end point of the straight line input by the user as both ends may be defined as the specified information. The operation of the information processing apparatus 100 when not limiting the input position or range will be described later with reference to FIG. 8.

[0031] Hereinafter, the operation of the imaging system according to this embodiment will be described with reference to FIG. 2. FIG. 2 is a flowchart showing an example of control processing by the information processing apparatus 100 according to this embodiment. In a form where the information processing apparatus 100 incorporates a processor and a memory, the processing flow shown in FIG. 2 is executed by a program for causing the processor to execute the procedure shown in FIG. 2. The processor incorporated in the information processing apparatus 100 according to this embodiment is a computer, and executes a program read from the memory incorporated in the information processing apparatus 100. The memory incorporated in the information processing apparatus 100 is a recording medium that stores this program so that this processor can read it.

[0032] In S201, the information processing apparatus 100 determines whether or not it has acquired an instruction for angle-of-view setting (changing the imaging range) from the operation terminal 110. If there is no instruction for angle-of-view setting (changing the imaging range) (NO in S201), the process repeats S201, and if not, the process proceeds to S202.

[0033] In S202, the image processing unit 102 acquires information specifying the imaging range on the captured image. Here, the information processing apparatus 100 performs a first straight-line detection operation, which is an operation of acquiring specified information based on the coordinate information specified by the user input on the operation terminal 110.

[0034] Next, the first straight line detection operation will be described with reference to FIG. 3. FIG. 3(A) shows an example of a display image displayed on the screen 305 of the operation terminal 110. The display image 304 is a captured image captured by the imaging unit 101 and displayed on the screen 305. The information processing apparatus 100 acquires the coordinate values of the start point 301 and the end point 302 on the captured image specified by the user on the operation terminal 110. The information processing apparatus 100 generates a first straight line 303 by connecting the two specified points with a straight line. Further, the information processing apparatus 100 sets a specified area 306 which is a rectangular area whose vertical side and horizontal side are parallel to the display image 304 respectively, with the first straight line 303 as the diagonal line. In FIG. 3, although the subject 310 is a person, different subjects such as other animals, doors, or passages may be imaged. Hereinafter, when simply expressed as "display", it refers to a display including a captured image such as 305 in FIG. 3(A). In FIGS. 3(A) to 3(E) and FIG. 4 described later, the same reference numerals are assigned to the common configurations, and the overlapping descriptions are omitted.

[0035] In the example of FIG. 3(A), the y coordinate of the start point 301 is less than the y coordinate of the end point 302, and the imaging unit is not rotated. On the other hand, for example, in FIG. 3(B), although a straight line is input on the captured image as in the example of FIG. 3(A), the y coordinate of the start point 301 is larger than the y coordinate of the end point 302. In such a case, the image processing unit 102 can rotate the imaging unit as described above. According to such processing, for example, when the subject is upside down as shown in FIG. 3(B) (for example, because the imaging unit 101 is installed upside down), based on the input of the user who has confirmed this, it is possible to perform a rotated display of the captured image. In particular, by determining whether to perform this rotation based on the same input as the input of two points for setting imaging parameters, it is possible to control the rotated display including the vertical inversion of the display with a simpler operation than before without requiring an additional operation.

[0036] In S203, the image processing unit 102 calculates a specified area 306, which is a rectangular area, based on two points on the captured image (here, the detected first straight line 303). In S204, the image processing unit 102 performs a portrait shooting determination based on the specified area 306. The portrait shooting determination is a determination as to whether the length of the vertical side (the side in the vertical direction) of the specified area 306 is greater than the length of the horizontal side (the side in the horizontal direction). If the length of the vertical side of the specified area is greater than the length of the horizontal side, the process proceeds to S205; otherwise, the process proceeds to S207.

[0037] When the length of the vertical side of the specified area is greater than the length of the horizontal side, the image processing unit 102 according to the present embodiment can control the imaging parameters so as to rotate the imaging sensor by 90° by rotation driving. According to such processing, for example, when a vertically long (the vertical side is longer than the horizontal side) specified area is specified, it is determined that the user has specified a vertically long subject to be included in the imaging range, and rotation driving can be performed so that the imaging sensor is oriented appropriately for such a subject.

[0038] In S205, the information processing apparatus 100 controls the imaging unit 101 so that the imaging posture of the imaging unit 101 rotates clockwise by 90° about the optical axis by rotation driving. FIG. 3(C) shows an example of the screen 305 of the operation terminal 110 after the rotation driving of the imaging unit 101 from the state of FIG. 3(A). In the state of FIG. 3(C), the imaging sensor of the imaging unit 101 has rotated counterclockwise by 90° about the optical axis from before the rotation driving, and as a result, the subject 310 has rotated clockwise by 90°.

[0039] In S206, the information processing apparatus 100 controls the rotation so as to rotate the captured image counterclockwise by 90°. This control of rotation is the control of the display orientation of the captured image, which is different from the rotation drive. In other words, this control of rotation is the control of the display orientation of the captured image so that the subject 310 is in an upright state when displayed on the operation terminal 110. FIG. 3(D) shows an example of the case of rotating the display image 304 shown in FIG. 3(C). The display image 304 in FIG. 3(D) is an image obtained by rotating the display image 304 in FIG. 3(C) counterclockwise by 90°. Note that S206 may be performed after S207. Also, S206 may be performed not on the information processing apparatus 100 side but on the operation terminal 110 side.

[0040] In S207, the image processing unit 102 controls the imaging parameters of the imaging unit 101 based on the information indicating two points on the display acquired in S202. Here, the image processing unit 102 can control the imaging parameters so that panning, tilting, and zoom driving are performed such that the designated area 306 shown in FIG. 3(D) becomes the imaging range. FIG. 3(E) shows an example of the screen 305 of the captured image displayed on the operation terminal 110 when the imaging parameters are controlled and the captured image is rotated. In the example of FIG. 3(E), the zoom ratio is changed to suit the subject 310, and also the panning direction (horizontal direction) of the imaging direction is changed, and the imaging parameters are controlled so that the center of the display image 304 and the center of the designated area 306 substantially coincide. Regarding the tilt drive as well, the imaging parameters are controlled in the same manner as the above-described panning change. When S207 ends, the process returns to S201 again. Note that if the end condition (for example, when the driving for a predetermined time has ended or when a user operation indicating end has been performed) is satisfied at the end of S207, the process ends without returning to S201. In this way, by controlling the imaging parameters including the rotation drive of the imaging sensor and the pan-tilt-zoom drive, and the rotation of the captured image, in accordance with the designated area defined according to two points corresponding to the user input, it is possible to adjust the imaging angle of view including the vertical shooting change more simply than in the conventional case.

[0041] Further, the operation terminal 110 may be configured to notify the user before any one or more of the processes in S205 to S207 and allow the user to select whether the change is allowed. For example, a pop-up display may be performed on the screen 305 to allow the user to select whether the change is allowed. Alternatively, a configuration may be adopted in which the user can pre-select whether the change is allowed by performing a pre-operation using a mouse or the like. By doing this, the user can perform a change according to his or her intention. Such processing may be controlled by the operation terminal 110 or may be controlled according to an instruction from the information processing apparatus 100.

[0042] In addition, in the present embodiment, the above-described designated area 306 has been described as a rectangular area whose vertical side is parallel to the vertical side of the display image 304. However, the rectangular area used for determining the imaging range here is not particularly limited in this way, and may be inclined within the range of 0° to 90° from the state shown in FIG. 3. FIG. 4 is a diagram for explaining an example of such an inclined designated area 306.

[0043] FIG. 4 shows an example of a GUI in a detailed setting mode for more detailed setting of the designated area 306 in the operation terminal 110. For example, the image processing unit 102 may acquire information indicating two straight lines as specified information and set a rectangular area having those straight lines as diagonals as the designated area. In the example of FIG. 4, the subject 310, which is a person, is imaged with a slight deviation and inclination from the upright position. Therefore, in addition to the first straight line 303 input by the user, the second straight line 401 is used as the specified information, and the designated area 306 is set. In the present embodiment, for example, in response to the pressing of the button 402 shown in FIG. 4, the mode may be shifted to a detailed setting mode for inputting a plurality of straight lines as the specified information. According to such processing, for example, even when the imaging sensor is inclined and the long side of the captured image is not parallel to the ground, it is possible to control the imaging posture so as to be at an angle corresponding to the user instruction.

[0044] Hereinafter, with reference to FIG. 5, the details of a method for defining a specified area 306 by designating a first straight line 303 and a second straight line 401 will be described. FIG. 5 is a flowchart showing an example of control processing by the information processing apparatus 100 according to the present embodiment, which is performed when two straight lines are used as definition information. Since the processing shown in FIG. 5 is performed in the same manner as the processing shown in FIG. 2 except that S501 to S503 are performed between S202 and S203, duplicate explanations will be omitted.

[0045] In S501 following S202, the information processing apparatus 100 determines whether it has shifted to the detailed setting mode. Here, it is determined whether the detailed setting mode has been specified by the user. If it has shifted to the detailed setting mode, the process proceeds to S502, and if not, the process proceeds to S203.

[0046] In S502, the image processing unit 102 determines whether an additional operation has been performed by the user using the operation terminal 110. Here, the image processing unit 102 determines whether definition information different from the first straight line has been input. If no additional operation has been performed, the process repeats S502, and if an additional operation has been performed, the process proceeds to S503.

[0047] In S503, the image processing unit 102 acquires information indicating two points on the display that is different from what was acquired in S202. Here, the image processing unit 102 acquires, as information indicating two points at both ends of the straight line, information additionally input by the user in the detailed setting mode.

[0048] In S203, the image processing unit 102 calculates a specified area 306, which is a rectangular area, based on the points on the display (here, the detected first straight line 303 and second straight line 401). Here, a rectangular area having the first straight line and the second straight line 401 as diagonals is set as the specified area 306.

[0049] By setting a plurality of straight lines as the specified information in this way and controlling the imaging parameters based on the rectangular area set based on such a plurality of straight lines, it becomes possible to easily perform fine adjustment of the rotation angle of the imaging unit 101 and adjustment of the imaging angle of view.

[0050] In the example of FIG. 4, in addition to the two points defining the first straight line, the description has been made assuming that information indicating the two points defining the second straight line is input. However, if it is possible to obtain an input for designating other vertices of the rectangular area having the first straight line as a diagonal line in addition to the two points defining the first straight line, an input different from the two points defining the second straight line may be obtained. For example, the image processing unit 102 may obtain another point in addition to the two points defining the first straight line, and calculate a rectangular area having the two points at both ends of the first straight line as diagonal vertices and the obtained another point as another vertex as the designated area 306. As described above, the image processing unit 102 according to the present embodiment can set the designated area based on information indicating points including at least one point in addition to the two points on the display.

[0051] In the present embodiment, the description has been made assuming that a straight line is obtained as the specified information and the imaging range is set (imaging parameter control) according to the obtained straight line. However, the specified information to be obtained is not limited to a straight line as long as information indicating two points can be obtained. For example, when an ellipse is input by the user using a mouse or the like, a rectangular area circumscribing such an ellipse may be set as the designated area. Further, the image processing unit 102 may set, as the designated area, the smallest rectangular area that includes the rectangular area circumscribing such an ellipse and has the same aspect ratio as the captured image (for example, shares the vertex at the upper left end). Here, it is assumed that information indicating two points is generated with the vertex of the rectangular area closest to the first input point when the ellipse is input as the starting point and the vertex at the diagonal position of that vertex as the ending point. According to such a configuration, it becomes possible for the user to specify the imaging range by a more intuitive operation.

[0052] For example, when a specified area 306 is set (e.g., by S203), a preview thereof may be displayed on the operation terminal 110 so that the specified area 306 can be edited by the user. Hereinafter, such an example will be described with reference to FIG. 6.

[0053] In FIG. 6, in addition to the display image 304, buttons 602 to 605 used for setting imaging parameters are displayed on the screen 305. Further, a variable area 601 is superimposed and displayed on the display image 304. The variable area 601 is a rectangular area representing a preview of a specified area that can be edited by the user. For example, according to user input, when the position of the variable area 601 is changed, the pan-tilt drive amount of the imaging unit 101 is changed, when the size of the variable area 601 is changed, the zoom magnification of the imaging unit 101 is changed, and when the variable area 601 is rotated, the rotation drive amount of the imaging unit 101 is changed. When changing the size of the variable area 601, it may be changed while maintaining the aspect ratio of the variable area 601 equal to that of the display image 304, or it may be possible to change the vertical and horizontal directions separately.

[0054] The button 602 is a button for shifting to a mode for changing the pan-tilt drive amount of the imaging unit 101. The button 603 is a button for shifting to a mode for changing the zoom magnification of the imaging unit 101. The button 604 is a button for shifting to a mode for changing the rotation drive amount of the imaging unit 101. The button 605 is a button for ending the editing of the variable area 601, and the variable area 601 at the time when the button 605 is pressed is set as the specified area 306. In this way, the information processing apparatus 100 can display the specified area 306 as the variable area 601 of the preview, and by acquiring the editing content for the variable area 601, correct the specified area based on the editing content. According to the operation using such a variable area 601, the user can adjust the imaging range without defining a straight line again.

[0055] Hereinafter, with reference to FIG. 7, the details of a method for performing an editing process after setting the specified area 306 will be described. FIG. 7 is a flowchart showing an example of control processing by the information processing apparatus 100 according to the present embodiment that performs such editing processing of the specified area. Since the processing shown in FIG. 7 is performed in the same manner as the processing shown in FIG. 2 except that S701 to S505 are performed between S203 and S204, duplicate explanations will be omitted.

[0056] In S701 following S203, the information processing apparatus 100 causes the operation terminal 110 to superimpose and display the variable area 601 on the display image 304. Here, it is assumed that the initial position and size of the variable area 601 are the same as those of the specified area 306.

[0057] In S702, the information processing apparatus 100 determines whether an editing operation has been performed on the variable area 601. If no editing operation has been performed, the process repeats S702, and if an editing operation has been performed, the process proceeds to S703. In S703, the information processing apparatus 100 determines whether the performed editing operation instructs the end of the operation. If the editing operation instructs the end of the operation, the process proceeds to S204, and if not, the process proceeds to S704.

[0058] In S704, the information processing apparatus 100 determines whether an operation to change the variable area 601 has been performed. If no operation to change the variable area 601 has been performed, the process returns to S702, and if an operation to change the variable area 601 has been performed, the process proceeds to S705. In S705, the information processing apparatus 100 changes the position, size, or rotation of the variable area 601 according to the change operation, and returns the process to S702.

[0059] According to such a configuration, the user can finely adjust the specified area, and can make changes according to a more user-preferred imaging angle of view.

[0060] In addition, in this embodiment, the description has been made assuming that the designated area is a rectangular area having the same aspect ratio as the captured image. However, it is also conceivable that a rectangular area having a line segment between two points of the specified information as a diagonal and each side parallel to that of the captured image may be an area having an aspect ratio different from that of the captured image. In such a case, the image processing unit 102 will describe an example of setting a corrected area obtained by correcting the designated area set from the two points indicated by the specified information with reference to FIG. 8.

[0061] FIG. 8 shows an example of a case where the designated area 306 is an area having an aspect ratio different from that of the captured image by the imaging unit 101. After calculating the designated area 306, when the aspect ratios of the designated area 306 and the display image 304 do not match, the information processing apparatus 100 corrects the designated area 306 to generate a corrected area 801 as shown in FIG. 8. The corrected area 801 includes the entire designated area 306 and is an area having an aspect ratio that matches the display image 304. For example, the image processing unit 102 may change the length of one of the vertical side and the horizontal side while maintaining the position of the center point of the designated area 306 (in particular, by selecting the one with the smaller area of these) to generate a corrected area 801 having an aspect ratio that matches the display image 304. In the example of FIG. 8, the corrected area 801 is generated from the designated area 306 by maintaining the length of the horizontal side and extending the length of the vertical side.

[0062] According to such processing, it is possible to display a captured image having an aspect ratio corresponding to the imaging sensor without limiting the position or range of the straight line specified by the user.

[0063] In addition, in this embodiment, the description has been made assuming that the imaging parameters are controlled to image the designated area 306. However, for example, the designated area 306 may be cut out and displayed. According to such processing, it is possible to display the imaging range intended by the user after acquiring the captured image by the imaging sensor.

[0064] [Embodiment 2] In the above-described embodiments, each processing unit shown in FIG. 1 or the like is realized by dedicated hardware. However, some or all of the processing units included in the information processing apparatus 100 may be realized by a computer. In the present embodiment, at least a part of the processing according to the above-described embodiments is executed by a computer.

[0065] FIG. 9 is a diagram showing a basic configuration of a computer. In FIG. 9, a processor 901 is, for example, a CPU and controls the operation of the entire computer. A memory 902 is, for example, a RAM and temporarily stores programs, data, and the like. A computer-readable storage medium 903 is, for example, a hard disk or a CD-ROM and stores programs, data, and the like in a long term. In the present embodiment, a program for realizing the functions of each part stored in the storage medium 903 is read into the memory 902. Then, the functions of each part are realized by the processor 901 operating according to the program on the memory 902.

[0066] In FIG. 9, an input interface 904 is an interface for acquiring information from an external device. An output interface 905 is an interface for outputting information to an external device. A bus 906 connects the above-described respective parts and enables data exchange.

[0067] The disclosure of this specification includes the following information processing apparatus, information processing method, and program. (Item 1) First acquisition means for acquiring information specifying an imaging range on an imaging image captured by an imaging unit that is rotatable around an optical axis; Determination means for determining whether to rotate the imaging unit around the optical axis based on the acquired information; Control means for controlling the imaging unit based on the determination result of the determination means; An information processing apparatus comprising the above. (Item 2) The information processing apparatus according to item 1, wherein the information is information on a line segment between two points specified by a user on the captured image. (Item 3) The information includes information indicating a start point and an end point of the line segment, The information processing apparatus according to item 2, wherein the determination means determines whether to rotate the imaging unit based on a positional relationship between the start point and the end point. (Item 4) When a coordinate system is defined on the captured image with the horizontal direction as the x-axis and the vertical direction as the y-axis, The information processing apparatus according to item 3, wherein the determination means determines whether to rotate the imaging unit based on a relationship between y coordinates of the start point and the end point in the coordinate system of the captured image. (Item 5) When the downward direction in the vertical direction is defined as the positive direction of the y coordinate on the captured image, The information processing apparatus according to item 4, wherein the control means controls the imaging unit to rotate 180° when the y coordinate of the start point is greater than or equal to the y coordinate of the end point. (Item 6) The information processing apparatus according to any one of items 2 to 5, wherein the control means controls the imaging unit to image a rectangular area having the line segment as a diagonal. (Item 7) The first acquisition means further acquires information indicating at least one point in addition to the two points, The information processing apparatus according to any one of items 2 to 6, wherein the control means controls the imaging unit to image a rectangular area having the two points as diagonal vertices and having the at least one point as a vertex different from the two points. (Item 8) The apparatus further comprises correction means for generating a corrected area obtained by correcting a rectangular area having the line segment between the two points as a diagonal, The information processing apparatus according to any one of items 2 to 6, wherein the control means controls the imaging unit to image the corrected area. (Item 9) The information processing apparatus according to item 8, wherein the correction area is an area obtained by correcting the rectangular area so as to have the same aspect ratio as the captured image. (Item 10) The information processing apparatus further comprising second acquisition means for acquiring editing content for the rectangular area. The information processing apparatus according to item 8, wherein the correction means corrects the rectangular area based on the editing content. (Item 11) The information processing apparatus according to any one of items 1 to 10, wherein the imaging unit is rotatable in pan or tilt and is capable of changing a zoom magnification. (Item 12) Notification means for notifying a user before rotation control of the imaging unit. The information processing apparatus further comprising third acquisition means for acquiring an input as to whether to execute rotation control of the imaging unit. The information processing apparatus according to any one of items 1 to 11, wherein the control means executes rotation control of the imaging unit when the third acquisition means has acquired an input that rotation control of the imaging unit is to be executed. (Item 13) A step of acquiring information for designating an imaging range on a captured image captured by an imaging unit rotatable in rotation around an optical axis. Determination means for determining whether to rotate the imaging unit around the optical axis based on the acquired information. A step of controlling the imaging unit based on a determination result of the determination means. An information processing method comprising: (Item 14) A program for causing a computer to function as each means of the information processing apparatus according to any one of items 1 to 12.

[0068] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, an ASIC) that implements one or more functions.

[0069] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0070] 101: Imaging unit, 102: Image processing unit, 103: Communication control unit, 104: Pan drive unit, 105: Tilt drive unit, 106: R drive unit, 107: Zoom drive unit, 108: Drive control unit

Claims

1. First acquisition means for acquiring information specifying an imaging range on an acquired image captured by an imaging unit that can rotate rotationally around an optical axis; Determination means for determining whether to rotate the imaging unit around the optical axis based on the acquired information; Control means for controlling the imaging unit based on the determination result of the determination means; An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, wherein the information is information on a line segment between two points specified by a user on the captured image.

3. The information includes information indicating a start point and an end point of the line segment, The determination means determines whether to rotate the imaging unit based on a positional relationship between the start point and the end point. The information processing apparatus according to claim 2.

4. When a coordinate system is defined on the captured image with the horizontal direction as the x-axis and the vertical direction as the y-axis, The determination means determines whether to rotate the imaging unit based on a relationship between the y-coordinates of the start point and the end point in the coordinate system of the captured image. The information processing apparatus according to claim 3.

5. When the downward direction in the vertical direction is defined as the positive direction of the y-coordinate on the captured image, The control means controls to rotate the imaging unit by 180° when the y-coordinate of the start point is greater than or equal to the y-coordinate of the end point. The information processing apparatus according to claim 4.

6. The control means controls the imaging unit to image a rectangular area having the line segment as a diagonal. The information processing apparatus according to claim 2.

7. The first acquisition means further acquires information indicating at least one point in addition to the two points, The control means controls the imaging unit to image a rectangular area having the two points as diagonal vertices and having the at least one point as a vertex different from the two points. The information processing apparatus according to claim 2.

8. Further comprising correction means for generating a corrected area obtained by correcting a rectangular area having the line segment between the two points as a diagonal, The control means controls the imaging unit to image the corrected area. The information processing apparatus according to claim 2.

9. The information processing apparatus according to claim 8, wherein the correction area is an area obtained by correcting the rectangular area so as to have the same aspect ratio as the captured image.

10. further comprising second acquisition means for acquiring editing content for the rectangular area; The information processing apparatus according to claim 8, wherein the correction means corrects the rectangular area based on the editing content.

11. The information processing apparatus according to claim 1, wherein the imaging unit is capable of panning rotation or tilting rotation and is capable of changing a zoom magnification.

12. notification means for notifying a user before rotation control of the imaging unit; further comprising third acquisition means for acquiring an input as to whether or not to execute rotation control of the imaging unit; The information processing apparatus according to claim 1, wherein the control means executes rotation control of the imaging unit when the third acquisition means has acquired an input indicating that rotation control of the imaging unit is to be executed.

13. a step of acquiring information specifying an imaging range on a captured image captured by an imaging unit that is rotatable about an optical axis; determination means for determining whether or not to rotate the imaging unit about the optical axis based on the acquired information; a step of controlling the imaging unit based on a determination result of the determination means; An information processing method comprising:

14. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 12.

Images